Re: [dev] music db editor

From: Evan Buswell <ebuswell_AT_gmail.com>
Date: Sat, 12 Oct 2013 01:18:50 -0700

mpd is actually about 50% orthogonal to what I want to do. It
maintains a database in order to play files. I'm pretty OK with the
actual players out there. The primary purpose here is not playing
music but managing and editing the metadata associated with music
files; I just threw in the call-an-external-player feature since, if
you already have a means to select a bunch of files based on tags (for
inspection/editing), this might literally be about three lines of code
extra. Although code is not the only consideration, so perhaps that
doesn't belong.

I kind of wondered whether I'd get this response. And of course no
worries if people prefer to go with existing tools. But I do think
there's a real need, at least for me. Maybe another way of thinking
about it is what I'm building would be to tools like id3v2 as an
interactive text editor is to sed. Sed is not replaceable by vi, and
in many ways is "better" and more flexible. But you wouldn't want sed
to be the only way to edit your file. Same for a music database.
Sometimes it's convenient to use things like for file in `find -name
*.mp3`; do if id3v2 -l $file|grep -q 'TPE1.*something'; then id3v2 -a
'something else' $file; fi; done. And indeed, wrapped in some scripts
that hid the boilerplate for the 90% cases, that's what I did for
years. But many typical tasks just don't map very well to this
paradigm, e.g. look through the entire music database and make sure
you don't have anything tagged incorrectly. What's incorrect? Hard to
say; you only really know it when you see it. It's that "seeing it"
that's made difficult by most command line tools. You can create
heuristics, but these only go so far. That being said, I'm not married
to having this functionality follow the "music manager" paradigm; in
fact I'd like to move away from that as much as possible, while still
keeping visualization/interactive editing. One possible less
monolithic way would be to have three programs; one which sucked tags
into a tabular database; one which tagged files based on the database;
and one which provided a sort of visual editing of a tabular database.
Potentially the last could even be a regular text editor (although
that wouldn't solve the issue of album art). But I get fidgety when I
think about nontrivial databases being replicated, edited, and then
overwritten. Potentially, one bug and boom, the whole thing's gone.

-Evan

On Sat, Oct 12, 2013 at 12:16 AM, Daniel Bryan <danbryan_AT_gmail.com> wrote:
> The reason all the existing solutions are so sucky is that they try to
> solve too many problems. I think that if you deconstruct what you want
> this program to do, the concept falls apart a little.
>
> I'm not sure how "suckless" it is in terms of the code - for all I know
> it's a maintainers' nightmare - but it seems like MPD already does
> a lot of what you want.
>
> Furthermore, it has a simple client-server architecture; MPD just binds
> to a socket and clients talk to it with a simple control protocol.
> Security can be handled at the network level. Anyone can write a client;
> I use mpc, a nice unix-Y utility, but there are clients written in GTK
> and QT and for Android and iOS etc. (they all suck compared to a CLI
> interface obviously). Some of the web interfaces aren't too bad.
>
> This is my entire mpd.conf:
>
> music_directory "/var/lib/mpd/music"
> db_file "/var/lib/mpd/mpd.db"
> log_file "/var/log/mpd/mpd.log"
> log_level "verbose"
> pid_file "/var/run/mpd/mpd.pid"
> state_file "/var/lib/mpd/mpdstate"
> user "mpd"
>
> audio_output {
> type "alsa"
> name "My Sound Card"
> }
>
>
>> 1) There is no actual "music database" that's updated. There is just a
>> cache db which automatically indexes the metadata embedded in music
>> files. The tags on the file are always authoritative.
>
> This is precisely what MPD does.
>
>> 5) Sorting. Searching. This could potentially be done through straight
>> sql (and use of sqlite), but then I don't like sql much, so...yeah. I
>> suppose this will likely just have
>> key--regex--sortkey/prevsortkey/prevprevsortkey/etc. It would be nice
>> to be completely general, but that would involve the creation of a
>> sql-like language that isn't awkward for 98% of actual queries.
>
> Searching:
>
> mpc search artist Fela
>
> Sorting and filtering:
>
> mpc search artist Fela | sort -r | grep 'Expensive Shit'
>
>> 7) Editing. Undecided.
>
> Specialised tools do this better.
>
> I'm confused by the perceived need for a "music manager" at
> all. There are fantastic scripts out there for munging idv3 tags and all
> that; combined with find and xargs you can do anything. For playback and
> browsing your library, MPD does a great job.
>
> I don't know what "management" there is to be done beyond that.. is this
> program designed to solve the fact that you didn't organise your music
> files into artist and album in the first place?
>
> It sounds like this could be a bit of a monolithic beast.
>
> I did once start (and abandon) a tool that did a small part of this: it
> would examine the tags in my music folder and identify:
>
> * files that weren't organised correctly into folders by artist and
> album
> * files that seemed to be missing key tags like track number, artist,
> album
>
> It would print the filename, the artist and the album, tab-delimited, so
> it was easy to then script moving the offending files into the right
> place with cut, xargs, awk, etc.
>
> Daniel
>
>
> On 2013-10-11, Evan Buswell wrote:
>> Hi all,
>>
>> New to this list, but a great admirer of some of the elegance I've
>> seen coming out of the suckless community. Apologies for a very long
>> first post, but it's because I'm trying to help a lot? We'll go with
>> that. Also, double apologies if this ought to go elsewhere. It seemed
>> like a proposal that would fit in well with the current set of
>> suckless programs and others on this list might be interested in it.
>>
>> In what began as a sort of rage against the immense suckiness of
>> rhythmbox, banshee, clementine, itunes, etc. etc., I've been thinking
>> about writing a music db management program that would do just that.
>> Existing software is so awful and agglomerative that almost anything
>> would be better, so all the more reason for caution at the beginning.
>> I know plenty of people probably just happily/unhappily use fs
>> organization + mplayer. Awesome if that works for you. It used to, but
>> the niceness of not having to use crap was finally outweighed by the
>> awkwardness of this method and its distance from hardware players.
>> Here's what I'm thinking so far:
>>
>> 1) There is no actual "music database" that's updated. There is just a
>> cache db which automatically indexes the metadata embedded in music
>> files. The tags on the file are always authoritative.
>>
>> 2) There is one configurable location with music files. If a user
>> needs them in more than one place, they can use symlinks. By default
>> this is just ~/Music.
>>
>> 3) The music folder is watched. The cache is updated/invalidated
>> automatically when a file is altered or a new file/folder is added
>> anywhere. This should be easy/lightweight enough with inotify,
>> although other more portable suggestions are welcome so long as they
>> don't involve something hideous that pulls in all of gnome. On startup
>> the whole shebang will have to be reindexed, but careful use of stored
>> mtimes, background reindexing, plus a fallback start-with-clear-cache
>> flag should keep this from getting too out of hand.
>>
>> 4) Support for meta-tags, like "artist" that are present everywhere
>> but differ in id3v1, the several id3v2s, flac, etc. Access to any tag
>> directly by name when that is needed.
>>
>> 5) Sorting. Searching. This could potentially be done through straight
>> sql (and use of sqlite), but then I don't like sql much, so...yeah. I
>> suppose this will likely just have
>> key--regex--sortkey/prevsortkey/prevprevsortkey/etc. It would be nice
>> to be completely general, but that would involve the creation of a
>> sql-like language that isn't awkward for 98% of actual queries.
>>
>> 6) Display. Needs to be graphical bc of embedded pictures. But should
>> be only one view: a simple table with configurable columns
>> corresponding to tags/metatags. Album art is a column in the table
>> just like everything else. If someone wants some itunes-like album
>> picture list view (and don't get me wrong, I kinda like that), there
>> should be a separate chooser program that gives that view and nothing
>> else.
>>
>> 7) Editing. Undecided. Could be simple double click on a field makes
>> it editable. Better might be an interactive sed-like regex, but this
>> has the potential to totally wreck massive numbers of files very
>> quickly. Any way about it I can't completely depend on analogy to
>> sed/ed/vi/whatever, as there's images to consider. Anything more
>> automagically complicated than opening a file chooser for images is
>> immediately barred as something to be implemented in a separate
>> program, if at all.
>>
>> 8) Playback. No playback. But can launch a specified command with a
>> group of files based on a search/sort.
>>
>> 9) Organize files. No file organization, but I'll likely build a
>> separate commandline program that arranges the filesystem based on
>> tag. Actually, this is the only part that I've started.
>>
>> 10) Sync. Should be able to manually sync with libgpod or libmtp
>> supported devices. As in: search, sync results with device x.
>> Automatic sync of a complex search should be supported through shell
>> scripts. Not decided, but atm I feel like taking files *from* these
>> devices is separate enough to jettison.
>>
>> Questions:
>>
>> 1) Most of these types of programs are now media rather than music
>> management programs. Does anybody actually want this? I'm all for more
>> generality if it can be done with minimal added complexity and in a
>> way which actually results in more simplicity, rather than just
>> idiotic agglomeration. But I suspect this is just agglomeration.
>> Nobody tags movies, AFAIK.
>>
>> 2) Toolkit? Is swk ready for a project like this? Is gtk2 too sucky? I
>> love C. A lot. But it may not be the best for quick, simple GUI
>> development (yet!). Fallback is either tcl/tk, or Python+tkinter.
>> Actually in all likelihood I'll use Python+tkinter, but I want to
>> explore other options first.
>>
>> 3) Suggestions on being more simple? Can this be usefully partitioned
>> into more than one app? Can I usefully not do some of this? I'm open
>> to just having a few command line tag/copy/etc programs, too, but
>> somehow the task of music management would need to be made as fast as
>> using something like Clementine is, on the rare occasions that it
>> actually works and does what you want instead of simplifying in all
>> the wrong places and then crashing. Really, my preferred way of doing
>> this would have been to just write a FUSE filesystem and call it a
>> day. Filesystems and databases have always seemed redundant to me. The
>> trouble is that actually quite a few tags have "/" embedded in them,
>> so this is not really workable atm. Why shells/filesystems/tools
>> didn't long ago provide for full binary filename support with escape
>> sequences is beyond me.
>>
>> 4) Suggestions on programs which already do all or a portion of this?
>> Possible reusable crossovers to other projects? It strikes me that 90%
>> of this could be implemented as a completely generic interactive
>> key--value database editor. But I'm not sure the glue and special
>> cases for the other 10% (cache vs. db, plus the extra variable of a
>> file corresponding to a data set) is worth it.
>>
>> Caveats:
>>
>> Just (!) that I'm in a PhD program and should be doing that instead of
>> thinking about this. And probably randomly will. For months at a time.
>> But it's possible that with a good, simple starting design and
>> judicious use of existing libraries this could be a pretty quick
>> development process. Especially with help. Any way about it, if enough
>> people are interested I'm fine taking more of a background role,
>> anything from co-maintainer to mindless (but shiftless) slave. I just
>> want to never use Rhythmbox/clementine/banshee/etc. ever again.
>>
>> For whoever made it this far, thank you!
>> -Evan
>>
>
Received on Sat Oct 12 2013 - 10:18:50 CEST