[emms-help] Album artist grouping in emms-browser

[Pierre Neidhardt]

I haven't looked into this properly, sorry about that, so I'll just drop
a quick question: is it possible to group by album-artist in the browser
view?

A quick search did not return much on the mailing list (did I miss it?)
and I found this hack on StackExchange:

https://emacs.stackexchange.com/questions/10412/sort-by-artist-in-emms-with-compilation-albums/10435

Can we do better?

Cheers!

[Petteri Hintsanen]

Post by Pierre Neidhardt
I haven't looked into this properly, sorry about that, so I'll just drop
a quick question: is it possible to group by album-artist in the browser
view?

I use this

(defun ph-emms-browser-get-track-field (track type)
"Return TYPE from TRACK.
This function uses 'info-albumartistsort, 'info-albumartist,
'info-artistsort, 'info-originalyear, 'info-originaldate and
'info-date symbols, if available for TRACK."
(cond ((eq type 'info-artist)
(or (emms-track-get track 'info-albumartistsort)
(emms-track-get track 'info-albumartist)
(emms-track-get track 'info-artistsort)
(emms-track-get track 'info-artist "(unknown)")))
((eq type 'info-year)
(let ((date (or (emms-track-get track 'info-originaldate)
(emms-track-get track 'info-originalyear)
(emms-track-get track 'info-date)
(emms-track-get track 'info-year "(unknown)"))))
(ph-extract-year-from-date date)))
(t (emms-track-get track type "(unknown)"))))


(setq emms-browser-get-track-field-function
#'ph-emms-browser-get-track-field)


Remove info-year cond if you don't want to use originalyear/originaldates.

HTH, Petteri

[Pierre Neidhardt]

That does not work for me.

(emms-track-get track 'info-albumartistsort)

is always nil. So it only ever uses 'info-artist. Something I missed?

--
Pierre Neidhardt

[Petteri Hintsanen]

Post by Pierre Neidhardt
That does not work for me.
(emms-track-get track 'info-albumartistsort)
is always nil. So it only ever uses 'info-artist. Something I missed?

Perhaps you don't have that symbol in your track database
(emms-cache-db; ~/.emacs.d/emms/cache)?

I'm using emms-print-metadata via emms-info-libtag to extract tags from
files; it returns info-albumartist and info-albumartistsort if they
exist in the input file. I don't know if other emms-info-functions
work.

I also forgot to include this defun in my previous mail. It grabs year
from somewhat obscure date formats like YYYY-MM. (You get that kind of
data from MusicBrainz).

(defun ph-extract-year-from-date (date)
"Try to extract year part from DATE.
Return DATE if the year cannot be extracted."
(let ((year (nth 5 (parse-time-string date))))
(if year (number-to-string year)
(if (string-match "^[ \t]*[0-9]\\{4\\}" date)
(substring date (match-beginning 0) (match-end 0))
date))))


regards,
Petteri

[Pierre Neidhardt]

Post by Petteri Hintsanen
Perhaps you don't have that symbol in your track database
(emms-cache-db; ~/.emacs.d/emms/cache)?

Indeed, I don't. See below as for why.

Post by Petteri Hintsanen
I'm using emms-print-metadata via emms-info-libtag to extract tags from
files; it returns info-albumartist and info-albumartistsort if they
exist in the input file. I don't know if other emms-info-functions
work.

Me too, and I have the latest taglib 1.11.1. The culprit is either

Post by Petteri Hintsanen
emms-print-metadata ~/music/Drive/2011.\ Drive/14.\ Cliff\ Martinez\ -\ Hammer.ogg

info-artist=Cliff Martinez
info-title=Hammer
info-album=Drive
info-tracknumber=14
info-year=2011
info-genre=Soundtrack
info-note=
info-playing-time=284

Post by Petteri Hintsanen
mediainfo ~/music/Drive/2011.\ Drive/14.\ Cliff\ Martinez\ -\ Hammer.ogg

General
Complete name : /home/ambrevar/music/Drive/2011. Drive/14. Cliff Martinez - Hammer.ogg
Format : Ogg
File size : 5.05 MiB
Duration : 4 min 44 s
Overall bit rate mode : Variable
Overall bit rate : 149 kb/s
Album : Drive
Album/Performer : Drive
Track name : Hammer
Track name/Position : 14
Performer : Cliff Martinez
Genre : Soundtrack
Recorded date : 2011
Writing application : Lavc57.89.100 libvorbis

mediainfo returns "Drive" as Album/Performer, which is correct. This is
missing from emms-print-metadata.

[Petteri Hintsanen]

Post by Pierre Neidhardt
mediainfo returns "Drive" as Album/Performer, which is correct. This is
missing from emms-print-metadata.

Then there is something different within tags, I guess. What does
ogginfo show?

Petteri

[Pierre Neidhardt]

Post by Petteri Hintsanen
Then there is something different within tags, I guess. What does
ogginfo show?

$ ogginfo 14.\ Cliff\ Martinez\ -\ Hammer.ogg
Processing file "14. Cliff Martinez - Hammer.ogg"...

New logical stream (#1, serial: 5672e153): type vorbis
Vorbis headers parsed for stream 1, information follows...
Version: 0
Vendor: Lavf57.71.100
Channels: 2
Rate: 44100

Nominal bitrate: 192.000000 kb/s
Upper bitrate: 4294967.295000 kb/s
Lower bitrate: 4294967.295000 kb/s
User comments section follows...
encoder=Lavc57.89.100 libvorbis
ALBUMARTIST=Drive
TRACKNUMBER=14
date=2011
genre=Soundtrack
title=Hammer
artist=Cliff Martinez
album=Drive
Vorbis stream 1:
Total data length: 5291569 bytes
Playback length: 4m:44.399s
Average bitrate: 148.848636 kb/s
Logical stream 1 ended

I tried adding a "performer" and "album_artist" command with ffmpeg:
ogginfo displays them, but not emms-print-info.

[Petteri Hintsanen]

Post by Pierre Neidhardt
ogginfo displays them, but not emms-print-info.

ALBUMARTIST should be enough. Can you send me a chunk, say the first 16
kBs, of your ogg file:

dd count=16 bs=1k if=your.ogg of=chunk.ogg

so that I can investigate what libtag does.

regards,
Petteri

[Pierre Neidhardt]

$ emms-print-metadata chunk.ogg
info-album=Drive
info-albumartist=Drive
info-artist=Cliff Martinez
info-date=2011
info-genre=Soundtrack
info-title=Hammer
info-tracknumber=14
info-playing-time=1
emms-print-metadata chunk.ogg

info-artist=Cliff Martinez
info-title=Hammer
info-album=Drive
info-tracknumber=14
info-year=2011
info-genre=Soundtrack
info-note=
info-playing-time=1

Notice the differences:

- You have 'info-date', I have 'info-year'.

- I have 'info-note' which I suppose is a bogus 'info-albumartist'.

This machine has Debian stable (Stretch) with taglib 1.11.1. So you
ought to get identical output.

I use an up-to-date Gentoo with emms 4.1, taglib 1.11.1.

ldd emms-print-metadata

linux-vdso.so.1 (0x00007ffe833fe000)
libtag_c.so.0 => /usr/lib64/libtag_c.so.0 (0x00007f0d4a0f9000)
libc.so.6 => /lib64/libc.so.6 (0x00007f0d49d60000)
libtag.so.1 => /usr/lib64/libtag.so.1 (0x00007f0d49a66000)
libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/libstdc++.so.6 (0x00007f0d4966d000)
libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/libgcc_s.so.1 (0x00007f0d49456000)
/lib64/ld-linux-x86-64.so.2 (0x00007f0d4a2fe000)
libz.so.1 => /lib64/libz.so.1 (0x00007f0d4923f000)
libm.so.6 => /lib64/libm.so.6 (0x00007f0d48f3b000)

Nothing strange there, or is it?

[Petteri Hintsanen]

Post by Pierre Neidhardt
I use an up-to-date Gentoo with emms 4.1, taglib 1.11.1.

You should get emms 4.2 or newer. Version 4.1 uses the old
emms-print-metadata, which is implemented using TagLib's C interface.
New emms-print-metadata is written in C++ and has more features,
including albumartist.

Post by Pierre Neidhardt
libtag_c.so.0 => /usr/lib64/libtag_c.so.0 (0x00007f0d4a0f9000)
libtag.so.1 => /usr/lib64/libtag.so.1 (0x00007f0d49a66000)

Here libtag_c is the C interface.

Please try again with a recent EMMS.

Thanks,
Petteri

[Pierre Neidhardt]

Damn! Somehow I was convinced that 4.1 was the latest version!
Got fooled by Gentoo...

I've requested a version bump: https://bugs.gentoo.org/635796

Problem fixed, thank you very much!

[Yoni Rabkin]

Post by Pierre Neidhardt
Damn! Somehow I was convinced that 4.1 was the latest version!
Got fooled by Gentoo...
I've requested a version bump: https://bugs.gentoo.org/635796

thank you; that's very useful

[Pierre Neidhardt]

I noticed that setting emms-browser-get-track-field-function as
suggested in this thread will fail to work on tracks that were already
cached.

Is this a problem on the display side or on the caching side?
I haven't investigated much yet.

[Petteri Hintsanen]

Post by Pierre Neidhardt
I noticed that setting emms-browser-get-track-field-function as
suggested in this thread will fail to work on tracks that were already
cached.

Your cache probably contains records from the older version of
emms-print-metadata. These records do not have albumartist and other
such tags.

Sadly enough the cache does not have versioning information, as far as I
know. So if metadata tools (in this case emms-print-metadata) get
updated, the cache is not invalidated. I usually do something like
(setq emms-cache (make-hash-table)) to clear the cache. After that it
gets rebuild. It would be nice to have this done automatically.

Caching in general is not particularly smart at the moment. Also the
metadata extraction could be orders of magnitude faster by batching; now
there is one exec per track.

Regards and season's greetings,
Petteri

[Petteri Hintsanen]

I usually do something like (setq emms-cache (make-hash-table)) to
clear the cache.

Sorry, the correct var is emms-cache-db

Petteri

[Pierre Neidhardt]

Post by Petteri Hintsanen
Your cache probably contains records from the older version of
emms-print-metadata. These records do not have albumartist and other
such tags.

You were right. I just tried again on a fresh cache and it works.

Post by Petteri Hintsanen
Caching in general is not particularly smart at the moment. Also the
metadata extraction could be orders of magnitude faster by batching; now
there is one exec per track.

Any specific example where this slowdown shows?

[Petteri Hintsanen]

Post by Pierre Neidhardt

Post by Petteri Hintsanen
Caching in general is not particularly smart at the moment. Also the
metadata extraction could be orders of magnitude faster by batching; now
there is one exec per track.

Any specific example where this slowdown shows?

Nothing particular, but consider, for instance, a case where you want to
populate the cache by inserting a large directory tree into the
playlist, say "~/Music". After that open the playlist, and you can see
how track metadata are extracted one-by-one with speed of something like
10 tracks per second. My fairly small music collection has about 4000
tracks, and it takes many minutes to extract tags from them.

By batching it would be significantly faster. emms-print-metadata and
others could take multiple files per each invocation and output the
corresponding tags. This is not as straight-forward as it sounds,
though, but certainly doable for at least some metadata backends.

Fortunately caching is (or should be) done only once. MPD users are
also lucky because they can populate the cache from MPD in an instant.

Petteri

[Pierre Neidhardt]

After that open the playlist, and you can see how track metadata are
extracted one-by-one with speed of something like 10 tracks per
second.

Wow, it's more like 2 tracks per second here on my machine :(
Am I doing something wrong?

Fortunately caching is (or should be) done only once. MPD users are
also lucky because they can populate the cache from MPD in an instant.

Is there any drawback in using MPD? I actually wonder if MPD shouldn't
be the natural backend for EMMS. Re-implementing caching and metadata
extraction seems like re-inventing the wheel.

--
Pierre Neidhardt

QOTD:
"Like this rose, our love will wilt and die."

[Petteri Hintsanen]

Post by Pierre Neidhardt
Wow, it's more like 2 tracks per second here on my machine :(
Am I doing something wrong?

Probably not. My "estimate" was really just a guess.

Post by Pierre Neidhardt
Is there any drawback in using MPD? I actually wonder if MPD shouldn't
be the natural backend for EMMS. Re-implementing caching and metadata
extraction seems like re-inventing the wheel.

No. I have used MPD on some machines, it works well. But the version I
ran did not recognize albumartist etc. Recent versions do, however.

I wouldn't dare to call any backend "natural", as I would expect users
to have wildly different setups. They are emacs users, after all...
Also, MPD takes more effort to set up properly.

Petteri

[Pierre Neidhardt]

Post by Petteri Hintsanen

Post by Pierre Neidhardt
Wow, it's more like 2 tracks per second here on my machine :(
Am I doing something wrong?

Probably not. My "estimate" was really just a guess.

Post by Pierre Neidhardt
Is there any drawback in using MPD? I actually wonder if MPD shouldn't
be the natural backend for EMMS. Re-implementing caching and metadata
extraction seems like re-inventing the wheel.

No. I have used MPD on some machines, it works well. But the version I
ran did not recognize albumartist etc. Recent versions do, however.
I wouldn't dare to call any backend "natural", as I would expect users
to have wildly different setups. They are emacs users, after all...
Also, MPD takes more effort to set up properly.

My latest fix for later-do dramatacally improves info query
performance. Default should be around 40 tracks per second (this is
customizable).
Considering this, it makes MPD far less necessary for Emms.

[Yoni Rabkin]

Post by Pierre Neidhardt

After that open the playlist, and you can see how track metadata are
extracted one-by-one with speed of something like 10 tracks per
second.

Wow, it's more like 2 tracks per second here on my machine :(
Am I doing something wrong?

Fortunately caching is (or should be) done only once. MPD users are
also lucky because they can populate the cache from MPD in an instant.

Is there any drawback in using MPD? I actually wonder if MPD shouldn't
be the natural backend for EMMS. Re-implementing caching and metadata
extraction seems like re-inventing the wheel.

There shouldn't be such a thing as a preferred backend for Emms. Since
Emms is potentially taken wherever Emacs goes, and Emacs goes
everywhere, Emms should remain as modular and flexible as possible in
this regard.

This isn't to say that we can't have excellent MPD support, but that
supporting any one backend shouldn't ever be at the expense of another.

I also acknowledge that this means that we would be reimplementing bits
and pieces of functionality and that the reimplementation may be
sub-par. But my stance is that it is a price worth paying for the
overall goal of supporting all of the free media playing software we
can.

[Pierre Neidhardt]

Post by Petteri Hintsanen
Caching in general is not particularly smart at the moment. Also the
metadata extraction could be orders of magnitude faster by batching; now
there is one exec per track.

I initially investigated in that direction to optimize info queries, but
that wasn't it. In fact, emms-print-metadata can return retur info for
10000 files in a couple of seconds.

The main bottle neck was with later-do which pause 0.5s every track.
I've added some batch-processing there. See my recent commit on later-do.

[Petteri Hintsanen]

Post by Pierre Neidhardt

Post by Petteri Hintsanen
Caching in general is not particularly smart at the moment. Also the
metadata extraction could be orders of magnitude faster by batching; now
there is one exec per track.

...

Post by Pierre Neidhardt
The main bottle neck was with later-do which pause 0.5s every track.
I've added some batch-processing there. See my recent commit on later-do.

I see. But there still is one process invocation per track? I would
prefer a design where metadata extractor would accept multiple track
file names and output tags for them in a batch. This would be easy to
do with emms-print-metadata via simple stdin / stdout. Of course it
requires some modifications to the elisp side, and things have to be
backwards compatible so that other info methods still work.

I can think about some solution, but don't hold your breath. Batching
later-do is probably good enough solution for most use cases.

Petteri

[Pierre Neidhardt]

Post by Petteri Hintsanen

Post by Pierre Neidhardt

Post by Petteri Hintsanen
Caching in general is not particularly smart at the moment. Also the
metadata extraction could be orders of magnitude faster by batching; now
there is one exec per track.

...

Post by Pierre Neidhardt
The main bottle neck was with later-do which pause 0.5s every track.
I've added some batch-processing there. See my recent commit on later-do.

I see. But there still is one process invocation per track? I would
prefer a design where metadata extractor would accept multiple track
file names and output tags for them in a batch. This would be easy to
do with emms-print-metadata via simple stdin / stdout. Of course it
requires some modifications to the elisp side, and things have to be
backwards compatible so that other info methods still work.

I initially went down that road before patching later-do: I implemented
multiple argument support for emms-print-metadata and only then I
realized it was not the bottleneck.
In fact, emms-print-metadata is extremely fast and adding batch support
to it would only result in a negligible performance boost.

Emms' design makes batch processing even more impractical: the first
thing all the file-query-functions (`emms-add-file',
`emms-add-directory', etc.) do is to split the file queue into
individual files before sending them down a long sequence of functions
all of which can only work on individual files. At the very end of the
function sequence waits the dreaded `later-do'.

Changing this design would imply a lot of refactoring... For little
benefit I believe.

--
Pierre Neidhardt

[Petteri Hintsanen]

Post by Pierre Neidhardt
Changing this design would imply a lot of refactoring... For little
benefit I believe.

I believe your analysis is correct. It would be stupid to break an
otherwise simple design (which is intrinsically good) and working
implementation.

Thanks,
Petteri

[Pierre Neidhardt]

Post by Petteri Hintsanen
I use this
(defun ph-emms-browser-get-track-field (track type)
"Return TYPE from TRACK.
This function uses 'info-albumartistsort, 'info-albumartist,
'info-artistsort, 'info-originalyear, 'info-originaldate and
'info-date symbols, if available for TRACK."
(cond ((eq type 'info-artist)
(or (emms-track-get track 'info-albumartistsort)
(emms-track-get track 'info-albumartist)
(emms-track-get track 'info-artistsort)
(emms-track-get track 'info-artist "(unknown)")))
((eq type 'info-year)
(let ((date (or (emms-track-get track 'info-originaldate)
(emms-track-get track 'info-originalyear)
(emms-track-get track 'info-date)
(emms-track-get track 'info-year "(unknown)"))))
(ph-extract-year-from-date date)))
(t (emms-track-get track type "(unknown)"))))

I've added and adapted version of the function to Emms and made it the
default.
I've documented the option in the manual.

Commit coming soon.

[Petteri Hintsanen]

Post by Pierre Neidhardt

Post by Petteri Hintsanen
I use this
(defun ph-emms-browser-get-track-field (track type)

I've added and adapted version of the function to Emms and made it the
default.

This is ok, except that it will display 'info-albumartistsort and
'info-artistsort in the browser lists. These may seem unnatural to some
users. Eg. "The Prodigy" will be displayed in the artist list as
"Prodigy, The". It would be good to at least mention this in the
manual.

A correct implementation would collate by 'info-albumartistsort (or
'info-artistsort) and display 'info-albumartist. For example Rhythmbox,
or whatever it is called nowadays, does that. I've been trying to
implement the same feature into the browser, but it's more difficult
than one might think. But OTOH I'm fairly novice with elisp, and
someone more familiar with the browser's internals might succeed
quickly.

regards,
Petteri

[Pierre Neidhardt]

Good point. I never use *sort tags, so I would never have noticed.
I'll work on it when I have time.

[Pierre Neidhardt]

Perfectly clear, thanks.
I'll work on it if I have time.
In the meanwhile, I hope the current behaviour at least _displays_ well :)