Bug 11780 - Scanner failing on 'common album names'
: Scanner failing on 'common album names'
Status: CLOSED FIXED
Product: Logitech Media Server
Classification: Unclassified
Component: Scanner
: 7.4.0
: PC Windows XP
: -- normal with 2 votes (vote)
: 7.4.0
Assigned To: Andy Grundman
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2009-04-13 08:22 UTC by snarlydwarf
Modified: 2009-10-05 14:30 UTC (History)
3 users (show)

See Also:
Category: ---


Attachments
test case for 'common album names' failure (15.91 MB, application/octet-stream)
2009-04-14 09:34 UTC, snarlydwarf
Details
Bugfix (short-term version, re-instates old logic for groupdiscs mode) (1.06 KB, patch)
2009-04-14 14:37 UTC, Moonbase
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description snarlydwarf 2009-04-13 08:22:16 UTC
These two albums used to be handled correctly, they are now merged as one:

TIT2 (Title/songname/content description): Nada sei (apnéia)
TPE1 (Lead performer(s)/Soloist(s)): Kid Abelha
TRCK (Track number/Position in set): 1/19
TALB (Album/Movie/Show title): Acústico MTV
UFID (Unique file identifier): http://musicbrainz.org, 36 bytes
TXXX (User defined text information): (ALBUMARTISTSORT): Kid Abelha
TXXX (User defined text information): (MusicBrainz Album Type): live
TXXX (User defined text information): (MusicBrainz Album Id): a4ce56c4-19e6-4f3f-86ca-be9967a9586f
TXXX (User defined text information): (MusicBrainz Album Status): official
TYER (Year): 2002
TXXX (User defined text information): (ASIN): B00007M00R
TPE2 (Band/orchestra/accompaniment): Kid Abelha
TXXX (User defined text information): (MusicBrainz Album Artist Id): e3e90633-b6f1-4376-b15b-2492c94b4cf7
TXXX (User defined text information): (MusicBrainz Album Release Country): BR
TXXX (User defined text information): (MusicBrainz Artist Id): e3e90633-b6f1-4376-b15b-2492c94b4cf7
TXXX (User defined text information): (MusicBrainz Album Artist): Kid Abelha
TCON (Content type): MPB;Live (255)

and

TPE1 (Lead performer(s)/Soloist(s)): Cidade Negra
TALB (Album/Movie/Show title): Acústico MTV
TIT2 (Title/songname/content description): Girassol
TRCK (Track number/Position in set): 1/17
TXXX (User defined text information): (MusicIP PUID):
TXXX (User defined text information): (MusicBrainz Artist Id): 4a95ceff-7445-42f1-b34f-01d680777d85
TXXX (User defined text information): (MusicBrainz Album Id): 6d29f81d-15d8-40cd-8c16-052bdf58e5da
TXXX (User defined text information): (MusicBrainz Album Status): official
TXXX (User defined text information): (MusicBrainz Album Artist Id): 4a95ceff-7445-42f1-b34f-01d680777d85
TXXX (User defined text information): (MusicBrainz Album Artist): Cidade Negra
TXXX (User defined text information): (MusicBrainz Album Artist Sortname): Cidade Negra
UFID (Unique file identifier): http://musicbrainz.org, 36 bytes
TXXX (User defined text information): (MusicBrainz Non-Album): 0
TYER (Year): 2002


They have different album ID's, different artists, different directories...

They should be, well, different, but they are picked up as the same:

Album detail page shows:
 Album:  Acústico MTV
 Album Artist: Cidade Negra, Kid Abelha
 Band/Orchestra: Kid Abelha
 Track Artist: Cidade Negra, Kid Abelha
 Year: 2002 

The year seems to be the break point, I have another 'pair' of these, too:

 Album:  Acústico MTV
 Album Artist: Lenine, Rita Lee
 Artist: Lenine, Rita Lee
 Year: 1998 

I believe this is related to Moonbase's changes to VA detection logic.
Comment 2 Chris Owens 2009-04-13 09:55:13 UTC
Moonbase, does this seem plausible, or do you think there's a different issue?  As always if you don't have the time or inclination to work on this, just let me or qa@slimdevices.com know.
Comment 3 ilmonstro 2009-04-13 10:20:47 UTC
Have the same problem with three MTV Unplugged albums. They only share the same album title. I use the debian version 7.3.3 25911.
Comment 4 Mike Walsh 2009-04-14 00:04:00 UTC
is this the right bug with moonbases changes?

https://bugs-archive.lyrion.org/show_bug.cgi?id=10583
Comment 5 Moonbase 2009-04-14 04:34:44 UTC
Sorry, have been away over Easter. I’ll take a look into this tomorrow and will try to set up a test case and verify more.

Please also read the comments in
http://forums.slimdevices.com/showthread.php?p=415339#post415339

snarlydwarf: Thanks for the detailed feedback.
Mike: Yes, 10583 was the basic one.
Comment 6 Moonbase 2009-04-14 04:36:23 UTC
Q to all having this problem:
Does this happen only with MP3, only with FLAC, or with BOTH?
Comment 7 snarlydwarf 2009-04-14 09:34:53 UTC
Created attachment 5115 [details]
test case for 'common album names' failure
Comment 8 snarlydwarf 2009-04-14 09:37:07 UTC
(In reply to comment #7)
> Created an attachment (id=5115) [details]
> test case for 'common album names' failure

The key here is to set 'Treat Multi-Disc Sets as a Single Album'.

Depsite no TPOS in either file, these appear to be detected as a multi-disc set (with just one disc, since neither gets a disc number assigned to it on the track info) and then combined.
Comment 9 Moonbase 2009-04-14 10:12:24 UTC
snarlydwarf: Thanks for the test data and the hint at "multi-disc albums". Will investigate that.

Can anyone say for sure that the same problem exists with FLAC and/or other file types? (Though I assume its schema-related, not a problem of a single file type.)
Comment 10 ilmonstro 2009-04-14 10:45:14 UTC
(In reply to comment #9)
> snarlydwarf: Thanks for the test data and the hint at "multi-disc albums". Will
> investigate that.
> 
> Can anyone say for sure that the same problem exists with FLAC and/or other
> file types? (Though I assume its schema-related, not a problem of a single file
> type.)

I only use FLAC.
Comment 11 Moonbase 2009-04-14 10:47:42 UTC
For further narrowing it down:

Anyone who has set "Treat multi-disc sets as a single album", would you be willing to try setting TPOS (or DISCNUMBER in FLAC files) to a sensible value for the same files (i.e. "1/1", "1/2", "2/2", ...), do a full rescan and see if it changes the behaviour (without modifying any other tags or SC settings)?
Comment 12 snarlydwarf 2009-04-14 11:06:48 UTC
Setting TPOS to 1/2 and 2/2, "fixes" the problem, making two albums

(which is actually backwards!  Since I have 'treat as single album' set, it should treat it as a single album I would think, at least since it does when they don't have TPOS set...)

Setting them both to be 1/1 works as well.

So it is the absence of a TPOS type field that confuses it into pairing the albums.
Comment 13 Moonbase 2009-04-14 12:12:04 UTC
Again, thanks for feedback! (The TPOS stuff was just to verify part of the internal logic.) I can now acknowledge this as a bug in current versions: When using MusicBrainz Album IDs PLUS "Group Discs" mode, the folder location of the tracks isn’t taken account of, which, in turn, can lead to different albums being merged into one if they have the exact same title and no (or no differing) TPOS/DISCNUMBER/DISC/DISCC/TOTALDISCS tags.

Q @ ALL (incl. Logitech dev & QA): Can we assume (like we assume other things) that all tracks from multi-disc albums are stored in the same file folder (whether using a DISCNUMBER/DISC+DISCC or not)?

(Which would make it a trivial change; basically re-instating "the old logic" in case of groupdiscs mode and completely ignoring MUSICBRAINZ_ALBUM_ID since that can’t easily be used with groupdiscs anyway in the current schema. I’d still like to get 2-3 days for doing exhaustive testing before making the change official. This would be a "short-term fix" but I’d still like to do some more work on "correctly" grouping albums even if they are not in the same folder, like maybe multi-disc compilations where tracks could be stored in "artists" folders.)

"New Schema" development: Please earnestly consider including MUSICBRAINZ_ALBUM_ID in the track table for the new schema! This would internally allow for much easier and safer grouping/joining of unique albums in spite of any "Best of" naming problems -- be it groupdiscs mode or not.
Comment 14 Mike Walsh 2009-04-14 13:20:48 UTC
personally, i think most people store multidisc albums in one disc per folder.

so, "the wall" is:

\pink floyd\the wall (CD1)\
&
\pink floyd\the wall (CD2)\

some people might also do:

\pink floyd\the wall\cd1\
\pink floyd\the wall\cd2\

i think people who put ALL the tracks into:

\pink floyd\the wall\

are the vast minority.
Comment 15 Moonbase 2009-04-14 14:37:44 UTC
Created attachment 5118 [details]
Bugfix (short-term version, re-instates old logic for groupdiscs mode)

Attached patch to Schema.pm fixes the problem for me, using SC 7.4-25946 on both Windows and Fedora Core 10. (By re-instating the exact previous logic for groupdiscs mode.)

QA (and interested svn users) might want to verify this. I'll do more tests and will come back tomorrow with the results and a suggestion whether to include it as an official change so 7.3.3 release can be more bug-free (until we have something even better).
Comment 16 Moonbase 2009-04-14 14:42:55 UTC
Someone with appropriate rights might want to set this bug's status to RESOLVED (waiting for QA approval), I can't.
Comment 17 Andy Grundman 2009-04-14 21:24:14 UTC
Applied in 7.4 change 25955.
Comment 18 Moonbase 2009-04-15 03:48:26 UTC
Andy, cheers for the "quick fix", BUT ... I’d NOT yet consider it "production quality FIXED" because using the "old logic" with groupdiscs mode again breaks compilations with tracks stored in different folders in some circumstances (I ran a long test using ~26k files).

You can leave the current patch in for the moment (it will better things but not cure everything), but I’m still working on a better change that should also handle the above situation(s).

I’ll keep this bug updated and let you know. Rough estimate with testing: 2-3 days.
Comment 19 Moonbase 2009-04-15 07:09:04 UTC
Ok, leave at RESOLVED FIXED. And consider including in 7.3.3 release after some more positive feedback (and/or QA testing).

I did a lot of tests now, but it seems that with the current schema/database structure there is no "one-fits-all" solution for each and every use case.

I find that ONE case is still unhandled: If you have a VA compilation, use "Treat multi-disc sets as a single album" (groupdiscs mode), have no DISC and DISCC set (or TPOS in MP3, or DISCNUMBER or TOTALDISCS...), PLUS store the single tracks of the compilation in different folders (i.e., under "Artist" folders) instead of grouping the release/disc in one folder, SC will show as many "albums" as there are tracks in the compilation.

There is currently no easy solution to that, EXCEPT letting users know to ALWAYS set at least a DISC number (better both DISC and DISCC, or TPOS or DISCNUMBER in "x/y" style) in this case. This will aggregate even these "scattered around" tracks into one album. Since we reverted the "Disc 1/1" special case to normal, even a discnumber of "1/1" can and SHOULD be set for this special case. Many applications (like for instance the dBpoweramp ripper) set disc numbers by default (even disc 1/1), so most of the user base should be happy with the results.

All other mentioned cases, like different folder structures (mentioned in the forum thread) I have tested as good as possible. Snarlydwarfs test files also work ok (no albums by different artists grouped together anymore). Previous logic (as of bug 10583) is unharmed and works as before.

Tested on SC 7.4-25946 (plus my patch) and SC 7.4-25965 on both Windows and Linux platforms (Fedora Core 10).

Let’s consider this closed and move on ... :-)
Comment 20 James Richardson 2009-10-05 14:30:22 UTC
This bug has been marked as fixed in the 7.4.0 release version of SqueezeBox Server!
    * SqueezeCenter: 28672
    * Squeezebox 2 and 3: 130
    * Transporter: 80
    * Receiver: 65
    * Boom: 50
    * Controller: 7790
    * Radio: 7790  

Please see the Release Notes for all the details: http://wiki.slimdevices.com/index.php/Release_Notes

If you haven't already, please download and install the new version from http://www.logitechsqueezebox.com/support/download-squeezebox-server.html

If you are still experiencing this problem, feel free to reopen the bug with your new comments and we'll have another look.