Bugzilla – Bug 2219
Genre scanning completly broken for whole-album flacs with embedded cuesheet metadata
Last modified: 2008-09-15 14:36:01 UTC
Browsing by genre has never worked for me with the 6.2 version of Slimserver. With whole-album flac files with embedded metadata cuesheets, the genre_track table gets corrupted and only ends up with a very small number of records linking genre to less than 10% of the tracks in my library. Tested version: Slimserver 6.2 svn 4458 on WindowsXP SP2 & Linux: Fedora FC4 1456 Steps to reproduce the bug: 1). Download the posted zip file and extract the test files to a folder. 2). Set Slimserver's music folder to the folder containing the test files. 3). Perform a "Clear library and rescan everything." 4). Browse by genre. You will observer that only a fraction of the albums and tracks show up.
Created attachment 874 [details] zipfile containing flacs that demonstrate the bug
As an additional comment, I have noticed this: 1) If I have a Flac file (whole-album) with standard tags of "Artist", "Album", "Genre" and "Year" and no embedded cuesheet, Slimserver shows the album under the corresponding genre and year. 2) If I embed a cuesheet (not containing genre and year, just a CD-TEXT cuesheet as generated by EAC) then the GENRE does not appear anymore on the list (unless theres another album in that same genre without an embedded cuesheet) Similar behaviour happens with the YEAR tag, but it seems like all the years get listed , but no album appears under any one of them.
Created attachment 875 [details] zipfile containing duplicate metadata from the previously posted test files
I think I've demonstrated that the problem must be with some difference between the embedded and external cuesheet scanning code. I'll post 3 additional zip files: one with the flacs with all metadata stripped out, but with external cuesheets for the flacs saved in ansi; the same, but with the cues saved as UTF8, and finally the same, but with the cues saved UTF8 with byte-order-marks. Slimserver svn 4473 doesn't have any trouble at all with any of the three versions (ANSI, UTF8, UTF8-with-BOM) with external cuesheets. Browsing by genre with external cuesheets seems to be working perfectly. The whole problem seems to be with the embedded cuesheet scanning code. I'm sorry that I don't know the code, but I'll ask a question anyway: Is it possible that there is a conflict between the parsing of the embedded cuesheet (tag: comment[0]:cuesheet=etc.) and the metaflac imported CUESHEET tag (METADATA block type 5, which contains just frame-offset info, if I'm not mistaken.)?? Is it possible that the comment[0]:cuesheet= data is parsed just fine, but then the tracks get thrown away when the type 5 cuesheet offsets are scanned?? (Just an idle thought..)
Created attachment 876 [details] Same flacs, no metadata, ansi external cuesheets
Created attachment 877 [details] Same flacs, no metadata, UTF8 external cuesheets
Created attachment 878 [details] Same flacs, no metadata, UTF8 with BOM external cuesheets
Ok - This should be fixed in subversion change 4495. You'll need to wipe & rescan.
Close, but only 1/2 a cigar. Try this on your windows box: 1). Wipe and rescan the db using my original flacs with the embedded cues. 2). Broswe by genre down to the album level (where you see the tracks for an album.) 3). On the navigation links below the "Browse Genre" header, click back on the genre you are browsing. 4). Watch as VERY BAD THINGS HAPPEN. Most of the albums, tracks, etc. dissapear from the db.
I should say, browse the 2nd artist in any genre down to the album level, then click back on the Home / Browse Genres / Whatever link. You'll see the albums disappear. While I'm thinking of it: cosmetic: Isn't the plural of "genre" "genre"? I don't think "genres" is a word.
Ok - subversion change 4497 should fix this.
Hummm...4496 didn't seem to do it either. I'm starting to feel like I'm persecuting you.
You are. :) Can you run with --d_info ? Look for "deleteing $file from cache as it no longer exists."
Ok, wiped db, rescanned with --d__info. Browsed to: Home/ Browse Genres / G_German_Baroque / Telemann, G P / Quintetti d'archi e Partite per due liuti Then Clicked on Home/ Browse Genres / G_German_Baroque Then clicked on Home/ Browse Genres and navigated to G_German_Baroque Only 1 artist with 1 album and 1 track shows up. Partial output of --d__info: 2005-09-30 18:01:54.6638 readCoverArtTags: Not a song, skipping: file:///C:/Reco rdings/Music_Beta6/g_German_Baroque/Telemann,%20G%20P/Quintetti%20d%27archi%20e% 20Partite%20per%20due%20liuti.flac 2005-09-30 18:02:00.4687 browsedb - hierarchy: genre,artist,album,track level: 1 2005-09-30 18:02:03.8593 DBI: Periodic commit - 157 dirty items 2005-09-30 18:02:03.8601 forceCommit: syncing to the database. 2005-09-30 18:02:33.3460 browsedb - hierarchy: genre,artist,album,track level: 0 2005-09-30 18:02:34.1408 DBI: Supressing periodic commit - no dirty items 2005-09-30 18:02:34.9375 browsedb - hierarchy: genre,artist,album,track level: 1 2005-09-30 18:02:35.9941 browsedb - hierarchy: genre,artist,album,track level: 2 2005-09-30 18:02:38.5294 browsedb - hierarchy: genre,artist,album,track level: 3 2005-09-30 18:02:38.5790 readCoverArtTags: Not a song, skipping: file:///C:/Reco rdings/Music_Beta6/g_German_Baroque/Telemann,%20G%20P/Quintetti%20d%27archi%20e% 20Partite%20per%20due%20liuti.flac 2005-09-30 18:02:38.6037 readCoverArtTags: Not a song, skipping: file:///C:/Reco rdings/Music_Beta6/g_German_Baroque/Telemann,%20G%20P/Quintetti%20d%27archi%20e% 20Partite%20per%20due%20liuti.flac 2005-09-30 18:03:04.1562 DBI: Supressing periodic commit - no dirty items 2005-09-30 18:03:16.7187 browsedb - hierarchy: genre,artist,album,track level: 0 2005-09-30 18:03:17.7593 browsedb - hierarchy: genre,artist,album,track level: 1 2005-09-30 18:03:18.6290 browsedb - hierarchy: genre,artist,album,track level: 2 2005-09-30 18:03:19.3326 browsedb - hierarchy: genre,artist,album,track level: 3 2005-09-30 18:03:19.3812 readCoverArtTags: Not a song, skipping: file:///C:/Reco rdings/Music_Beta6/g_German_Baroque/Telemann,%20G%20P/Quintetti%20d%27archi%20e% 20Partite%20per%20due%20liuti.flac 2005-09-30 18:03:19.4031 readCoverArtTags: Not a song, skipping: file:///C:/Reco rdings/Music_Beta6/g_German_Baroque/Telemann,%20G%20P/Quintetti%20d%27archi%20e% 20Partite%20per%20due%20liuti.flac 2005-09-30 18:03:34.1719 DBI: Supressing periodic commit - no dirty items
Created attachment 889 [details] File who are missing when using sserver 6+ Seems to be a linux problem?
I just performed a quick test using using svn #4569 and my first posted test files. After a wipe and scan, the genre_track table contained 197 records. Then, after browsing by genre in each genre down to an album view (seeing a list of tracks) and then clicking on the "navigation" link in the "medgray" "statusarea" back to the genre, the genre_track table contained only 5 records. When the navigation is performed, something is obviously killing records in the genre_track table!!!
Gordon - I just checked in a change 4610 - which might help you out. Could you please give it a try? Thanks.
Dan: I can't get svn 4610 to scan my test library without crashing. Here's the output at the DOA moment: erBeta/server/Slim/DataStores/DBI/DBIStore.pm line 1230) frame 1: Slim::DataStores::DBI::DBIStore::_checkValidity (C:/Program Files/Sl imServerBeta/server/Slim/DataStores/DBI/DBIStore.pm line 200) frame 2: Slim::DataStores::DBI::DBIStore::objectForId (C:/Program Files/SlimS erverBeta/server/Slim/Music/Import.pm line 226) frame 3: Slim::Music::Import::artScan (C:/Program Files/SlimServerBeta/server /Slim/Utils/Scheduler.pm line 99) frame 4: Slim::Utils::Scheduler::run_tasks (slimserver.pl line 617) frame 5: main::idle (slimserver.pl line 569) frame 6: main::main (slimserver.pl line 1214) 2005-10-14 00:53:26.3613 Couldn't create track for file:///C:/Recordings/Music_B eta6/g_German_Baroque/Telemann,%20G%20P/Quintetti%20d%27archi%20e%20Partite%20pe r%20due%20liuti.flac#0-0.0133333333333333 : Can't insert new Slim::DataStores::D BI::Track: DBD::SQLite::st execute failed: column url is not unique(1) at dbdimp .c line 401 at C:\Program Files\SlimServerBeta\server\CPAN/DBIx/ContextualFetch. pm line 51. at C:/Program Files/SlimServerBeta/server/Slim/DataStores/DBI/DBIStore.pm line 429 2005-10-14 00:53:27.2343 DBD::SQLite::st execute failed: column url is not uniqu e(1) at dbdimp.c line 401 at C:\Program Files\SlimServerBeta\server\CPAN/DBIx/Co ntextualFetch.pm line 51. 2005-10-14 00:53:27.2356 Backtrace: frame 0: Slim::DataStores::DBI::DBIStore::newTrack (C:/Program Files/SlimServ erBeta/server/Slim/DataStores/DBI/DBIStore.pm line 1230) frame 1: Slim::DataStores::DBI::DBIStore::_checkValidity (C:/Program Files/Sl imServerBeta/server/Slim/DataStores/DBI/DBIStore.pm line 200) frame 2: Slim::DataStores::DBI::DBIStore::objectForId (C:/Program Files/SlimS erverBeta/server/Slim/Music/Import.pm line 226) frame 3: Slim::Music::Import::artScan (C:/Program Files/SlimServerBeta/server /Slim/Utils/Scheduler.pm line 99) frame 4: Slim::Utils::Scheduler::run_tasks (slimserver.pl line 617) frame 5: main::idle (slimserver.pl line 569) frame 6: main::main (slimserver.pl line 1214) 2005-10-14 00:53:27.2363 Couldn't create track for file:///C:/Recordings/Music_B eta6/g_German_Baroque/Bach,%20J%20S/Die%20Kunst%20der%20Fuge%20BWV%201080%20-%20 Leonhardt%20-%20Disk%201.flac#0-0.0133333333333333 : Can't insert new Slim::Data Stores::DBI::Track: DBD::SQLite::st execute failed: column url is not unique(1) at dbdimp.c line 401 at C:\Program Files\SlimServerBeta\server\CPAN/DBIx/Context ualFetch.pm line 51. at C:/Program Files/SlimServerBeta/server/Slim/DataStores/DBI/DBIStore.pm line 429 Operation `bool': no method found, argument in overloaded package Class::DBI::Ob ject::Has::Been::Deleted at C:/Program Files/SlimServerBeta/server/Slim/Music/Im port.pm line 229.
Also: if I restart Slimserver using the db as scanned before the crash, I'm still getting the same behavior: lots of albums and artists dissapear if you click back to the genre navigation link if you've browsed by genre down to the album level. Also also: I can't browse down to the track level with svn 4610: seemingly, the album object makes it into the db, but the tracks don't.
Ok - more changes made to subversion. Please sync up. And can you run with --d_info ? Thanks.
Nevermind - i was able to reproduce here.
Ok - subversion change 4616 should be better for you.
Checking with svn 4618: looks good so far!!! I can browse genre and click back on the nav links without stuff dissapearing. I can't give it a really thorough tire-kickking until tomorrow, which I promise to do. Scanning the library, Slimserver now spits out this message: 2005-10-14 08:54:14.6250 Parse error on line 100: DE Zufällige Jahrgänge 2005-10-14 08:54:14.9011 Parse error on line 100: DE Zufällige Jahrgänge This, I believe, isn't related to this bug, but is a parsing error of strings.txt.
Gordon, I found the bad string in RandomPlay and put in a correction at change 4619. marking this as fixed, but please re-open if the tire leaks ;)
Hi, Im adding this info just to check if I need to open a new bug for my issue or it can be worked on this one: I'm using FLAC with embeded cuesheets. My embeded cuesheets are EAC CD-TEXT generated, and they do NOT contain GENRE, YEAR (DATE) nor REPLAY GAIN info. All that info is present as Vorbis comments in the flac file. As an example: . . . vendor string: reference libFLAC 1.1.2 20050205 comments: 9 comment[0]: ARTIST=Astor Piazzolla comment[1]: ALBUM=Live at Cine Teatro Gran Rex, Buenos Aires comment[2]: DATE=1981 comment[3]: GENRE=Nuevo Tango comment[4]: REPLAYGAIN_TRACK_PEAK=0.85516357 comment[5]: REPLAYGAIN_TRACK_GAIN=-2.20 dB comment[6]: REPLAYGAIN_ALBUM_PEAK=0.85516357 comment[7]: REPLAYGAIN_ALBUM_GAIN=-2.20 dB comment[8]: CUESHEET=CATALOG 0000000000000 PERFORMER "Astor Piazzolla" TITLE "Live at Cine Teatro Gran Rex, Buenos Aires" FILE "Astor Piazzolla - Live at Cine Teatro Gran Rex, Buenos Aires.flac" WAVE TRACK 01 AUDIO TITLE "Invierno Porteño" PERFORMER "Astor Piazzolla" INDEX 01 00:00:00 TRACK 02 AUDIO TITLE "Otoño Porteño" PERFORMER "Astor Piazzolla" INDEX 01 06:47:47 TRACK 03 AUDIO TITLE "Concierto para Quinteto" PERFORMER "Astor Piazzolla" INDEX 01 13:01:62 TRACK 04 AUDIO TITLE "Viyuya" PERFORMER "Astor Piazzolla" INDEX 01 22:11:70 TRACK 05 AUDIO TITLE "Muerte del Angel" PERFORMER "Astor Piazzolla" INDEX 01 28:25:42 TRACK 06 AUDIO TITLE "Adiós Nonino" PERFORMER "Astor Piazzolla" INDEX 01 31:38:37 TRACK 07 AUDIO TITLE "Tangata" PERFORMER "Astor Piazzolla" INDEX 01 40:08:30 TRACK 08 AUDIO TITLE "Verano Porteño" PERFORMER "Astor Piazzolla" INDEX 01 49:52:70 TRACK 09 AUDIO TITLE "Libertango" PERFORMER "Astor Piazzolla" INDEX 01 56:59:67 The current behaviour of Slimserver is to ignore the GENRE, DATE, and REPLAYGAIN comments , I guess because it assumes everything is on the cuesheet, so I end up with no genres in my slimserver. Is there any chance for this comments (GENRE , DATE, etc) present on the flac file, at the comment level, to be considered the default values , unless they are superseeded by information inside the cuesheet comment? (The idea is that every song in the cuesheet "inherits" the GENRE for the album). Currently is pretty straigthforward to change the Genre comment for example by using a Taging tool, without touching the cuesheet itself. (Gordon has provided me scripts to modify cuesheets, but I would prefer to modify just a comment in the file than modify the cuesheets). Also I prefer the replay_gain tag to be at the comment level than at the cuesheet level. Any idea if this is possible? Should I open a new bug? Thanks a lot!
Nestor: My personal feeling is that you ought to file a new bug on this one. This seems like an enhancement issue to me: getting slimserver to recognize "mixed" tags. I'm hardly the appropriate arbiter on this, though. I am working on a new utility that will perform the cuesheet fixups that fixcue performs and then translate the cuesheet into the "stacked" comment style and tag the flac accordingly..i.e. get rid of the embedded cuesheet altogeather. At my present rate, however, it may be some months before I've got it ready. PS: now that Piazolla is gone, who is carrying the "Nuevo Tango" torch? And would they be "Neo-Nuevo Tango"? ;-)
Gordon, just opened a new bug (2304), lets see how lucky I get. Originally I used those (Album, Artist, Genre , and Year) at the comment level beacuse they were the 4 classes you can Browse music with SlimServer. Then I added the replaygain , and embedded the cuesheet. If Im not wrong up until some point it used to work ok, but dont remember if it was in 5.4 , 6.1 or just bad memory form my side :-) >PS: now that Piazolla is gone, who is carrying the "Nuevo Tango" torch? And >would they be "Neo-Nuevo Tango"? ;-) Hmmm..., Piazzolla has been gone for about 13 years now, and there are many other people going around, trying differente things... but... As someone said some time ago: Forget the discussions about if what Piazzolla played was tango or not... he is a genre himself. ;-)