Bug 656 - Last track in external FLAC cuesheet is dropped
: Last track in external FLAC cuesheet is dropped
Status: CLOSED FIXED
Product: Logitech Media Server
Classification: Unclassified
Component: Formats
: 5.x or older
: All All
: P2 normal (vote)
: ---
Assigned To: Dan Sully
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2004-11-10 08:37 UTC by Jason Holtzapple
Modified: 2008-12-18 11:54 UTC (History)
3 users (show)

See Also:
Category: ---


Attachments
patch for reading ending time on files with external cuesheets (901 bytes, patch)
2004-12-04 19:41 UTC, michael
Details | Diff
Zip file with external cuesheet and tags from flac file (1.10 KB, application/zip)
2005-01-05 07:41 UTC, Nestor
Details
External cuesheet, and internal flac (only replaygain) (1.07 KB, application/zip)
2005-01-05 07:57 UTC, Nestor
Details
External cuesheet, no embeded cuesheet, internal flags for artist,album,year, genre. The name of the artist include a non-english character (1.10 KB, application/zip)
2005-01-05 08:26 UTC, Nestor
Details
Windows Locale program. (290 bytes, text/plain)
2005-03-02 15:09 UTC, Dan Sully
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jason Holtzapple 2004-11-10 08:37:13 UTC
The last track in an album defined in an external FLAC cuesheet seems to always
be dropped. I think I've narrowed down the cause, but don't know how to fix it.

In Formats/Parse.pm, lines 229-231:

        my $lastpos = (defined $tracks{$currtrack}->{'END'})
                ? $tracks{$currtrack}->{'END'}
                : Slim::Music::Info::durationSeconds($filename);

For a flac cuesheet without a "REM END" tag (i.e. most of them),
Slim::Music::Info::durationSeconds($filename) where $filename
is the URL of the flac file) returns a null value
instead of the total number of seconds. I checked mp3+cue sheets and
Slim::Music::Info::durationSeconds($filename) works fine for mp3
files.

Because of this null value, the first track parsed in the cuesheet
(which is the last since they are parsed in reverse) gets dropped.

So if Slim::Music::Info::durationSeconds($filename) can be fixed to
work for flac files it should fix this bug.

I verified in Formats/FLAC.pm that SECS is being read correctly, so
I don't think it's a problem with parsing the actual bits out of the
FLAC file.
Comment 1 KDF 2004-11-10 09:05:06 UTC
I think this is partly due to the CUE parsing interfering with the normal
process of readTags.  The line you speak of is called before readTags is done. 
This shoudl then trigger an extra readTags call.  seems to highlight a rather
problematic inefficiency as the parseCUE routine calls another readTags later in
the routine.
Comment 2 michael 2004-11-11 18:03:21 UTC
Yeah, the problem seems to be related to parsing the cue file before the flac
file it points to has been parsed. The total length of the file (and thus the
ending point of the last track) isn't known yet. Performing an extra readTags()
at this point will make that information available, so long as you're careful to
not create an infinite loop in the process.
The following snippet illustrates the issue
("browse music folder" will still show the wrong info, but the last track will
show properly in browse artist/album.)

diff -u -r1.23 Parse.pm
--- server/Slim/Formats/Parse.pm        15 Oct 2004 23:48:24 -0000      1.23
+++ server/Slim/Formats/Parse.pm        12 Nov 2004 01:45:16 -0000
@@ -238,7 +238,11 @@
        foreach my $key (sort {$a <=> $b} keys %tracks) {
 
                my $track = $tracks{$key};
-               if (!defined $track->{'START'} || !defined $track->{'END'} ||
!defined $filename ) { next; }
+#              if (!defined $track->{'START'} || !defined $track->{'END'} ||
!defined $filename ) { next; }
+               if (!defined $track->{'START'} || !defined $filename ) { next; }
+               if (!defined $track->{'END'}) {
Slim::Music::Info::readTags($filename); $track->{'END'} =
Slim::Music::Info::durationSeconds($filename); }
+               next unless defined $track->{'END'};
+
                my $url = "$filename#".$track->{'START'}."-".$track->{'END'};
                $::d_parse && Slim::Utils::Misc::msg("    url: $url\n");



The bottom line is that metadata parsing is a mess, and need to be reworked at
it's very basic levels. Cuesheet parsing is even more of a mess, and won't
really have a clean implementation without some changes to the basic parsing
logic first. and none of this is likely to be really clean before the sql
backend comes in.
In the meantime, I'll see if I can't come up with a fix that isn't too ugly.
Comment 3 michael 2004-12-04 19:41:54 UTC
Created attachment 221 [details]
patch for reading ending time on files with external cuesheets

I suggest applying this patch as a workaround until we have the chance to
re-work how parsing is handled. It seems to fix the immediate problem and
correctly read the last track of the file referenced by the cue sheet.
Comment 4 michael 2004-12-07 13:54:07 UTC
I've found some cases where the patch above isn't sufficient, so even with this
patch applied, there is still some work to do before closing this issue.
Comment 5 michael 2004-12-07 16:25:32 UTC
Ok, I've tested the above patch with BRANCH_5_4_x and it works there too.

The corner case mentioned in the last comment seems to apply when using an
external cuesheet, and also having "stacked" tags internal to the flac file.
This case is no worse off than without the patch, and is caused by a compeletely
different problem than the END detection that we're dealing with here.

Overall, I think this can be applied to both the trunk and 5.4.x branch. It's a
small change that will have a big impact for those using external cuesheets.
Comment 6 Nestor 2004-12-08 10:56:48 UTC
Hi ,  is there a different bug for the case of having tags inside flac files
besides the external cuesheet? Thats my case and would like that situation to be
considered if its possible.

Thanks
Nestor
Comment 7 michael 2004-12-08 12:23:43 UTC
bug 717 is probably what you're looking for. as noted above, some variations of
tagging can cause that bug to appear similar to the symptoms of this one.
Comment 8 Dan Sully 2005-01-02 22:13:38 UTC
I've applied this patch to the 6.0 tree - will try and get it into the 5.4
branch as well.
Comment 9 Nestor 2005-01-03 12:02:35 UTC
My setup: 1 Flac file per CD, with external cuesheet ( I have many albums with
internal cuesheets also but you can forget about them for this case).

I've tried the last nightly release, and while I now get all songs in the
cuesheet to display, it seems that in many cases (not all :-(, and thats the
strange part ) when I choose to play the last song, I get the whole Flac file
being played , from the begining, and the last song is not played at all. 
Also, sometimes for the last song has the correct name , and other times the
name of the whole album appears as the name of the last song.
Comment 10 KDF 2005-01-03 12:05:05 UTC
Nestor, are you certain you downloaded the 6.0 nightly?  This should have caused
a crash (as just recently posted to the dev list). 5.4 nightly does not yet have
this patch, as that requires a bit more careful review.
Comment 11 Nestor 2005-01-03 12:50:41 UTC
Hi,
I donwnloaded 5.4.1. I have re-checked and the parse.pm of this nightly release
does include the lines mentioned in the patch above
(https://bugs-archive.lyrion.org/attachment.cgi?id=221&action=view)

Am I wrong? looking at the wrong place? :-)  Please, let me know.
Comment 12 KDF 2005-01-03 13:02:45 UTC
yes, you are correct.  This went in to 5.4.1 last night apparently.
This might be the case Michael mentions above where this patch isn't sufficient
in certain cases. If you haven't already, try wiping the cache (server
settings->performance) and see if that changes anything.  The track->{END} info
would change the infocache data, and it doen't appear that DBVERSION was
incremented to force a cache wipe.
Comment 13 Nestor 2005-01-03 13:27:20 UTC
I tried the wipe cache, and the problem remains.
Whats strange is that it happens with many (most?) of the albums, but not all.
Comment 14 KDF 2005-01-03 13:38:47 UTC
as mentioned above, some cases remain where this fix doesn't work.  clearly you
have some of those cases.
Comment 15 michael 2005-01-04 12:39:32 UTC
Right now slim server works best if you're using internal OR external
cuesheets/metadata for a particular flac file. having both (internal and
external) for the same piece of audio can often produce odd listings. The issue
with "stacked tags" mentioned above is one such case. For the time being, I'd
suggest either use an embedded cuesheet and vorbis tags, or alternately, use an
external cuesheet with CDTEXT entries and no vorbis tags on the flac itself.

If you're still haveing trouble, I'd like to peek at one of the files that's
causing problems.
Comment 16 Nestor 2005-01-05 07:41:13 UTC
Created attachment 240 [details]
Zip file with   external cuesheet and    tags from flac file

I will upload a couple of differente cases.

First case (ejemplo1.zip):
external cuesheet , no embedded cueesheet, flac flags with artist, album, year,
genre, and replaygain values.

In this case, SlimServer shows the right number of songs, but the last song
instead of having the right name "Catalina, Bahía", appears as "15. Andres
Calamaro - Alta Suciedad ", ie, artist-albumname pair. If you choose to play
just the last song , its played OK, even when the name its not right.
Comment 17 Nestor 2005-01-05 07:57:29 UTC
Created attachment 241 [details]
External cuesheet, and internal flac (only replaygain)

Second case (ejemplo2.zip):
external cuesheet , no embedded cueesheet, flac flags with  replaygain values.

In this case, browsing by album gives you 2 albums "Alta suciedad" and "No
Album".
Choosing "Alta suciedad" album you see all 15 songs with the right name, but if
you choose to play the last one you get the first song of the flac being
played.
Choosing "No album" , you get only one song called "15. Andres Calamaro - Alta
Suciedad" (artist-albumname pair) and if you play it you get the last song of
the album.

Browsing by artist you get two: "Andres Calamaro" and "No artist" and the
behaviour is similar to the one above, ie: if you choose "Andres Calamaro" you
get all names right, but the last song cant be played ( you get the first one)
and if you choose "No artist" you get only one song with the wrong name , but
when you play it , its the last song of the album.




In this case, SlimServer shows the right number of songs, but the last song
instead of having the right name "Catalina, Bahía", appears as "15. Andres
Calamaro - Alta Suciedad ", ie, artist-albumname pair. If you choose to play
just the last song , its played OK, even when the name its not right.
Comment 18 Nestor 2005-01-05 08:26:48 UTC
Created attachment 242 [details]
External cuesheet, no embeded cuesheet, internal flags for artist,album,year, genre. The name of the artist include a non-english character

third case (ejemplor.zip):
external cuesheet , no embedded cueesheet, flac flags with  artist,album, year,
genre, replaygain.
In this case I used the right artist name, including an "é" (alt-130)
non-english character, both in the internal artist name flag, and in the name
of the files.

Browsing by album, I get 1 album (it says 1 album with 16 songs by 2 artists).
When you enter the album you find 16 songs instead of 15. The first one is
called "15. Andrés Calamaro - Alta Suciedad" and when you play it you get the
last song of the album. From the second line on, you get all the songs in the
album with the right number , but if you choose to play the last song, you get
the first song in the album.

Browsing by artist you get two: "Andrés Calamaro" and "Andrés Calamaro". 
Choosing "Andrés Calamaro", you get 1 album and 1 song, and the song (using
the name of artist-album) is the last song of the album.
Choosing  "Andrés Calamaro" you get 1 album with the right number and name of
songs (15), again, if you go to play the last one, you get the first one on the
 flac file.
Comment 19 Dan Sully 2005-02-25 16:35:18 UTC
Nestor - can you test with the latest 6.0 build?

Also - external FLAC cuesheets should be encoded in the same character set as
your locale. Preferably UTF-8
Comment 20 Nestor 2005-02-28 10:53:47 UTC
Hi, 

I have tested last night build. 

In this case, no embedded cuesheet, no "strange characters" on artist or album
name. These are the tags for the flac file:

Format:  FLAC
Details: 44100 Hz Stereo, 4294966970 kbps, playtime 01:00:24
Tag:     FLAC
Title:
Artist:  Andres Calamaro
Album:   Alta Suciedad
Year:    1997
Track:
Genre:   Rock nacional
Comment:
REPLAYGAIN_TRACK_PEAK=0.99896240
REPLAYGAIN_TRACK_GAIN=-6.42 dB
REPLAYGAIN_ALBUM_PEAK=0.99896240
REPLAYGAIN_ALBUM_GAIN=-6.42 dB



The external cuesheet is something like this:

PERFORMER "Andres Calamaro"
TITLE "Alta Suciedad"
FILE "Andres Calamaro - Alta Suciedad.flac" WAVE
  TRACK 01 AUDIO
    TITLE "Alta Suciedad"
    PERFORMER "Andres Calamaro"
    ISRC ES5019753000
    INDEX 01 00:00:00
  TRACK 02 AUDIO
    TITLE "Todo Lo Demás"
    PERFORMER "Andres Calamaro"
    ISRC ES5019753001
    INDEX 01 04:27:44
  TRACK 03 AUDIO
    TITLE "Donde Manda Marinero"
    PERFORMER "Andres Calamaro"
    ISRC ES5019753002
    INDEX 01 07:18:36
.
.
.
.

Slimserver list this album like this:

All Songs      
1. Alta Suciedad      
2. Todo Lo Demás      
3. Donde Manda Marinero      
4. Loco      
5. Flaca      
6. Quien Asó La Manteca      
7. Media Verónica      
8. El Tercio De Los Sueños      
9. Comida China      
10. Elvis Está Vivo      
11. Me Arde      
12. Crímenes Perfectos      
13. Nunca Es Igual      
14. El Novio Del Olvido      
15. Andres Calamaro - Alta Suciedad      
15. Catalina, Bahía      
 

So, I got two  "15th" songs: the first 15 is the name of the artist and name of
album. The second "15" is the correct last song of the record.

I havent tried yet with having both external and embedded cuesheets, nor using
accented characters on the name.  Will let you know as soon as I can test that.

Regards
Néstor
Comment 21 michael 2005-02-28 11:06:40 UTC
That extra track at the end may be part of an issue dan and I have been
discussing, where tracks with external cuesheets are not having their base file
hidden in the ui. Exactly where this extra "song" appears seems dependend on the
tags attached to the base file. This is mentioned in bug 836, and is a direct
decended of bug 717.
Comment 22 Nestor 2005-02-28 11:12:49 UTC
I just rechecked the same case , but adding the right accented character both in
the directory name, and flac file:

Dir: "C:\prueba\Andrés Calamaro - Alta Suciedad"
file = "Andrés Calamaro - Alta Suciedad.flac"

The cuesheet ("Andrés Calamaro - Alta Suciedad.cue")  goes like this :

PERFORMER "Andrés Calamaro"
TITLE "Alta Suciedad"
FILE "Andrés Calamaro - Alta Suciedad.flac" WAVE
  TRACK 01 AUDIO
    TITLE "Alta Suciedad"
    PERFORMER "Andrés Calamaro"
    ISRC ES5019753000
    INDEX 01 00:00:00
  TRACK 02 AUDIO
    TITLE "Todo Lo Demás"
    PERFORMER "Andrés Calamaro"
    ISRC ES5019753001
    INDEX 01 04:27:44
  TRACK 03 AUDIO
    TITLE "Donde Manda Marinero"
    PERFORMER "Andrés Calamaro"
    ISRC ES5019753002
    INDEX 01 07:18:36
.
.
.
.

In this case, Slimserver does not show the list of songs at all.
I just get this:



 Home / Browse Artists / Andrés Calamaro / Alta Suciedad   
 
 1 album with 15 songs by 1 artist.  
  
 
  

With no list of songs being displayed.

Hope this helps...
Néstor


Comment 23 Dan Sully 2005-03-02 13:15:01 UTC
Nestor - what are your locale settings? And is the cuesheet written out in that
locale, or in UTF-8?
Comment 24 Nestor 2005-03-02 14:43:14 UTC
Hi Dan,

if you let me know how to check for the "locale" in Windows , I would be able to
tell :-)

The only locale I can set is: Spanish (Argentina) , and thats what Im using
Im not sure how to relate that to UTF8 in Windows, or how to check im ok with UTF8.
Any help would be appreciated.
Comment 25 Dan Sully 2005-03-02 15:09:39 UTC
Created attachment 298 [details]
Windows Locale program.
Comment 26 Dan Sully 2005-03-02 15:10:07 UTC
If you have ActiveState perl installed, could you run this small program from
the command prompt?
Comment 27 Nestor 2005-03-02 15:17:06 UTC
perl locale.pl
Locale is: cp1252


Comment 28 Nestor 2005-03-13 19:51:31 UTC
The latest behaviour (as to 6.0b1 , windows executable) seems to be:

List of songs on Slimserver and squeezebox looks like this :

1. "Song 1"
2. "Song 2"
.
.
.
12. "Song 12" (or "Album title"  in a few cases)
12. "Song 12"

If you play the whole album, starting at (1), it plays 1 to 11 without any
problem. Then , when it reachs the first (12) it plays the WHOLE album,
INCLUDING the real song (12), and then it plays the second (12): the last song
of the album.
In other words, the whole album is inserted between the next to last song and
the last song.

Initial conditions: whole album FLAC file, and external cueeshet.

Hope this helps!
Néstor
Comment 29 Vidur Apparao 2005-03-14 18:23:33 UTC
Michael, any chance you'd like to own this bug and work on fix for 6.0?
Comment 30 michael 2005-03-14 21:55:16 UTC
well, I'm not currently seeing this behaviour with my test files, and some of
the comments above lead me to believe that dan's UTF8 knowledge may be more
useful here.  But I'll take another look and try out the cuesheet nestor
attached to see if I can at least reproduce the problem.
Comment 31 michael 2005-03-17 03:24:33 UTC
I would say that this is definitely a character encoding issue.

I grabbed the cuesheet and tags posted in the last attachment. I changed the
FILE line to point to an untagged flac file I have.
slimserver picked up the new artist, and although the accented character didn't
display correctly on my system, I chalked that up to my locale settings (en_US).
Browsing into the album showed the correct number of tracks, and the names
seemed to match what the cuesheet indicated they should (apart from handling of
non-ascii characters). Hovering the mouse over the play icon for each song
showed a url in the status bar than included a plausible anchor, and thus should
play a single song if I had the actual flac this cuesheet refers to.

Then I tried to add the tags provided.  The tags file that was in the attachment
isn't formatted as what metaflac expect to import (or export) so I had edit it
into an appropriate format while trying to preserve the tags it seemed to
indicate.  Then I ran
metaflac --no-utf8-convert --import-tags-from=tags.txt  testfile.flac
and I got the message
testfile.flac: ERROR: memory allocation failure

This is likely a problem with metaflac itself, and not necessarily anything to
do with slimserver, but it likely narrows down the problem we're seeing as well.
removing the accented character from the tags file caused it to import just fine.
I must admit that I used --no-utf8-convert out of habit, as it avoid a lot of
'##' marks in my tags. Interestingly, if I omit that from the metaflac line, the
import works. When looking at the newly tagged file in slimserver, I now see two
artist entries that differ by their handling of the accented character. The old
one, which displays weird characters where the accented character should be, but
has the correct tracks.  and an additional artist that has an accented e in that
spot, and has one album (named correctly) but which doesn't list any songs.
I didn't even try to see what it would do if I had non-ascii characters in the
file name.
Unfortunately, I don't know much at all about i18n, so I'll have to defer to dan
on this one.  But as far as I can tell it certainly seems to be a character set
issue.
Comment 32 Blackketter Dean 2005-06-07 13:10:58 UTC
Dan, can you comment on the status of this or update the bugs status?
Comment 33 Blackketter Dean 2005-06-10 15:37:15 UTC
dan says that this is fixed for the general case.  Nestor: your cue sheet with it's different character set 
should probably be attached to a new bug.
Comment 34 Chris Owens 2008-03-11 11:27:56 UTC
This bug was marked resolved in Slimserver 6.1, which is several versions ago.  If you're still seeing this bug, please re-open it.  Thanks!
Comment 35 Chris Owens 2008-12-18 11:54:38 UTC
Routine bug db maintenance; removing old versions which cause confusion.  I apologize for the inconvenience.