Is there anything unusual about the albums that don't get scanned?  Also, please turn off the plugins logging for now if you would.  That's basically all that's in your log snippet.  Feel free to upload the entire log text as an attachment so we can have a good look at it.

Do you use any plugins that would affect scanning such at Moodlogic or MusicIP?

If the scanner is really crashing, it may be useful to run the scanner from the windows command prompt by going to the slimserver directory and typing:

scanner.exe --wipe --d_import --progress

We have quite a few 6.5 users with more than 120 albums, so it seems likely it's going to be somewhat specific to your system.

I should have specified the directory where scanner.exe is located is (typically) \program files\slimserver\server\, unless you installed it somewhere else! :)

also --d_scan, if you would.

Created attachment 1477 [details]
The CUE file

The file that causes the problem is a CUE file.  One of the tracks is in a FLAC file that had a corrupted file name.

The really weird thing is that scanner crashes when I run as follows:

scanner.exe --wipe --d_import --progress --d_scan --force "\Documents and Settings\Nigel\My Documents\My Music\Brahms, Johannes\Brahms Symphonies 3 and 4, Violin Concertos, Haydn Variations - Cleveland Orchestra, Christoph von Dohnanyi.CUE"

However, when I copy the offending CUE and FLAC files somewhere else, the scanner works fine, correctly reporting "ERROR: parseCUE: Couldn't fine referenced FILE: [\test\Brahms Symphonies ... \07... ] on disk! Skipping!".  I tried copying the files to a really long directory name with spaces in the names and the scanner still worked.  So what is wrong with the original location in the "My Music" folder?

I could not find any log files with stack dumps or anything like that.  As I am running on XP, the OS offered to send the crash info to Microsoft but didn't offer to send the dumps anywhere else!  Is there anyway I can get a dump?

I tried uploading the FLAC files to my ftp site but there was not enough space.  I have attached the CUE file herewith.  The track 7 file was missing but in its place was a file called 07CONC~1.FLA.  The corruption probably happened while copying the files from the old computer to my new.

Thanks, we love it when we get sample files to work with!

Ross, could you have a look at this one?

Nigel,

I'm trying to reproduce this bug and I'm having a tough time. I might need your FLAC files associated with this cuesheet. 

Are you sure they're FLAC? In the cuesheet it refers to WAV files... 

You could zip the files up and send them via dropload. Please send them to ross AT slimdevices DOT com. 

It appears to me that the problem is in the length of the full file name (including directory) and the way the files are copied around, rather than the files themselves.

Getting a repro is tricky.

- create a folder (I will call it "My Music") on the root of a NTFS drive.

- create a long directory path of, say, 130 characters total length in the "My Music" folder.  In my case the directory is "Brahms, Johannes\Brahms Symphonies 3 and
4, Violin Concertos, Haydn Variations - Cleveland Orchestra, Christoph von
Dohnanyi"

- copy in the attached CUE file

- copy in 7 flac files and rename them to the seven file names in the CUE file BUT with a flac extension, not a wave extension (the wave extensions in the cue file are no doubt because I mis-used EAC).  I don't think the problem is the flac files themselves, so just take 7 random files of yours and rename them.  I could upload my flac files but the problem happens in all cases where the file names are too long (about 30 out of my 8000 tracks).

Note that the full path to the longest file (track 7) is under 255 which seems to be the NTFS limit.

- create a folder of length about 50 characters (eg. /Documents and Settings/User/My Documents/My Music

- open explorer, select everything inside the /My Music folder, copy it and paste it into the /Documents and Settings/User/My Documents/My Music folder (do not use xcopy)

If we have got these repro steps correct then the 07 track will show as 07CONC~1.FLA because the full name exceeds the NTFS limit.  The other files have slightly shorter names and appear with their full names.  Run the scanner against the deeper My Music folder and if we have a repro then it will crash.

It appears to me that NTFS to some extend created the file with the full name and there is some residual trace of that full name in the directory.  If I xcopy the files (cue and flac files) elsewhere and run the scanner on the new directory then no crash happens (it correctly says the file for track 7 cannot be found).  My guess is that in the new directory there is just the 8.3 file name, no residual long file name.  Perhaps Perl is being told a file exists, but then cannot open it, or something like that, caused by confusion over handling of the long and short file names.

I note also that scanner.exe seems to find the files when processing the cue sheets even though the cue sheet says .wave and the files are in fact .flac.  Is this because the scanner is looking for files of any extension with the given name?

Note also this problem has a bypass (move the music folder to the root of the drive or otherwise shorten the path lengths).

Ross, did you tell me you tried this even with the giant pathnames?

That is correct. I tried it with a directory that had 244 characters in the directory name, and the song name 7 characters. 

I was unable to reproduce this issue. 

Subject: Re:  Scanner stops scanning after 120 albums

Ross - did you try with the steps Nigel suggested?

Thanks

I talked to Ross and he says he did try following Nigel's steps.  I'm going to move this to 7.0 since it doesn't seem important enough to hold up the 6.5 release for, but will try some more myself to reproduce it.  It's exactly the kind of bug that I wouldn't see since I typically don't test with that many albums, and I need to fix that.

I previously said that the problem occurred with all files which had their names converted to the 8.3 format.  However, that appears not to be the case as the crash did not occur with one of such files, which makes the problem even more elusive.  I will see if I can come up with a command file that runs the steps and gets a reliable repro from a fixed starting point.

As I am the only one who seems to have come across it and it is not easy to get a repro, it would not be reasonable to expect even an extensive test program to find this one.

I also have problems with the scanner.exe, it stops after 1120 tracks. When I remove the track the scanner continues, but stops again after a number of additional tracks. This way I removed about 30 tracks to keep the scanner running. I am 100% sure the tracks are OK, also the Tag's are OK. When I use SlimServer 6.5 dated 11 September 2006 the scanner works without any problems. Any newer version sincs 6.5 was released gives me problems. This is including the official 6.5 release. I also installed the latest 6.5.1 and 7.0 versions (dated 27 September 2006) but both have the same problem. 

After testing I found out the problem only occurs with .ape files (Monkeysaudio) and those files are from compilation albums. Also these files have 'Various Artists' as description in the albumArtist field, while in the artist field the artist of the track is stated.

I run SlimServer on a Windows 2003 R2 server. This server is a Virtual Server on a Win2003 R2 host using MS virtual Server 2005 R2.

Subject: Re:  Scanner stops scanning after 120 albums

Can you attach a few of those problem files to this bug?

Thanks

The files are approx 30MB each. Do you want me to attach them or can I send them to you by FTP or some other option?

Subject: Re:  Scanner stops scanning after 120 albums

FTP them to ftp://slimdevices.com/incoming/

Thanks

Saturday I uploaded 3 tracks.

Is there any update on this issue?

Fred - I took a look at those files, and scanned them - everything appears fine here.

Nigel - In the FILE listing for the CUE sheet, part of the line contains: "\01", which I believe is being parsed incorrectly.

Could you try removing that, and renaming the file to match?

Thanks

I just installed the latest 6.5 version from 17 Octover 2006. With this version the scanner works without problems, but only with the Mysql instance installed by SlimServer.

As soon as I change SlimServer.pref to use my seperate Mysql server version 5.0.26 on Windows std Server 2003 r2, again I have the same problems. The scanner stops, after reading song 1020 (the first track) I've send to you by FTP. The scanner reads the track and adds it to the DB, but the album field of the track table is 'null' for the last track. When I manually fill-in the album field, and run the scanner again to look for new and changed music, it continues, but then I get very strange results. The scanner finds another 2000 tracks, but skips about 18000 other tracks. This problem only happens when I use a seperate mysql server.

Patches welcome, but we can't reproduce this.

This bug is being closed since it was resolved for a version which is now released!  Please download the new version of SqueezeCenter (formerly SlimServer) at http://www.slimdevices.com/su_downloads.html

If you are still seeing this bug, please re-open it and we will consider it for a future release.