Bugzilla – Bug 17450
MusicIP import fails due to upper/lower case differences in the file path (Windows)
Last modified: 2011-08-19 06:32:32 UTC
Once the MusicIP importer has been run (or MusicIP flags are all set to true) - see http://forums.slimdevices.com/showthread.php?t=89462 ) you can click an 'M' icon in the track list. The left side panel redisplays and you get: at the top: Music IP Mix next line: Home > XXX MusicIP Mix (where XXX is whatever you mixed on) next line: All Songs ...And the rest of the left panel (where the playlist should be displayed) is blank - or occasionally a few tracks will be listed. I *think* this might be a timing issue. No errors are generated in the server.log file when creating a MusicIP mix, but when I hover over "All Songs" and press either the 'play' or 'add to playlist' icon, there's a long pause before the tracks are added to the "now playing" list (right hand side) - and errors in the following format appear in the server log %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% [11-08-14 08:43:54.9481] Slim::Formats::Playlists::M3U::write (262) Error: Couldn't retrieve objectForUrl: [file:///d:/[music]/lossless-music-1/Avishai%20Cohen%20Trio%20-%20Gently%20Disturbed/02%20-%20Chutzpan.flac] - skipping! [11-08-14 08:43:54.9527] Slim::Formats::Playlists::M3U::write (262) Error: Couldn't retrieve objectForUrl: [file:///e:/[music]/lossless-music-2/Stefon%20Harris%20&%20Jacky%20Terrasson%20-%20Kindred/13%20-%20Body%20&%20Soul.flac] - skipping! [11-08-14 08:43:54.9571] Slim::Formats::Playlists::M3U::write (262) Error: Couldn't retrieve objectForUrl: [file:///d:/[music]/lossless-music-1/Acoustic%20Alchemy%20-%20This%20Way/11%20-%20Now%20I%27m%20on%20My%20Way.flac] - skipping! [11-08-14 08:43:54.9617] Slim::Formats::Playlists::M3U::write (262) Error: Couldn't retrieve objectForUrl: [file:///d:/[music]/lossless-music-1/Habib%20Koite%20&%20Bamada%20-%20Ma%20Ya/03%20-%20Ma%20Ya.flac] - skipping! [11-08-14 08:43:54.9662] Slim::Formats::Playlists::M3U::write (262) Error: Couldn't retrieve objectForUrl: [file:///e:/[music]/mp3-collections/[Putumayo]/Putumayo%20-%20Blues%20Lounge/Putumayo%20-%20Blues%20Lounge%20-%2006%20-%20Moby%20-%20Run%20On.mp3] - skipping! %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% I have verified that these files exist - and even the ones reported as skipped seem to be added to the playlist. Finally, items added to the playlist do not have 'M' (mixable) icons, even though they are flagged as mixable in the database. From 'Information' in the webGUI Version: 7.6.1 - r33044 @ Thu Aug 11 02:05:41 PDT 2011 Hostname: server4 Server IP Address: 172.16.51.16 Server HTTP Port Number: 9000 Operating system: Windows Server 2003 - EN - cp1252 Platform Architecture: 586 Perl Version: 5.10.0 - MSWin32-x86-multi-thread Database Version: DBD::SQLite 1.34_01 (sqlite 3.7.7.1) Total Players Recognized: 2 Library Statistics Total Tracks: 72,901 Total Albums: 5,669 Total Artists: 1,962 Total Genres: 261 Total Playing Time: 8205:34:19 I'd be glad to reproduce this with other logging on - please let me know what to turn on. Thanks!
I've encountered an issue where MIP would fail on Windows because it sometimes uses uppercase drive letters where SBS would have stored lowercase and vice versa. Could you check the track information of one of these files, and see what the path SBS is reporting is spelled like?
(In reply to comment #1) >Could you check the track information of one of these files, and see > what the path SBS is reporting is spelled like? I've just dumped all track info (sqlite3 squeezebox.db "select url from tracks;") Yes there's a case mismatch in volume letters. Further, looking at all the URLs, there's a mix of uppercase and lowercase volume letters I'm not sure why there's a mix. I haven't done any "Look for new and changed music" scans. For these last 6.1 dailies, I've installed the app, set to start when system starts, stopped the service, deleted the cache, and restarted (so nothing left over from previous). Last done with b33044 Do you want the list? Do you want me to do this with the latest build? It appears that MusicIP is always using lower case drive letters - is that the actual case?
I wrote a bit of Python to just set all those uppercase volume letters to lowercase, and MusicIP works (pretty much) I get 1 error: [11-08-16 23:04:23.6519] Slim::Plugin::MusicMagic::Plugin::getMix (1008) MIP attempted to mix in a song at e:\[music]\lossless-new-2\Ottmar Liebert & Luna Negra - Up Close (binaural) (24.96 downsampled)\03 - Cocteau.flac that can't be found at that location So maybe I borked something in the db - or I need to refresh the musicIP db (started that now) It seems slower than with 7.5 (40 tracks), and, if a track is playing, output goes silent for about 20 sec, then resumes - seems to pick up that much later in the track rather then just stop. Here's thePython, on the off chance it might help... import sqlite3 conn = sqlite3.connect('test.db') c = conn.cursor() # get all id:url tuples into the list 'allURLs' c.execute ("""select id,url from tracks""") #c.execute ("""select id,url from tracks where id<10""") allURLs=[] for row in c: if(len(row[1])) > 0: allURLs.append(row) updated=0 for row in allURLs: if row[1][8].isupper(): newURL=row[1][0:8]+row[1][8].lower()+row[1][9:] #print(row[1]) #print(newURL) update="update tracks set url='"+newURL+"' where id="+str(row[0]) #print(update) c.execute (update) updated=updated+1 conn.commit() c.close() print('updated '+str(updated)+' URLs')
Great feedback! Thanks Bobo. I think I've identified the underlying issue: in MySQL the url column in the db was case insensitive, while in SQLite it is. I'll run a few more tests today, but this finding might help us fix several other issue like "all my playlists are empty" too.
Ran MusicIP refresh. Got a lot of [11-08-17 08:08:35.9680] Slim::Utils::Scanner::Local::deleted (545) Handling deleted track file:///d:/[music]/blah blah blah messages in the server.log Now pressing the 'M'ix button on a track gets a mix pretty quickly, but (minor issue) only returns 20 tracks (system is set for 40). No errors in the server.log after a mix. Michael - kudos on spotting a fundamental issue. *Somebody* ought to be buying you a beer - and if nobody at SB central is doing that, I'd be glad to.
OK - that minor issue When mixing, and system is set to make a mix of 40 tracks, only 20 tracks are shown in the left Win GUI window. No album art is displayed. BUT if the 'play' icon next to 'All Songs' is clicked, 40 tracks get added to the playlist. The left (shorter) list isn't a truncation of the right hand one, but appears to be every other one When I 'save' now playing, I get an m3u file. In that m3u file, some files have uppercase volume letters and some have lowercase. The ones with uppercase were the ones that were displayed in the leftmost window when the mix was created. Also, clicking 'play' causes a number of errors in the server.log. e.g. [11-08-17 08:57:13.0408] Slim::Formats::Playlists::M3U::write (262) Error: Couldn't retrieve objectForUrl: [file:///d:/[music]/lossless-music-1/Fourplay%20-%20Yes,%20Please!/07%20-%20Fortress.flac] - skipping! Note that the errors were for URLs with lower cased volume letters - and I know that the URLs in the db all have lower case volume letters. So, I'm not sure what exactly is going on, but, yeah, a change in case enforcement looks like it's causing at least some of those playlist issues
Actually, I don't think it's clicking 'play' that caused those errors but clicking 'save'. I'll have to experiment, but can't get to it for a bit as I really have to get some work done.
I checked in a schema change which should fix this. Unfortunately this will trigger a full re-scan. Please give it a try and let me know how it works for you. Thanks!
Is that in a nightly, or somewhere else? I'm asking because I don't see anything in the bug fixes. Are there experimental builds I haven't been clever enough to discover around somewhere?
Yes, it's in the release candidate: http://downloads.slimdevices.com/SqueezeboxServer_v7.6.1_rc3/ I somehow managed to check the change in without leaving a note here - I'm sorry about that. r33105 should fix this.
Found your comment in 17457 that the changes had been checked in, just nothing to the bug report, so I tried 7.6.1-33110. I deleted my entire cache and ran a 'find everything' scan. - Did a select on tracks.url: all volume letters are capitalized (yeah, that was the point - just looking) - MusicIP mix generation, works great, and no errors to the server.log - saving / retrieving playlists (m3us) works great and no errors in the server log Also - these latter two are dramatically faster. What I did not test: upgrading one tracks table to another to see that all volume letters got upcased. Sorry. I'd already upcased all the volume letters manually, decided deleting the cache was the right thing to do, and afterwards realized that I had an unmodified copy of squeezebox.db I could have copied back in. My bad.
Thanks for the testing! > What I did not test: upgrading one tracks table to another to see that all > volume letters got upcased. That's not important any more, as the important change was to make the comparison of those paths case insensitive. > Sorry. I'd already upcased all the volume letters > manually, decided deleting the cache was the right thing to do, and afterwards > realized that I had an unmodified copy of squeezebox.db I could have copied > back in. My bad. No problem! Thanks!
Oh right - LIKE in SQLie is case insensitive. Well, so long as the characters are not international. I was wondering about the mismatch between all MusicIP lists having lower case volume letters and the db having all uppercase letters and figured you went with something like that. I don't know that there are any windows users with above-ascii-127 volume letters. bit maybe it's worth adding to the list of known limitations for the support folks? With millions of users, there's (by definition) a couple of "one in a million" types in there.
> Oh right - LIKE in SQLie is case insensitive. Well, so long as the characters > are not international. We've added the ICU libraries to overcome this issue (http://www.sqlite.org/faq.html#q18)