Bugzilla – Bug 664
iTunes Playlists won't go away
Last modified: 2009-09-08 09:23:01 UTC
When I delete a playlist in iTunes, it doesn't actually go away in SlimServer. Newly created playlists show up within about a minute, and in fact it's so efficient that even playlists that are created only very temporarily and quickly deleted (by accidental drag and drop to the playlist window in iTunes, for instance) still end up getting stuck in SlimServer. The only way that I've been able to get rid of these playlists is to do a "Wipe Cache" from the Server Settings, which of course forces a rescan at which point everything is fine, but not only is that a manual process, but it takes a while to run. In my configuration, SlimServer is running on RedHat Linux 8.0, and iTunes is running on Windows XP SP2. The iTunes music and library database are stored directly on the Linux server running SlimServer and accessed from iTunes using Samba 3.0.
KDF, you want to take a crack at this? The problem is that when we start scanning in the itunes playlists, we don't remove the old ones from the db.
Slim::Music::Info::clearPlaylists(); is being called, but I guess you mean they need to be removed from %infoCache. I can try to figure it out, but I have no idea at the moment how anything is removed from the db itself. There is something that generates the playlists, and it steps in a loop through everything in teh cache. on rescan, we could call something that steps through and removes anything with a playlistURL, but that's really distasteful
Created attachment 205 [details] delete playlists before rescan we do a quick scan of the db before rescan to mark entries as invalid. this is a good place to delete the playlists.
comitted to cvs for nov 25 build. please check and confirm so it can be merged into teh 5.4.x release plan.
Installed 25 Nov build first thing this morning. Initial visit to "Browse Playlists" showed all the correct playlists, plus two additional entries for what appear to be streaming sources (ShoutCast URIs, I believe), but they didn't appear to be actual playlists, just URIs. Did a Wipe Cache and rescan, and since then I haven't been able to get any of the iTunes playlists to come back. Even tried creating a new playlists in iTunes with no success. Will probably end up rolling back to the previous nightly, since I basically now have no playlists.
I've tried rescan and wipe cache and the iTunes playlists seem intact. I'm using Mandrake, so I'm surprised that your experience is so different. before you give up on today's build, could you try running slimserver.pl --d_iTunes --logfile itunes.log, do a wipe cache, and past the last bit of the iTunes scan from the playlist to the end: 2004-11-25 10:21:28.3906 iTunes: Finished scanning iTunes XML 2004-11-25 10:21:29.0781 iTunes: done Scanning: unlocking and closing
Actually, I had already gone back to the Nov 20 build by the time I got your message (and for the record, everything worked fine as soon as I went back). Just dropped this morning's build in again so that I could do the logging, and below is what came out of it following a wipe cache and rescan. Again, no playlists show up in SlimServer. 2004-11-25 20:45:46.9811 iTunes: starting playlist parsing 2004-11-25 20:45:47.6069 got a playlist array of 3131 items 2004-11-25 20:45:47.6081 got a playlist (itunesplaylist:Library) named Library 2004-11-25 20:45:47.6096 playlists now has 0 items... 2004-11-25 20:45:47.6287 got a playlist array of 24 items 2004-11-25 20:45:47.6303 got a playlist (itunesplaylist:Audible) named Audible 2004-11-25 20:45:47.6318 playlists now has 0 items... 2004-11-25 20:45:47.6453 got a playlist array of 79 items 2004-11-25 20:45:47.6465 got a playlist (itunesplaylist:Christmas) named Christmas 2004-11-25 20:45:47.6479 playlists now has 0 items... 2004-11-25 20:45:47.6513 got a playlist array of 11 items 2004-11-25 20:45:47.6524 got a playlist (itunesplaylist:Comedy) named Comedy 2004-11-25 20:45:47.6539 playlists now has 0 items... 2004-11-25 20:45:47.6684 got a playlist array of 104 items 2004-11-25 20:45:47.6697 got a playlist (itunesplaylist:iTrip%20Stations) named iTrip Stations 2004-11-25 20:45:47.6711 playlists now has 0 items... 2004-11-25 20:45:47.6888 got a playlist array of 127 items 2004-11-25 20:45:47.6900 got a playlist (itunesplaylist:Jazz) named Jazz 2004-11-25 20:45:47.6914 playlists now has 0 items... 2004-11-25 20:45:47.6976 got a playlist array of 34 items 2004-11-25 20:45:47.6988 got a playlist (itunesplaylist:JDH%20Mix) named JDH Mix 2004-11-25 20:45:47.7002 playlists now has 0 items... 2004-11-25 20:45:47.7097 got a playlist array of 56 items 2004-11-25 20:45:47.7110 got a playlist (itunesplaylist:My%20Top%20Rated) named My Top Rated 2004-11-25 20:45:47.7124 playlists now has 0 items... 2004-11-25 20:45:47.7167 got a playlist array of 16 items 2004-11-25 20:45:47.7180 got a playlist (itunesplaylist:Party%20Shuffle) named Party Shuffle 2004-11-25 20:45:47.7193 playlists now has 0 items... 2004-11-25 20:45:47.7916 got a playlist array of 575 items 2004-11-25 20:45:47.7930 got a playlist (itunesplaylist:Recently%20Added) named Recently Added 2004-11-25 20:45:47.7944 playlists now has 0 items... 2004-11-25 20:45:47.8169 got a playlist array of 146 items 2004-11-25 20:45:47.8182 got a playlist (itunesplaylist:Recently%20Played) named Recently Played 2004-11-25 20:45:47.8196 playlists now has 0 items... 2004-11-25 20:45:47.8257 got a playlist array of 33 items 2004-11-25 20:45:47.8269 got a playlist (itunesplaylist:Slow%20Mix) named Slow Mix 2004-11-25 20:45:47.8283 playlists now has 0 items... 2004-11-25 20:45:47.8377 got a playlist array of 25 items 2004-11-25 20:45:47.8390 got a playlist (itunesplaylist:Top%2025%20Most%20Played) named Top 25 Most Played 2004-11-25 20:45:47.8404 playlists now has 0 items... 2004-11-25 20:45:47.8448 got a playlist array of 13 items 2004-11-25 20:45:47.8461 got a playlist (itunesplaylist:Unplayed%20Hit%20List) named Unplayed Hit List 2004-11-25 20:45:47.8475 playlists now has 0 items... 2004-11-25 20:45:48.2169 got a playlist array of 3049 items 2004-11-25 20:45:48.2182 got a playlist (itunesplaylist:Unrated) named Unrated 2004-11-25 20:45:48.2196 playlists now has 0 items... 2004-11-25 20:45:48.2239 iTunes: Finished scanning iTunes XML 2004-11-25 20:45:51.2215 iTunes: done Scanning: unlocking and closing
This is working for me, so I'm not sure what else to try.
At this point, I've tried just about everything, uninstalling and reinstalling the Nov 25 build, and finally the Nov 26 build. I've also changed parameters for iTunes integration, turning it off, turning it on, changing the XML libration locations, but nothing seems to make much of a difference. I've even wiped the DB and CONF file and done a complete fresh installation of the Nov 26 build, with the same results. The only minor difference is that I found that if I specify a "Music Folder" location in the main Server Settings page (which shouldn't *technically* be required with iTunes integration on), I will very initially (for about 10 seconds) get a playlist that looks something like the following: + KJAZ + ShoutcastBrowser Recently Played http://64.236.34.4:80/stream/2008 http://64.236.34.67:80/stream/2008 http://www.slimdevices.com/update/?version=5.4.0&lang=EN + iTunes: Audible + iTunes: Christmas + iTunes: Comedy + iTunes: iTrip Stations + iTunes: Jazz + iTunes: JDH Mix + iTunes: Library + iTunes: My Top Rated + iTunes: Party Shuffle + iTunes: Recently ed + iTunes: Recently ed + iTunes: Slow Mix + iTunes: Top 25 Most ed + iTunes: Uned Hit List + iTunes: Unrated Once the rescan starts running, however, these all disappear and never come back, except for the first two (which are physical M3U files in the playlists directory). Note those three URLs as well. Not sure where they're coming from. They're not contained in any of my playlists -- even the temporary ones, nor in my iTunes XML file, and they never showed up anywhere prior to the Nov 25 build, and in fact the slimdevices one didn't show up until today's build (I saw the first two yesterday when I first installed the Nov 25 build). This seems to imply that the playlist entries are in fact in the DB (and a quick peek at the .slimserver.db file confirms this, in fact, along with those three "strange" URLs). The limitation therefore seems to be that it's not *displaying* the playlists (remember I've physically deleted the DB completely and done a rescan, so it's obviously picking up the information from somewhere).
Tried turning off the cache. No difference.
A bit of further investigation revealed that those two IP-based URLs are actually *indirectly* part of the ShoutCast Recently Played playlist... They're not in the M3U file itself, but rather in the .PLS file that's downloaded from the ShoutCast URL. The "Living Room.m3u" file (contained within the ShoutCast Playlist sub directory) contains the following: #EXTM3U #EXTINF:-1,128 kbps: SKY . FM- Smooth Jazz - the world's smoothest jazz 24 hours a day http://www.shoutcast.com/sbin/tunein-station.pls?id=1354&filename=playlist.pls #EXTINF:-1,48 kbps: the spirit of KJAZ San Francisco http://www.shoutcast.com/sbin/tunein-station.pls?id=835445&filename=playlist.pls If I download the PLS file from this first URL, two of the URLs that it contains (out of five) match those first two "phantom" playlist entries. In my previous testing, I had changed the location of the playlist directory in server settings, and that didn't seem to make any difference, so I don't think the mere presence of the Shoutcast recent playlists is affecting the problem. If I remove the Shoutcast directory, and then I re-run the test mentioned above (define the path to my music folder under server settings), I no longer get any of those URLs in the initial list (everything looks just fine), nor do they appear in the .slimserver.db file. However, the iTunes playlists still disappear once the scan is finished (although as I indicated above, they do remain in the DB). Note as well that the above test only works if I define the real path to my music folder (/var/music in my case). If I define any other path, then only my normal playlists will show up.
ok, here is a long shot, based on something I saw posted in the list. Are you runnign the iTunes Updater plugin while iTunes is also running and being updated live? If so, do you also have the iTunes update interval set to 60? If all this, or something similar, exists then you are possibly losing the playlists because the server always things iTunes file has changed and it starts a simple rescan (not a wipe cache, but a background scan).
Actually, I had considered that the iTunes plugin might be interfering, so I had disabled it and it didn't seem to make a difference. I wasn't running it in direct mode anyway, but rather indirect (where it writes a history file and then a Windows Perl script updates the actual iTunes data). I also shut down iTunes and anything else related to it just to make sure that there wasn't a file open conflict. However, I wouldn't think that would make a difference in terms of the update interval unless iTunes was actively writing to the library (either because music was playing or I was in the middle of managing things). Also, I don't know if this is conclusive, but normally when a background scan is performed, I get the "SlimServer is currently scanning" message at the top of the window, which I'm not seeing in this case.
Okay, at this point I've completely removed SlimServer and all traces of it from my Linux box. I've reinstalled the Nov 26 build from scratch, even going so far as to reconfigure it from scratch, rather than using my existing config file. It still behaves exactly as I've indicated above. However, if I simply drop in the Nov 24 build, everything's okay again.
ok, first...bad new is, my main server STILL works fine. Good news is, on the windows test machine that I have, oddly with the same code, I get the results you are reporting. at least now I have some hope of finding out what is going on.
Okay, if I add the code in the attached patch to the Nov 24 build, it actually works exactly as it's supposed to. This leads me to believe something [I]else[/I] has changed in the Nov 25 build that's causing this problem.
ok, think I have an idea what is going on. ITs not related to the fix for this problem at all. It looks like interference from Musicmagic stuff, so assigning back to me.
Jesse, can you test something out for me. Open up and edit /usr/local/slimserver/Slim/Music/Import.pm remove line 60, which looks like this: $importCount--; then restart your server and check your itunes playlists. if this fixes things 9and it seems to for me) I'll merge it in tonight
Yup, that did it for me. (Specifically, I installed today's build from the RPM, loaded it up to confirm that the problem was still present, then modified that line and restarted SlimServer. I had to do a rescan before anything showed up, but after a couple of rescans, and adding an deleting a couple of iTunes playlists, everything looks cool).
These changes don't appear to have made it into the 5.4.1 branch. The old behaviour has been back since the 5.4.x stuff was branched off. Currently running the 23 Dec 04 build.
it will be merged over for tomorrow. some patches got missed when they started building the branch.
something else must be going on here. The changes for this ARE in 5.4.1 as of Dec 2.
as far as I can see with my setup, this bug is still fixed in 5.4.1
Apologies for the delay in responding, as I've been unavailable. The problem still exists for me. I'm running SlimServer on Windows XP now, still on the 23 Dec 04 build. Wiping the cache and rescanning of course will naturally get rid of the deleted playlists, but then if more playlists are renamed or deleted, they continue to linger until the next "Wipe Cache" process. This is the same behaviour that was showing up before the problem was originally fixed. I'm going to try the most recent build (05 Jan 05) and see if that makes a difference.
when using the rescan button under server settings, they definately go away and show changed names, etc. Are you speaking of the itunes scan at the reload interval?
ok, this is what I think is happening here. when the checker finds the itunes library has changed, it rescans the itunes file, adding to the %infoCache. Then regenerates the playlists from the %infoCache at the end of scan. Thus, removed files are NOT removed. changed files show up as added ones. without forcing 5.4.1 to undergo the same reworking of the Importers, the simlest way I can think of to solve this is either to live with it until 6.0 OR to call Slim::Music::Import::startScan instead jsut as if a user were to press the rescan button. Dean..what do you think?
Yes, I've confirmed that pressing the rescan button makes the playlists go away. I'm not on the 05 Jan 05 build, BTW. Personally, I'm content to live with the problem for now, particularly since calling the full rescan process would probably take longer and be more of a performance hit than a simple refresh.
The iTunes importer should throw out all itunes playlists in the db when it starts importing from the file, rather than throw out all playlists in general.. Let's fix this for 6.0 and use the DB appropriately
*** Bug 799 has been marked as a duplicate of this bug. ***
There appears to be no way to delete a url entry from the DB. when we 'genereateExternalPlaylists', we then read all the itunesplaylist: urls, even old ones that don't appear in newer rescans of the itunes xml. we have to be able to delete them, or mark a group of urls by mask as invalid, so that the xml can re-validate. Then we'd still need a way to not display or clean out invalid data.
bouncing to Dan for the db-side handling for this.
oh yeah, this shoudl be a target for 6.0 release
I've commited a fix in svn change 1993 for this. We now clear external change lists from the db.