Bugzilla – Bug 9554
Controller lockup/reboot when browsing artists list
Last modified: 2008-12-15 12:40:37 UTC
Running 7.2 r 2873 controller firmware with 7.3 SC. Music Library, Artists, Select an Artist causes the display to mostly switch to the artist's album list but then it locks up about 7/8th of the way to sliding the new menu/list over. After about 15 seconds it reboots the controller without any user action.
Doug: a few Q's reproducible with any artist? reproducible 100% of the time? Is there anything out of the ordinary about your system (plugins, demonic possession, etc)?
Well, every artist that I've tried. Every time. I randomaly picked about 5 different artists from various places in my artist list prior to reporting. I don't think that I have any plugins laying around. I just recently had to manually blow away all of the SC folders (except bin, to keep my lame.exe file) and reinstall SC. Would plugins be somewhere else other than the program files folder on Windows XP? I was getting some sort of socketholder error on the Windows box and that seemed to be somewhat related to my one-time upgrade issue (not sure if it was cause or effect, though). I haven't seen it since I wiped the folder and reinstalled. I did NOT clean the registry when I wiped the folders. I have recently rebooted the Windows box several times and restarted SC after changing some logging settings and have done several clear and full rescans (and partial scans) - no impact on this issue's reproducibility, though. Albums are flac, ripped with EAC and AccurateRip verified (for the most part). Individual files per track. They have been tagged with Jaikoz - potentially his newest version. No embedded artwork (located externally in the folder as either cover.jpg or foler.jpg (depending upon the program that found the artwork)). I run SC as a Windows Service with no user (local system account). I also use MusicIP integration - the new way with it only providing "mixable" information. Would either of those count as "demonic possession"? :) When the artist's album list is sliding into view and it hangs it MIGHT be about where the MusicIP mixible icon would be coming into view from the right... The controller freezes at that point and then reboots after a short time. It has almost fully slid into view and there are only a couple of characters in size remaining on the left side from the previous artist list screen.
I'm confused on the MIP stuff-- the "new" way? Am I missing something with that? I added the MIP support for SBC and haven't changed anything significantly in a while. I also don't believe there's any support for displaying a 'mixable' icon on SBC--that's only in the playerUI for displayed players (i.e., everything but SBR). Can you elaborate on what you meant by that? Further, it'd probably be good to eliminate MIP from the picture and see if you can still reproduce. Turn off the MIP server, disable the MIP plugin, and restart SC. Another thing that would be helpful is to turn SC's log setting for network.cometd to DEBUG, reproduce the issue, then attach the server log to this bug. FYI, I use MIP on both of the SC's I run at home here and am not seeing the issue. I'm using 99% MP3s with (typically) cover.jpg files.
In SC 7.3 MIP integration just recently got a major change checked in. There is a config. option in the MIP area that lets you select between the old way (full pull of all track info. from MIP) or the "new" way - just pulling over the "mixable" information from MIP and all of the rest from the SC scan. The changes needed for me to use the "new" way were to define a music folder for SC (it was blank to prevent duplicates and speed up the scanning process in the past) and change the new MIP inteface option in SC 7.3 to mixable only. The reasons for the MIP interface change were scanning performance (massivly faster when adding a few albums) and better "grouping"? (apparently MIP changes some things/tags when it pulls tracks into its library that differ in how SC handles things, not sure of the details of this, though). I thought that the controller displayed a "mm" graphic? when something was mixable. I could easily be mistaken, though, going from memory is always spotty. I generally do my mixes from the web UI. I will turn on the logging this evening and recreate the issue.
Created attachment 4051 [details] log file with controller lockup.
I added the log file. One other note - I managed to get one to work - an artist with only one album appears to work without hanging the controller. Multiple albums for an artist still hangs every time I try it. However, if you scroll down on the one album list to the add to favorites it lock up the controller and causes it to reboot as well. I didn't find this out until after I'd captured the log file so this case isn't in it. Let me know if you need that one as well.
Created attachment 4053 [details] section of previously attached log that requests and receives data for Artists->ABC Doug, can you confirm that going into Artists->ABC reproduced the problem? The attached request/response for that artist, extracted from your log, doesn't look amiss (SC successfully queried for the artist, constructed a properly-formatted JSON response, sent back the correct list count). So, if Artists->ABC "shot that poison arrow" through your controller, the next thing to get a gander at is the controller log.
The best way to capture a log from the controller is using an SD card. I wrote up a wiki page on this a while back-- see section 3.1: http://wiki.slimdevices.com/index.php/SqueezeboxControllerMessagesFile After you boot with the SD card, you may as well turn on some extra debugging to capture to the log file. When an SD card is present, you should see a menu item at Settings->Advanced->Logging. Set player.browse.data to DEBUG, then reproduce the issue and attach the log here (when you attach, set the mime type to text/plain, which will allow me to look at the log directly in my browser) Sorry to put you through so many hoops...I'd really like to get to a root cause with this one right away and haven't been able to reproduce the issue on a similar setup here.
It was something in the A's, so that is probably it. I didn't want to put too many extra steps in the log file. I only did the one action wiht the log file I uploaded. I hadn't played anything after rebooting SC that morning. I'll do the controller logging this evening, no problem. That's why I run beta code... :) One note, my controller is a beta version. I THINK that it is one of the last beta ones, though because I got into that program late. Not sure, though. If that may be important then please let me know how I could gather the specific information about what mfg. run it is from.
beta hardware, esp. late beta hardware, shouldn't be an issue. Thanks for helping out with the log files. One last request-- make your playlist much smaller than the 200 tracks you had in it in the first log file. Long playlists end up adding a ton of unneeded data in the log file.
Created attachment 4057 [details] controller log file with player.browse.data set to debug
Log file from controller attached. This was when I picked ABC artist again. I also noticed a delay when I actually selected Artist from the main menu this time. That did not occur previously.
Hmmm...what I'm seeing in the log is that the a request for 200 items at position 200 in the list went out right before the controller rebooted itself. This seems odd if user was trying to browse to an artist with much less than 200 items in its list. That request should never be made... Also strange that nothing was being reported from player.browse.data debug. Sep 24 20:19:31 SqueezeboxController user.debug jive: (DB.lua:54) - DB:__init() Sep 24 20:19:31 SqueezeboxController user.debug jive: (DB.lua:156) - DB {Artists} menuItems() Sep 24 20:19:31 SqueezeboxController user.debug jive: (DB.lua:222) - DB {Artists} missing accel=false index=nil dir=nil Sep 24 20:19:31 SqueezeboxController user.info jive: (Wireless.lua:564) - iwconfig power off Sep 24 20:19:37 SqueezeboxController user.debug jive: (DB.lua:156) - DB {Artists} menuItems() Sep 24 20:19:37 SqueezeboxController user.debug jive: (DB.lua:121) - ..store invalid, different count Sep 24 20:19:37 SqueezeboxController user.debug jive: (DB.lua:222) - DB {Artists} missing accel=false index=nil dir=0 Sep 24 20:19:40 SqueezeboxController user.debug jive: (DB.lua:156) - DB {Artists} menuItems() Sep 24 20:19:40 SqueezeboxController user.debug jive: (DB.lua:222) - DB {Artists} missing accel=false index=3 dir=1 Sep 24 20:19:40 SqueezeboxController user.debug jive: (DB.lua:260) - DB {Artists} missing 200 items from pos 200 Sep 24 20:20:33 SqueezeboxController syslog.info syslogd started: BusyBox v1.7.1.svn adding Michael-- Michael, I think we're into the nasty chunked request are again. Do you know if anything changed on the SC side that might make our requests return incorrect data to queries? Doug, do you have a recollection that this worked fine in 7.2 SC?
*** Bug 9461 has been marked as a duplicate of this bug. ***
Dang, how did I miss the earlier report? I did search for "reboot." Sorry about that. Could I not have changed the log settings properly? On the controller and went to the (going from memory) settings, advanced, logging and selected on the player.brose.data until it changed to debug, then went out (left arrow). Do you need to hit play or something to make them take effect? Reboot? I didn't see much in the way of details in the wiki (maybe I was looking in the wrong section?). It was mainly about the SD card setup. I don't normally use the Artist list very often but I do use it upon occasion. I'm pretty sure that I used it in the late beta 7.2 timeframe. I doubt that I used it with the GA 7.2 since I was only on that for a couple of weeks and I was away a lot of that time. The artist (ABC) in question has 1 album and 2 various artists album tracks in my collection.
For the record, 9461, which was marked as a duplicate of this, has slightly different behavior. The hang that I experience is seen when scrolling down with the wheel after I've gotten into the album list for an artist. The bug described here involved the sideways transition from the artist list to the album list for an artist. My problem was also reported on the current production 7.2 release of software. But yes, the end result is the same - controller hangs and restarts. And for me it is with only a handful of artists (3 so far). BTW, 99% of the time, I navigate genre->artist->album. In my case, no matter how I navigate to a "problematic" artist, I see the same behavior.
I think the thing to do would be to reproduce the lockup/reboot and then immediately pop the battery out so the bottom of the log is where the failure occurred.
I had to do that (knock the batter out) to collect that logs from the bug I reported. Otherwise the log files were overwritten when the controller came back up.
Just so I understand - create the error and then while it is locked up pull the battery? OR wait for the black screen when it is about to reboot?
probably doesn't matter, but might as well wait for the reboot. for the record, I don't think the messages file gets overwritten on reboot if you are using an SD card. AFAIK, it should append to the bottom of the existing messages file.
The logs I uploaded were pulled out of the SD card after the controller had rebooted from the error and then shut down normally.
okay, you definitely captured the crash with the first log you attached right above the log output I mention in comment#13, I see this: Sep 24 20:19:11 SqueezeboxController user.info jive: (LogSettingsApplet.lua:59) - Setting log category player.browse.data to WARN Sep 24 20:19:12 SqueezeboxController user.info jive: (LogSettingsApplet.lua:59) - Setting log category player.browse.data to ERROR Sep 24 20:19:13 SqueezeboxController user.info jive: (LogSettingsApplet.lua:59) - Setting log category player.browse.data to DEBUG that tells me that I got everything I'm going to get from player.browse.data (which is to say, not much) I don't think another log at this point is necessary. Let's take a different tack: instead of going My Music->Artists->ABC, just go to My Music->Artists and scroll all the way to the bottom. Did it still reboot? The log file makes me think that diving into the artist is a red herring, that the lockup is caused by the big all Arists query a level up.
Note that I said earlier that I could also create the issue when I went into a one item artist's list and scrolled down to the "add to favorite" item. So scrolling to the bottom of that, very short, list caused an issue - may be related. I will check scrolling to the bottom of my full artist list tonight.
(In reply to comment #20) > probably doesn't matter, but might as well wait for the reboot. > for the record, I don't think the messages file gets overwritten on reboot if > you are using an SD card. AFAIK, it should append to the bottom of the existing > messages file. Hi Ben - When I tried collecting logs with an SD card, I was logged into the controller. Prior to the reboot, I saw the file reach a certain size. When it came back up, the file was significantly smaller. I also tried copying the file to another file name before the reboot. I saw it there, but it was gone when the reboot was complete. That was why I did the "knock the battery out" thing. I would have expected it to append and was surprised by this behavior. Those were my observations - perhaps the behaivor is due to the problem at hand - not sure. Thanks you guys for looking into this. Rich
Created attachment 4066 [details] Album list when locking up One more thing I've noticed: while SqueezePlay would display an icon with the "All items" entries, Controller does not. Plus, the "All tracks" item is badly laid out on Controller. The crash imho only occurs with artists having more than one album, thus the above items are displayed.
Created attachment 4067 [details] same list on SqueezePlay
Created attachment 4068 [details] same list on SqueezePlay
Created attachment 4069 [details] OKish layout (though false position), not locked up yet Ok, my latest bet is the additional items' layout: - If I disable the additional menu items ("all..", "Favorites"), everything's fine. I can scroll through the full list of items. - If I disable the "All..." menus, I can scroll through the full list, _until_ I scroll to the favorites item. That moment the menu's layout will break and SBC will reboot. The last item's icon will shift to the right by a few pixels. I guess it does by default lock up immediately, as there's a "Play all" item at the top, displayed first. Ben - could you imagine a SC side change in 7.3 which required a SqueezePlay change too? Thus it's fine in SP on PC, but not in the rather oldish firmware images we have on our Controllers?
Created attachment 4070 [details] Broken layout after scrolling down to the Favorites item - Controller locking up
Looks like you guys might have gotten past my requested test, but when I go to Music Library (not My Music) and the Artists and scroll to the bottom it does NOT cause a lockup. I can scroll all the way down and it pauses and goes to the top as it should. I can also go up all the way and it pauses and goes to the bottom just fine as well. If I pick a different artist I do get a lockup. I did once get a lockup on an artist where the full next list displayed instead of only partially sliding in from the right. That artist had more than a page of entries. On an artist with only 1 entry it locks up when you scroll down to the "add to favorites" menu but not before.
"Ben - could you imagine a SC side change in 7.3 which required a SqueezePlay change too? Thus it's fine in SP on PC, but not in the rather oldish firmware images we have on our Controllers?" YES. I think we may have a smoking gun... https://bugs-archive.lyrion.org/show_bug.cgi?id=9412 So, what I need to do at this point is get a first beta of 7.3 firmware pushed out to anyone running SC 7.3 so anyone running 7.3 SC is matched with 7.3 controller firmware. Secondly, I need to resolve the issue in 7.2, because the checkin to fix 9412 was in the 7.2 branch (post 7.2 release, so it will be in 7.2.1 or anyone updating their 7.2 SVN checkout). I think the correct path would be to pair another controller revision to push out with a SC 7.2.1 release. That, fingers crossed, may solve this lock up issue for all users. What I believe is happening is that SC is sending a munged style for the "special" items in this list (play all, add to favorites), and as a result the controller doesn't know how to render the window. In those situations, misc bad things can happen. many thanks for the input, Michael
see bug 9630 for addressing the underlying crash issue. Sending problematic menu styles from SC should not crash the SBC UI. We need better behavior on that front.
FYI, I just checked SC 7.2 release code with current top-of-7.2-tree SBC firmware, r3080. The problem doesn't happen in this combination. This means that the bug manifests itself particularly with the combination of new SC (SC 7.2.1/7.3) with old SBC firmware, and not the reverse. In other words, IMO there are two things we should do to sanely get this resolved for all users: 1. get 7.3 SBC firmware pushed out to SC 7.3 users a.s.a.p. (this is being actively worked on) 2. push 7.2.1 firmware out with any official 7.2.1 SC release that may come.
FWIW: using 7.3 I've now manually upgraded to FW3070. This combination is working fine.
how long until r3080 is available on SN?
I'm experiencing this issue as well. How do I manually upgrade my controller firmware? Or can the newest version be pushed out to controllers?
> I'm experiencing this issue as well. How do I manually upgrade my controller > firmware? Or can the newest version be pushed out to controllers? Download http://update.slimdevices.com/update/firmware/7.2.1/jive_7.2_r3070.bin and follow the instructions here: http://wiki.slimdevices.com/index.php/SqueezeOS_Upgrades#Firmware_Update_with_SD_Card
Cool, just installed 3080, and I can confirm the bug is fixed for me.
(In reply to comment #38) > Cool, just installed 3080, and I can confirm the bug is fixed for me. I just installed 3080 myself and it has resolved the issues I've been seeing. Nicely done!! :-).
7.3 SC now pushes 7.3 controller firmware so this bug is effectively dead.
Verified, no longer locks up and reboots 7.3 - r3403.
This bug has been fixed in the 7.3.0 release version of SqueezeCenter! Please download the new version from http://www.slimdevices.com/su_downloads.html if you haven't already. If you are still experiencing this problem, feel free to reopen the bug with your new comments and we'll have another look.