Bugzilla – Bug 15681
Scanning progress web ui bugs
Last modified: 2011-05-12 10:11:32 UTC
Created attachment 6508 [details] Screen capture during discovery pass of a new & changed scan During any type of scan, the times displayed for any given pass are incorrect, usually showing some oddly formatted negative number. When the pass is finished, the correct times are then shown.
Created attachment 6509 [details] Screen capture following new & changed scan Here's another screen capture showing the correctly formatted time after the new & changed scan is finished. Another small problem shown in this capture is that after the scan, the last file scanned remains displayed. If you leave the page and return, this isn't shown any longer.
I don't think a 7.5.0 target should be set for any 7.5 embedded branch bugs. It's my understanding that this branch will be released as 7.6 or 8.0.
Thanks Jim, correct. That's not that important for 7.5.
== Auto-comment from SVN commit #30553 to the slim repo by mherger == == http://svn.slimdevices.com/slim?view=revision&revision=30553 == Fixed Bug: 15681 Description: make Total display the current Done value if there's no total
Created attachment 6758 [details] Progress displayed during a new & changed scan No, still seeing the same thing at r30558. This screen cap shows progress during a new & changed scan. Look at the line 'Discovering files' (the three lines above are left over from, I guess, the most recent clear & rescan run last night).
Created attachment 6760 [details] Progress display after new & changed scan Here again is a screen cap of the progress screen following the completion of the new & changed scan. Problems: - During the 'discovering' pass, the elapsed time displayed is negative and doesn't grow. It's corrected when the scan completes. - During the 'discovering' pass, the progress line is displayed in all black and doesn't move. - Status lines from the passes from a previous clear/rescan are shown both during and after the new & changed scan. - After completion, the elapsed times from that previous scan are added into the total time displayed for the new & changed scan. In this example the total time should be approximately 50 seconds, not 17:57. - After completion, the path of one file remains displayed below the 'Discovering files' status bar. Let me know if you'd like any other bugs opened for any of this. Progress for a clear/rescan has similar, but different problems.
Reopening.
*** Bug 15682 has been marked as a duplicate of this bug. ***
Those issues are why I left 15682 open. But if you prefer this one, I'll keep it open :-P
Created attachment 6762 [details] New & changed scan where new content is added to library Sorry for the confusion. Here's a cap from a new & changed scan where two new albums were added to the library. It's a bit different from a new & changed scan that discovers no new files. - Passes are displayed in the wrong order - discovering files should be first. - Does not display passes from a previous scan. - Total time is calculated correctly.
Andy - the web UI is simply pulling this data from the progress table afaik. Is the old data correctly removed when a new scan starts?
Created attachment 6766 [details] SQLite progress table In the last example, if the order of the passes is displayed incorrectly, it suggets that the records aren't being sorted properly. Sorting them by ID should fix it it. Here is a view of the SQLite progress table following a new & changed scan with two new albums. The records appear to be in the correct order.
Ah, I think I see the problem. Take a look at that last attachment. The third record shows a start time that precedes the other three records. I'm not sure how that can happen, but if you're sorting them by start time then it would explain why they're out of order.
The existing code is sorting by start,id: my @progress = Slim::Schema->rs('Progress')->search( $args, { 'order_by' => 'start,id' } )->all; Looking at your snippet I see that the playlist scan is the oldest (judging from the start timestamp), which makes me think the table isn't wiped on new scans.
OK, I probably forgot to put in a call to Slim::Utils::Progress->clear for all types of scans.
Jim - do you see this only when triggering a rescan from the web UI or in (m)any cases?
== Auto-comment from SVN commit #30582 to the slim repo by mherger == == http://svn.slimdevices.com/slim?view=revision&revision=30582 == Bug: 15681 Description: scan progress fixes - don't overwrite file discovery with playlist discovery - reset progress when triggering a rescan manually - replace redundant Slim::Music::Import->clearProgressInfo with Slim::Utils::Progress->clear -
== Auto-comment from SVN commit #30583 to the slim repo by mherger == == http://svn.slimdevices.com/slim?view=revision&revision=30583 == Bug: 15681 Description: add missing strings for new playlist discovery step
== Auto-comment from SVN commit #30584 to the slim repo by mherger == == http://svn.slimdevices.com/slim?view=revision&revision=30584 == Bug: 15681 Description: fix active flag handling when running external scanner
Jim - I've fixed a few issues related to the scan progress. But I still can't reproduce the negative time. Do you still see it? If so - do you have simple steps to reproduce it?
Created attachment 6773 [details] SQLite progress table Yes on the negative time. In this instance I'm running a new & changed scan from the web interface with no new content. Stays at -3 seconds and doesn't change during the course of the discovery pass. See the attached shot of the SQLite table during the scan, which may help explain it - it's the data, not a formatting error. The finish time is correct (1271269007, 54 seconds) only when the discovery pass completes. Will do more testing.
I just ran a test 'new & changed' scan, with new content. No negative times, but a similar problem during the discovering_directory pass. What I'm seeing is that with the directory_new (scanning new files) and precacheArtwork passes, while they're active the finish time remains NULL and is only populated when the pass finishes. That's not the case with the discovery_directory pass, which has the finish time populated with either a time preceding or equal to the start while it is active.
Jim - what do you see in the UI? If the display is correct, then we don't have to bother about what's in the db. From your two last comments I'm not sure you're still seeing anything wrong.
(In reply to comment #23) > Jim - what do you see in the UI? I'm seeing the negative times during the discovery pass when there's no new content. And when there is new content then the elapsed time during the discovery pass stays at zero.
Are you using MySQL or SQLite? Is this a rather large or small collection? How long would the discovery step take?
SQLite, 27k FLAC files. The discovery step generally takes from 50 to 55 seconds.
Here's a fix that only uses the finish time if the pass is active, otherwise it uses the current time. Obviously, it doesn't fix the problem with the scanner putting bad data into the progress table for the discovering_directory step. Line 47 of server/Slim/Web/Pages/Progress.pm #my $runtime = ($p->finish || time()) - $p->start; my $runtime = ((!$p->active && $p->finish) ? $p->finish : time()) - $p->start;
Is a total file count no longer found before the discovery step? This would be the 'of Y' value when displaying 'X of Y'. If so, that's fine, but IMO the numbers should be displayed differently when the total is unknown and you're simply counting. Instead of '(1 of 1)', '(268 of 268)', '(4773 of 4773)', '(30815 of 30815)', etc. it should display (1), (268), (4773), (30185). Lack of a total also presents a problem of what to do with the progress bar. It can't really be used properly without a total count, but right now is displayed all black during the discovery step. IMO it should be either all gray or maybe 1/2 gray and 1/2 black while the step is active.
== Auto-comment from SVN commit #30652 to the slim repo by mherger == == http://svn.slimdevices.com/slim?view=revision&revision=30652 == Bug: 15681 Description: improve scan progress display. Don't show "x of y" values unless there's an y value.
== Auto-comment from SVN commit #30653 to the slim repo by mherger == == http://svn.slimdevices.com/slim?view=revision&revision=30653 == Bug: 15681 Description: need to convert progress values to integer before comparing them. Hide progress bar if there's no Total value
I'm still seeing the left over text showing a file path after the completion of a 'new & changed' scans. Have not seen this following a clear/rescan.
== Auto-comment from SVN commit #31271 to the slim repo by mherger == == http://svn.slimdevices.com/slim?view=revision&revision=31271 == Fixed Bug: 15681 Description: hide Info element if it's empty in the current data set. This should hide the left over path information when the scan has finished without any new file found.
A couple of minor string issues in the English translation. Not all of the verbs are in progressive tense. I'm not sure how many are actually in use, but may as well fix them all. Corrections below: DIRECTORY_DELETED_PROGRESS EN Remove deleted files => Removing deleted files DIRECTORY_NEW_PROGRESS EN Scan new files => Scanning new files PLAYLIST_DELETED_PROGRESS EN Remove deleted playlists => Removing deleted playlists PLAYLIST_NEW_PROGRESS EN Scan new playlists => Scanning new playlists
Another minor issue that someone recently mentioned in the forums: When there are no playlists to be found, the 'Discovering playlists' phase displays (1 of 1) when finished instead of (0 of 0), or just (0).
== Auto-comment from SVN commit #31275 to the slim repo by mherger == == http://svn.slimdevices.com/slim?view=revision&revision=31275 == Fixed Bug: 15681 Description: use progressive tense for all progress information
Created attachment 7276 [details] 32398 log
Comment on attachment 7276 [details] 32398 log No negative time displayed on 7.6.0 32398.