Bug 17930 - New artists missing on new and changed scan
: New artists missing on new and changed scan
Status: RESOLVED FIXED
Product: Controller App
Classification: Unclassified
Component: User Interface
: 1.1
: All Debian Linux
: -- normal
: ---
Assigned To: Michael Herger
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-19 15:06 UTC by zarniwoop75
Modified: 2014-03-14 15:34 UTC (History)
2 users (show)

See Also:
Category: ---


Attachments
Bump lastScanTime value after every scan step (792 bytes, patch)
2014-03-14 10:16 UTC, Michael Herger
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description zarniwoop75 2012-03-19 15:06:27 UTC
When a new artist is added to the library using the new and changed scan it does not appear in the Artist list of the Android controller app. A search for the artist is successful but it does not appear in the list. A full clear and rescan restores the artist to the list but is hardly practical.
Comment 1 Michael Herger 2012-03-19 22:41:29 UTC
Is this reproducible? Does it always happen when you add new music? How did you add it: browse music folder or rescan?  What server version?
Comment 2 Michael Herger 2012-03-20 02:15:08 UTC
Ok, in the forum you mention it was reproducible. What if you force-quit the app and restart?

The app should receive "lastscantime" timestamps in the status updates, triggering a flush of the local cache if needed. While it seems to be working fine in most cases, it seems to miss that update every now and then.
Comment 3 zarniwoop75 2012-03-20 03:58:47 UTC
(In reply to comment #2)
> Ok, in the forum you mention it was reproducible. What if you force-quit the
> app and restart?
> 
> The app should receive "lastscantime" timestamps in the status updates,
> triggering a flush of the local cache if needed. While it seems to be working
> fine in most cases, it seems to miss that update every now and then.

I've tried restarting the phone but I haven't tried force quitting the app, is that different to a normal exit? The album appears under albums and under new music so the cache is updated but the artist does not appear under artists. A search of artists reveals it so the app knows it is there. When I say it is reproducible it has happened both times I have consciously added new artists and then used the app.
It happens on rescan but I also tried browsing to the music folder and then returning to artists but no luck.
Comment 4 Michael Herger 2012-03-20 04:11:50 UTC
> It happens on rescan but I also tried browsing to the music folder and then
> returning to artists but no luck.

How much time was between the rescan/browse music folder and returning to artists?
Comment 5 zarniwoop75 2012-03-20 04:28:41 UTC
(In reply to comment #4)
> > It happens on rescan but I also tried browsing to the music folder and then
> > returning to artists but no luck.
> 
> How much time was between the rescan/browse music folder and returning to
> artists?

I just browsed to the folder then immediately returned to artists. The rescan was last night and then I tried browse music folder this morning. One thing I haven't tried is clearing the cache in Android "manage apps". I could try that tonight if you think it would prove anything.
Comment 6 zarniwoop75 2012-03-20 10:40:31 UTC
(In reply to comment #5)
> (In reply to comment #4)
> > > It happens on rescan but I also tried browsing to the music folder and then
> > > returning to artists but no luck.
> > 
> > How much time was between the rescan/browse music folder and returning to
> > artists?
> 
> I just browsed to the folder then immediately returned to artists. The rescan
> was last night and then I tried browse music folder this morning. One thing I
> haven't tried is clearing the cache in Android "manage apps". I could try that
> tonight if you think it would prove anything.

Neither force quitting or clearing the cache fix it.
Comment 7 Michael Herger 2012-03-21 01:25:36 UTC
> Neither force quitting or clearing the cache fix it.

Now that latter is hard to believe: if the cache is wiped, the app should fetch a new list from the server. Could you please enable debugging for network.jsonrpc on LMS, wipe the app's data, and try again? Would you see the requests for the new lists being sent to the server?
Comment 8 zarniwoop75 2012-03-21 01:54:47 UTC
(In reply to comment #7)
> > Neither force quitting or clearing the cache fix it.
> 
> Now that latter is hard to believe: if the cache is wiped, the app should fetch
> a new list from the server. Could you please enable debugging for
> network.jsonrpc on LMS, wipe the app's data, and try again? Would you see the
> requests for the new lists being sent to the server?

The cache I cleared was the Android cache and I am not even sure if that cache contains the library. I didn't try clearing data to avoid reentering mysb info. Should I try clearing data instead?
Comment 9 Michael Herger 2012-03-21 01:57:14 UTC
> The cache I cleared was the Android cache and I am not even sure if that cache
> contains the library. I didn't try clearing data to avoid reentering mysb info.

Ah, this might be it. Yes, if you flush the data, you should definitely get the most recent lists. But that's just to make sure it's a client side issue, and not server side. The problem remains...
Comment 10 zarniwoop75 2012-03-21 02:02:00 UTC
(In reply to comment #9)
> > The cache I cleared was the Android cache and I am not even sure if that cache
> > contains the library. I didn't try clearing data to avoid reentering mysb info.
> 
> Ah, this might be it. Yes, if you flush the data, you should definitely get the
> most recent lists. But that's just to make sure it's a client side issue, and
> not server side. The problem remains...

I'll try tonight. Have you seen the problem yourself?
Comment 11 zarniwoop75 2012-03-21 10:33:12 UTC
(In reply to comment #9)
> > The cache I cleared was the Android cache and I am not even sure if that cache
> > contains the library. I didn't try clearing data to avoid reentering mysb info.
> 
> Ah, this might be it. Yes, if you flush the data, you should definitely get the
> most recent lists. But that's just to make sure it's a client side issue, and
> not server side. The problem remains...

Clearing the app data adds the artist to the artist list as you suspected.
Comment 12 Geert Kloosterman 2014-03-13 07:48:09 UTC
Hi.

I've stumbled on the same problem quite a while ago.  I was able to make a workaround (a hack, a "good enough for me" fix) that I'm using for about a year now. I'm posting my findings here in the hope it will be useful as a starting point to make a proper fix.

I looked into this for 7.7.2 and haven't updated since.  Since the bug is still open I assume it still holds for 7.7.3.

The missing albums in the squeezebox controller app were indeed caused by 'lastRescanTime' not being updated after a partial rescan. Adjusting the 'lastRescanTime' field in the SQLite library.db file by hand would make the missing albums show up.

Further debugging showed Local.pm::markDone() --the function that among others updates the lastRescanTime--  was in fact called after scanning my music directory, but did an early exit because there was still a "pending action".  I found out this pending action was the scanning of the playlist directory.  

So here we come to my hack: I disabled the part that says "scan playlist folder too" in Control/Command.pm.  This indeed ensures the lastRescanTime is updated correctly, and new albums found in partial rescans now also automatically show up in the app.

Obviously this is not the proper way to fix it, but it does the trick for me...

Hope this helps,

Cheers,
Geert
Comment 13 Geert Kloosterman 2014-03-13 20:49:38 UTC
I can reproduce the problem using logitechmediaserver_7.8.0-1394636298: a partial rescan does not update the lastRescanTime, which in turn causes updates not to appear within the Android app.

My quick hack for 7.7.2 as described in my earlier comment doesn't work for 7.8.0 however.
Comment 14 Michael Herger 2014-03-14 05:09:29 UTC
(In reply to comment #13)
> I can reproduce the problem using logitechmediaserver_7.8.0-1394636298: a
> partial rescan does not update the lastRescanTime, which in turn causes
> updates not to appear within the Android app.

What do you mean by "partial" rescan? Did you cancel it halfway through?
Comment 15 Geert Kloosterman 2014-03-14 06:27:49 UTC
> (In reply to comment #13)
> What do you mean by "partial" rescan? Did you cancel it halfway through?

Oh, I'm sorry for the confusion.  With "partial" I meant the option "Look for new and changed media files".  This in contrast with what I would call a "full" rescan: "Clear library and rescan everything".  The problems do not seem to occur after a full rescan.

I'm using the following settings:

 - a single media folder with only the "Music" checkbox checked (having the others checked used to cause scans not to finish on my system, not sure whether that's still the case)
 - so the second media folder field is empty
 - a playlist directory filled in (pointing outside of the media folder tree)
Comment 16 Michael Herger 2014-03-14 10:16:36 UTC
Created attachment 7719 [details]
Bump lastScanTime value after every scan step

Here's what I'm seeing: the timestamp is only set at the very end of the scan process. But the variable holding the number of changes ($changes) is reset after every step. Therefore if there was more than one folder to scan (eg. playlists), but the last folder had no changes, then the timestamp would not be bumped, as $changes was zero.

This patch would bump the value after every scan step. Could you please give this a try?
Comment 17 Geert Kloosterman 2014-03-14 12:49:01 UTC
(In reply to comment #16)
> This patch would bump the value after every scan step. Could you please give
> this a try?

Yes, this works.  The lastRescanTime field is updated now.
Comment 18 Michael Herger 2014-03-14 15:21:54 UTC
Ok, thanks for testing! Let's give this a try. I've committed the change to both 7.7 and 7.8. Expect it to show up in tomorrow's nightly builds.
Comment 19 Geert Kloosterman 2014-03-14 15:34:33 UTC
(In reply to comment #18)
> Ok, thanks for testing! Let's give this a try. I've committed the change to
> both 7.7 and 7.8. Expect it to show up in tomorrow's nightly builds.

Thanks for picking it up so quickly!

Note I tested using 7.8.0.  I assume it will work for both.  As an aside, the reason I mentioned earlier that "my hack" didn't work for 7.8.0 was because I didn't restart LMS after changing the code... :-/