Bug 17734 - install stalls on windows due to huge FileCache folder
: install stalls on windows due to huge FileCache folder
Status: CLOSED FIXED
Product: Logitech Media Server
Classification: Unclassified
Component: Windows Installer
: 7.6.2
: PC Windows Home Server
: P2 major with 1 vote (vote)
: 7.7.1
Assigned To: Michael Herger
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-11-06 06:52 UTC by pgrosman
Modified: 2011-12-13 23:00 UTC (History)
5 users (show)

See Also:
Category: ---


Attachments
Make sure we run the initial purge cycle (778 bytes, patch)
2011-11-09 04:02 UTC, Michael Herger
Details | Diff
Use SQLite to store the cache instead of the file system (15.98 KB, patch)
2011-11-10 00:24 UTC, Michael Herger
Details | Diff
Use SQLite to store the cache instead of the file system (16.16 KB, patch)
2011-11-10 00:56 UTC, Michael Herger
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description pgrosman 2011-11-06 06:52:55 UTC
install stalls at the beginning (create directory phase)

disk is busy

I have waited for several hours

nothing happens

the program files/squeezebox directory is created but remains empty
Comment 1 Michael Herger 2011-11-06 21:52:27 UTC
could you please check how large your cache folder is? In particular the FileCache sub-folder? You might want to delete the latter if it has thousands of files. Shouldn't harm.
Comment 2 pgrosman 2011-11-07 13:59:09 UTC
(In reply to comment #1)
> could you please check how large your cache folder is? In particular the
> FileCache sub-folder? You might want to delete the latter if it has thousands
> of files. Shouldn't harm.

you're right, directory was huge (30 000 files)

took me hours to delete it

how is that possible ?

I will try tomorrow to do the install (still waiting for the delete to finish)

thanks
Comment 3 Michael Herger 2011-11-07 23:02:39 UTC
> you're right, directory was huge (30 000 files)
> took me hours to delete it
> how is that possible ?

Are you using any 3rd party plugins?

Andy/Triode - are we doing any purging on the cache? Eg. remove expired data? This is not the first report I've seen where the FileCache folder had grown to a huge size, causing long delays during installation. MeSue has reported the same for her system.

Unfortunately the Windows installer checks every single file while it's setting permissions for the SBS cache.
Comment 4 pgrosman 2011-11-07 23:54:40 UTC
(In reply to comment #3)
> > you're right, directory was huge (30 000 files)
> > took me hours to delete it
> > how is that possible ?
> 
> Are you using any 3rd party plugins?
> 
> Andy/Triode - are we doing any purging on the cache? Eg. remove expired data?
> This is not the first report I've seen where the FileCache folder had grown to
> a huge size, causing long delays during installation. MeSue has reported the
> same for her system.
> 
> Unfortunately the Windows installer checks every single file while it's setting
> permissions for the SBS cache.

no 3rd party plugins installed

just a windows home server (always on, very few reboots)
Comment 5 Jim McAtee 2011-11-08 18:46:31 UTC
FWIW... I installed the 7.7.0 release on Windows tonight from the installer. It was a fresh install, with no old cache folder, and even on a fast server it's agonizingly slow. I've never seen a Windows program install as slowly. There's a point during the process where it appears to stall for a minute or more.
Comment 6 pgrosman 2011-11-09 00:24:39 UTC
(In reply to comment #5)
> FWIW... I installed the 7.7.0 release on Windows tonight from the installer. It
> was a fresh install, with no old cache folder, and even on a fast server it's
> agonizingly slow. I've never seen a Windows program install as slowly. There's
> a point during the process where it appears to stall for a minute or more.

after deleting manually the cache folder, installation was ok

but the huge number of files is still mysterious to me

very annoying because my system whs partition did get full because of this and
no systems update were installed since several months because of this
Comment 7 Michael Herger 2011-11-09 01:13:37 UTC
> FWIW... I installed the 7.7.0 release on Windows tonight from the installer. It
> was a fresh install, with no old cache folder, and even on a fast server it's
> agonizingly slow. I've never seen a Windows program install as slowly. There's
> a point during the process where it appears to stall for a minute or more.

Jim - we know you hate it. You've state it many times before. But it doesn't help resolving this issue. Please do not dilute bug reports with unrelated comments. Thanks.
Comment 8 Jim McAtee 2011-11-09 01:15:59 UTC
??? I was trying to say that it stalls even with no cache.
Comment 9 Michael Herger 2011-11-09 04:02:31 UTC
Created attachment 7552 [details]
Make sure we run the initial purge cycle

We always had code in place to run a purge cycle on startup, but it was buggy,
skipping that initial purging. Subsequent tasks could have been skipped after a
bunch of circumstances. In particular users who never shut down their players
could easily end up with thousands of cached items.

This change make sure we do run an initial purging at startup, and reduces the
interval from 24h to 6h, in order to reduce the risk of having the purge cycle
be executed always at the same time, when a player was in use.

Please note that the first purging of such a large file cache could potentially
delay startup by several minutes. Under normal circumstances it shouldn't take
longer than a few seconds.
Comment 10 Michael Herger 2011-11-10 00:24:50 UTC
Created attachment 7556 [details]
Use SQLite to store the cache instead of the file system

Let's store the cached data in a database instead of the file system. This should not only fix this issue, but speed up purging etc. considerably.

Patch attached for review.
Comment 11 Michael Herger 2011-11-10 00:56:14 UTC
Created attachment 7557 [details]
Use SQLite to store the cache instead of the file system
Comment 12 SVN Bot 2011-11-10 07:17:54 UTC
 == Auto-comment from SVN commit #33683 to the slim repo by mherger ==
 == http://svn.slimdevices.com/slim?view=revision&revision=33683 ==

Bug: 17734
Description: rewrite cache to use a db instead of the file system
Comment 13 SVN Bot 2011-11-11 06:02:01 UTC
 == Auto-comment from SVN commit #33694 to the slim repo by mherger ==
 == http://svn.slimdevices.com/slim?view=revision&revision=33694 ==

Bug: 17734
Description: let the installer remove the legacy FileCache folder. Add message with a warning to the user - this can take several minutes.
Comment 14 Michael Herger 2011-11-11 08:28:41 UTC
*** Bug 17598 has been marked as a duplicate of this bug. ***
Comment 15 Michael Herger 2011-11-17 05:40:39 UTC
This is fixed in the upcoming 7.7.1 release.

Please note that the installation of 7.7.1 might still require a lot of time, as the old cache is being cleared.
Comment 16 Dinanath Kumar Singh 2011-12-13 23:00:20 UTC
Verification passed on 7.7.1 r33735.