Bugzilla – Bug 17791
Large libraries have severe performance problem.
Last modified: 2014-10-13 21:51:45 UTC
I have a very large library, 350K+ tracks. Ever since 7.6, the server freezes under some usages: - My Music-->New Music (almost all the time) - My Music-->Albums, Artists. (seen sometimes). The web page will attempt to connect and wait (2-3 minutes). During this time the connected player will continue to play until emptying their buffers, then go silent. The display will say 'Can't connect to server. Left to go back, right to try again.' I leave it be, and it will reconnect in 2-3 minutes and be ok after that. This was working perfectly in 7.5.X, and was broken in 7.6 and 7.7. (When Mysql support was dropped.....) Logitech Media Server Version: 7.7.0 - r33614 @ Tue Oct 18 11:07:25 PDT 2011 Hostname: XXXXredactedXXXX Server IP Address: 192.168.1.110 Server HTTP Port Number: 9000 Operating system: Debian - EN - utf8 Platform Architecture: x86_64-linux Perl Version: 5.10.1 - x86_64-linux-gnu-thread-multi Database Version: DBD::SQLite 1.34_01 (sqlite 3.7.7.1) Total Players Recognized: 6 No images, No videos. Ubuntu 11.10 - 64bit.
Are you already using the 'High' setting under Settings > Performance > Database Memory Config? Maybe an enhancement request for additional 'Very High' memory settings would be worthwhile, although I'm not sure how much difference the existing High setting makes. At the current time you can still run MySQL in 7.7, although it's not officially supported. You must run your own instance of MySQL server and create an empty database before doing the following: 1. Update your LMS installation to a recent 7.7.1 nightly. It contains some missing database definition scripts for MySQL. 2. Manually install DBD::mysql in your Perl installation. 3. With the server stopped, edit the server.prefs file, changing the database settings to include the following: dbpassword: <your_MySQL_LMS_user_password> dbsource: dbi:mysql:hostname=127.0.0.1;port=3306;database=<your_MySQL_LMS_database> dbtype: MySQL dbusername: <your_MySQL_LMS_user_name> If the MySQL server is running on another computer, enter that computer's IP address instead of 127.0.0.1. The 'dbtype' setting is new to 7.6/7.7 and must be capitalized exactly as shown. 4. Restart the server.
(In reply to comment #1) > dbpassword: <your_MySQL_LMS_user_password> > dbsource: > dbi:mysql:hostname=127.0.0.1;port=3306;database=<your_MySQL_LMS_database> > dbtype: MySQL > dbusername: <your_MySQL_LMS_user_name> Note that the "dbsouce: dbi:mysql:hostname=..." entry should on a single line in the server.prefs file.
Thanks Jim, Yes I am using the 'High' setting. I'll try the running MySQL, but would much prefer an official solution.
I experience the same behaviour, but on windows with a 45K+ library. Sometimes it happens after a search, sometimes when changing tracks. Can't find the cause.
I ran strace when the hang occurs (This is just a small sample, it will go on for 2-3 minutes: read(16, "\r\0\0\0\2\1 \0\2\221\1 \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024 lseek(16, 4810752, SEEK_SET) = 4810752 read(16, "\r\0\0\0\2\1*\0\2\211\1*\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024 lseek(16, 269996032, SEEK_SET) = 269996032 read(16, "\r\0\0\0\2\0\360\0\2s\0\360\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024 lseek(16, 192765952, SEEK_SET) = 192765952 read(16, "\r\0\0\0\2\0\366\0\2}\0\366\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024 lseek(16, 457278464, SEEK_SET) = 457278464 read(16, "\r\0\0\0\2\0\340\0\2n\0\340\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024 lseek(16, 286321664, SEEK_SET) = 286321664 read(16, "\r\0\0\0\2\0|\0\2L\0|\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024 lseek(16, 298785792, SEEK_SET) = 298785792 read(16, "\r\0\0\0\2\0\322\0\2J\0\322\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024 lseek(16, 297192448, SEEK_SET) = 297192448 root@MusicServer:~# ls -l /proc/23649/fd/16 lrwx------ 1 squeezeboxserver nogroup 64 2012-01-01 01:49 /proc/23649/fd/16 -> /var/lib/squeezeboxserver/cache/library.db
I finally got a chance to try this. I'm running Suse 12.1 - 64-bit /etc/squeezeboxserver/server.conf is a symbolic link to /var/lib/squeezeboxserver/prefs/server.prefs I stop server (/etc/init.d/squeezeboxserver stop), and edit the prefs file, changing: bpassword: '' dbsource: dbi:SQLite:dbname=/var/lib/squeezeboxserver/cache/library.db dbusername: slimserver To: dbpassword: ___MYPASSEWORD____ dbsource: dbi:mysql:hostname=127.0.0.1;port=3306;database=slimserver dbtype: MySQL dbusername: slimserver I've created a MySQL db: mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | slimserver | | test | +--------------------+ 5 rows in set (0.00 sec) mysql> use slimserver; Database changed mysql> show tables; Empty set (0.00 sec) --------------------------------------------- It's there & empty. Now I start server...and wait a bit. when I look at the prefs its overwritten the db* items with the original sqlite versions: dbpassword: '' dbsource: dbi:SQLite:dbname=/var/lib/squeezeboxserver/cache/library.db dbusername: slimserver Back on the webpage, after the normal setup (mysqueezebox.com, entering music & playlist dirs, All tracks are there immediately, I'm assuming its using the old (sqlite db). I ran strace and the server writes to server.prefs.tmp and then move server.prefs.tmp to server.prefs. Any idea what I'm doing wrong??
One more thing: I have perl-DBD-mysql installed.
Never mind... I missed the comment about dbi: not being on a separate line. Fixed this & now I'm getting: 12-01-30 17:16:31.8822] Carp::Clan::__ANON__ (214) Warning: Carp::Clan::__ANON__(): Carp::Clan::__ANON__(): DBI Connection failed: DBI connect('hostname=127.0.0.1;port=3306;database=slimserver','slimserver',...) failed: Access denied for user 'slimserver'@'localhost' (using password: YES) at /usr/share/squeezeboxserver/CPAN/DBIx/Class/Storage/DBI.pm line 999 pretty sure I can track this down...
Well I got the server working with mysql & hit 2 problems. 1) Scan took 50 hours (vs ~10 for 7.5.x). 2) 'look for new & changed' rescan causes each track to be listed twice. I'm going back to 7.5.6 for now. I am requesting this serious regression is fixed.
*** This bug has been confirmed by popular vote. ***
Performance with large collection should be vastly improved in 7.9. Make sure you're setting the performance/memory preference to maximum. The bottle neck with large collections often was disk I/O, as sqlite had to read indices from disk during queries. Giving it more memory to keep that data in RAM greatly improves performance. Please give it a try.
Works like a dream. Squeezboxes are a pleasure to use again. Thanks for all your hard work.