Bug 3019 - synched players do not advance to next song
: synched players do not advance to next song
Status: RESOLVED FIXED
Product: Logitech Media Server
Classification: Unclassified
Component: Playlists
: 6.3.0
: PC Other
: P2 normal with 2 votes (vote)
: ---
Assigned To: Andy Grundman
:
Depends on: 259
Blocks:
  Show dependency treegraph
 
Reported: 2006-02-16 12:21 UTC by Russell Blaine
Modified: 2011-03-16 04:34 UTC (History)
2 users (show)

See Also:
Category: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Russell Blaine 2006-02-16 12:21:36 UTC
When I have two players (a SB3 and a PC with softsqueeze) synched, they will play exactly one song and then stop. I can manually advance them by going to the next song in the playlist and hitting "play" - they will then play one more song and stop.

I've seen this with 6.2.1 and 6.2.2. (Couldn't get nightly 02-16 working enough to try it there)
Comment 1 KDF 2006-02-16 14:30:17 UTC
I am sure this is a dupe of bug 1869
Comment 2 Dan Sully 2006-04-22 13:48:24 UTC

*** This bug has been marked as a duplicate of 1869 ***
Comment 3 Russell Blaine 2006-04-28 12:14:20 UTC
I don't think this is the same a 1869. In 1869, players do not start. In my case a sb2 and softsqueeze 2.3 fail to advance to the next song when theyre synched. They play a song fine, synched well, but then stop at the end without moving to the next song at all. They both sit there at the end of the current song. d_sync says this:

...
2006-04-28 12:10:13.7519 3f:72:37:03:54:f9 : is synced with its slave
00:04:20:06
:42:a1
2006-04-28 12:10:15.9449 3f:72:37:03:54:f9 : is synced with its slave
00:04:20:06
:42:a1
2006-04-28 12:10:15.9563 3f:72:37:03:54:f9 : is synced with its slave
00:04:20:06
:42:a1
2006-04-28 12:10:15.9656 3f:72:37:03:54:f9 : is synced with its slave
00:04:20:06
:42:a1
2006-04-28 12:10:16.0112 3f:72:37:03:54:f9 : is synced with its slave
00:04:20:06
:42:a1
2006-04-28 12:10:16.0243 00:04:20:06:42:a1 has run out of data, checking to see
i
f we can push on...
2006-04-28 12:10:16.0251 00:04:20:06:42:a1 has run out of data, checking to see
i
f we can push on...
2006-04-28 12:10:16.0257 00:04:20:06:42:a1 has run out of data, checking to see
i
f we can push on...
2006-04-28 12:10:17.0117 3f:72:37:03:54:f9 : is synced with its slave
00:04:20:06
:42:a1
2006-04-28 12:10:18.0218 3f:72:37:03:54:f9 : is synced with its slave
00:04:20:06
:42:a1
2006-04-28 12:10:19.0216 3f:72:37:03:54:f9 : is synced with its slave
00:04:20:06
:42:a1
2006-04-28 12:10:20.0317 3f:72:37:03:54:f9 : is synced with its slave
00:04:20:06
:42:a1
2006-04-28 12:10:21.0413 3f:72:37:03:54:f9 : is synced with its slave
00:04:20:06
:42:a1
...

This is with the 6.5 beta from April 25th. Let me know if you need more debuging info.
Comment 4 Russell Blaine 2006-04-28 12:14:56 UTC
and I've seen this on 6.2.2 as well as the most recent 6.5b1.
Comment 5 Eric Bost 2006-06-02 13:40:55 UTC
(In reply to comment #0)
I had the same issue for SB3.  I got a SB3 in December 2005 and installed Slim Server V6.2.1. on a XP-SP2 PC.  All worked great.  We wanted a second SB3.

Got the new SB3 a few days ago.  Upgraded to Slim Server V6.2.2.  The old SB3 needed firmware upgrade.  Pressed and held the brighteness button to do the upgrade to firmware v48.

Then I encountered the bug.  

To fix the problem:

1. I uninstalled Slim Server.
2. Rebooted PC
3. Installed Slim Server V6.6.2
4. Did factory reset on both SB3s.  (Hold the "+" while powering up SB3s.)
5. Reentered my PSK in the wireless set up.
6. When I synched the players in Slim Server everything worked.

I have had absolute stability for two days.  I'm not sure whether the clean install of Slim Server or the factory reset of the SB3 was the trick.  The synch feature is so sweet!

Hope this helped.
Comment 6 Blackketter Dean 2006-06-06 16:55:41 UTC
Is this still happening with the 6.3 beta release?
Comment 7 Mike Gilpin 2006-06-06 16:55:47 UTC
Subject: Out of Office AutoReply:  synched players do not advance to next song

I am out of the office on business through Friday evening June 9th, so may not be able to reply to your message as quickly as usual. If you have an urgent matter, try my mobile number, 240-899-5886.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7638.1">
<TITLE>Out of Office AutoReply: [Bug 3019] synched players do not advance to next song</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->

<P><FONT SIZE=2>I am out of the office on business through Friday evening June 9th, so may not be able to reply to your message as quickly as usual. If you have an urgent matter, try my mobile number, 240-899-5886.</FONT></P>

</BODY>
</HTML>
Comment 8 Eric Bost 2006-06-06 19:17:53 UTC
Subject: RE:  synched players do not advance to next song

I haven't tried the 6.3 beta.  Note that the original message was about the
simulator not SB3.  My SB3's now work great in sync mode following the steps
that I posted.

-----Original Message-----
From: Slim Devices Bugzilla [mailto:bugs@web.slimdevices.com] 
Sent: Tuesday, June 06, 2006 4:56 PM
To: svejbost@surewest.net
Subject: [Bug 3019] synched players do not advance to next song

https://bugs-archive.lyrion.org/show_bug.cgi?id=3019





------- Comment #6 from dean@slimdevices.com  2006-06-06 16:55 -------
Is this still happening with the 6.3 beta release?




------- You are receiving this mail because: -------
You are a voter for the bug, or are watching someone who is.

Comment 9 Chris Owens 2006-06-07 17:03:04 UTC
Russell (or anyone) are you still seeing this issue with the 6.3 release?

If so, do you have fade in/fade out enabled?
Comment 10 Russell Blaine 2006-06-07 17:50:26 UTC
I just tried server 6.3 from today (june 7th), and the problem is still there. Not using fade-in or fade-out on the squeezebox.
Comment 11 Blackketter Dean 2006-06-11 20:31:29 UTC
Andy, did your recent change fix this?
Comment 12 Mike Gilpin 2006-06-11 20:31:36 UTC
Subject: Out of Office AutoReply:  synched players do not advance to next song

I am out of the office on business through Tuesday evening June 13th, so may not be able to reply to your message as quickly as usual. If you have an urgent matter, try my mobile number, 240-899-5886.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7638.1">
<TITLE>Out of Office AutoReply: [Bug 3019] synched players do not advance to next song</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->

<P><FONT SIZE=2>I am out of the office on business through Tuesday evening June 13th, so may not be able to reply to your message as quickly as usual. If you have an urgent matter, try my mobile number, 240-899-5886.</FONT></P>

</BODY>
</HTML>
Comment 13 Dan Sully 2006-06-11 20:33:53 UTC
Russell - there have been some fixes to syncing that went in on June 9th. Could you try the most recently nightly again? June 11th or 12th (depending on when you get this).

Thanks.
Comment 14 Andy Grundman 2006-06-11 20:51:22 UTC
I haven't seen this bug myself.
Comment 15 Mike Gilpin 2006-06-12 00:38:57 UTC
(In reply to comment #11)
> Andy, did your recent change fix this?
> 

I installed the Saturday night build of 6.3 yesterday. So far so good - but sometimes it takes a few days
before the problem shows up. So I'll keep using it this week and let you know what happens.
Comment 16 Andy Grundman 2006-06-13 22:07:30 UTC
Mike, how's it been doing?
Comment 17 Mike Gilpin 2006-06-14 02:10:33 UTC
I was out of town Monday-Tuesday, installed the Monday night build of 6.3 last night. Will be playing music all day today and let you know how it goes. Until that point, the build from Saturday had been working fine, but it's too early to declare victory.
Comment 18 Russell Blaine 2006-06-14 07:19:51 UTC
I'm away from home, so I can't try an updated version until next week.
Comment 19 Mike Gilpin 2006-06-14 14:38:35 UTC
So I've been testing this all day, and it hasn't frozen up, but it has exhibited some "undesirable" behavior. Still, it's an improvement on the freeze-up problem, which would hang up SlimServer and require a reboot.

So here's what it's doing today:
-- Every now and then the players get out of synch. I hear the music playing on one but not the other, or they are both playing the same song, but not at the same point.
-- Sometimes there's a long pause between songs before advancing to the next. There appears to be some correlation between these two issues, but it's not totally one-to-one.
Comment 20 Mike Gilpin 2006-06-19 15:29:32 UTC
(In reply to comment #13)
> Russell - there have been some fixes to syncing that went in on June 9th. Could
> you try the most recently nightly again? June 11th or 12th (depending on when
> you get this).
> 
> Thanks.
> 

OK, testing completed, in the sense that the bug is still there. Just had my first "freeze-up" since installing the 6-17 build of 6.3. I had also been experiencing other synch-related problems as before. So to be clear, here are all the specific failure modes:

1) Freeze-up: part way through playing a track (which happened to be 60th in a playlist of 4820), playback freezes. The displays on all four players are also frozen at the same point - scrolling of the text on the display has stopped. Go to the Preferences Pane on the Mac and click "Stop", and it never comes back. The process is hung somehow. Shutting down only takes down part of the Finder, leaving the kernel running. I have to use the power switch to turn it off. I looked in the log before shutting down and there was nothing there related to this problem.

2) Pause between tracks: A few times over the last couple of days, it pauses for a bit - a minute or so - before going on to the next track.

3) Loss of synch: I hear the same song playing at different points in the track on various players.

Once failure modes 2 and 3 start to happen, they happen more and more often. So there's some deterioration.

Stopping SlimServer and restarting it seems to temporarily halt this downward spiral - music plays OK for a few hours before the next problem happens.
Comment 21 Mike Gilpin 2006-06-20 08:06:34 UTC
This (6-17 build of 6.3) has been getting the "freeze up" failure scenario pretty consistently, actually a bit more frequently than what I was experiencing with earlier builds. But perhaps this is good news, insofar as an ability to recreate the problem, at least on my end? It's been very unpredictable before. Now it seems to fail in the range of 40-50 tracks into the playlist.

So are there some diagnostics you'd like me to try to get, for when it fails? Not sure what options to turn on that would help finding the problem, but I'm around all this week and willing to help.
Comment 22 Andy Grundman 2006-06-20 09:59:46 UTC
Your freeze-up issue might be bug 3581 which should be fixed.  I know you didn't have an empty playlist but I've seen that same issue with a populated playlist.
Comment 23 Andy Grundman 2006-06-20 12:09:27 UTC
I realize I didn't answer your question.  You should use "--d_source --d_sync --d_playlist --logfile sync.log" if you want to create a log of this happening.  I'm going to let a couple players run on a long playlist and see if I can reproduce.
Comment 24 Andy Grundman 2006-06-21 08:25:53 UTC
I put a 120-song playlist on repeat and synced it on 2 players overnight.  This morning it had wrapped around at least once and is now on song 90/120.  Both players are still in perfect sync and display the correct track.  I wonder if the problem only occurs if there is some sort of network dropout.  I'm going to try playing with the network connections and signal strength to see if I can break it.
Comment 25 Andy Grundman 2006-06-21 08:33:47 UTC
Mike, I have a few other questions.

Are you using all SB2/3 or do you have an SB1 in the mix?  Is SoftSqueeze involved?
What format of music are you playing?
Is any internet streaming involved? 
Are they all wireless, if so, is it 11g?

My test was done with an SB2 and SB3, 11g wireless, and a combination of mp3, ogg, and flac files from networked storage.
Comment 26 Mike Gilpin 2006-06-21 08:45:24 UTC
 ------- Comment #25 from andy@slimdevices.com  2006-06-21 08:33
> -------
> Mike, I have a few other questions.
>
> Are you using all SB2/3 or do you have an SB1 in the mix?  Is 
> SoftSqueeze involved?
> What format of music are you playing?
> Is any internet streaming involved?
> Are they all wireless, if so, is it 11g?
>
> My test was done with an SB2 and SB3, 11g wireless, and a combination 
> of mp3, ogg, and flac files from networked storage.

Hi, I just got a chance to download a new build this morning - so I 
got the 6-21 version of 6.3. So far so good, although it's too soon to 
tell.

On your questions:
-- I have four SB2s
-- All are connected via wired network (Cat 5e connected to Gigabit 
Ethernet switches)
-- No Softsqueeze
-- All the music is MP3s. All are high bit-rate (192 or above - most 
of what I've ripped lately is 320K). I do this do it all works across 
a mix of other players (an iPod as well as a Creative Labs device)

I doubt I'm getting any interruptions in network connectivity. At the 
max these connections should be using only a tiny fraction of the 
network capacity.
Comment 27 Andy Grundman 2006-06-21 09:16:00 UTC
Does the problem ever happen if you use only 2 or 3 players?  I'm going to try it with 4 players and see if that makes a difference.
Comment 28 Andy Grundman 2006-06-21 12:35:04 UTC
Just reproduced this.  On starting song 25 of the playlist, one player became unsynced by a fraction of a second.  The other 3 stayed in sync.
Comment 29 Mike Gilpin 2006-06-21 12:42:08 UTC
(In reply to comment #28)
> Just reproduced this.  On starting song 25 of the playlist, one player became
> unsynced by a fraction of a second.  The other 3 stayed in sync.
> 

I also have experienced this synch problem once today, as well as the "slow start" problem (meaning, it took a mysteriously long time to transition from one track to the next).

For me, though the synch problem (4 players) was different. One player just didn't go along with the others. It didn't appear frozen; the display had moved to the next track (the one that was being player in the other rooms), but whereas in the rooms where music was still playing, the progress indicator was filling in normally, on the player that was not playing, the progress bar remained empty. But the track name was still scrolling by.

Do you still want me to try just running two players, for a while? I will also be turning on the logging, when I get a chance (I'm on the phone a lot today).
Comment 30 Andy Grundman 2006-06-21 12:51:33 UTC
On the next track after the problem occurred, things were back in sync.  I haven't seen the long delay or wrong title display issues yet.  I have a suspicion that it's related to a network delay in all the unpause commands being sent out, this could be caused by heavy load on the server as well.  This issue may never be fully solved until bug 259 is implemented.

I'm streaming FLAC to 4 wireless players, so it's pushing 4 Mbit of data over my wireless network, with 3 laptops on the same network being used.  While the network should be able to handle up to about 20Mbit, that much load could cause some issues.  This could be why 2 players never caused the problem for me.  That shouldn't be an issue on your wired network though as you pointed out.  The player that unsynced was the one with the lowest signal (blocked by a metal filing cabinet near my desk).

Do you have the web interface open when you run into this problem?  I've been leaving the web UI open during my testing today.  I bet if a webpage such as the playlist frame refreshes at exactly the time when it's trying to send out all unpause commands, this may also create a delay.
Comment 31 Mike Gilpin 2006-06-21 12:54:54 UTC
(In reply to comment #30)
> On the next track after the problem occurred, things were back in sync.  I
> haven't seen the long delay or wrong title display issues yet.  I have a
> suspicion that it's related to a network delay in all the unpause commands
> being sent out, this could be caused by heavy load on the server as well.  This
> issue may never be fully solved until bug 259 is implemented.
> 
> I'm streaming FLAC to 4 wireless players, so it's pushing 4 Mbit of data over
> my wireless network, with 3 laptops on the same network being used.  While the
> network should be able to handle up to about 20Mbit, that much load could cause
> some issues.  This could be why 2 players never caused the problem for me. 
> That shouldn't be an issue on your wired network though as you pointed out. 
> The player that unsynced was the one with the lowest signal (blocked by a metal
> filing cabinet near my desk).
> 
> Do you have the web interface open when you run into this problem?  I've been
> leaving the web UI open during my testing today.  I bet if a webpage such as
> the playlist frame refreshes at exactly the time when it's trying to send out
> all unpause commands, this may also create a delay.
> 


I do have the Web interface open - I use it as a remote control as I'm in an adjacent room to where the nearest player is located.

Perhaps my problem is partly a wimpy server - it's a Mac Mini, one of the earlier ones. It has a gig of RAM, but the processor's not that fast. So if there are server-overload potential issues, that could be related. I'll try closing the web interface to see if that makes a difference.
Comment 32 Andy Grundman 2006-06-21 13:12:30 UTC
Cool.  I think the problem I saw is one of the unpause commands not being received at the right time.  This could be relatively easily solved by using broadcast packet(s) that would reach all players at the same time.  This will not make it into 6.3 however, so I'm moving this back to 6.5.

I'm also changing the title since we've deviated quite a bit from the original bug reported, which I believe has been resolved.

It will still be good to get a log of the long pause and wrong title display issues.
Comment 33 Andy Grundman 2006-06-21 13:14:05 UTC
*** Bug 3616 has been marked as a duplicate of this bug. ***
Comment 34 Mike Gilpin 2006-06-21 13:18:51 UTC
Posting from another email:
On Jun 20, 2006, at 3:20 PM, Gilpin, Mike wrote:

> Thanks. It's been a while since I've run from the command line instead 
> of the Preferences Pane (Max OS X). Remind me how that works? What 
> command name do I use, is it Slim or Slimserver? (sorry to be
> ignorant)

Sure, this should work for you in Terminal:

cd ~/Library/PreferencePanes/SlimServer.prefpane/Contents/server
./slimserver.pl --d_source --d_sync --d_playlist --logfile sync.log

You can also add --daemon if you want it to go into the background so you don't need to leave a Terminal window open.  The log file will be located in the same directory.

Please try with tonight's nightly, since it will include the fix for the lockup issue.
Comment 35 Andy Grundman 2006-06-21 13:32:27 UTC
Just saw the 30 second delay.  In fact it was *exactly* 30 seconds, very interesting.  It happened when I manually selected a different track from the playlist.  Also found an easy way to get the players out of sync: enable --d_select and start a track.
Comment 36 Mick 2006-06-21 13:39:57 UTC
Andy,

I'm not sure exactly how the SB's stayed synched without and accurate clock on board and using something like the NTP so that each player exactly were it sholud be in a song at an accurate point in time.
1 player would be deemed the synch master and all other players would try to match to it.

I accept that this is a difficult problem to solve, especially on a wirless network, but buffering should make things easier.

In my case I have 1 SB3 which communicates over 2 hops to the Slimserver
192.168.2.9  SB3 --(802.11G)-->Wireless Router --(Cat5)--> Slimserver

and another SB2 which uses a wireless bridge

192.168.2.10 SB2 --(802.11G)-->AP Bridge --(802.11G)-->Wireless Router --(Cat5)--> Slimserver.

Ping times from slimserver to devices on the network seems to vary quite a bit.
So a broadcast telling all players to unpause, may take 1ms to get to one player and 10ms to get to the other.

C:\>ping 192.168.2.9

Pinging 192.168.2.9 with 32 bytes of data:

Reply from 192.168.2.9: bytes=32 time<1ms TTL=64
Reply from 192.168.2.9: bytes=32 time=1ms TTL=64
Reply from 192.168.2.9: bytes=32 time=31ms TTL=64
Reply from 192.168.2.9: bytes=32 time<1ms TTL=64

Ping statistics for 192.168.2.9:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 31ms, Average = 8ms

C:\>ping 192.168.2.10

Pinging 192.168.2.10 with 32 bytes of data:

Reply from 192.168.2.10: bytes=32 time<1ms TTL=64
Reply from 192.168.2.10: bytes=32 time<1ms TTL=64
Reply from 192.168.2.10: bytes=32 time=2ms TTL=64
Reply from 192.168.2.10: bytes=32 time=3ms TTL=64

Ping statistics for 192.168.2.10:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 3ms, Average = 1ms

C:\>ping 192.168.2.9

Pinging 192.168.2.9 with 32 bytes of data:

Reply from 192.168.2.9: bytes=32 time=7ms TTL=64
Reply from 192.168.2.9: bytes=32 time<1ms TTL=64
Reply from 192.168.2.9: bytes=32 time<1ms TTL=64
Reply from 192.168.2.9: bytes=32 time<1ms TTL=64

Ping statistics for 192.168.2.9:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 7ms, Average = 1ms

C:\>ping 192.168.2.10

Pinging 192.168.2.10 with 32 bytes of data:

Reply from 192.168.2.10: bytes=32 time=4ms TTL=64
Reply from 192.168.2.10: bytes=32 time=1ms TTL=64
Reply from 192.168.2.10: bytes=32 time=1ms TTL=64
Reply from 192.168.2.10: bytes=32 time=1ms TTL=64

Ping statistics for 192.168.2.10:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 1ms, Maximum = 4ms, Average = 1ms

C:\>
Comment 37 Andy Grundman 2006-06-21 14:49:26 UTC
You're right that NTP is the only real solution here.  Right now we're betting on LAN latency being under 10ms where the sync difference won't be audible.  Any time you get more than 10ms delay between all unpause packets being received, you'll have sync problems.  Using broadcast packets would only remove any delay caused by the server being slow to send these packets in sequence, which in my brief testing is hard to do.  Network latency, especially wireless, is a far worse enemy in this situation.
Comment 38 Mike Gilpin 2006-06-22 05:33:59 UTC
I've been running successfully (on the same build) with only three out of the four SB2 players as part of the synch group, and without the Web interface running, since yesterday afternoon. Left it running overnight with the sound turned off. This morning everything still seems properly synched, with no freeze-ups either. Currently at track 214 out of the 4820 in the playlist.

So this looks like an acceptable workaround until the bug is fixed.

Say, Andy, is there an easy way to find out when 6.5b1 will be sufficiently stabilized for me to try that again? I think I was having better luck with it than 6.3, a few weeks ago - but I no longer have that version around, sadly. Did a cleanup a bit too soon. It might make sense (re: the server utilization explanation you floated yesterday), if using MySQL for the DBMS is more efficient than SQLite, at least insofar as resource usage by the main thread. I did notice that scanning was much faster in 6.5b1. Or perhaps there are other reasons why 6.5b1 is faster. 

Dean had mentioned that 6.5b1 was somewhat less stable during this timeframe when you're trying to lock down 6.3 and get it out the door. So, perhaps, just wait about a week after 6.3 ships?
Comment 39 Mick 2006-06-22 06:11:43 UTC
Mike,

Leaving players on overnight does not really tell you whether the players have stayed synched for the whole night.

It just tells you that they are synched since the start of the current song.
Slimserver attempts to re-synch the players at the start of every song.
If the players got out of synch during the night, your test just shows that they sucessfully re-synched.

I was thinking about this problem a bit more last night.
The current way that the SB buffers data means that the more Wireless players on a network, the more of a load on the network at the start of each song.

At the moment, the SB will buffer up until the end of a song, and only start pulling down the next song once the buffer is drained.
For 1 SB, that's fine. For 2, you have twice the network traffic being initated at the same time.
It then takes longer for the buffer to fill up and since playback kicks off once about 10% of the track is bufferred, there's more of a chance of an underrun as the 2 SB's try to fill their buffers at the same time.

Why not begin buffering the next song when time remaining <20% of the song length on a round robin basis.
For an average 4 or 5 minute song, that gives each SB a window of 80-100 seconds to buffer a certain amount of the next track. The round robin approach could continue until the buffer on each SB is 100% full.
Comment 40 Mike Gilpin 2006-06-22 06:32:26 UTC
(In reply to comment #39)
> Mike,
> 
> Leaving players on overnight does not really tell you whether the players have
> stayed synched for the whole night.
> 
> It just tells you that they are synched since the start of the current song.
> Slimserver attempts to re-synch the players at the start of every song.
> If the players got out of synch during the night, your test just shows that
> they sucessfully re-synched.
> 
> I was thinking about this problem a bit more last night.
> The current way that the SB buffers data means that the more Wireless players
> on a network, the more of a load on the network at the start of each song.
> 
> At the moment, the SB will buffer up until the end of a song, and only start
> pulling down the next song once the buffer is drained.
> For 1 SB, that's fine. For 2, you have twice the network traffic being initated
> at the same time.
> It then takes longer for the buffer to fill up and since playback kicks off
> once about 10% of the track is bufferred, there's more of a chance of an
> underrun as the 2 SB's try to fill their buffers at the same time.
> 
> Why not begin buffering the next song when time remaining <20% of the song
> length on a round robin basis.
> For an average 4 or 5 minute song, that gives each SB a window of 80-100
> seconds to buffer a certain amount of the next track. The round robin approach
> could continue until the buffer on each SB is 100% full.
> 


I hear you, except that in my experience with this particular release, and my particular
configuration, when things start to go wrong (with the synch problem) they just keep getting 
worse and worse until I get a freeze-up. So although I agree it's theoretically possible
that there could have been a synch problem during the night that I missed, the fact that
things are still working so well this morning seems to validate the view Andy expressed
yesterday that there is some correlation to the load on the server (which in my case is 
a wimpy Mac Mini).

What you say about the network load may well be true (as an issue for folks with wireless
networks), but I have a wired Gigabit Ethernet network - which should have plenty of
capacity. I do like your suggestion for changing the way the player cache gets loaded,
though - at least from my point of view as a "used to be a programmer" who doesn't 
know Perl. It seems likely that such a cache-management strategy could minimize the
potential for issues on wireless networks - which I gave up over a year ago for Slimserver
usage, as I got tired of interference from the microwave oven. Perhaps the 802.11n stuff
will be worth trying, once the standard is finally approved and we get out of this "pre-n"
phase. I understand the MIMO technology is much less susceptible to 2.4 GHz interference.
Comment 41 Mike Gilpin 2006-06-23 03:27:51 UTC
Crashed overnight. I left it running again. There was nothing in the log - the machine was hung. The one difference was that the player screens had all gone blank, rather than staying frozen on the same display. This was probably just another instance of the "freeze-up" problem, but since I was sleeping the players eventually timed out or something and went dark when they couldn't find the server. Normally I would have rebooted long before then.

At the point of the crash (not being entirely sure when it happened) I was a few hundred tracks into a playlist, but not the same one I was playing before (my wife came home - now playing music she likes). This is consistent with a correlation I've seen before - switching to another playlist seems to hasten the arrival of the next freeze-up problem. Could there be something the server does when it sorts a large playlist that uses more resources, which are then not released? Does Perl do garbage collection? Is there a memory leak, or is that something irrelevant to Perl? Java is the most recent language I actually used.

So in your recreate efforts (if you're still looking at this problem), you might try this:
-- Have two different playlists of several thousand songs each
-- Be in "shuffle by song" mode
-- Play one of the playlists for a while
-- Switch to the other one
-- Play it for a while
-- Keep switching back and forth until it freezes up.

And if you had some probe on the server code to track memory allocation, or to catch infinite loops, so much the better.

Pardon my amateur attempt at assistance with diagnostics!

Oh, and talking about this recreate scenario reminds me of an idea for an enhancement: retained state for multiple playlists. It would be really cool if, when you switched from one playlist to another, it would persist the current playlist somewhere (in its shuffled state), so when you switched back to it, you'd still be in the same position you were before and could continue from there. I've tried manually doing this with saved playlists, but it never seems to work quite right.

If you think this is a good suggestion, let me know, and I'll add it to the "enhancement suggestion" forum.
Comment 42 Mike Gilpin 2006-06-27 09:36:21 UTC
So far so good on 6.3 (8118). I have not had as much of a chance to do a lot of testing as I would like, due to excessive work and travel.

But, Andy, should I join that 4th player back into the synch group? I noticed in the beta forum that Michael O'Reilly indicated he had experienced better synch performance and that you had checked in some changes in this area. 

If you think I should try this again (4 players, plus the Web interface), I will.
Comment 43 Andy Grundman 2006-06-27 09:41:39 UTC
There really isn't going to be better sync performance until we have an NTP implementation.  Some major sync bugs were fixed but they don't address the fact that players will sometimes start playing at slightly different times.  In a situation where all players are on a low-latency LAN (like your setup) with <10ms latency to the server, things should be good most of the time.  For those using wireless where some players may have higher latency, differing signal strength, packet loss, and so on, you'll end up with more sync issues.
Comment 44 Mike Gilpin 2006-07-05 07:21:59 UTC
I just installed the final 6.3.0 production release, so I'll be testing again to see what happens when I'm running 4 SB2 players. My previous testing with the build from 6/24 found that my synch problems did not appear if I was running only 3 players, and no browser interface. Adding the 4th player and browser interface brought out the synch problems pretty quickly.

Question: is there any reason to think the final-production version of 6.3.0 will behave any differently? In particular, I'm thinking that since (in my case) this problem seems to be (at least in part) one of the capacity (speed, etc.) of the machine running the server, if the final code is faster because of not running any debug code, then it could be worth more testing. I know the beta releases I get from Microsoft are slower because of debug code, but I don't know if that applies to Perl-based apps.
Comment 45 Andy Grundman 2006-07-05 07:23:45 UTC
This issue still exists so 6.3 won't change anything for you unfortunately.
Comment 46 Mike Gilpin 2006-07-05 07:26:04 UTC
Re: still exists. Yes, I realize that, yet I have been successfully working around the issue by running only 3 players - and it really does make a difference. Do you think the final 6.3 will run any faster (or use fewer system resources) than the 6-24 beta build of 6.3 was?
Comment 47 Andy Grundman 2006-07-05 07:27:16 UTC
Nope, it should be close to the same.
Comment 48 Russell Blaine 2006-07-26 16:58:02 UTC
My bug was hijacked. I filed this bug because, as my original description states, synched players do not advance to the next song. During song playback, the sync is perfect. When they get to the end of the song, they stay there with the display at the end of the song, until the skip-forward button is pressed. If folks are having other synchronization problems, please file a different bug.

I am still seeing this problem with the official 6.3.1 release, there has been no change in behavior since it was first reported. I'd still like to know if there is anything I can do to help root-cause this issue (instead of just trying successive releases and hoping it is fixed accidentally - that is not a strategy for success).

BTW, this is on a PC running softsqueeze 2.8 under Solaris 11.
Comment 49 Mike Gilpin 2006-07-26 16:58:06 UTC
Subject: Out of Office AutoReply:  synched players do not advance to next song

I am out of the office on business through Friday evening July 28th, so  may not be able to reply to your message as quickly as usual. If you have an urgent matter, try my mobile number, 240-899-5886. Otherwise, I will reply to your message at the earliest opportunity.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7650.5">
<TITLE>Out of Office AutoReply: [Bug 3019] synched players do not advance to next song</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->

<P><FONT SIZE=2>I am out of the office on business through Friday evening July 28th, so&nbsp; may not be able to reply to your message as quickly as usual. If you have an urgent matter, try my mobile number, 240-899-5886. Otherwise, I will reply to your message at the earliest opportunity.</FONT></P>

</BODY>
</HTML>
Comment 50 Andy Grundman 2006-07-26 19:05:03 UTC
Sorry for getting sidetracked on this bug. :)  I'm not able to reproduce this bug at all with 6.5.  I've just tested with 1 SB2 synced with the latest Softsqueeze 3.0b2 running on OSX.  Both players played several songs in sequence just fine.

Are you able to test with the latest 6.5 nightly?
Comment 51 Andy Grundman 2006-07-28 16:08:46 UTC
1869 has been fixed, and was almost certainly the same cause as this bug.  Please test!
Comment 52 Mike Gilpin 2006-07-28 16:29:52 UTC
Will check it out this weekend. In 6.5?
Comment 53 Andy Grundman 2006-07-28 18:22:31 UTC
Yep, 6.5.  Thanks.
Comment 54 Mike Gilpin 2006-07-29 13:49:54 UTC
I tried to upgrade to 6.5b1 (the 6-29 build) but failed miserably. Here's what happened:

1) Installed the upgrade.
2) The library rescan (to build a new MySQL database) started automatically and seemed to be running OK, so I let it run.
3) Because it now runs in a separate thread, I was able to track its progress by watching the Album count go up.
4) I also went around the house updating the firmware on my players, which proceeded smoothly until I got to the fourth player. For some reason it would repeatedly fail part way through and reboot. So I decided that it might be competing for CPU with the rescan, and decided to leave it alone until the rescan finished.
5) About five hours later it finally did finish rescanning (I was surprised it took so long - the mid-May builds of 6.5b1 could rescan my whole library in less than 2 hours). The counts looked correct.
6) I then tried to update firmware on the one remaining player, and had to retry about a dozen times - then finally it got all the way through. I have never experienced this problem before, so I'm pretty sure the player is OK.
7) I had noticed while I was updating the players that after they booted up under the new firmware, the screen would go dark a short while later. I tried unplugging all of them then plugging them back in, but still had that problem.
8) I took the players that were in a synch group out so I could work with one individual player.
9) I then played one of my normal playlists through that player. It shuffled the list OK and started playing, and the display showed what was playing with normal progress indicators.
10) Then the display of that player went dark, but the music kept playing. A short while later the same song started over again at the beginning, without having gotten all the way through.
11) I tried doing a Skip (from the browser interface) to see if it could make it all the way through a song. It couldn't. No matter which track I was playing, it would get part way through, then start over at the beginning of that song.
12) During this behavior, the display would periodically go dark, then come back a little while later, showing the song that was playing. It did not appear to reboot the player, it was just the display going away and coming back.
13) I also noticed that when the display came back, the position of the progress indicator (the bar showing remaining time) was wrong - the bar was full like it should have been at the end of the song, but it wasn't at the end - because it had started over. I guess the progress indicator got confused about where it was due to the restarting.
14) I couldn't get the other players to stay on, either, their displays would go dark after a short while.
15) I also noticed that the browser interface was not working properly. It sort of did, in that I could do a Skip. But when I clicked Stop, the music didn't stop until a bit later on (30-45 seconds later).

So now I'm going back to 6.3.1 and rescanning... 

I'm willing to try to take this one fix and patch it in to 6.3.1, except I don't know how to do that. Is there someplace I can go to look at how to do that? I have good skills but am not a Perl developer.

In case you need the details (don't have them from last time):
-- Running 4 SB2s.
-- Over wired Gigabit Ethernet (Cat 5e)
-- Mac Mini with 1GB RAM, an early Motorola-based model
-- Latest OSX patched up to current level
-- Running the browser interface from Firefox on a PC

6.3 had been working pretty well for me, as long as I only used 3 players. But it did freeze up every now and then.

Thanks,
Mike
Comment 55 Mick 2006-07-29 17:31:32 UTC
Mike,

Andy has added a comment to bug 1869 showing the file that needs to be patched and you can see the changes in Subversion.
I've midified the file and can send to you if you're not able to make the changes yourself.

I'm not sure how Slimserver runs on the Mac. Does it run from within perl.

https://bugs-archive.lyrion.org/show_bug.cgi?id=1869#c27
Comment 56 Chris Owens 2006-07-31 10:49:04 UTC
*** Bug 3844 has been marked as a duplicate of this bug. ***
Comment 57 Andy Grundman 2006-08-01 10:02:00 UTC
Marking this fixed.
Comment 58 Mike Gilpin 2006-08-01 10:02:07 UTC
Subject: Out of Office AutoReply:  synched players do not advance to next song

I am out of the office on business through Wednesday evening August 2nd, so  may not be able to reply to your message as quickly as usual. If you have an urgent matter, try my mobile number, 240-899-5886. Otherwise, I will reply to your message at the earliest opportunity.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7650.5">
<TITLE>Out of Office AutoReply: [Bug 3019] synched players do not advance to next song</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->

<P><FONT SIZE=2>I am out of the office on business through Wednesday evening August 2nd, so&nbsp; may not be able to reply to your message as quickly as usual. If you have an urgent matter, try my mobile number, 240-899-5886. Otherwise, I will reply to your message at the earliest opportunity.</FONT></P>

</BODY>
</HTML>