Bug 15672 - Logging failing at logrotate... (7.5.0-embedded, maybe other trees)
: Logging failing at logrotate... (7.5.0-embedded, maybe other trees)
Status: RESOLVED FIXED
Product: Logitech Media Server
Classification: Unclassified
Component: Debian
: 7.4.2
: PC Debian Linux
: P2 major (vote)
: 8.0.0
Assigned To: Michael Herger
http://forums.slimdevices.com/showthr...
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2010-02-09 20:44 UTC by snarlydwarf
Modified: 2010-03-01 06:55 UTC (History)
1 user (show)

See Also:
Category: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description snarlydwarf 2010-02-09 20:44:36 UTC
from lsof -p:

squeezebo 10152 squeezeboxserver    6w   REG      3,6   392259   271035 /var/log/squeezeboxserver/server.log.1 (deleted)

Looks like the rotation isn't correctly being recognized by SBS, the file is left open after it has been gzip'd away.

logrotate.d/squeezeboxserver has this:
      # send USR1 to squeezebox PID to reset logging
    /bin/kill -USR1 `pgrep squeezebox-serv 2>/dev/null` 2>/dev/null || :

This will need to be changed.. from 'man pgrep':

NOTES
       The process name used for matching is limited to the 15 characters present in the output of /proc/pid/stat.

The code for logrotate.d is in error two ways: there is no process named 'squeezebox-serv' so it will fail.  And the real process name is 'squeezeboxserver' which is 1 character too long to properly identify it.

This should probably use the contents of /var/run/squeezeboxserver.pid instead of the pgrep.

At any rate, logging dies at first logrotate until SBS is restarted.
Comment 1 snarlydwarf 2010-02-09 20:57:29 UTC
ugh, actually, i see no code to handle USR1 as a signal for log rotation... nor anything.  HUP does a re-init, but I don't see that as changing the open log file, my server is still logging to the deleted server.log.1

So, um, yeah, well, logging is broken on embedded as soon as logrotate tries to rotate.
Comment 2 snarlydwarf 2010-02-11 08:43:13 UTC
It looks like this goes back to 7.4: the name of the process is wrong there, too.

(whether the USR1 signal works at all there is another question.)
Comment 3 snarlydwarf 2010-02-23 13:27:48 UTC
Has this been fixed in 7.4.2?

Could someone with confirm abilities please confirm this?

It's trivial:

Start SBS.

Rotate your logs.

Watch SBS stop logging, but still using disk space as it logs to a deleted filehandle.

This seems pretty serious to me, in that it makes finding bugs impossible since no logs will exist after the first time logrotate runs on a *nix machine.
Comment 4 snarlydwarf 2010-02-24 21:20:33 UTC
wow this is sad... does no one actually look at bugzilla anymore?

It's been two weeks and its not even flagged as suitable for anyone to look at.
Comment 5 KDF 2010-02-24 21:58:38 UTC
the code is there to handle USR1.  It just doesn't respond.  Care so revert and find out where it stopped working?
Comment 6 SVN Bot 2010-02-24 22:23:09 UTC
 == Auto-comment from SVN commit #30249 to the  repo by kdf ==
 == https://svn.slimdevices.com/?view=revision&revision=30249 ==

bug: 15672
description: assume 'squeezeboxserve' first match is for squeezeboxserver as it's not predictable what pid we can use and the process names are too stupidly long to isolate.  btw, this is likely messsed for at least fedora and redhat too as these seem to still rely on 'slimserver' naming
Comment 7 snarlydwarf 2010-02-25 19:53:01 UTC
(In reply to comment #5)
> the code is there to handle USR1.  It just doesn't respond.  Care so revert and
> find out where it stopped working?

There we go: USR1 does work if and only if you actually move the file and then generate some log message.

(Ie, it doesn't actually close/reopen until there is a new log entry, the signal just tells it "next time you write to the log file, you will need to re-open it)

And, yes, all the other builds with logrotate scripts need to change: none will find the correct process name.

Fixing the logrotate script should fix it.
Comment 8 Michael Herger 2010-02-25 20:53:16 UTC
didn't kdf's change fix this?
Comment 9 snarlydwarf 2010-02-26 09:38:33 UTC
(In reply to comment #8)
> didn't kdf's change fix this?

For debian, yes.

For Redhat, no.

    postrotate
        /bin/kill -USR1 `pgrep slimserver.pl >/dev/null 2>&1` >/dev/null 2>&1 || true
    endscript

There is no slimserver.pl of course.
Comment 10 SVN Bot 2010-03-01 06:55:46 UTC
 == Auto-comment from SVN commit #30285 to the slim repo by mherger ==
 == https://svn.slimdevices.com/slim?view=revision&revision=30285 ==

Fixed Bug: 15672
Description: we need to grep for squeezeboxserver, not slimserver.pl. That script is renamed on some platforms.