Created attachment 2041 [details]
Example FLAC file

This is a short FLAC file I have prepared that demonstrates extreme clipping. The Replaygain value is +21.9dB, but the peak level is close to 0dB. The clipping is about half a second long and starts at 4 seconds in t the file.

Warning: make sure you play this file at low levels!

Clive, I am not sure if this is practical to implement, but will look into it.  Do you have a real world example of such extreme clipping, not just a modified file?  One could also do something similar with the "--apply-replaygain-which-is-not-lossless" function in FLAC even though I know it is not the same thing.

Steven, thanks for your comments. Firstly, I see that this report has been classified as an enhancement. With all due respect, I think it really should be categorised as a bug (perhaps with a severity of "minor"). The bottom line is that when applying Replaygain in the Squeezebox, clipping can be introduced even though the necessary metadata is available to prevent it.

Of course I have no idea how the firmware is coded, and I can quite appreciate that it may be extremely difficult to do anything about it. All I can say is that if the "replaygain_track_peak" and "replaygain_album_peak" tags are sent to the Squeezebox as well as the "replaygain_track_gain" and "replaygain_album_gain" tags, then I'd have hoped a fairly simple calculation can be used to modify the gain applied.

Or is it that the RG tags are extracted by SlimServer's scanner and stored in the database? If that is the case, and there is nowhere in the database to store the peak data, then I can see how it might be tricky to fix this. One suggestion I could make is to modify the scanner so that it reads the peak as well as gain tags, and amend the volume adjustment data stored in the database accordingly. That way there'd be no need to change the firmware at all.

Regarding a real-world example. I found one in my library fairly quickly. The first movement of Beethoven's Appassionata Sonta has a peak level of -0.9dB but a Replaygain value of +2.05dB (album) and +1.5dB (track). So this would clip when played back on a Squeezebox. I've extracted the short section of this track that has the -0.9dB peak and will upload it as another attachment. (The original extreme example I prepared was done to make what is going on very obvious).

Finally, concerning your suggestion of using "--apply-replaygain-which-is-not-lossless". This would require server-side decoding, which would stop FFWD and REW working, and increase network bandwidth requirements. So I'd prefer not to do this.

Created attachment 2045 [details]
Real-world example

This is a short extract from the first movement of Beethoven's Appassionata Sonata, which has a peak level of -0.9dB and a RG value of +1.5dB (track) and +2.05dB (album).

I've done a bit more research, including examining what's in the SlimServer database. I can see that both the Replaygain gain and peak values are stored in the ALBUMS and TRACKS tables.

A simple way to avoid the clipping is to modify these entries so that the Replaygain value is limited to be no greater than the peak value, thus:

  update albums
    set replay_gain = -20*log10(replay_peak)
    where -20*log10(replay_peak) < replay_gain;

  update tracks
    set replay_gain = -20*log10(replay_peak)
    where -20*log10(replay_peak) < replay_gain;

Can I propose that this might be added as the final stage of a music library scan (and also after a new album is added to the database via browse music folder)?

We'll look at this post-7.0.

Is attachment 3 [details] to bug 4391 a real-world example of a track that would exhibit the behaviour described here?  It's got +20.32/+0.105469 for track/peak and and + 17.36/+0.202728 for album/peak.

Re. Nigel's comment: Yes, those two replaygain settings are a good example of this problem.

The track peak of 0.105469 means that the peak value for the track is -19.54dB, but if 20.32dB of replaygain boost is applied, then you're clipping by a factor of 0.78dB.

The album gain figures are even more alarming. A peak value of 0.202728 equates to -13.86dB. So applying a replaygain boost of 17.36dB will clip by 3.5dB, which is a pretty large amount - rather worse than the Beethoven example I attached.

(Incidentally, that's an extremely quiet album. What is it?)

> (Incidentally, that's an extremely quiet album. What is it?)

It's Arvo Pärt's Alina on ECM.
(http://www.ecmrecords.com/Catalogue/New_Series/1500/1591.php?cat=%2FArtists%2FP%E4rt+Arvo%23%23Arvo+P%E4rt&we_start=0&lvredir=712)

Caleb, Dean notes we'd like your input on this bug.  What is the 'right' thing to do with the 'peak' tag?

There's no simple solution here, really what's needed is a form of soft clipping or compression to allow the gain to work but the clipping to be reduced. 

Feedback from caleb on the feasibility of this most welcome.

 We'll get there.  It's just software :-)  There are several means of doing dynamic range compression, and it'll be some time before I can work out how to do it right.  May be for MP, maybe post MP.

With all due respect, applying dynamic range compression is NOT the solution. (Apart from anything else, does the CPU in the Squeezebox even have the necessary horsepower to make a good job of compression?)

One could reasonably argue that using compression to achieve a desired loudness is a valid option to offer, but this isn't what ReplayGain is about, which is a means of applying a LINEAR amplitude adjustment. By all means offer compression as a separate option, but please don't apply it by default.

In the meantime, my comment dated 19th June 2007 describes precisely how to adjust the tags post-scan so that clipping is avoided. Perhaps it could be added to the end of SCHEMA_OPTIMIZE.SQL?

Digital clipping is totally unacceptable under ANY circumstances. We MUST limit positive replygain such that the peaks do not exceed 0dB.

Also if possible, this should take into account the user's volume control setting, allowing for higher gains when the volume has been attenuated (which would create more headroom). I'm not sure if that is possible given crossfading considerations and such.

Reclassifying this as a bug. But is this a firmware or server issue?

if more datafrom other values would help, maybe the new schema should be aware of this, so at least the data is available for a future fix to use.

http://forums.slimdevices.com/showthread.php?t=54004

(In reply to comment #15)
> if more datafrom other values would help, maybe the new schema should be aware
> of this, so at least the data is available for a future fix to use.

The necessary data is already in the SqueezeCenter schema - the ALBUMS and TRACKS tables each have a column named REPLAY_PEAK as well as REPLAY_GAIN.

It seems to me that there are some trouble understanding how replaygain works.

There are four tags in question (all of them are currently stored in the database according to Clive)


If track replaygain is used:
REPLAYGAIN_TRACK_PEAK
REPLAYGAIN_TRACK_GAIN

If the album dynamics should be preserved the following tags are used:
REPLAYGAIN_ALBUM_PEAK
REPLAYGAIN_ALBUM_GAIN

Today only the GAIN value seems to be used. The peak values are there to avoid clipping. The peak value is represented by a value from 0-1 in lossless files and 0-xx (My "worst" mp3 have a peak value of 2,64 if I remember correctly) in lossy files, where anything above 1 basically means clipping with zero replaygain. 
This means that if the PEAK value is originally 1 and you apply positive replay gain then there will be clipping. This will also happen if you have lower peak values than 1 but higher positive replay gain values.


The PEAK value is not directly comparable to the value in the gain field as the gain field is stored in decibels. 
To find the representative dB value from the PEAK tags you need to do the following:

-20*LOG(PEAK_VALUE)

This value gives the maximum db "headroom" left according to the peak value. 
If this value is lower than the real replay gain value in the REPLAYGAIN_TRACK_GAIN and/or REPLAYGAIN_ALBUM_GAIN  then this value must be used and not the real replay gain value in the tags. If this is not the case then the original replay gain in the tag could be used.

I have attached an .xls document with a simple formula to try different scenarios and see the resulting and non-clipping replay gain value according to the logic I used above.

Created attachment 4403 [details]
Excel sheet implementing simple non-clipping replay gain logic

Applying Hans-Kristian Bakke's formula to adjust the requested gain going to the player to avoid clipping seems like a good and somewhat simple to implement compromise to this issue.  I know this is not an ideal solution but is much better then what we do now which is nothing.  This will make some of the quieter tracks play at a lower volume overall to avoid clipping when replaygain is enabled but seems like an acceptable trade off to me.

It would also be nice to be able to show the new computed replaygain value while playing.  For example "3.29 (adjusted to 2.89)" or similar.

I absolutely second the notion to handle this as a bug. Mostly in classical music (an audiophile = paying market) and in world music the situation arises that a track will clip when »blindly« applying (positive) ReplayGain. That’s what the Peak values are for. And that’s what can quickly ruin a product’s credibility (as opposed to supporting RG and ShoundCheck which is a BIG Sales Plus).

I would recommend doing the following:

1. Be sure to only apply ONE type of Replay Gain to any one file (in case both SoundCheck—which is Apple’s Replay Gain—AND ReplayGain are present). ReplayGain should be preferred. (This currently seems another bug, btw.)

2. I’d be very much for a SETTABLE clipping prevention (enable/disable). Let the user decide. It’s really not hard to calculate. The »switching it off capability *could* be an option, I’m a little unsure why anyone would *want* clipping. Nevertheless, an (advanced options) switch is cheap, provided that the default is »prevent clipping«.

3. Additionally, a »default« Gain should be settable in SC that defines how much a track should be gained if NO ReplayGain or SoundCheck tags are found. (Typically set for -6 dB as default, like in Winamp.) This reduces unwanted sudden loadness »jumps« in case we don’t have gaining data for a track and overall tends to make users »happier«. Since the user could SET it, (s)he can easily adjust this to personal preferences.

4. Refrain from doing any »normalising«, gating and/or Automatic Gain Control. This *could* be okay for the mass or broadcasting market which this product is not targeted at. It will definitely ruin some good credibility towards the product that IS out there.

Keep it LINEAR and accept the pros and cons of »garbage in—garbage out«: If he puts garbage in, (unchanged) garbage comes out. Basta. If she puts high quality music in, LINEAR and UNCHANGED high quality will come out—even on the most high-end systems. Hello, audiophiles, you CAN buy our product! And the rest of us can just enjoy great music, no »pumping« el-cheapo AGC ruining it.

Best regards,
Matthias

You must of course keep the option to disable all gain manipulation, some of us wants to pass bit perfect information to the digital output.

I absolutely agree, Mikael. RG/SC as a whole MUST of course have a "disable". Got so carried away with marketing hype that I really forgot :-)

Another one I almost forgot:

The user needs to be able to select between »Album« (audiophile) and »Track« (radio) Replay Gain!

Reason:
When listening to a complete album, one wants to listen to the album as it was done by the artist. Some songs might intentionally have a different loudness. This is what album replay gain is for.

When listening to mixed music, one usually wants to have every title »sound equally loud« (as is done in radio broadcasts). This is what track replay gain is for.

Also, this CANNOT be decided by the software. Please don’t try to »find out« if one listens to a complete album and then use »the other RG«! It should ENTIRELY be the decision of the listener if (s)he wants to listen

– without RG applied
– with RG-Track applied (with or without clipping prevention)
– with RG-Album applied (with or without clipping prevention)

(RG Track and RG Album are NOT cumulative but either-or!)

I’d propose to set »clipping prevention on/off« and »default gain« in SC (maybe advanced settings) and have at least the possibility to set »No RG«, »Track RG« or »Album RG« on the remote/from the menu.

A user would usually set clipping prevention and default gain ONCE while (s)he might listen to mixed music (using RG Track) during the day and enjoy a complete album from his leisure chair in the evening (using RG Album).

I don't think this needs a preference.  If both gain and peak values are tagged in a file, the peak formula will be used to reduce the amount of gain the player is instructed to add, if necessary.  That's all this will be, it won't affect any of the existing RG settings.

re: comment 23

this already works the way you described it - if you like it or not ;-) (i like it) 

have a look at the different settings at settings->player->audio->Volume Adjustment/Replay Gain.

The Replay Gain Logic and Fallback Structures.

A. User has selected »Replay Gain: None«

Don’t apply any RG (Replay Gain) or SC (SoundCheck).


B. User has selected »Replay Gain: Track«

a) Check if RG track value is there. If so, use, goto (d).
b) Check if SC track value is there. If so, use, goto (d).
c) Use default gain stored in settings.
d) Check if »prevent clipping« is set in SC as overall option. If not, goto (h).
e) Check if RG track peak value is there. If so, use, goto (h).
f) Check if SC track peak value is there. If so, use, goto (h).
g) Assume 1.0 as track peak (full gain).
h) Calculate gain and output stream.


C. User has selected »Replay Gain: Album«

a) Check if RG album value is there. If so, use, goto (f).
b) Check if SC album value is there. If so, use, goto (f).
c) Check if RG track value is there. If so, use, goto (f). *
d) Check if SC track value is there. If so, use, goto (f). *
e) Use default gain stored in settings.
f) Check if »prevent clipping« is set in SC as overall option. If not, goto (l).
g) Check if RG album peak value is there. If so, use, goto (l).
h) Check if SC album peak value is there. If so, use, goto (l).
i) Check if RG track peak value is there. If so, use, goto (l). *
j) Check if SC track peak value is there. If so, use, goto (l). *
k) Assume 1.0 as album peak (full gain).
l) Calculate gain and output stream.

* Using the track values if the album values aren’t available STILL gives a much better listening result than using the default gain or none. This is why I propose to fall back to track values if album values aren’t stored. (Often happens when you only have ONE track from an album. Tools like metamp3 will only store track values in this case, so album values will be undefined.)

All the above checking logic can be greatly simplified by putting some of the logic into the file scanner modules that populate the database. Each database row should store RG-Track, Track-Peak, RG-Album, Album-Peak. (The latter two might be in the album data row, don’t know about the DB structure right now.)

These could be filled in (and the logic above simplified) by having the file scanner do:

a) Check if RG track tag is there. If so, use, goto (d).
b) Check if SC track tag is there. If so, use, goto (d).
c) Set RG-Album to »unknown«. ***
d) Populate RG-Track in DB row.

e) Check if RG track peak tag is there. If so, use, goto (h).
f) Check if SC track peak tag is there. If so, use, goto (h).
g) Assume 1.0 as track peak (full gain).
h) Populate Track-Peak in DB row.

i) Check if RG album tag is there. If so, use, goto (n).
j) Check if SC album tag is there. If so, use, goto (n).
k) Check if RG track tag is there. If so, use, goto (n). **
l) Check if SC track tag is there. If so, use, goto (n). **
m) Set RG-Album to »unknown«. ***
n) Populate RG-Album in DB row.

o) Check if RG album peak tag is there. If so, use, goto (t).
p) Check if SC album peak tag is there. If so, use, goto (t).
q) Check if RG track peak tag is there. If so, use, goto (t). **
r) Check if SC track peak tag is there. If so, use, goto (t). **
s) Assume 1.0 as album peak (full gain).
t) Populate Album-Peak in DB row.

** Again, setting album to track values in case they are missing is better than using a default or none.

*** »unknown« needs to be something safe! It cannot do to use 0, -1 or the like since all these would constitute legal gain corrections. BUT we must be able to see if the track HAS a valid RG value, in order to apply the default gain later on if it has not.


This said, the above RG playing logic could be abbreviated to:

A. User has selected »Replay Gain: None«

Don’t apply any RG values from the DB.


B. User has selected »Replay Gain: Track«

a) Check if RG-Track in DB is set. If so, use, goto (d).
c) Use default gain stored in settings.
d) Check if »prevent clipping« is set in SC as overall option. If not, goto (h).
e) Use Track-Peak from DB. (Must be set, we did it before.)
h) Calculate gain and output stream.


C. User has selected »Replay Gain: Album«

a) Check if RG-Album in DB is set. If so, use, goto (f).
e) Use default gain stored in settings.
f) Check if »prevent clipping« is set in SC as overall option. If not, goto (l).
g) Use Album-Peak from DB. (Must be set, we did it before.)
l) Calculate gain and output stream.


Does anybody see a fault with this logic? Please comment.

Why does 'prevent clipping' need to be an option?  I think it should always be enabled if the peak value is present in a file.  If you don't want clipping prevention you could strip the peak tags from your files.

Also, what do you mean by "SC track tag" and "SC album tag"?  If a file doesn't have RG tags, no gain will be applied.  There is no default gain preference.

Basically, everything will remain as it is today with the addition of the peak calculation when a peak value is available.

Sorry I see you meant SoundCheck.  I believe the SoundCheck logic is already correct, is this not the case?  There was an old bug where the values were added together but that was fixed a while ago.

(In reply to comment #26)
With all due respect, Moonbase, I think perhaps you may be unaware of what facilities already exist. The Squeezebox system already has the options you talk about: no RG, track RG, album RG. In addition, it also has "smart RG", whereby track gain is used except when consecutive tracks from an album are played, in which case album gain is used. The user is free to choose whichever he wishes on a per-player basis.

Moreover, I don't think here is the place to discuss these issues. This bug report is specifically about clipping caused when there is a positive ReplayGain value. That's what has to be fixed.

Being new here, it’s still a bit difficult for me to see what Logitech wants WHERE. Some things apparently belong here for the developers, some into the forums.

Just thought it’d be a more sensible place here to have it clear once and for all, especially since the forum threads stated that it might be »unclear to the developers«. The logic describes what has to be done, regardless of the RG value being positive or not.

Sorry if I was being too elaborate.

Hadn’t yet the chance to look over the code, but if you say all is already working, great. I’ll check.

i just wanted to add a datapoint to this bug.

winamp has the following modes in its RG options:

apply gain
apply gain / prevent clipping
normalize
prevent clipping

you can also pick a preferred source, track or album, and disable if that source isn't available.  (i think SC's "smart gain" is better)

it ALSO has a dB adjustment for files WITHOUT RG.

in any case, it looks to me like the solution for this bug exists and was described earlier, so i look forward to the new version!  :)

I think that we have what's necessary already as several others are saying.
We can manually select to disable, use only track, only album or a so called smart logic. 
To implement logic to do something with replay gain when there are no replay gain tags is not the correct thing to do in my opinion. The tags are there for a reason for the persons that really cares.

I think everything should stay the same as today as we already have all the choices we need, with the exception of adding non-clipping logic for the songs that have replay gain tags when replaygain is activated in some form. 

I can't see that someone really WANTS clipping, but to please everyone maybe the choices should stay the same as today and with a separate selectable (default on of course) global parameter for "Avoid clipping when replay gain is active" or something like that, even though this should be rather unnecessary in my eyes.

Fixed in change 24314.  Please test!

Looks good, verified fixed in Version: 7.3.1 - 24372

This bug has been fixed in the 7.3.1 release version of SqueezeCenter!

Please download the new version from http://www.slimdevices.com/su_downloads.html if you haven't already.  

If you are still experiencing this problem, feel free to reopen the bug with your new comments and we'll have another look.

Reduce number of active targets for SC