I think this might be something that would be great as part of the bandwidth
limiting code.  Instead of just simply transcoding via LAME, if the server knew
that an MP3 existed of the same exact file, then it could send that and use a
lot less CPU resources.  This could work on other formats if any end up being
handled natively as well.  

We'd have to restrict this and only support for matching filename, and all tags.
(or some other specific definition of 'matching') It would probably be very bad
to be picking a wrong file just because of matching filename (sans extension).
This would confuse generic users who wouldn't necessarily have careful tagging
or filenaming. 

Ian - is there any reason you don't use the transcoding feature of the server to
convert your FLAC files to mp3 using lame?

Just remove the MP3 files at that point.

>Ian - is there any reason you don't use
>the transcoding feature of the server to
>convert your FLAC files to mp3 using lame?

>Just remove the MP3 files at that point.

I need the MP3s for iPods etc -- for things that don't seamlessly transcode as
required.

Plus, I really want the server to use the file format that the SLIMP3 supports
natively, when the track is available in that format.  Fast forward and rewind
work, uses less juice on the (underpowered, in terms of CPU) server, etc.  I
think it's a good thing to offload the job of 'playing music' to the device
designed for 'playing music' :).

I suppose I could try to maintain parallel hierarchies, but I think the issue
here is really more general than that -- when the same track (need to work out
how closely the tags have to match for two files to be regarded as representing
the same track) is available in multiple formats, the server should have
predictable behaviour.  At the moment, it's not predictable.

I think the problem is also that which format gets presented to the user seems
to be inconsistent, picking some mp3 and some flac to be available to the user.
 regardless of how we handled playback of multi-format, the server should
probably display them in a consistent way.  I'd say it should list them all, in
both formats.  chosing one for playback might be able to be re-filed as a
specific enhancement if that is not a trivial thing.

>I'd say it should list them all, in both formats.

I think this might have negative effects when playing whole albums (to give one
example).  Whilst I may have two copies (one .mp3, one .flac) of each track in a
given 12-track album, I definitely don't want 24 tracks to be enqueued when I
tell the system to play the album.

This is why I agree with comment 1 -- the server (ideally) would have a concept
of which files represent the same track (in the same album, etc).  That is to
say, which files differ only in the encoding formats.

In the short term (and in my specific instance) having it examine .mp3 files
after .flac files would do the trick, so the data from the .flac was overridden
by that from the .mp3, but this hardly seems like the cleanest possible option.

I've agreed to take this one over from Dean.  I also have a collection of CDs
that are ripped to both MP3 and FLAC, so my own collection is an idea testbed.

I've been programming perl for 10 years, but I have no experience with this
codebase.  This didn't seem to bother Dean, so I guess I'm good enough for the job.

Having received no feedback from my email about what policy to implement, I
think I'm going to just make a fix, and wait for somebody to hate it.

The policy of the fix will be "mp3 trumps everything, otherwise go by
alphabetical order".  So MP3 > AAC > FLAC > OGG > ZIPPY_NEW_FORMAT

This is really lazy on my part, and I'm hoping that this laziness with enrage
someone to the point of coming up with a real policy.

What about giving the lossless formats priority, followed by mp3, followed by
everything else?

I figure that if someone has lossless copies, that's probably what they want to
hear through the squeezebox and additional copies are probably there for use
with less advanced devices.  Putting mp3 second may avoid unnecessary
transcoding in some circumstances. After that I'd guess everything else falls
into the "pick any one" category.

not having seen the email, thus not having the opportunity to reply...
I think you might want to consider following the bitrate limiting settings, at
least to a certain extent.  If a user has their player set to 'no limit', it
woudl be clear that they want high quality audio, thus lossless codecs shoud
come first.  WAV, FLAC, AppleLossless, WMA Lossless (don't care what order).  If
limited then mp3 first...the rest later (maybe based on cpu overhead to
transcode, perhaps).

Odds are this patch is only going to come into play for those who have a 'good
copy' and 'streamable copy' rather than users who just happen to like to back up
their FLAC files with WAV files :)

Re: comment 7 -- the original report (mine) says quite clearly that the problem
in my case is not that sometimes it doesn't pick the lossless ones, but that
sometimes it _does_ pick the lossless ones.

This is, of course, because I have SLIMP3s, not Squeezeboxes.  The .FLACs are
for possible future Squeezeboxen.

kdf's suggestion above about bitrate limiting could solve the issue of having a
different behaviour for slimp3 (i.e. prefer the mp3 version to avoid
transcoding) as well.

There are a couple of things going on here.  Tell me if I've got it right:

1. Sometimes we want to ignore some file types globally (i.e. they get ignored in the scan).
This should be able to be accomplished by modifying the types.conf file.  

2. The database should recognize that there are two files that correspond to one song and then use 
some cleverness to decide which format to stream to a given player.  (i.e. Send MP3 to SLIMP3, FLAC to 
Squeezebox).  Choosing which format for which player may be configurable (or maybe we already have 
all the configuration we need via the transcoding framework.

What worries me here, is what happens when the server fins two files of
identical filenames, only differing in teh extension.  If we send the MP3 to
Slimp3 and the FLAC to squeezbox and these file are NOT truly identical, the
user is in for an annoying surprise.  This is also an issue for synchronisation.
 The more I think about this, the more I shy away from any server-side
"automatic" solution, and the possible combinations for user options gets a
little crazy as well. I personally blame several recent headaches on conflicting
option requests.

not to imply that file names are the only/best way to match two songs, but for
the sake of argument if that's the mechinism that's chosen, then it's not too
unreasonable to expect the user who turns this feature on to have well named
files. Just as current users are expected to have the correct file extension on
a song, or have annoying surprises.

I'm the first to say...do it the way the server likes, and stop whining. 
However, I've been blasted to no end for that, so I'm just sharing the
experience in the form of devils advocate :)

The main thing I hope to see from this is really that the files are consistently
handled.  Right now, you can load an album and get a mix of FLAC and MP3. 
Eventualy, I plan to get rid of dupes, so it really wont matter to me after that.

Re: Comment 11

Issue (1): I don't actually want to ignore some filetypes globally :).
           As stated in the original report, I only want to ignore
           those filetypes (.FLAC, in my case) if there is a
           "matching" .MP3 (for some definition of "matching" still
           to be determined).  This is because:
             (a) I have SLIMP3s, not Squeezeboxes;
             (b) The music server is underpowered, in terms of CPU; and
             (c) The music collection breaks down as follows:
                  + For lots of tracks:
                      Collection contains .FLAC and .MP3;
                  + For some tracks:
                      Collection contains .MP3 only;
                  + For a few tracks:
                      Collection contains .FLAC only.

Issue (2): This is the "matching" question, and I freely admit I don't know
           how to do this.  I would be happy with 'contains exactly the
           same tags' being the definition of "matching", but even there
           there are problems (what if the .MP3 contains fields for which
           no corresponding .FLAC field exists?).

Re: Comment 14, consistent behaviour.

Yes -- I agree that the most important thing is to get the server to at
least behave consistently (per my original report, I get an confusing mix
of .MP3s and .FLACs when I play an album).  The second most important thing
is to have the server choose the most appropriate version of a track to
play.  Comment 12 had a great point about synchronisation in mixed
SLIMP3/Squeezebox environments, which adds yet another layer of complexity.


This is a much larger can of worms than I had initially suspected!

I think that the real bug here is:   SlimServer should notice when there are multiple encodings for a 
single file and only show one instance when browsing and creating playlists.  When playing back a song 
that has multiple encodings, the preferred formats as expressed by the various players and the 
transcoding framework should take advantage of all the formats available.

How's that sound?

Re: Comment 16

I think that's a good description.

The arrival of the new Squeezebox2 makes this issue even more interesting --
what's the correct behaviour in a mixed network of SLIMP3s, Squeezebozen, and
Squeezebox2en when some tracks are available in MP3 and FLAC (for example), some
only in FLAC, and some only in MP3?

There's self-interest in this question, of course -- not only do I have such a
collection (per earlier comments), but I'm also about to have a mixed network of
SLIMP3s and Squeezebox2en, and would really like the Squeezebox2en to get the
FLACs natively, but still have the SLIMP3s get the matching MP3s natively (I
imagine that synchronization is off the table for the purposes of this discussion).

Fun fun.

I agree.  This issue quickly went over my head, and is only getting more
complicated.  I no longer feel qualified to field it.  It seems simple when
considering only one's own home configuration.  Coming up with a policy engine
that fits all possible configurations quickly becomes a large undertaking.

And I still don't feel qualified to decide on the policy.

Perhaps the original suggestion to wait until the 6.0 release was best.

well Dean, I guess its back to you for now :)

dan to find the dup of this and deal.

*** Bug 707 has been marked as a duplicate of this bug. ***

Another related behaviour of the current system -- when I browses the music
collection directly (via 'Browse Music Folder') and add a directory to the
playlist from there, two copies of each track is put in the playlist (one MP3,
one FLAC -- see earlier comments on my collection).

Technically, this is what 'add' is supposed to do in this context (put
everything in the directory on the playlist), but it's not what the user (this
user, anyway) expects to happen.  What I think should ideally happen is that one
copy of each track (using whatever system is chosen to detect multiple copies of
the same track, and to select which of those multiple copies to use in a given
situation) is added to the playlist.

Pushing off to 6.1.

Patches welcome.

*** Bug 1119 has been marked as a duplicate of this bug. ***

Another aspect of this occurred to me today when trying to play "The Wall" -- an
album of which I have three copies in my collection:
  (1) MP3, one file per track.
  (2) FLAC, one file per track.
  (3) MP3, one file per disk, with cuesheet.
Several days ago I diddled types.conf so that (2) is not seen any more, but (as
one might expect) two copies of each track still show up in the album listing --
one from (1), and one from (3).

This is the same bug, but it's another aspect of it.

suggestion: quick, highly limited workaround that might take care of Ian's issue
in Comment #26

have SS recognize a marker of some sort that would skip some tracks on scan. 
perhaps a text file named something like 'ignore.txt'.  creating this file in
any subdirectory of the music directory would cause SS's scan to skip that
directory and any subs.  

presumably simple to implement and would take care of a small but annoying class
of library management issues.

possibly extensible to the greater issue if, for instance, scripts that
automatically generate mirror FLAC/MP3 encodings of the same material were to
mark one of these to be ignored.  not ideal, but perhaps useful in the interim.

I agree with comment 16.  I would like to see the following:

If there are multiple duplicates of a track in a directory (duplicates either from exact tag matches, exact 
filename matches (except extension) - then show on browse and/or search just the files that match a 
given player's prefered type.

If there is both FLAC and MP3, stream the best fit to the given player
If just MP3 or FLAC, then stream (and/or transcode) to deliver the track to the player.

I would be nice to have filtered view of my music library.
One of the filter could be display highest quality format for any single tracks.
I'm sure many audiophiles would rip songs in a lossless format such as FLAC and one in mp3 for portable devices.
So, as is now, it's a major pain to have to scroll through two copies of everything in the lib.
Or the only other way, is the manually seperate them and never mix them in the lib.

I just received my Squeezebox3, moving from SoundBridge, and have exactly the same problem.  All the songs in my music server are in Apple Lossless (for anything) and MP3 (for iPod).  With the SoundBridge I had two mt-daapd's running, one for m4a's, and one for mp3's.

Simple option to ignore certain file types would be sufficient enough, anything more complex would be even better.

Re: Comment 30

The workaround is to comment out the filetypes that you don't want to support from types.conf and convert.conf.  In my case, I commented out all the FLAC-related things to get MP3s everywhere.  In your case, perhaps you'll want to comment out all the MP3-related things.

I have been running with these local changes since the time of the bug report.

Music Magic has an interesting way of doing something similar with 'exclude' lists.

Basically you can tell it not to load certain wildcards - even if it seems them in the directory or from reading iTunes etc.

Micght be worth considering (it would allow me to link SlimServer with iTunes - which I can't today without adding all the AAC files to the DB - however I can have MM integrate with iTunes without trouble).

Note that just extensions (or using the .conf file) doesn't work for ALE vs AAC since both are .m4a (!).

I keep my 'unique' AAC/mp3 files in a different directory from my 'available in both formats' AAC/mp3 - so in my care this would do most of what people are asking for.

A partial "fix" for this bug has been added in change 8171 - an option to exclude file extensions during scan time.

While digging through the BMF code I found reference to a preference called "ignoreDirRE" which allows to skip given item names (Slim::Utils::Misc::readDirectory()). It's been there since revision 2, but isn't used anywhere else. There's no GUI for it.

Was this an early attempt to do what we're discussing here? 

Nice, guys.  If you are going to just close it saying you will never fix it, it would be nice for you least to comment on why, given that a number of your customers indicated enough interest to document this.  I may be wrong, but I don't think the original crowd would have done this with nary a comment.  Please tell me this isn't the way that the new company is going to behave.

Chris

7.9 comes with an option to prefer lossless over lossy copies of the same track. It's part of the "Extended Browse Modes" plugin.