Bug 5285 - Duplicate albums and missing tracks when importing from MusicIP
: Duplicate albums and missing tracks when importing from MusicIP
Status: CLOSED FIXED
Product: Logitech Media Server
Classification: Unclassified
Component: MusicIP
: 6.5.4
: Macintosh Other
: P2 normal (vote)
: ---
Assigned To: Michael Herger
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2007-08-18 14:08 UTC by Michael Robinson
Modified: 2008-12-18 11:12 UTC (History)
3 users (show)

See Also:
Category: ---


Attachments
MusicIP API and Slimserver library information (27.57 KB, text/rtf)
2007-08-20 14:58 UTC, Michael Robinson
Details
MusicIP import debug logs (20.37 KB, text/rtf)
2008-01-04 09:18 UTC, Michael Robinson
Details
MusicIP import file (21.67 KB, text/rtf)
2008-01-04 12:44 UTC, Michael Robinson
Details
MusicIP import file - uneditted (14.64 KB, text/plain)
2008-01-07 06:08 UTC, Michael Robinson
Details
Compressed Squeezecentre log files (434.34 KB, text/plain)
2008-01-07 06:47 UTC, Michael Robinson
Details
SqueezeCenter logs after MusicMagic/Importer.pm edits (1.19 MB, application/zip)
2008-01-07 08:41 UTC, Michael Robinson
Details
Fix file path conversion on Mac (1.04 KB, patch)
2008-01-08 04:45 UTC, Michael Herger
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Robinson 2007-08-18 14:08:15 UTC
SlimServer_6.5_v2007-08-18
iMac Intel OSX 10.4.10
MusicIP 1.8

I upgraded from SlimServer_v6.2.2 to 6.5_v2007-08-18 nightly and discovered that a large number of tracks are not imported correctly from MusicIP.

This did work correctly in 6.2.2.

To test this, I created a MusicIP database with 1 album by 1 artist containing 17 tracks.

In Home / Server Settings / Basic Settings :
- MUSIC FOLDER is blank
- PLAYLISTS FOLDER is blank
- Do Not Use iTunes is selected
- Use MusicMagic is selected

I then selected "Clear library and clear everything"

After the scan had finished, Slimserver said "Your music library contains 2 albums with 2 songs by 1 artist"

Home / Artists / contained duplicate albums each one containing 1 track.

So the library has duplicated albums and 15 tracks are missing. 

This is happening with other albums as well.

I tried the Home / Server Settings / Debugging options with d_musicmagic set however this did not show any tracks being imported during the clear and rescan.  This used to work in previous versions of Slimserver.

Please let me know what debugging information would assist.

I'm going back to 6.2.2 as this release works.
Comment 1 Michael Robinson 2007-08-18 15:26:50 UTC
ps. I estimate there are 55 duplicate albums in my correction of 877 albums
Comment 2 Michael Herger 2007-08-19 12:14:33 UTC
Do these albums have non-latin characters in there path, are stored in a linked folder or is there anything else common to these, but different to the big bunch?

Could you please post what you get when running the following request against your MusicIP server: http://yourserver:10002/api/songs - make sure you do this with that little sample collection, as it will return a list of _all_ songs
Comment 3 Michael Robinson 2007-08-20 14:57:55 UTC
The problem does appear to be related to accents in the artist name or song title.

I did the following for a number of problem albums:
1. Created a MusciIP file containing only the problem album
2. Captured the MusicIP list of songs with command http://localhost:10002/api/songs
3. Cleared and rescanned the Slimserver libary with settings to import this from MusicIP
4. Examined the Slimserver library

There appear to be a couple of different things happening.

When the artist name has an accent (Sinéad O'Connor, Sigur Rós), a number of tracks are missing from the Slimserver library

When the song name has an accent, it appears to split the songs into multiple duplicate albums.

For example, if the 6th track has an accent, the first album will have tracks 1-6 and the second album will have the remaining tracks.

If there are two tracks with accents, then the songs will be split into 3 duplicate albums.

I haven't checked all 55 duplicated albums, but it appears that this is happening to AAC and Apple Lossless albums and not to MP3.

Also, the problem with songs containing an accent appears to be happening with multi-disk albums... however I haven't checked all albums to verify this.

See attached file with MusicIP songs and resulting Slimserver library information.
Comment 4 Michael Robinson 2007-08-20 14:58:59 UTC
Created attachment 2114 [details]
MusicIP API and Slimserver library information
Comment 5 Michael Herger 2007-08-20 22:06:41 UTC
Seems to be a problem with file paths. If you rename one of theses folders not to have any non-latin character, does it work then?
Comment 6 Michael Robinson 2007-08-21 11:47:07 UTC
It looks like Slimserver can't deal with accented characters in the file path.

I removed accented characters in the artist name and song title from two problem albums and they appeared correctly in the slimserver library after a clear and rescan.

This did work in 6.2.2 and I am using the same MusicIP database file.
Comment 7 Chris Owens 2007-09-04 11:44:43 UTC
Steven, could you look at this on Slimserver 7?
Comment 8 Chris Owens 2007-12-10 09:44:05 UTC
ping steven
Comment 9 Blackketter Dean 2007-12-28 05:16:48 UTC
ping steven
Comment 10 Michael Robinson 2007-12-30 10:19:39 UTC
I tried to reproduce this using SC7 (SqueezeCenter_trunk_v2007-12-30)
 and I still get a number of duplicated albums.


Comment 11 Michael Herger 2008-01-03 01:10:48 UTC
This is IMHO an issue with MusicIP not correctly encoding file paths on non-Windows systems.
Comment 12 Michael Robinson 2008-01-03 14:32:22 UTC
(In reply to comment #11)
> This is IMHO an issue with MusicIP not correctly encoding file paths on
> non-Windows systems.
> 

I don't think so.  MusicIP import using MusicIP 1.8 worked correctly with Slimserver 6.2.2 but doesn't work with Slimserver 6.5 or Slimcenter 7 so the MusicIP version hasn't changed.

From the MusicIP API (http://localhost:10002/api/songs) there seems to be the same encoding for one of the problem albums:

/Volumes/LaCie500G/Music/Tracks/ALAC/Sigur RoÃÅs/Takk.._/01 Takk....m4a
/Volumes/LaCie500G/Music/Tracks/ALAC/Sigur RoÃÅs/Takk.._/02 GloÃÅsoÃÅli.m4a
/Volumes/LaCie500G/Music/Tracks/ALAC/Sigur RoÃÅs/Takk.._/03 HoppiÃÅpolla.m4a
/Volumes/LaCie500G/Music/Tracks/ALAC/Sigur Rós/Takk.._/04 Með Blóðnasir.m4a
/Volumes/LaCie500G/Music/Tracks/ALAC/Sigur RoÃÅs/Takk.._/05 SeÃÅ Lest.m4a
/Volumes/LaCie500G/Music/Tracks/ALAC/Sigur Rós/Takk.._/06 Sæglópur.m4a
/Volumes/LaCie500G/Music/Tracks/ALAC/Sigur RoÃÅs/Takk.._/07 MiÃÅlanoÃÅ.m4a
/Volumes/LaCie500G/Music/Tracks/ALAC/Sigur RoÃÅs/Takk.._/08 Gong.m4a
/Volumes/LaCie500G/Music/Tracks/ALAC/Sigur RoÃÅs/Takk.._/09 Andvari.m4a
/Volumes/LaCie500G/Music/Tracks/ALAC/Sigur RoÃÅs/Takk.._/10 Svo HljoÃÅtt.m4a
/Volumes/LaCie500G/Music/Tracks/ALAC/Sigur RoÃÅs/Takk.._/11 HeysaÃÅtan.m4a

In Slimcenter, the album is split into two and there are 4 tracks missing:

Home > Artists > Sigur Rós > Takk...
1. Takk... by Sigur Rós from Takk...
8. Gong by Sigur Rós from Takk...
9. Andvari by Sigur Rós from Takk...
10. Svo Hljótt by Sigur Rós from Takk...
11. Heysátan by Sigur Rós from Takk...

Home > Artists > Sigur Rós > Takk...
2. Glósóli by Sigur Rós from Takk...
3. Hoppípolla by Sigur Rós from Takk...

Browsing to a track in the first instance of the album does not show the artwork and has the following information:

Home > Takk... > Heysátan

Album:	Takk...
Year:	2005
Track:	11
File Format:	Apple AAC or Movie File
Duration:	4:09
Location:	/Volumes/LaCie500G/Music/Tracks/ALAC/Sigur Ro/Takk.._/11 Heysaan.m4a (Download)
Date Modified:	Thu 3 Jan 2008, 22.02.59

The file path of this track is wrong and when I return to browsing by album, the track is not listed.

Browsing to a track in the second instance of the album DOES show the artwork and has the following information:

Home > Sigur Rós > Takk... > Glósóli

Artist:	Sigur Rós
Composer:	Jón Þór Birgisson/Kjartan Sveinsson/Orri Páll Dýrason
Album:	Takk...
Genre:	Downtempo
Year:	2005
Track:	2
File Format:	Apple AAC or Movie File
Duration:	6:15
Volume Adjustment:	-14.64
File Length:	43,520,333 Bytes
Bitrate:	898kbps VBR
Sample Rate:	44.1 kHz
Location:	/Volumes/LaCie500G/Music/Tracks/ALAC/Sigur Rós/Takk.._/02 Glósóli.m4a (Download)
Date Modified:	Sat 12 Aug 2006, 11.55.06

Doing a "Clear library and rescan everything" to re-import the tracks from MusicIP, the same album is duplicated but now there are a different set of tracks in the two instances:

Home > Artists > Sigur Rós > Takk...
1. Takk... by Sigur Rós from Takk...   
8. Gong by Sigur Rós from Takk...
9. Andvari by Sigur Rós from Takk...
10. Svo Hljótt by Sigur Rós from Takk...
11. Heysátan by Sigur Rós from Takk...

Home > Artists > Sigur Rós > Takk...
2. Glósóli by Sigur Rós from Takk...   
3. Hoppípolla by Sigur Rós from Takk...
4. Með Blóðnasir by Sigur Rós from Takk...
5. Sé Lest by Sigur Rós from Takk...
6. Sæglópur by Sigur Rós from Takk...
7. Mílanó by Sigur Rós from Takk...












Comment 13 Michael Herger 2008-01-04 07:41:52 UTC
Could you please upload Library/Caches/SqueezeCenter/mmm-song-data.txt - that's MusicIP's output SC is using. I'd like to have a look at it and its encoding.

And please make sure you're only testing with the latest SqueezeCenter 7. As you're still writing "SlimServer" I'm a bit confused and fear you're not using SqueezeCenter yet. Thanks.
Comment 14 Michael Robinson 2008-01-04 09:16:58 UTC
(In reply to comment #13)
> Could you please upload Library/Caches/SqueezeCenter/mmm-song-data.txt - that's
> MusicIP's output SC is using. I'd like to have a look at it and its encoding.
> 
> And please make sure you're only testing with the latest SqueezeCenter 7. As
> you're still writing "SlimServer" I'm a bit confused and fear you're not using
> SqueezeCenter yet. Thanks.
> 

I'm using SqueezeCenter_trunk_v2007-12-30.

I can't find the file mentioned.

There are directories called
~/Library/Application Support/SqueezeCenter/,  
~/Library/Caches/SlimServer/
~/Library/Logs/SqueezeCenter 

but none contain a file called mmm-song-data.txt or similar.

I turned on MusicIP debug logs while importing 2 problem albums and these are attached.


Comment 15 Michael Robinson 2008-01-04 09:18:57 UTC
Created attachment 2607 [details]
MusicIP import debug logs
Comment 16 Michael Robinson 2008-01-04 12:42:31 UTC
ok, I discovered that mmm-song-data.txt is only visible during the MusicIP import and gets deleted when the import is complete.  I don't know how to recover the deleted files so I made a copy of mmm-song-data.txt when the scan was in progress so it may not have all of the imported tracks in it.

I've editted the file to include the tracks that are imported correctly (that is, they can play) and those that aren't imported correctly (that is, they won't play and disappear from the browser interface after they are selected)
Comment 17 Michael Robinson 2008-01-04 12:44:23 UTC
Created attachment 2610 [details]
MusicIP import file
Comment 18 Michael Herger 2008-01-07 02:20:15 UTC
Could you please upload a copy you didn't edit? Editing can already change the encoding. thanks!
Comment 19 Michael Robinson 2008-01-07 06:08:01 UTC
Created attachment 2628 [details]
MusicIP import file - uneditted

Uneditted file attached - this was copied during the scan so may not contain all of the imported tracks.

Songs imported ok were:

Home > Artists > Sigur Rós > Ágætis Byrjun
1. (Intro) by Sigur Rós from Ágætis Byrjun
2. Svefn-g-englar by Sigur Rós from Ágætis Byrjun
3. Starálfur by Sigur Rós from Ágætis Byrjun
4. Flugufrelsarinn by Sigur Rós from Ágætis Byrjun
5. Ný batterí by Sigur Rós from Ágætis Byrjun
6. Hjartað hamast by Sigur Rós from Ágætis Byrjun
7. Viðrar vel til loftárása by Sigur Rós from Ágætis Byrjun
8. Olsen Olsen by Sigur Rós from Ágætis Byrjun
9. Ágætis byrjun by Sigur Rós from Ágætis Byrjun
10. Avalon by Sigur Rós from Ágætis Byrjun

Home > Artists > Sigur Rós > Hvarf/Heim
3. Í gær by Sigur Rós from Hvarf/Heim
5. Hafsól by Sigur Rós from Hvarf/Heim
7. Starálfur (Live) by Sigur Rós from Hvarf/Heim
9. Ágætis byrjun (Live) by Sigur Rós from Hvarf/Heim

Home > Artists > Sigur Rós > Hoppípolla - EP
1. Hoppípolla by Sigur Rós from Hoppípolla - EP
2. Með Blóðnasir by Sigur Rós from Hoppípolla - EP
3. Hafsól by Sigur Rós from Hoppípolla - EP

Home > Artists > Sigur Rós > Sæglópur - EP
1. Sæglópur by Sigur Rós from Sæglópur - EP
2. Refur by Sigur Rós from Sæglópur - EP
3. O Fridur by Sigur Rós from Sæglópur - EP
4. Kafari by Sigur Rós from Sæglópur - EP

Home > Artists > Sigur Rós > Takk...
2. Glósóli by Sigur Rós from Takk...
3. Hoppípolla by Sigur Rós from Takk...
4. Með Blóðnasir by Sigur Rós from Takk...
5. Sé Lest by Sigur Rós from Takk...
6. Sæglópur by Sigur Rós from Takk...
7. Mílanó by Sigur Rós from Takk...

Home > Artists > Sita > L'envers du décor
6. Rien à perdre by Sita from L'envers du décor

Songs not imported ok were:

Home > Artists > Sigur Rós > Hvarf/Heim
1. Salka by Sigur Rós from Hvarf/Heim
2. Hljómalind by Sigur Rós from Hvarf/Heim
4. Von by Sigur Rós from Hvarf/Heim
6. Samskeyti (Live) by Sigur Rós from Hvarf/Heim
8. Vaka (Live) by Sigur Rós from Hvarf/Heim
10. Heysátan (Live) by Sigur Rós from Hvarf/Heim
11. Von (Live) by Sigur Rós from Hvarf/Heim

Home > Artists > Sigur Rós > Takk...
1. Takk... by Sigur Rós from Takk...     
8. Gong by Sigur Rós from Takk...
9. Andvari by Sigur Rós from Takk...
10. Svo Hljótt by Sigur Rós from Takk...
11. Heysátan by Sigur Rós from Takk...

Home > Artists > Sita > L'envers du décor
1. Ce qui nous rend fous by Sita from L'envers du décor
2. Hello by Sita from L'envers du décor
3. L'envers du décor by Sita from L'envers du décor
4. Red Guitar by Sita from L'envers du décor
5. Si on court by Sita from L'envers du décor
7. Jerk by Sita from L'envers du décor
8. Selfish by Sita from L'envers du décor
9. With You by Sita from L'envers du décor
10. Come With Me by Sita from L'envers du décor
11. My Kitchen by Sita from L'envers du décor
12. Bizarre Love Triangle by Sita from L'envers du décor
13. Happy by Sita from L'envers du décor
Comment 20 Michael Herger 2008-01-07 06:27:08 UTC
Thanks for the log. One more thing... could you please upload plugin/musicmagic.log as well? Enable plugin.musicmagic (logging settings) and launch a full rescan. Then upload that file. 
Comment 21 Michael Robinson 2008-01-07 06:47:48 UTC
Created attachment 2629 [details]
Compressed Squeezecentre log files
Comment 22 Michael Herger 2008-01-07 08:02:17 UTC
Kevin - I've noticed the importer does not always de/encode filenames the same way. But shouldn't this be the same for the full importing file?

 /Volumes/LaCie500G/Music/Tracks/AAC/Sigur Rós/Hvarf_Heim/03 Í gær.m4a
 /Volumes/LaCie500G/Music/Tracks/AAC/Sigur Rós/Hvarf_Heim/05 Hafsól.m4a
 /Volumes/LaCie500G/Music/Tracks/AAC/Sigur Rós/Hvarf_Heim/07 Starálfur (Live).m4a
 /Volumes/LaCie500G/Music/Tracks/AAC/Sigur Rós/Hvarf_Heim/09 Ágætis byrjun (Live).m4a
 /Volumes/LaCie500G/Music/Tracks/AAC/Sigur Roフ《/Hvarf_Heim/01 Salka.m4a
 /Volumes/LaCie500G/Music/Tracks/AAC/Sigur Roフ《/Hvarf_Heim/02 Hljoフ[alind.m4a
 /Volumes/LaCie500G/Music/Tracks/AAC/Sigur Roフ《/Hvarf_Heim/04 Von.m4a
 /Volumes/LaCie500G/Music/Tracks/AAC/Sigur Roフ《/Hvarf_Heim/06 Samskeyti (Live).m4a
 /Volumes/LaCie500G/Music/Tracks/AAC/Sigur Roフ《/Hvarf_Heim/08 Vaka (Live).m4a
 /Volumes/LaCie500G/Music/Tracks/AAC/Sigur Roフ《/Hvarf_Heim/10 Heysaフ》an (Live).m4a
 /Volumes/LaCie500G/Music/Tracks/AAC/Sigur Roフ《/Hvarf_Heim/11 Von (Live).m4a

The first four songs do not scan correctly. And in the mmm export file they look the same. So I guess that the utf8decode_guess or whatever isn't always returning the same value, depending on the filename, though the path name should be the same.
 
 Mike - if you look at your disk, are these files really in the same folder? iTunes sometimes plays games and creates odd sub-folders based on tags.
Comment 23 Michael Herger 2008-01-07 08:05:13 UTC
Mike - are you comfortable editing one of the files? Could you please apply the following change, restart, enable plugin.musicmagic, redo a scan, and send scanner.log?

Index: /Users/mh/Documents/workspace/Teststoff/Slim/Plugin/MusicMagic/Importer.pm
===================================================================
--- /Users/mh/Documents/workspace/Teststoff/Slim/Plugin/MusicMagic/Importer.pm	(revision 15966)
+++ /Users/mh/Documents/workspace/Teststoff/Slim/Plugin/MusicMagic/Importer.pm	(working copy)
@@ -299,6 +299,8 @@
 
 		my $enc = Slim::Utils::Unicode::encodingFromString($songInfo{$key});
 
+		$log->debug("encoding: $enc, '$songInfo{$key}', '" . Slim::Utils::Unicode::utf8decode_guess($songInfo{$key}, $enc) . "'") if $key eq 'file';
+
 		$songInfo{$key} = Slim::Utils::Unicode::utf8decode_guess($songInfo{$key}, $enc);
 	}
 
You basically add that "$log->..." line around line 299 in 
/Library/PreferencePanes/SqueezeCenter.prefPane/Contents/server/Slim/Plugin/MusicMagic/Importer.pm
Comment 24 Michael Robinson 2008-01-07 08:41:54 UTC
Created attachment 2630 [details]
SqueezeCenter logs after MusicMagic/Importer.pm edits

I have checked that all the files concerned are in the same folder based upon the file path /AAC/Artist/Album/ or /ALAC/Artist/Album/

I think I have made the correct edits to MusicMagic/Importer.pm

I stopped and started SqueezeCenter, enabled debug level loggin on plugin.musicmagic, did a clear and rescan and have attached the logs.

The files imported/not imported correctly were:

Imported correctly:
Home > Artists > Sigur Rós > Ágætis Byrjun
1. (Intro) by Sigur Rós from Ágætis Byrjun
2. Svefn-g-englar by Sigur Rós from Ágætis Byrjun
3. Starálfur by Sigur Rós from Ágætis Byrjun
4. Flugufrelsarinn by Sigur Rós from Ágætis Byrjun
5. Ný batterí by Sigur Rós from Ágætis Byrjun
6. Hjartað hamast by Sigur Rós from Ágætis Byrjun
7. Viðrar vel til loftárása by Sigur Rós from Ágætis Byrjun
8. Olsen Olsen by Sigur Rós from Ágætis Byrjun
9. Ágætis byrjun by Sigur Rós from Ágætis Byrjun
10. Avalon by Sigur Rós from Ágætis Byrjun

Home > Artists > Sigur Rós > Hoppípolla - EP
1. Hoppípolla by Sigur Rós from Hoppípolla - EP
2. Með Blóðnasir by Sigur Rós from Hoppípolla - EP
3. Hafsól by Sigur Rós from Hoppípolla - EP

Home > Artists > Sigur Rós > Hvarf/Heim
3. Í gær by Sigur Rós from Hvarf/Heim
5. Hafsól by Sigur Rós from Hvarf/Heim
7. Starálfur (Live) by Sigur Rós from Hvarf/Heim
9. Ágætis byrjun (Live) by Sigur Rós from Hvarf/Heim

Home > Artists > Sita > L'envers du décor
6. Rien à perdre by Sita from L'envers du décor

Home > Artists > Sigur Rós > Sæglópur - EP
1. Sæglópur by Sigur Rós from Sæglópur - EP     
2. Refur by Sigur Rós from Sæglópur - EP
3. O Fridur by Sigur Rós from Sæglópur - EP
4. Kafari by Sigur Rós from Sæglópur - EP

Home > Albums > Takk...
2. Glósóli by Sigur Rós from Takk...
3. Hoppípolla by Sigur Rós from Takk...
4. Með Blóðnasir by Sigur Rós from Takk...
5. Sé Lest by Sigur Rós from Takk...
6. Sæglópur by Sigur Rós from Takk...
7. Mílanó by Sigur Rós from Takk...

Not imported correctly: 

Home > Artists > Sigur Rós > Hvarf/Heim
1. Salka by Sigur Rós from Hvarf/Heim
2. Hljómalind by Sigur Rós from Hvarf/Heim
4. Von by Sigur Rós from Hvarf/Heim
6. Samskeyti (Live) by Sigur Rós from Hvarf/Heim
8. Vaka (Live) by Sigur Rós from Hvarf/Heim
10. Heysátan (Live) by Sigur Rós from Hvarf/Heim
11. Von (Live) by Sigur Rós from Hvarf/Heim     

Home > Artists > Sita > L'envers du décor
1. Ce qui nous rend fous by Sita from L'envers du décor
2. Hello by Sita from L'envers du décor
3. L'envers du décor by Sita from L'envers du décor
4. Red Guitar by Sita from L'envers du décor
5. Si on court by Sita from L'envers du décor
7. Jerk by Sita from L'envers du décor     
8. Selfish by Sita from L'envers du décor
9. With You by Sita from L'envers du décor
10. Come With Me by Sita from L'envers du décor
11. My Kitchen by Sita from L'envers du décor
12. Bizarre Love Triangle by Sita from L'envers du décor
13. Happy by Sita from L'envers du décor

Home > Artists > Sigur Rós > Takk...
1. Takk... by Sigur Rós from Takk...
8. Gong by Sigur Rós from Takk...
9. Andvari by Sigur Rós from Takk...
10. Svo Hljótt by Sigur Rós from Takk...
Comment 25 Michael Herger 2008-01-07 09:02:55 UTC
Thanks, that gives me some more information. Interestingly I see three different encoding: utf8, utf-8 and shift_jis. I thought the latter was Japanese or something. Nothing you're using, is it?
Comment 26 Michael Robinson 2008-01-07 09:51:26 UTC
(In reply to comment #25)
> Thanks, that gives me some more information. Interestingly I see three
> different encoding: utf8, utf-8 and shift_jis. I thought the latter was
> Japanese or something. Nothing you're using, is it?
> 

Not knowingly! 

All the problem tracks have been interpreted as shift_jis.

Of the problem tracks, the ones with the ALAC file path have been ripped from CD (using iTunes) and the ones with the AAC file path have been purchased from iTunes (DRM free iTunes plus)
Comment 27 KDF 2008-01-07 10:05:37 UTC
To me, it's the same old problem. Encode::Detect having to deal with file metadata that is sourced from a different codeset than what is found on the running SC system.  clearly this is another case where the detection is failing. Forcing utf8 for $enc would likely make all of these tracks work.  
Comment 28 Michael Herger 2008-01-07 10:15:30 UTC
I've tried to do a test script using Slim::Utils::Unicode. Guess what? It's returning different encodings (cp1252 - Windows) as if running SC on the same machine, same input file. Do you have an idea what outside S::U::Unicode might affect detection?
Comment 29 Michael Robinson 2008-01-07 10:29:50 UTC
(In reply to comment #27)
> To me, it's the same old problem. Encode::Detect having to deal with file
> metadata that is sourced from a different codeset than what is found on the
> running SC system.  clearly this is another case where the detection is
> failing. Forcing utf8 for $enc would likely make all of these tracks work.  
> 

The problem tracks have been in an "all OSX" environment as I'm running SC on a Mac and I did rip and purchase these files myself on my Mac and didn't get them from another source.  However I think the problem tracks are from when I had a G4 mac and I've since moved to an Intel machine.  The OSX and iTunes versions will have changed many times as well.

If there is a problem in the metadata, then it is what iTunes provides - either from purchased tracks or from metadata when ripping a CD on iTunes.
Comment 30 Michael Herger 2008-01-07 10:44:15 UTC
Mike - another try...

Index: /Users/mh/Documents/workspace/Teststoff/Slim/Plugin/MusicMagic/Importer.pm
===================================================================
--- /Users/mh/Documents/workspace/Teststoff/Slim/Plugin/MusicMagic/Importer.pm	(revision 15966)
+++ /Users/mh/Documents/workspace/Teststoff/Slim/Plugin/MusicMagic/Importer.pm	(working copy)
@@ -315,9 +315,7 @@
 	$log->debug("Exporting song: $songInfo{'file'}");
 
 	# Both Linux & Windows need conversion to the current charset.
-	if ($OS ne 'mac') {
-		$songInfo{'file'} = Slim::Utils::Unicode::utf8encode_locale($songInfo{'file'});
-	}
+	$songInfo{'file'} = Slim::Utils::Unicode::utf8encode_locale($songInfo{'file'});
 
 	my $fileurl = Slim::Utils::Misc::fileURLFromPath($songInfo{'file'});
 
Comment 31 Michael Robinson 2008-01-07 10:59:24 UTC
(In reply to comment #30)

That didn't seem to make any difference.  I've shown my edits to Importer.pm below.  I stopped and started SC and did a clear & re-scan  but there are still the same problem tracks and scanner.log still shows these tracks as "shift_jis" - see example:

[08-01-07 18:50:16.5335] Slim::Plugin::MusicMagic::Importer::processSong (302) encoding: shift_jis, '/Volumes/LaCie500G/Music/Tracks/AAC/Sita/L'envers du décor/13 Happy.m4a', '/Volumes/LaCie500G/Music/Tracks/AAC/Sita/L'envers du deフ…or/13 Happy.m4a'

$log->debug("Exporting song: $songInfo{'file'}");

	# Both Linux & Windows need conversion to the current charset.
	
$songInfo{'file'} =
Slim::Utils::Unicode::utf8encode_locale($songInfo{'file'});

	my $fileurl = Slim::Utils::Misc::fileURLFromPath($songInfo{'file'});

Comment 32 Michael Herger 2008-01-08 04:45:58 UTC
Created attachment 2632 [details]
Fix file path conversion on Mac

Here's another try. Please revert your Importer.pm file before applying this patch. 

I did two successful tests with this change, once against my own collection (with lots of Umlauts and accented characters, like Björk, Züric West, Ali Farka Touré, La Rançon du Succès), and with your export file. Of course I wasn't able to really test the latter, but the logged filenames (plugin.musicmagic=debug) did look good.
Comment 33 Michael Robinson 2008-01-08 15:28:25 UTC
(In reply to comment #32)
> Created an attachment (id=2632) [edit]
> Fix file path conversion on Mac
> 
> Here's another try. Please revert your Importer.pm file before applying this
> patch. 

This seems to work.

All albums appeared correctly with my test MusicIP file, so I imported my entire collection of over 900 albums from MusicIP and the SC album counts agreed with iTunes and MusicIP so there doesn't seem to be any duplicate albums or missing tracks.

thanks!

Comment 34 Michael Herger 2008-01-09 00:19:55 UTC
change 16050 - thanks for testing!
Comment 35 Chris Owens 2008-03-07 09:04:16 UTC
This bug is being closed since it was resolved for a version which is now released!  Please download the new version of SqueezeCenter (formerly SlimServer) at http://www.slimdevices.com/su_downloads.html

If you are still seeing this bug, please re-open it and we will consider it for a future release.