Bug 4485 - Artwork files are not picked up in unicode-named folders
: Artwork files are not picked up in unicode-named folders
Status: CLOSED FIXED
Product: Logitech Media Server
Classification: Unclassified
Component: Database
: 6.5.1
: PC Windows XP
: P2 normal with 1 vote (vote)
: ---
Assigned To: Michael Herger
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2006-11-07 01:28 UTC by Yuly Milner
Modified: 2008-12-18 11:12 UTC (History)
8 users (show)

See Also:
Category: ---


Attachments
directory as viewed via WebUI (48.88 KB, image/jpeg)
2006-11-13 23:07 UTC, Yuly Milner
Details
directory as viewed via Windows Explorer (78.32 KB, image/jpeg)
2006-11-13 23:07 UTC, Yuly Milner
Details
Sample music library (44.02 MB, application/octet-stream)
2007-01-15 23:08 UTC, Yuly Milner
Details
scanner.log (29.81 KB, text/plain)
2008-01-16 07:12 UTC, Yuly Milner
Details
scanner log with line 143 in artworks.pm commented (65.83 KB, text/plain)
2008-01-17 05:11 UTC, Yuly Milner
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Yuly Milner 2006-11-07 01:28:32 UTC
Starting from about 2 week old 6.5.1 nightly release, artwork file is not recognized inside the folder, if the folder has a non-english (cyrillic) name. It can be seen using browse music folders - cover.jpg is not visible there at all. Changing the folder name to english cleares the problem.

VERSION INFO
SlimServer Version: 6.5.1 - 10602 - Windows Server 2003 - EN - cp1251
Server IP address: 192.168.11.6
Perl Version: 5.8.8 MSWin32-x86-multi-thread
MySQL Version: 5.0.22-community-nt
Comment 1 Chris Owens 2006-11-13 16:18:22 UTC
Possibly related to bug 4470.

What is the name of the folder, by the way?
Comment 2 Yuly Milner 2006-11-13 23:06:27 UTC
Yes, definetely looks like it's the same bug.
Have a number of those; as an example, this is one:

������ ��������� � �����a ���������� - ������� ������(1998)[FLAC]

Not sure if this is going to be visible, so I've attached tho screenshots: the music folder as viewed via WebUI, and the windows explorer view.
Comment 3 Yuly Milner 2006-11-13 23:07:17 UTC
Created attachment 1713 [details]
directory as viewed via WebUI
Comment 4 Yuly Milner 2006-11-13 23:07:51 UTC
Created attachment 1714 [details]
directory as viewed via Windows Explorer
Comment 5 Yuly Milner 2007-01-11 00:38:08 UTC
Any news/expectations?
Comment 6 Spies Steven 2007-01-15 11:38:50 UTC
Yuly, are you able to add any brand new unicode-named files or folders to Slimserver?

The reason I ask is that I suspect this is another symptom of bug 2475.
Comment 7 Yuly Milner 2007-01-15 11:41:56 UTC
Oh, yes! I do it on a daily basis. New unicode-named folders with unicode-named files are recognized during rescan, without covers.
Comment 8 Spies Steven 2007-01-15 16:24:23 UTC
Yuly, would you take the time to upload a small archive file maintaining the directory structure you use that exhibits the behavior?
Comment 9 Yuly Milner 2007-01-15 23:08:49 UTC
Created attachment 1773 [details]
Sample music library

Attached is a zip archive with 3 directories in it, each representing one album, with 1 track.
One directory uses is named using latin characters, and two are named using cyrillic.
I have pointed my slimserver to this directory structure as my library, stopped the server, cleared the cache directory and restarted. It picked my new library, and the bug is apparent: no covers or thumbs in the cyrillic-named albums
Comment 10 Yuly Milner 2007-01-15 23:18:06 UTC
BTW, the same music library can be used for bugs 3727 and 4630 illustration/verification.
How do I create a link to this upload in those?
Comment 11 Spies Steven 2007-01-16 10:50:47 UTC
Yuly, thank you for uploading the files. However I think the archiving process changed the name of the files. Since the names of the files were changed I am not able to reproduce the problem. You may want to try again with a different archiver such as RAR. If I do rename the files based on the tag information I run into a situation that Slimserver will not see the files at all let alone the artwork. Investigating further.
Comment 12 Spies Steven 2007-01-16 12:13:50 UTC
Yuly, don't bother uploading new files. The problem I was seeing had to do with the Regional and Language settings in Windows. Investigating further.

(In reply to comment #11)
> Yuly, thank you for uploading the files. However I think the archiving process
> changed the name of the files. Since the names of the files were changed I am
> not able to reproduce the problem. You may want to try again with a different
> archiver such as RAR. If I do rename the files based on the tag information I
> run into a situation that Slimserver will not see the files at all let alone
> the artwork. Investigating further.
> 

Comment 13 Yuly Milner 2007-01-16 12:37:43 UTC
Steven,
Just in case it matters: my Regional settings are set to <tab-option>: Regional-Russian; Advanced-Russian
Comment 14 Spies Steven 2007-01-16 13:56:04 UTC
Yuly, I was just about to ask you what your Regional setting are, you must be reading my mind ;) 

(In reply to comment #13)
> Steven,
> Just in case it matters: my Regional settings are set to <tab-option>:
> Regional-Russian; Advanced-Russian
> 

Comment 15 Spies Steven 2007-01-16 16:18:23 UTC
I am now able to reproduce the behavior.

1. In Win XP set "Language for non-Unicode programs" to "Russian" To do this go to Control Panel -> Regional and Language Options -> Advanced Tab
2. Download "Sample music library" above and expand into a directory that Slimserver sees
3. Navigate to the new files in Slimserver using Music Folder or Rescan Music Library
4. Notice the new files are available but does not display cover artwork

If I switch the "Language for non-Unicode programs" back to "English" the files in question disappear from Slimserver altogether and cannot be seen using the browse Music Folder. The file names on the desktop appear fine but the tags do not.

If I try to open the archive while I have "Language for non-Unicode programs" set to "English" the file names in the archive become garbled. These files will not load in Slimserver.

I am going to update bug 4470 as a duplicate as this one instead of bug 2475.

Chris, back to you.
Comment 16 Spies Steven 2007-01-16 16:20:30 UTC
*** Bug 4470 has been marked as a duplicate of this bug. ***
Comment 17 Yuly Milner 2007-04-10 23:39:34 UTC
Bump!
Any plans for fixing this?
Comment 18 Yuly Milner 2007-07-13 01:15:59 UTC
And another bump:( Will this be fixed someday? 
Comment 19 Chris Owens 2007-07-13 10:45:27 UTC
We'll look at it for 6.5.4.  Thanks for the reminder!
Comment 20 Michael Herger 2008-01-08 10:48:12 UTC
*** Bug 6498 has been marked as a duplicate of this bug. ***
Comment 21 Michael Herger 2008-01-09 00:53:15 UTC
Yuly - before I start digging - do you still see this issue with the latest SC7?
Comment 22 Yuly Milner 2008-01-09 02:32:31 UTC
Yes, I just installed the latest 7.0, and the problem is still there.
Comment 23 Michael Herger 2008-01-16 03:57:41 UTC
Are you using an all Windows system? I just tried to reproduce the issue with a Hebrew folder name and it did work fine for me.

Wallace - could you try to reproduce this? I'm sure you have some Chinese test machine around? Thanks!
Comment 24 Yuly Milner 2008-01-16 04:46:08 UTC
(In reply to comment #23)
> Are you using an all Windows system? I just tried to reproduce the issue with a
> Hebrew folder name and it did work fine for me.
> Wallace - could you try to reproduce this? I'm sure you have some Chinese test
> machine around? Thanks!

Yes, the machine is a Windows 2003 SP2 with cyrillic locale:
SqueezeCenter Version: 7.0 - 16286 - Windows Server 2003 - EN - cp1251
Server IP address: 192.168.11.6
Perl Version: 5.8.8 MSWin32-x86-multi-thread
MySQL Version: 5.0.22-community-nt 

Platform Architecture: 586

Hostname: yuly-home-new

Server Port Number: 9000

Total Players Recognized: 3

Cache Folder: C:\Documents and Settings\All Users\Application Data\SqueezeCenter\Cache

Plugin Folders: C:\Program Files\SqueezeCenter\server\Slim\Plugin, C:\Program Files\SqueezeCenter\server\Plugins

Comment 25 Michael Herger 2008-01-16 05:46:12 UTC
Yuly - please delete your scanner.log, turn on artwork=debug and run a scan again. Then upload scanner.log to this report. Thanks.
Comment 26 Yuly Milner 2008-01-16 07:12:07 UTC
Created attachment 2669 [details]
scanner.log

This is a scanner log with artwork debug enabled. A test library was used, with 3 folders: one named with latin characters and two - with cyrillic.
Comment 27 Michael Herger 2008-01-17 04:10:16 UTC
It looks as if the file name got incorrectly encoded in Slim::Music::Artwork::getImageContentAndType

Do you have ActivePerl installed? Could you try commenting out the following line (line 143 in Artwork.pm) and run slimserver.pl from a command window?
Comment 28 Yuly Milner 2008-01-17 05:11:42 UTC
Created attachment 2680 [details]
scanner log with line 143 in artworks.pm commented

This has done the trick! Disabling this line made squeezecenter correctly recognize and display the artwork files in the folders with cyrillic names.
Comment 29 Michael Herger 2008-01-17 05:47:10 UTC
Great news! Now let's try to make this a reasonable patch. Would this change work:

Index: /Users/mh/Documents/workspace/trunk/server/Slim/Music/Artwork.pm
===================================================================
--- /Users/mh/Documents/workspace/trunk/server/Slim/Music/Artwork.pm	(revision 16375)
+++ /Users/mh/Documents/workspace/trunk/server/Slim/Music/Artwork.pm	(working copy)
@@ -139,7 +139,7 @@
 	# path into the current locale.
 	my $locale = Slim::Utils::Unicode::currentLocale();
 
-	if ($locale ne 'utf8') {
+	if ($locale ne 'utf8' && !-e $path) {
 		$path = Slim::Utils::Unicode::encode($locale, $path);
 	}


This would only apply the encoding if the file can't be found without.
Comment 30 Yuly Milner 2008-01-17 05:49:40 UTC
(In reply to comment #29)
I must confess I do not know how to apply this patch. Can you provide me a link to some instructions?

> Great news! Now let's try to make this a reasonable patch. Would this change
> work:
> Index: /Users/mh/Documents/workspace/trunk/server/Slim/Music/Artwork.pm
> ===================================================================
> --- /Users/mh/Documents/workspace/trunk/server/Slim/Music/Artwork.pm   
> (revision 16375)
> +++ /Users/mh/Documents/workspace/trunk/server/Slim/Music/Artwork.pm   
> (working copy)
> @@ -139,7 +139,7 @@
>         # path into the current locale.
>         my $locale = Slim::Utils::Unicode::currentLocale();
> -       if ($locale ne 'utf8') {
> +       if ($locale ne 'utf8' && !-e $path) {
>                 $path = Slim::Utils::Unicode::encode($locale, $path);
>         }
> This would only apply the encoding if the file can't be found without.

Comment 31 Michael Herger 2008-01-17 06:11:11 UTC
> I must confess I do not know how to apply this patch. Can you provide me a link
> to some instructions?

In this simple case it's enough to replace the line marked with a "-" with the one marked with "+":

> > -       if ($locale ne 'utf8') {
> > +       if ($locale ne 'utf8' && !-e $path) {
> >                 $path = Slim::Utils::Unicode::encode($locale, $path);

...and uncomment the line you commented out previously :-). We're just adding a test whether the file does exist or not, before we try to "fix" the path using some encoding. "-e" is an operator to test a file's existence. Is this ok?

Otherwise something like windiff might help:
http://support.microsoft.com/kb/159214/en-us
Comment 32 Yuly Milner 2008-01-17 07:40:36 UTC
Worked for me!
Comment 33 Michael Herger 2008-01-17 07:56:18 UTC
Steinar - you reported bug 3245 which lead to the above encode() line. Could you please test this patch, whether it would break your system? Otherwise I'd like to get it in for the 7.0 release. Thanks!
Comment 34 sbjaerum 2008-01-17 12:48:01 UTC
Works for me.
Comment 35 Michael Herger 2008-01-18 00:08:16 UTC
change 16425 - thanks for testing!
Comment 36 Chris Owens 2008-03-07 09:04:13 UTC
This bug is being closed since it was resolved for a version which is now released!  Please download the new version of SqueezeCenter (formerly SlimServer) at http://www.slimdevices.com/su_downloads.html

If you are still seeing this bug, please re-open it and we will consider it for a future release.
Comment 37 Gordon Harris 2008-03-07 09:53:21 UTC
I'm still seeing this bug with flac files with embedded cuesheets.  But, in this case, it's not just artwork that is effected, but audio playback as well (which also completely fails.)  The bug involves mangling of filenames on UTF8 encoded file systems during the scanning process...i.e. the filename is "double utf8 encoded" when the filename is posted to the db.  Subsequent attempts to extract artwork or play back audio from the incorrectly formed pathname file.

Should this bug be kept open, or should this be considered a new bug?

Comment 38 Chris Owens 2008-03-07 13:27:35 UTC
This bug was specifically about the artwork; it sounds like your new bug is more pervasive, Gordon.  Could I ask you to file a new bug for it?  Thanks.