Bugzilla – Bug 6621
Can't store non-latin characters in preference file
Last modified: 2009-07-31 10:15:59 UTC
I entered a name for a player via the web interface using Firefox with a swedish character, which ended up encoded incorrectly in some way. "Kök" (kitchen) became "Kök". SqueezeCenter-version: 7.0 - 16284 - Debian - SV - utf8 Perl-version: 5.8.8 i486-linux-gnu-thread-multi MySQL-version: 5.0.45-Debian_1ubuntu3.1-log Platform Architecture: i686-linux
Works for me. What are your browser's language settings?
Ok, now I see it. It only changes once the screen has been left and re-entered.
Adrian - didn't we have this case already? Looks as if values aren't correctly written or read as utf8 encoded. Any idea?
Yeah, exactly, it needs a refresh to be visible for some reason. Haven't had time to check the database yet, so I'm not sure where the problem is.
Michael - see comment 5, 13 and 14 in bug 5173. I believe we may need to set the ImplictUnicode flag. Previously I was arguing (possibly on campfire) for a newer version of YAML::Syck and to turn on escaping of binary as well with that. I think Andy stores binary prefs for SN? Anyway Chris did not want to do it "this close to 7.0"...
> Michael - see comment 5, 13 and 14 in bug 5173. Thanks. I knew we've had this one already. I should not have closed that other bug, as it did (as you mentioned in your comment) cover various issues. We would have had enough time since september to fix it :-(. > I believe we may need to set the ImplictUnicode flag. Previously I was arguing > (possibly on campfire) for a newer version of YAML::Syck and to turn on > escaping of binary as well with that. I think Andy stores binary prefs for SN? I'll ping him. Thanks!
change 16333 - enabled ImplicitUnicode, working for me now. Please test. Thanks Adrian!
Cool, thanks. Will test as soon as it hits the Debian repo.
If you don't mind, the change is fairly simple: just add "$YAML::Syck::ImplicitUnicode = 1;" around line 29 in Slim/Utils/Prefs/Namespace.pm: --- trunk/server/Slim/Utils/Prefs/Namespace.pm (original) +++ trunk/server/Slim/Utils/Prefs/Namespace.pm Wed Jan 16 01:58:35 2008 @@ -28,6 +28,8 @@ use Slim::Utils::Prefs::Client; use Slim::Utils::Log; + +$YAML::Syck::ImplicitUnicode = 1; my $log = logger('prefs');
Hmm, Version: 7.0-16329, made the change you said, and now I can't save anything in the settings pages. This is the same even when I reverted the change. When clicking Apply I just get an empty maincontent and the change isn't saved.. I did, however, upgrade the version as well, so it's probably something between r16284 and r16329.
Thanks for the hint regarding the save problem. Can you confirm that you didn't encounter it before 16284? Are you accessing your server from the same machine?
Hmm, tried the computer at home now, which is on the same network, and it works fine there. With Firefox under Windows here at work, it doesn't work (saving that is). IE7 does work though. Can't remember if I've tried to save settings remotely with FF earlier, sorry. The name problem is still there though, after saving it appears fine, but is displayed incorrectly once I click "Players" again. This is with the changes from r16333 made to the file /usr/share/perl5/Slim/Utils/Prefs/Namespace.pm.
Did you update your installation to the latest nightly?
Just upgraded to: SqueezeCenter-version: 7.0 - 16373 - Debian - SV - utf8 Still not able to name it "Kök" though :/ But no problem saving now. It appears as "Kök" in the webinterface and "K\xC3\xB6k" in /etc/squeezecenter/server.prefs. Maybe it's a missing meta charset tag?
> It appears as "Kök" in the webinterface and "K\xC3\xB6k" in > /etc/squeezecenter/server.prefs. Is your browser set to display utf8 (or auto)? What you're seeing is what I get if I force eg. iso8859-1.
Hmmmm.. My browser automatically selects UTF8.. If I force it to iso8859-1 it displays as "Kök".. I'm using the swedish translation, and it displays everything as it should when using UTF-8, but also displays errors when using iso8895-1, it displays the letter "ö" the same way as when I save it (ö instead of ö)..
What does the following tell you: perl -MYAML::Syck -e 'print "$YAML::Syck::VERSION\n"'
oscar@dragon:/etc/squeezecenter$ perl -MYAML::Syck -e 'print "$YAML::Syck::VERSION\n"' Can't locate YAML/Syck.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.8.8 /usr/local/share/perl/5.8.8 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8 /usr/share/perl/5.8 /usr/local/lib/site_perl .). BEGIN failed--compilation aborted.
Ok, let's make this: perl -I/usr/share/squeezecenter/CPAN -MYAML::Syck -e 'print "$YAML::Syck::VERSION\n"' Please replace the include path with wherever SC has its own CPAN folder. Thanks!
oscar@dragon:/etc/squeezecenter$ perl -I/usr/share/squeezecenter/CPAN -MYAML::Syck -e 'print "$YAML::Syck::VERSION\n"' 0.64
change 16386 - reverting change 16333 as it didn't fix your issue, but caused negative side-effects. Steinar wasn't able to scan his collection any more after this change. Why ever this would influence the scanner... This won't be fixed in 7.0.
:/ Okay. Is it possible to manually enter the name in the correct format in the config file?
> Okay. Is it possible to manually enter the name in the correct format in the > config file? I doubt it. It would be overwritten the next time _any_ change to the file is done. But I'll try to find a solution post 7.0. Thanks for testing!
Some additional information using WinXP. I change a player's name to æøå. This is rendered correctly until SC is restarted. After a restart, the player's name is displayed as æøå. It is the same behavior both in the web interface and on the player display. This is on 7.0 r16779.
The problem here is that the preferences aren't correctly encoded when saved to disk. As long as SC is running, it keeps them in the memory. Thus you only see the issue after a restart.
changing summary to reflect the root cause of the issue reported. We're seeing more and more related problems.
*** Bug 7390 has been marked as a duplicate of this bug. ***
we seem to be getting variations of this issue on a daily basis - increasing severity
change 17846 I'm sorry Steinar. Checking this in again. We need some more testing. But I successfully tested the change once again, using FF2 (Win, OSX, Ubuntu), IE6/7, Safari as clients, and Windows XP, Ubuntu, OSX 10.5, and a ReadyNAS as servers, restarting SC after the changes. It didn't miss one non-latin character. I even used a Hebrew Windows system to change the setting on the ReadyNAS. Please test again one of the next 7.0.1 nightlies so we can track this down. Report back as soon as possible, with information about your client/server configuration etc.
QA - could you please go through the list of related bugs to see whether this patch fixes them, too?
I see the same side effect of the line checked in at r17846 as the last time it was checked in. Files with accented characters (at least the Norwegian æøå characters) are not picked up by the scanner. To me this side effect is much more severe than the bug this line is supposed to fix. Try to see if you are able to scan a file with a filename containing one of the characters æøå on Windows. My system information is: SqueezeCenter Version: 7.0.1 - 16718 - Windows XP - EN - cp1252 Server IP address: 10.0.0.4 Perl Version: 5.8.8 MSWin32-x86-multi-thread MySQL Version: 5.0.22-community-nt Platform Architecture: 586
Michael - there's another bug here related to the player name. Squeezebox.pm will send the player name to the player for saving as a pack Z* which is an Ascii string. This means once we support synching with the player again it doesn't actually save non ascii strings correctly on the player.
Looks as if this will need a bit more work. The patch seems to work for _some_ non-latin characters only while completely breaking writing the file in other cases. Additionally Triode just made me understand that the player name must _not_ contain non-ascii characters. There's a lot more work to do. Thanks for testing.
change 17864 - one more step... this will make sure a player's name is utf8 when it's read from the player. Steinar - the issue you mention (file name) is a different issue. The problem we're seeing here is not being able to store any preference with non-latin characters. Player name is a special case, but you should now be able to call it "æøå". Can you confirm this?
I can confirm that I now can save a player name of æøå. But I don't like the fix. On my system, a direct consequence of adding the line $YAML::Syck::ImplicitUnicode = 1; to Namespace.pm, is that the scanner does not include files with accented characters into the library. This is weird, but it is a fact. So the change introduces a more severe bug than it fixes.
change 17866 - don't use File::Slurp to write the file, as it can't handle utf8 encoded data Steinar - I'll do some more testing with file names tomorrow. Thanks for specifying the issue.
Verified fixed in 7.0.1 - 19422 If anyone else see strange behavior with UTF-8 please open a new bug with details of the errors.
This bug has recently been fixed in the latest release of SqueezeCenter 7.0.1 Please try that version, if you still see the error, then reopen this bug. To download this version, please navigate to: http://www.slimdevices.com/su_downloads.html
Reduce number of active targets for SC