Bugzilla – Bug 5207
I18n: Corrupt chars in Favorites
Last modified: 2011-04-28 08:49:29 UTC
Repro steps: 1) Go to Browse/Favorites 2) Click "New Favorite" 3) In the Name field, type (copy/paste): Спутник 4) Click Save 5) The entry turns into "%u0421%u043F%u0443%u0442%u043D%u0438%u043A", but it should remain "Спутник" Thanks.
Same as 5173 (https://bugs-archive.lyrion.org/show_bug.cgi?id=5173)?
Michael, this is not a duplicate - this is a new problem, specific to version 7. As I understand, the Favorites have been completely rewritten in version 7. This bug: - non-ASCII chars change to '%uXXXX' representation. Bug #5173: - non-ASCII chars are turned into ASCII (each UTF-8 octet becomes an ASCII character)
What codepage does Slimserver think it's running? This information is at the bottom of the 'Server Settings' page in the Slimserver web ui.
SlimServer Version: 7.0a1 - 12431 - RedHat - EN - utf8 Perl Version: 5.8.8 i386-linux-thread-multi MySQL Version: 5.0.21-standard
Alright, the fix in two parts: 1) In [Slim/Plugin/Favorites/HTML/EN/plugins/Favorites/html/favorites.js], replace all 'escape' calls with 'encodeURIComponent'. See [http://search.cpan.org/dist/URI/URI/Escape.pm] and [http://www.w3schools.com/jsref/jsref_encodeURIComponent.asp] for more info. 2) In [Slim/Web/HTTP.pm], need to replace: $value = Slim::Utils::Unicode::utf8on($value); $value = Slim::Utils::Unicode::utf8encode_locale($value); with: $value = Slim::Utils::Unicode::decode($value); According to [http://search.cpan.org/~dankogai/Encode-2.23/Encode.pm], 'encode' turns a string into octets. We need to turn octets into a string. Hence, decode is the right function to call. Thanks.
That change to HTTP.pm is interesting. In fact I've been encountering another issue with corrupted characters (https://bugs-archive.lyrion.org/show_bug.cgi?id=5236) and proposed a similar "fix" to the guy reporting it (Hungarian). He's been happy so far. I'll have to take a closer look at this.
Actually, the fix in HTTP.pm that I've suggested above is a reversal of [http://svn.slimdevices.com/trunk/server/Slim/Web/HTTP.pm?rev=5919&view=rev]: -------------- Revision 5919 Modified Sat Jan 28 13:37:34 2006 PST (18 months, 3 weeks ago) by dsully File length: 60884 byte(s) Diff to previous 5878 Bug: 2617 Description: Encode non-ascii paths correctly. -------------- After reading the description of bug #2617, I think the problem was bogus, and the change was misguided. Thanks.
There is a typo in Comment #5, I meant: $value = Slim::Utils::Unicode::utf8decode($value); Thanks.
Can you still reproduce this issue with the latest SC7 nightly? I've been successfully testing with some German umlauts, Hebrew and Hungarian text. Even your cyrillic example was stored correctly. Can we close this bug?
Michael, v7.0 is still broken as of 2007-12-23. REPRO: 1. Go to Home->Favorites->Edit 2. Click "New Favorite" 3. Copy/paste "Спутник" into the Name field 4. Click "Save" 5. The new entry turns into gibberish "СпÑÑник", however it should read "Спутник" THE FIX: 1) In [/usr/lib/perl5/vendor_perl/Slim/Plugin/Favorites/HTML/EN/plugins/Favorites/html/favorites.js] replace all 'escape' calls with 'encodeURIComponent'. Note: According to [http://search.cpan.org/dist/URI/URI/Escape.pm], "Javascript has a function called escape() that produce the sequence "%uXXXX" for chars in the 256 .. 65535 range. This function has really nothing to do with URI escaping but some folks got confused since it "does the right thing" in the 0 .. 255 range. Because of this you sometimes see "URIs" with these kind of escapes." 2) In [/usr/lib/perl5/vendor_perl/Slim/Web/HTTP.pm], replace: $value = Slim::Utils::Unicode::utf8on($value); with: $value = Slim::Utils::Unicode::utf8decode($value); Note: Variable $value should be converted from bytes to Unicode characters. Simply marking the string of bytes as Unicode, will turn each byte into a separate Unicode character - which is wrong. Thanks.
Hmm... this would be reversion of change 5920: http://svn.slimdevices.com/branches/BRANCH_6_2_x/server/Slim/Web/HTTP.pm?rev=5920&view=diff&r1=5920&r2=5919&p1=branches/BRANCH_6_2_x/server/Slim/Web/HTTP.pm&p2=/branches/BRANCH_6_2_x/server/Slim/Web/HTTP.pm
Can you still reproduce this? change 15871 might have fixed this. BTW: isn't this a dupe of bug 6297?
Michael, yes, change 15871 fixes this particular bug 6297. However, it is wrong in general: the plugin code should use strings as is, not decode them. Currently, the parameter value containing Unicode characters is passed around as raw bytes (against Perl support for Unicode) and then it has to be converted as the value is needed. The parameter value should be converted (utf8decode) once from raw bytes to UTF8 when it is read into the Perl realm (in HTTP.pm), and then used as a regular Perl string. Please undo change 15871, and apply the fix as described in Comment #10
Is your problem fixed?
Well, you seem to disregard my comments. I'll get back to you later.
I'm sorry you felt disregarded. I might have been a bit tired last night... Please give change 16051 another try. It should fix this issue as well as some others which have repeatedly cropped up. Thanks for insisting ;-)
16051 has broken the artwork param in Nokia770 skin. investigating for more details.
param is now fixed at change 16106
This bug is being closed since it was resolved for a version which is now released! Please download the new version of SqueezeCenter (formerly SlimServer) at http://www.slimdevices.com/su_downloads.html If you are still seeing this bug, please re-open it and we will consider it for a future release.
Created attachment 6499 [details] screenshot a repro screenshot
Still broken after I reported it 3 years ago. Some things change, some don't.. -------- Version: 7.4.1 - r28947 @ Tue Oct 20 08:13:15 PDT 2009 Operating system: Windows Home Server - EN - cp1252 Platform Architecture: 586 Perl Version: 5.10.0 - MSWin32-x86-multi-thread MySQL Version: 5.0.22-community-nt
QA to reproduce
Michael, I'm assigning this to you for now. It is reproducible on 7.5.1 r30796 on my Windows 7 system. Note, looking at the comments, it looks as though this is one of the bugs that is connected to one or more bugs in a way that fixing one breaks the other, so please act with caution!
Tested as working in 7.6 under Linux.
== Auto-comment from SVN commit #31786 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31786 == Fixed bug 5207: I18n: Corrupt chars in Favorites
Verified on 7.6.0 (r 32366) on XP with SB Toucch (In reply to comment #25) > == Auto-comment from SVN commit #31786 to the slim repo by ayoung == > == http://svn.slimdevices.com/slim?view=revision&revision=31786 == > > Fixed bug 5207: I18n: Corrupt chars in Favorites