Bug 5196 - I18n: Chars corrupted by Search
: I18n: Chars corrupted by Search
Status: REOPENED
Product: Logitech Media Server
Classification: Unclassified
Component: Web Interface
: 7.0
: All Windows XP
: P2 normal with 1 vote (vote)
: ---
Assigned To: Squeezebox QA Team email alias
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2007-07-27 22:26 UTC by Real name
Modified: 2011-05-30 07:58 UTC (History)
2 users (show)

See Also:
Category: ---


Attachments
Cyrillic characters show corrupt. (5.80 KB, image/png)
2007-07-27 22:26 UTC, Real name
Details
Cyrillic characters show corrupt. (2) (3.41 KB, image/png)
2007-07-27 22:48 UTC, Real name
Details
patch, with possible manual search fix. (1.13 KB, patch)
2007-07-29 13:09 UTC, KDF
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Real name 2007-07-27 22:26:01 UTC
When I use Search function with Cyrillic characters, some parts of the page contain corrupt text -- see the attached image.
Comment 1 Real name 2007-07-27 22:26:55 UTC
Created attachment 2075 [details]
Cyrillic characters show corrupt.
Comment 2 Real name 2007-07-27 22:30:31 UTC
SlimServer Version: 7.0a1 - 12431 - RedHat - EN - utf8
Perl Version: 5.8.8 i386-linux-thread-multi
MySQL Version: 5.0.21-standard

Web-browser: FireFox 2.0.0.5
Comment 3 Real name 2007-07-27 22:48:14 UTC
Created attachment 2076 [details]
Cyrillic characters show corrupt. (2)
Comment 4 Real name 2007-07-27 22:48:45 UTC
Comment on attachment 2076 [details]
Cyrillic characters show corrupt. (2)

this is after hitting the 'Search' button
Comment 5 Real name 2007-07-27 22:49:08 UTC
Comment on attachment 2075 [details]
Cyrillic characters show corrupt.

this is before hitting the 'Search' button
Comment 6 Michael Herger 2007-07-28 11:29:54 UTC
What browser are you using? If you have another browser available, could you please try it in eg. FireFox?
Comment 7 Real name 2007-07-28 12:34:00 UTC
see comment #2
Comment 8 Real name 2007-07-28 12:55:52 UTC
Internet Explorer 6 and 7 cannot open SlimServer at all -- they just hang "Opening page http://slimserver..."

In server.log I see the following error:

[12:53:48.0764] Slim::Networking::Select::select (237) Error: Select task failed: Undefined subroutine &Slim::Utils::Prefs::get called at /usr/local/slimserver/Slim/Web/HTTP.pm line 660.

Is this a known bug??

Firefox 2.0.0.5 opens the SlimServer page fine, and there are no errors in the log.
Comment 9 Michael Herger 2007-07-28 13:07:38 UTC
Sorry for not reading your reports... 

Concerning MSIE: that error message rather sounds like there was some code still using legacy prefs code. Do you have some third party plugins installed? What skin are you using? Does the page load if you force default (http://yourserver:9000/Default)?
Comment 10 Michael Herger 2007-07-28 13:16:09 UTC
One more false alert: it's not a plugin, but in the http code itself. But it is indeed MSIE only code. I've checked in a fix - thanks for reporting this back.

Back to your encoding issue: are you familiar with HTML code? Do you know Firebug by chance (an extension to Firefox)?

If you do so:
- do a search
- hit the right mouse button over a correct representation of the string in question, select "Inspect Element" from the context menu. What's the code for your character?
- do the same with a broken representation - do you see a difference?
Comment 11 Real name 2007-07-28 16:01:36 UTC
One immediate problem is that the following query fails (I can see it via FireBug):

http://slimserver:9000/Default/search.xml?query=%D0%9F%D0%B0%D1%80%D0%BA&player=127.0.0.1

Try it! I get a malformed XML error.

The query '%D0%9F%D0%B0%D1%80%D0%BA' corresponds to word "Парк" UTF-8 encoded, where:

0xD0 0x9F == letter 'П'
0xD0 0xB0 == letter 'а'
0xD1 0x80 == letter 'р'
0xD0 0xBA == letter 'к'

I think someone forgot to specify 'UTF-8' encoding while processing XML, and it's defaulting to ASCII.
Thanks.
Comment 12 Real name 2007-07-28 16:15:55 UTC
You can try to access http://www.3amok.com:32180 (it might be up)
Comment 13 Real name 2007-07-29 11:05:46 UTC
Michael, well-formed xml error was because of the unescaped '&' character. Could you please replace:

    '&contibutor.role=ALL' --> '&contibutor.role=ALL'

in two files:

     Slim/Web/Pages/LiveSearch.pm
     Slim/Web/Pages/Search.pm

Thanks!
Comment 14 Real name 2007-07-29 11:42:26 UTC
I found the culprit of this bug -- LiveSearch.pm:

                push @xml, sprintf("<div class=\"even\">\n<div class=\"browsedbListItem\"><hr width=\"75%%\"/><br/>%s \"$query\": $total<br/><br/></div></div>",
                        Slim::Utils::Strings::string(uc($type . 'SMATCHING'))
                );

It looks like $query is encoded in UTF-8 (since it's part of the URL). When it's inserted into the sting, it's butchered into ASCII. The code that works better is:

                push @xml, sprintf("<div class=\"even\">\n<div class=\"browsedbListItem\"><hr width=\"75%%\"/><br/>%s \"%s\": $total<br/><br/></div></div>",
                        Slim::Utils::Strings::string(uc($type . 'SMATCHING')),
                        Slim::Utils::Unicode::utf8decode($query)
                );


There is at least one more place that has the problem (when hitting the Search button, and "manualSearch=1") -- but this should give enough clue to a person who knows Perl and SlimServer. Thanks!
Comment 15 KDF 2007-07-29 12:51:23 UTC
search html/xml fixed added to trunk at change 12441
Comment 16 KDF 2007-07-29 13:09:15 UTC
Created attachment 2078 [details]
patch, with possible manual search fix.

try this and let us know how this works out for you. Do you have ok results with Advanced Search results?
Comment 17 Real name 2007-07-29 14:47:12 UTC
The patch fixed the simple search :)

Advanced search has a similar problem: the result is found ok, but the following fields are turned to ASCII 'Парк':

    Track, Artist, Album, Comment, Lyrics

Thanks!
Comment 18 KDF 2007-07-29 15:40:48 UTC
patch committed to trunk at change 12445 along with possible fix for advanced search.
please test and verify various results as I do not have a test set locally to use with this.
Comment 19 Real name 2007-07-29 22:59:29 UTC
To repro the issue/confirm the fix, all you need to do is copy/paste word 'Парк' into all the mentioned fields, and then to do Search. If the fields come back as 'Парк' -- it's fixed. If you see garbage 'Парк' -- it's still broken.

I'll test the next nightly build tomorrow...

Thanks!
Comment 20 Real name 2007-07-30 21:02:49 UTC
The build of 2007-07-30 appears to be fixed... both Simple Search and Advanced Search. -- Thanks a lot for your help!

By the way, version 6.5.3 has the same issue, plus the search itself is broken: it always yields 0 results when searching with Cyrillic characters. -- can these fixes be back-ported to the 6.x branch?
Comment 21 Michael Herger 2007-07-31 01:58:24 UTC
I'm not sure it's really fixed. The search still is not (or no longer?) working on Windows. We'll have to give it some more testing.
Comment 22 Chris Owens 2008-03-07 09:04:50 UTC
This bug is being closed since it was resolved for a version which is now released!  Please download the new version of SqueezeCenter (formerly SlimServer) at http://www.slimdevices.com/su_downloads.html

If you are still seeing this bug, please re-open it and we will consider it for a future release.
Comment 23 Real name 2011-05-30 05:03:44 UTC
It's not working, and I actually never saw this work. Checked:

   Version: 7.5.4 - r32171

Fucking assholes!