Bug 4851 - Artist view in Web UI replaces national characters with ? sign
: Artist view in Web UI replaces national characters with ? sign
Status: CLOSED FIXED
Product: Logitech Media Server
Classification: Unclassified
Component: Web Interface
: 6.5.2
: PC Windows Server 2003
: P5 normal with 3 votes (vote)
: 7.4.1
Assigned To: Squeezebox QA Team email alias
: charset_issues
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2007-03-23 03:08 UTC by Yuly Milner
Modified: 2009-10-30 10:28 UTC (History)
1 user (show)

See Also:
Category: ---


Attachments
Artists page - corrupted (89.56 KB, image/jpeg)
2007-03-26 00:37 UTC, Yuly Milner
Details
artists page - ok (98.28 KB, image/pjpeg)
2007-03-26 00:38 UTC, Yuly Milner
Details
unicode-tagged track (1.73 MB, application/octet-stream)
2007-03-26 22:55 UTC, Yuly Milner
Details
screen shot (10.70 KB, image/jpeg)
2007-03-27 16:36 UTC, Ross Levine
Details
A corrupted artist view (31.44 KB, image/jpeg)
2008-03-13 08:05 UTC, Yuly Milner
Details
A correct artist view (31.48 KB, image/jpeg)
2008-03-13 08:05 UTC, Yuly Milner
Details
A correct artist view (29.70 KB, image/jpeg)
2008-04-10 03:44 UTC, Yuly Milner
Details
Another corrupted artist view (40.27 KB, image/jpeg)
2008-04-21 08:43 UTC, Yuly Milner
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Yuly Milner 2007-03-23 03:08:06 UTC
This issue is discussed in the following forum thread:

http://forums.slimdevices.com/showthread.php?t=33449

It seems that, starting from 6.5.1, at some point after initial scan, the first character line in Artist view gets corrupted, and all the national (Russian, Hebrew, etc.) characters are replaced with question mark.

I have been not able to reliably reproduce this. When it happens, the way I use to get rid of this problem is to stop the server, delete the cache directory, and start the server again. This triggers full rescan, after which the letter line looks OK.

In my case, I have automatic clear ad rescan running every 24 hours; might be that on some point it screws up something.

My current server version:

SlimServer Version: 6.5.2 - 11586 - Windows Server 2003 - EN - cp1251
Server IP address: 192.168.11.6
Perl Version: 5.8.8 MSWin32-x86-multi-thread
MySQL Version: 5.0.22-community-nt
Comment 1 Ross Levine 2007-03-23 12:38:54 UTC
This will not be easy to reproduce if it is intermittent as you describe. Could you please attach a sample file that will show this issue?
Comment 2 Yuly Milner 2007-03-26 00:37:39 UTC
Created attachment 1864 [details]
Artists page - corrupted

Here is an example, how the artists page looks when the problem shows itself
Comment 3 Yuly Milner 2007-03-26 00:38:18 UTC
Created attachment 1865 [details]
artists page - ok

Here is how the artists page "should" look
Comment 4 Yuly Milner 2007-03-26 00:46:05 UTC
I'm afraid, there is no file. I (and others, who experience this bug) have a library, with enough artists to fill in more that one page in WebUI (with short artist list, the ABC line is not shown, and there is no problem). Those artists are tagged with english and Unicode names.
When everything is OK, the artists page looks as in my second attachement, with top ABC pointer line showing both english and (in my case) Cyrillic first letters of respectful names of artists.
When the problem appears, the artist page looks as in my first attachement; non-english letters in the ABC string are replaced with "?" sign.

Exactly same behavior is seen with genres page too; I suppose it is a common problem in every page where this ABC pointer line is present.

Full rescan does not solve the problem, but server restart does! I have discovered this just today; previously I have cleaned the cache too, but now it look like this is not necessary.
Comment 5 Ross Levine 2007-03-26 12:22:45 UTC
Yuly could you please attach a file with unicode characters that will display this issue? I'll start building an image with a large list of artist names and then add your file, I think I might be able to reproduce this. 
Comment 6 Yuly Milner 2007-03-26 22:55:51 UTC
Created attachment 1866 [details]
unicode-tagged track

This file is tagged with cyrillic characters; in the artists page/ABC top line it should be presented with a letter "Ю"
Comment 7 Ross Levine 2007-03-27 16:36:28 UTC
Thanks Yuly! I wasn't able to reproduce this, I see the character you describe, and not a ?. I'm attaching a screenshot. Yuly can you please test against the latest nightly build of 6.5.2 to see if this is still an issue?
Comment 8 Ross Levine 2007-03-27 16:36:49 UTC
Created attachment 1878 [details]
screen shot
Comment 9 Yuly Milner 2007-03-27 22:57:56 UTC
Yes, this looks fine.

The problem does not appear immediately. In my case, it takes a week or two before it shows up. I don't know, what sequence of events causes it. When the problem is there, server restart fixes it.
Comment 10 Ross Levine 2007-03-28 11:40:03 UTC
Interesting. Let us know if you manage to identify steps to reproduce this. I'll keep an eye out for this. 
Comment 11 Yuly Milner 2007-03-30 04:30:05 UTC
Is this pointer ABC line is generated in real-time when the specific view is selected in WEB UI? Are there any debugs that one could enable to trace the problem?
Comment 12 Ross Levine 2007-04-02 17:44:00 UTC
Sorry but there aren't any debugs that I think would help us with this. I think this probably has a lot to do with bug 2475, Yuly perhaps you should consider removing the Unicode characters in the file name and see if this is reproducible. Leave Unicode in the tags, just limit the actual file names to Ascii, and see if this is no longer an issue. If so, this likely has something to do with bug 2475. 
Comment 13 Greg Alton 2007-04-07 00:56:31 UTC
Terrible handling of all non-standard ASCII characters - Russian, French with accents, various others. This is a major, major problem and will keep me from recommending to other potential buyers.

In my case, two problems arise:
1) Tracks with non-standard characters do not show up after scanning at all. They just aren't seen by slimserver.
2) Sometimes it picks these tracks up, but then will not play them.
Comment 14 Yuly Milner 2007-04-10 23:46:30 UTC
(In reply to comment #12)
> Sorry but there aren't any debugs that I think would help us with this. I think
> this probably has a lot to do with bug 2475, Yuly perhaps you should consider
> removing the Unicode characters in the file name and see if this is
> reproducible. Leave Unicode in the tags, just limit the actual file names to
> Ascii, and see if this is no longer an issue. If so, this likely has something
> to do with bug 2475. 

I do not think those two are so closely related.
In my case files with unicode characters are handled correctly all the way: they are recognized by the Slim server, tags are correctly picked up, and I'm able to play them. There are other issues with unicode-named files, but not in the content of this bug.
I can run Slimserver for weeks without seing the symptoms of this bug: ABC pointer becoming corrupted. Then, out of a sudden, it happens! For example, right now I have it on my server. If I restart it, all becomes normal again, for some time. Clearly, at some point these web pages stop building the ABC line correctly. The question is how it can be tracked down?
Comment 15 Real name 2007-04-11 00:21:25 UTC
I saw the symptoms of this bug with strictly ASCII file names. ID3 tags were v2.3 with cyrillic characters in them.
Comment 16 Chris Owens 2007-05-15 14:38:49 UTC
Does this happen with all foreign-character tracks, Yuly, or just some of them?  But it doesn't happen all the time?

We're not able to reproduce this in time for the 6.5.2 release.  We'll keep working on it for 7.0.
Comment 17 Yuly Milner 2007-05-16 01:43:08 UTC
(In reply to comment #16)
> Does this happen with all foreign-character tracks, Yuly, or just some of them?


Since we are talking about the ABC pointer line becoming corrupted in artist and genre views of the Web UI, yes: it does happen with all foreign-character ARTISTS and GENRES, not tracks.

I feel that this bug is not correctly understood; probably due to the subject not reflecting the actual symptoms.
This bug talks about the ABC pointer line (shown in artist and genres view pages) becoming corrupted. The actual artist names and genres are displayed correctly, including those with naional characters in them. Attachements 1864 and 1865 show the ABC pointer in a normal and corrupted form

>  But it doesn't happen all the time?

As I mentioned in my earlier posts, this problem shows itself after some time (usually 1-2 weeks) of the server being up. Restart of the server brings it back to normal.

> We're not able to reproduce this in time for the 6.5.2 release.  We'll keep
> working on it for 7.0.
Comment 18 Martins Livens 2007-08-04 01:40:01 UTC
it was time without rescans at all (no auto schedul),
maybe some new files from "Music Folders".
took less that one week to show up again (after restart).

VERSION INFO
SlimServer Version: 6.5.3 - 12376 - Debian - EN - utf8
 Perl Version: 5.8.8 i486-linux-gnu-thread-multi
 MySQL Version: 5.0.22-Debian_0ubuntu6.06.3
Comment 19 Real name 2007-08-20 20:57:29 UTC
BTW, version 7.0 doesn't have this problem
Comment 20 Ross Levine 2007-10-23 13:05:37 UTC
If this works in SC7 then I think it appropriate to close this bug. If anyone has any issue please feel free to re-open and share your thoughts. 
Comment 21 Chris Owens 2008-03-07 09:05:04 UTC
This bug is being closed since it was resolved for a version which is now released!  Please download the new version of SqueezeCenter (formerly SlimServer) at http://www.slimdevices.com/su_downloads.html

If you are still seeing this bug, please re-open it and we will consider it for a future release.
Comment 22 Yuly Milner 2008-03-13 08:04:31 UTC
I'm sorry to say this bug is still present in the 7.0 Squeezecenter.
After some week of running, the artist page looks as shown in attachment artists_7.0-corrupted. Also, at the same time I was unable to find anything containing cyrillic name using SC search function (for example, if I looked for "Дольский", the search returned 0 results).
After the SC service was stopped and started again, the artist page started to look OK (like in attachment artists_7.0_correct). Also, the search for "Дольский" have correctly returned 1 artist.
Running:
SqueezeCenter Version: 7.0.1 - 17813 - Windows Server 2003 - EN - cp1251
Server IP address: 192.168.11.6
Perl Version: 5.8.8 MSWin32-x86-multi-thread
MySQL Version: 5.0.22-community-nt 

Platform Architecture: 586

Hostname: yuly-home-new

Server Port Number: 9000

Total Players Recognized: 3

Cache Folder: C:\Documents and Settings\All Users\Application Data\SqueezeCenter\Cache

Preferences Folder: C:\Documents and Settings\All Users\Application Data\SqueezeCenter\prefs

Plugin Folders: C:\Program Files\SqueezeCenter\server\Slim\Plugin, C:\Program Files\SqueezeCenter\server\Plugins
Comment 23 Yuly Milner 2008-03-13 08:05:28 UTC
Created attachment 3083 [details]
A corrupted artist view
Comment 24 Yuly Milner 2008-03-13 08:05:58 UTC
Created attachment 3084 [details]
A correct artist view
Comment 25 James Richardson 2008-04-09 09:58:27 UTC
PING ross
Comment 26 Ross Levine 2008-04-09 17:50:09 UTC
Sorry for the delay. Yuly I'm confused by your 2 recent screen captures, they look identical to me. The correct artist view image also shows the ?, right? 

I'm still unable to reproduce. 
Comment 27 Yuly Milner 2008-04-10 03:44:07 UTC
Created attachment 3219 [details]
A correct artist view

Sorry; the correct one is this one
Comment 28 Yuly Milner 2008-04-10 03:49:06 UTC
As a matter of fact, the last 7.0.1 I have (SqueezeCenter-7.0.1-17981.exe) is running for 14 days now, and I still don't see the problem myself. As Michael mentioned in one of his posts on the forums (http://forums.slimdevices.com/showthread.php?t=45822&highlight=bug+4851), this bug might have been fixed.
Comment 29 Ross Levine 2008-04-10 12:05:29 UTC
In that case lets close this bug. Yuly please feel free to re-open this bug if you  see this issue again. Thanks Yuly!
Comment 30 Yuly Milner 2008-04-17 04:17:52 UTC
I'm on 18806, and the problem is back on me just 1 day after the server is started. 
All the symptoms are as in comment 22. :(
Not sure if if was really fixed, though. It kind of pops in at some stage; this time it happened very quickly.
Comment 31 Chris Owens 2008-04-18 09:25:15 UTC
One of our developers asks if the oddity occurs immediately after a restart?

Comment 32 Yuly Milner 2008-04-18 13:57:31 UTC
(In reply to comment #31)
> One of our developers asks if the oddity occurs immediately after a restart?

No, after restart everything is fine. The server has to run for undetermined amout of time (sometimes a few days, sometimes a few weeks) before it happens.
Comment 33 KDF 2008-04-21 07:51:56 UTC
Is iTunes or MusicIP involved in any of these cases?  Any changes to those could trigger a re-import of that data and might not be enough to bring an awareness of a rescan, yet could overlay data badly.

May be possible to add some debug to the page generation code so that we have an idea of what's being grabbed from the db along the way.
Comment 34 Yuly Milner 2008-04-21 08:41:48 UTC
(In reply to comment #33)
> Is iTunes or MusicIP involved in any of these cases?  Any changes to those
> could trigger a re-import of that data and might not be enough to bring an
> awareness of a rescan, yet could overlay data badly.
> May be possible to add some debug to the page generation code so that we have
> an idea of what's being grabbed from the db along the way.

No, neither iTunes nor MusicIP are on my system. Debug sounds like a good idea
Comment 35 Yuly Milner 2008-04-21 08:43:29 UTC
Created attachment 3271 [details]
Another corrupted artist view

This morning the artists string showed a funny square character - never happened before.
Comment 36 Chris Owens 2008-06-04 10:18:33 UTC
Ross, have you ever been able to reproduce this reliably?
Comment 37 Ross Levine 2008-06-04 11:36:47 UTC
(In reply to comment #36)
> Ross, have you ever been able to reproduce this reliably?
> 

No I have not. I think we should consider reducing the priority of this bug. Yuly do you have any other pointers that might help to increase the frequency of reproducing this? I'll keep trying. 
Comment 38 Yuly Milner 2008-06-04 12:39:17 UTC
No, unfortunately no clues as for what triggers that.
I suppose only running some debug in the background could help.
Comment 39 Ross Levine 2008-11-10 18:46:33 UTC
Marking this bug as monitor, will need more input to reproduce this. 
Comment 40 James Richardson 2009-10-30 08:39:48 UTC
Yuly: Have you upgraded to 7.4.1 yet?  does this version still exhibit this issue.
Comment 41 Yuly Milner 2009-10-30 08:44:51 UTC
I'm on 7.4.1 now, and I don't see this bug. As a matter of fact, running 7.3.3 for quite a long time before, I also do not recal this happening. I suggest marking it as resolved :)
Comment 42 James Richardson 2009-10-30 10:28:38 UTC
Thanks for the confirmation, marking as closed