Bug 6341 - Chinese charactors shows up as blocks in Jive.
: Chinese charactors shows up as blocks in Jive.
Status: CLOSED FIXED
Product: SqueezePlay
Classification: Unclassified
Component: UI
: unspecified
: PC Other
: -- normal with 11 votes (vote)
: 7.6.0
Assigned To: Ben Klaas
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2007-12-12 16:16 UTC by Wallace Lai
Modified: 2011-05-16 16:53 UTC (History)
11 users (show)

See Also:
Category: Bug


Attachments
Chinese charactors are fine on SC but not on Jive. (1.69 MB, audio/x-ms-wma)
2007-12-12 16:21 UTC, Wallace Lai
Details
Chinese show as mubo jumboes on MediaPlayer, SlimServer, and Jive. (4.79 MB, audio/mpeg)
2007-12-12 16:28 UTC, Wallace Lai
Details
Script to concatenate font files (5.43 KB, text/x-perl)
2008-10-10 16:02 UTC, Christopher Key
Details
Script to concatenate font files (5.90 KB, text/x-perl)
2008-10-10 17:20 UTC, Christopher Key
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Wallace Lai 2007-12-12 16:16:41 UTC
Jive FW:  1 r1123 (from Richard)

Problem:
There are many Chinese songs have their titles, artists, and albums show up on SlimServer and MS MidiaPlayer as mumbo jumboes.  This problem has been around for a while.  Now, they show up on Jive as mumbo jumboes also.  That is bad, but not totally unexpected.

However, this user has finally aquired a Chinese song that shows on SqueezeCenter correctly, but the charactors show on Jive as blocks.
Comment 1 Wallace Lai 2007-12-12 16:21:17 UTC
Created attachment 2511 [details]
Chinese charactors are fine on SC but not on Jive.
Comment 2 Wallace Lai 2007-12-12 16:28:50 UTC
Created attachment 2513 [details]
Chinese show as mubo jumboes on MediaPlayer, SlimServer, and Jive.
Comment 3 Richard Titmuss 2008-01-11 05:25:58 UTC
andy i'm not sure how the shoutcast stations are imported, can you comment on this.
Comment 4 Richard Titmuss 2008-01-11 05:46:52 UTC
opps, commenting on wrong bug. Marking this for future, Chinese fonts are not yet supported.
Comment 5 Richard Titmuss 2008-03-12 04:53:35 UTC
Reset priority
Comment 6 Christopher Key 2008-10-10 16:02:51 UTC
Created attachment 4129 [details]
Script to concatenate font files

I ran into this limitation, and have come up with a temporary solution.  Attached is a script that will combine two font files.

When invoked as,

fontcat.pl FontA.ttf FontB.ttf Out.ttf

it will copy across any characters in FontB that are missing in FontA, and write the output to Out.ttf.  The range of characters it attempts to copy can be limited by appending one or more ranges of the form U+1234-U+5678 to the argument list.


The output font is a little buggy, for example, Character Map doesn't show the missing characters, and there are some structures that I'm not sure how to combine correctly.  Nevertheless, combining FreeSans and CODE2000 yields a font that seems to work perfectly happily in Jive.


I've not attached the resulting font file because CODE2000 isn't freely redistributable.  Instead, here's an overview of the necessary steps to recreate it:

Download fontcat.pl

Install Font::TTF  from http://search.cpan.org/~mhosken/Font-TTF-0.45/

Obtain FreeSans.ttf and FreeSansBold.ttf from http://www.gnu.org/software/freefont/index.html and place them in the same dir as fontcat.pl

Obtain CODE2000.ttf from http://www.code2000.net/code2000_page.htm and place it in the same dir as well.

Combine the two fonts:  (be prepared to wait a few minutes)
./fontcat.pl FreeSans.ttf CODE2000.ttf FreeSans_CODE2000.ttf
./fontcat.pl FreeSansBold.ttf CODE2000.ttf FreeSansBold_CODE2000.ttf

Copy the resulting files to the squeezebox controller:
scp FreeSans_CODE2000.ttf root@SqueezeboxControllerIP:/usr/share/jive/fonts/FreeSans.ttf
scp FreeSansBold_CODE2000.ttf root@SqueezeboxControllerIP:/usr/share/jive/fonts/FreeSansBold.ttf


Reboot the Controller

Done
Comment 7 Christopher Key 2008-10-10 17:20:12 UTC
Created attachment 4130 [details]
Script to concatenate font files

Bugfix: Compound glyphs weren't always copied correctly.
Comment 8 James Richardson 2009-05-18 15:30:46 UTC
Resetting bug target/assigned
Comment 9 Richard Titmuss 2009-09-11 02:08:38 UTC
*** Bug 13624 has been marked as a duplicate of this bug. ***
Comment 10 Biu Chan 2010-02-28 09:08:07 UTC
The main problem is the included font not supporting asian languages.  It is probably due to file-size and license issue that logitech decided not to include this support.  However, we found that the open-source Google Android font is small in size, displays very well on small screens and supports a wide range of languages and character sets.  It works perfectly fine.  Please see details in this thread in the forum:

http://forums.slimdevices.com/showthread.php?t=74977

It would be great if this could be incorporated in the firmware upgrade in the near future.
Comment 11 signor_rossi 2010-02-28 10:49:13 UTC
I have tried the Android font and there are still two problems, one minor but one grave:
Minor: Some German 'Umlaute' are missing.
Grave: Scrolling still becomes terribly slow and almost unusable on the Controller when  CJK fonts are displayed, scrolling is fine otherwise.
Comment 12 Uwe Schröder 2010-03-21 13:47:49 UTC
As the name implies, the DroidSansFallback font seems to be intended to be used as a secondary font, filling in the gaps of the regular DroidSans font. You would have to combine DroidSans and DroidSansFallback into one to get the full Unicode range. Does anybody know whether the license allows derivative works?
Comment 13 Blackketter Dean 2010-03-21 17:43:39 UTC
I don't think there is any licensing issue here.  The fonts can be used as is.
Comment 14 Uwe Schröder 2010-03-21 18:08:17 UTC
Unfortunately not. DroidSansFallback lacks many characters from the normal iso-8859-1 (western) range, for example the German letters ä and ö. That is unacceptable for German, Swiss, and Austrian users, so these would have to be added first.
Comment 15 Sam Martin 2010-09-17 18:25:27 UTC
DroidSansFallback has the issues with German and other characters and also results in slow scrolling of CJK characters in the playlists for me.

I found and use another font, which needs to be licensed in commercial use but is free for personal use: Bitstream Cyberbit
Logitech might want to buy a license for the future - but we as users should be allowed to put it now ourselves.

- Downloaded here: http://aol-4.vo.llnwd.net/pub/communicator/extras/fonts/windows/ (link is taken from Wikipedia entry http://en.wikipedia.org/wiki/Bitstream_Cyberbit )
- The font is contained in the Cyberbit.ZIP, named Cyberbit.ttf
It is rather large (13MB) but can be used in the /jive/fonts folder with two symbolic links (same method as for the Android font).
Scrolling appears much faster than Android whilst supporting any characters I tried so far (German, Englisch, CJK, Spanish,...)
Comment 16 g1234512345g 2010-09-28 11:32:54 UTC
I've installed DroidSansFallback.ttf on the Touch. Chinese characters now display properly on the Now Playing screen, as well as when browsing directories when it's connected to SBS. However, when the Touch is connected to its own server, I am unable to browse to any directories where there is a file with Chinese characters. It just sits there showing a spinning icon when I press Right, instead of opening the directory. I can still Play the entire directory though, and the Now Playing screen would still display correctly.
Comment 17 julian 2011-01-29 18:49:31 UTC
Bitstream Cyberbit is too big for the Squeezebox Radio, and trying to copy it over will make you end up having to do a factory reset.

But, being a little disappointed with the bugs in DroidSansFallback.ttf (missing certain characters, etc.) I decided to do some digging. You can use the open-source program FontForge to merge FreeSans.ttf with DroidSansFallback, so you can have the best of both worlds.

Steps:
1. Open DroidSansFallback.ttf with FontForge. Go to Element >> Font Info >> General and then adjust the Ascent to 948 -- the Em Size will now be 1000. This ensures that DroidSansFallback and FreeSans are the same sizes when merged. Click OK, and wait while the conversion takes place.

2. File >> Generate Fonts. Turn off verification, and save the modified DroidSansFallback font (with a different name, if you like).

3. Now close that font, and load FreeSans.ttf (you can copy it from your Radio, or download it freely on the net). Go to Element >> Merge Fonts. Select the modified DroidSansFallback font.

4. Now, File >> Generate Fonts and save the merged font. Mine was 3.8MB -- small enough to fit on the Radio. Now copy it to Radio in the same way you do with the DroidSansFallback font.

Works perfectly so far.
Comment 18 Ben Klaas 2011-01-31 07:44:46 UTC
Wow, Julian-- very cool info
Comment 19 julian 2011-02-01 06:07:32 UTC
(In reply to comment #18)
> Wow, Julian-- very cool info

Additionally, I just realized that there's still space left over in the Radio for a modified FreeSansBold.ttf that also contains all the characters from DroidSansFB. Just added that, and now the Radio looks just the same as factory default, only can display everything :)

Scrolling Chinese characters is still slower than regular latin text, though -- but that's not really a big deal for me.
Comment 20 Ben Klaas 2011-02-01 07:27:34 UTC
For production firmware to the masses we will need to take a more detailed look at the memory implications of the merged font, particularly on Radio and Controller where memory is more at a premium.

Also on the table is moving to the Droid font exclusively

Thanks again for the font merging idea, it's an intriguing one
Comment 21 Ben Klaas 2011-04-22 12:41:51 UTC
I've created a 3rd party applet to deliver the merged FreeSans/DroidSansFallback and FreeSansBold/DroidSansFallback. I'm going to contact the 3rd party devs to see about getting it included in the applet installer list.

If I can get the applet in that set, it will be installable via Settings->Advanced->Applet Installer->Font Replacer (and after bug 17155 is resolved, that will change to App Gallery->3rd Party Apps->Font Replacer)

The way the applet works is the following: 

Applet Installer installs the applet and restarts the squeezebox.
An item is displayed on the top of the home menu for Font Replacer
Follow this item and follow the steps to copy the font files to the correct location
Squeezebox will restart, and after reboot the new font files will be loaded. Font Replacer item will no longer appear on the home menu
Comment 22 Ben Klaas 2011-04-22 12:44:07 UTC
I should note that there was much consideration using this font in the stock firmware, but the CJK character set increases the size of the font file by about 8x, which is not going to be a good solution given some of our devices tight memory constraints.

What I'm attempting to do to resolve this is provide a fairly simple means for people desiring a fuller font, esp. with regards to CJK characters, to achieve that goal.
Comment 23 Ben Klaas 2011-04-22 12:55:06 UTC
I posted to the forums about getting the FontReplacer distributed via the 3rd party Applet Installer
http://forums.slimdevices.com/showthread.php?p=627028#post627028
Comment 24 Ben Klaas 2011-04-22 15:21:35 UTC
The FontReplacer applet is live now, and can be accessed via Settings->Advanced->Applet Installer->Font Replacer (make sure you uncheck the box for "recommended applets" as this applet has not yet been vetted into the reocmmended list)

Font Replacer will overwrite the stock FreeSans and FreeSansBold fonts with the merged fonts as described by Julian in comment#17.

After the font has been copied and the squeezebox restarted, the new fonts will be in use.

If you perform a Factory reset, this will put the system back to the stock fonts again.
Comment 25 Paul Chandler 2011-05-16 16:53:35 UTC
INstalled the Font Replacer on both the TOuch and the Jive: was able to view Chinese characters. 7.6.0 r 32419