Bugzilla – Bug 16683
Non-ASCII characters in file and directory names
Last modified: 2012-03-03 09:24:41 UTC
This bug is a catch-all to handle the issues associated with handling pathnames with elements containing non-ASCII characters. Pathnames include both directory-name (folder) and file-name components. The pathnames under consideration are those that contain audio files and playlist files. That is, those pathnames that make up the audiodir and playlistsdir preferences, and paths subordinate to them. The functionality under consideration includes the scanner, browse-music-folder (all UIs) and transcoding. It does not include the contents of files: such as pathnames that are included within playlist files or tags within audio files. The following bugs are currently under consideration: Bug 9897 Bug 13153 Bug 15460 Bug 15537 Bug 6754 Bug 10199
== Auto-comment from SVN commit #31562 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31562 == bug 16683: Non-ASCII characters in file and directory names Correct debug message.
== Auto-comment from SVN commit #31563 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31563 == bug 16683: Non-ASCII characters in file and directory names Do not decode value returned from Slim::Music::Info::fileName()
== Auto-comment from SVN commit #31564 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31564 == bug 16683: Non-ASCII characters in file and directory names In general, do all pathname manipulation as native byte-strings and do not decode to internal (utf8) character-string representation. Use Slim::Utils::Misc::getAudioDir() for all access to audiodir pref. Use Slim::Utils::Misc::getPlaylistDir() for all access to playlistsdir pref. These return native encoded strings. The idea is to hide the underlying storage mechanism / encoding for these preferences which may change. Use Slim::Utils::Unicode::encode_locale() whenever a pathname (component) must be generated from a (potentially) UTF-8 string. Still use Slim::Utils::Unicode::utf8encode_locale() when source string may be non-decoded UTF-8.
Also bug 16662
Also 16180
== Auto-comment from SVN commit #31566 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31566 == Fixed bug 16662: Incorrect UTF8 encoding returned by CLI bug 16683: Non-ASCII characters in file and directory names
== Auto-comment from SVN commit #31572 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31572 == bug 16683: Non-ASCII characters in file and directory names Pass folder_id instead of folder path as param to jiveplaytrackalbum so that encoding does not mash it around.
== Auto-comment from SVN commit #31573 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31573 == bug 16683: Non-ASCII characters in file and directory names Simplify encode_locale()
== Auto-comment from SVN commit #31576 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31576 == bug 16683: Non-ASCII characters in file and directory names Do not decode Track->title as it should already be in internal UTF-8 format if necessary. Use 'file:///' URLs for the pathnames in M3U playlist files that we write so that arbitrary character encodings will be preserved.
== Auto-comment from SVN commit #31577 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31577 == bug 16683: Non-ASCII characters in file and directory names Only encode artwork pathnames that may have been generated from tag data.
== Auto-comment from SVN commit #31578 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31578 == bug 16683: Non-ASCII characters in file and directory names Ensure that utf8encode always unsets UFT8 flag. Use simple alias for encode and decode.
== Auto-comment from SVN commit #31580 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31580 == bug 16683: Non-ASCII characters in file and directory names use ut8fdecode_local for making pathname visible in UIs.
== Auto-comment from SVN commit #31582 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31582 == bug 16683: Non-ASCII characters in file and directory names Need to use utf8decode_locale to put UTF-8 pathname in XML skin.
== Auto-comment from SVN commit #31583 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31583 == bug 16683: Non-ASCII characters in file and directory names Use utf8::decode, which is idempotent, instead of Slim::Utils::Unicode::utf8on.
== Auto-comment from SVN commit #31584 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31584 == bug 16683: Non-ASCII characters in file and directory names Simplify implementation of utf8on, which may actually be obsolete.
== Auto-comment from SVN commit #31609 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31609 == bug 16683: Non-ASCII characters in file and directory names Fixed Bug 15805 - Artists sorting incorrectly on 7.5-embedded SQLite Fixed Bug 14800 - Sorting not done correctly with Swedish characters Bug 13153 - Both web interface and squeezebox "hang" when one browses folders containing non-ascii chars Set LC_COLLATE so that SQLite database sorting using perlcollate works for different languages.
== Auto-comment from SVN commit #31611 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31611 == bug 16683: Non-ASCII characters in file and directory names Some simplifications.
== Auto-comment from SVN commit #31612 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31612 == bug 16683: Non-ASCII characters in file and directory names Fixed Bug 14906 - Browse music folder: Wrong folder sort order Use native locale for sorting filenames.
== Auto-comment from SVN commit #31680 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31680 == bug 16683: Non-ASCII characters in file and directory names Revert earlier change which used file-URL for local files instead of plain paths. Add new special comment type: "#EXTURL:" which is used to specify the URL of the entry. For local files this will be a file-URL, which uses native character-set encoding for pathnames (URL-encoded if needed). #EXTURL: comments are added when writing (before #EXTINF:) and are used in preference to the plain entry when reading.
== Auto-comment from SVN commit #31681 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31681 == bug 16683: Non-ASCII characters in file and directory names Mark bugs 6754, 9897, 13153 as fixed
== Auto-comment from SVN commit #31848 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31848 == bug 16683: Non-ASCII characters in file and directory names Assume pathnames are encoded in locale character-set encoding when guessing tags from pathnames.
== Auto-comment from SVN commit #31868 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31868 == bug 16683: Non-ASCII characters in file and directory names Use Slim::Utils::Unicode::utf8decode_locale() for Windows paths that were not '~'-encoded and when using cp1252 (the default).
== Auto-comment from SVN commit #31873 to the slim repo by ayoung == == http://svn.slimdevices.com/slim?view=revision&revision=31873 == Fixed bug 16683: Non-ASCII characters in file and directory names I know of no outstanding issues.
Working as expected now----files are not being hidden by non-ascii characters in the path (In reply to comment #23) > == Auto-comment from SVN commit #31873 to the slim repo by ayoung == > == http://svn.slimdevices.com/slim?view=revision&revision=31873 == > > Fixed bug 16683: Non-ASCII characters in file and directory names > I know of no outstanding issues.
This bug is NOT fixed! As of Logitech Media Server Version 7.7.1 there is still a problem with SOME non-ASCII characters. For example: a directory named "Múm" scans fine, but a directory named "Les Misérables" is completely invisible to SBS scans. Can somebody please confirm, and then re-open this bug?