Ubunu - Linux - Unicode - encoding

wxPython Users
1

Hello, a little longer question, and a little OT, only partially related to wxPython.

I know the Unicode pages in wiki, and they are very good.

But nevertheless, some open question I have.

I'm working on our project DrPython or try fix bugs in Linux,
(on windows, it works very good now with latin-1 encoding).

On Windows, it works good now, using setappdefaultencoding and the right
encoding for open with styled text control with the right encoding the files.
(I see the german Umlauts äöü and the "strong 's'" "ß")

The case:
I have a file on a WindowsXP partition which has as contents german umlauts
and the filename itself has umlauts like iÜüä?k.txt

If I want to append this file to a list,
I get somehow latin-1, cannot decode 'utf-8'.

I know, Robin created the wiki page, using small samples in questions,
(and I confirm it it is very useful),
but I don't have it yet, I can try to make a small one to duplicate this
issue, if needed.

sys.setappdefaultencoding(self.prefs.defaultencoding)
would be the easiest solution
which should be the same aus sys.setdefaultencoding in linux.

Why is there a setappdefaultencoding on Windows and sys.setdefaultencoding on linux.

I googled, and I found a strange solution (sys.setdefaultencoding is not available)

import sys
reload (sys)
then this function is available.

(Also Filemanagers like Nautilus or Krusader cannot display the files correctly).

Is there a system wide linux language setting (encoding), which I have to install and adjust?

I know, there are the methods encode, unicode, decode, but how do I know,
when they are needed, I don't want to replace all the source for encode, ... for string access.
So setappdefaultencoding would be the easiest way.

Why is this setdefaultencoding not working on linux?

Should I use also/instead the wx.SetDefaultPyEncoding in DrPython?

This would be the easiest solution, setappdefaultencoding, (getting it from preferences) but it doesn't work.

Beside I tried other editors like spe, pype, boa, ulipad, but none of them displayed the file,
which have german umlauts in the filesnames, correctly.

Thank you verrrry much in advance for a possible solution.

···

--

Franz Steinhaeusler

2

If you can get correct filename, try:

>>> a=u'i\xdc\xfc\xe4k.txt'
>>> file(a)
<open file u'i\xdc\xfc\xe4k.txt', mode 'r' at 0x00E54530>

···

--
I like python!
UliPad <<The Python Editor>>: http://wiki.woodpecker.org.cn/moin/UliPad
My Blog: http://www.donews.net/limodou

3

Hi limodou,

I want to display the file name in a list, you perhaps now, Dan from DrPython
has created a user written file open dialog, and this files cannot be displayed.

···

On Thu, 1 Feb 2007 15:40:05 +0800, limodou <limodou@gmail.com> wrote:

If you can get correct filename, try:

>>> a=u'i\xdc\xfc\xe4k.txt'
>>> file(a)
<open file u'i\xdc\xfc\xe4k.txt', mode 'r' at 0x00E54530>

--

Franz Steinhaeusler

4

If you use unicode I think it'll ok. for example:

import os
os.listdir(u'.')

if the path is unicode, then python will return files encoded in
unicode. so you should use unicode version of wxPython to show the
list I think.

···

On 2/1/07, Franz Steinhaeusler <franz.steinhaeusler@gmx.at> wrote:

On Thu, 1 Feb 2007 15:40:05 +0800, limodou <limodou@gmail.com> wrote:

>If you can get correct filename, try:
>
> >>> a=u'i\xdc\xfc\xe4k.txt'
> >>> file(a)
> <open file u'i\xdc\xfc\xe4k.txt', mode 'r' at 0x00E54530>

Hi limodou,

I want to display the file name in a list, you perhaps now, Dan from DrPython
has created a user written file open dialog, and this files cannot be displayed.

--
I like python!
UliPad <<The Python Editor>>: UliPad - Woodpecker Wiki for CPUG
My Blog: http://www.donews.net/limodou

5

Thanks, I will try this out.

Not directly related, but I would be intersted, if you try your ulipad,
create on a windows xp partition a file for example äöü.txt (latin-1 or cp1252),
can ulipad show this file on linux correctly in your file open dialog?

Do you have otherwise a function or plugin dealing with showing filenames
in wxpython self? This would be very interesting for me! :slight_smile:

···

On Thu, 1 Feb 2007 15:59:01 +0800, limodou <limodou@gmail.com> wrote:

On 2/1/07, Franz Steinhaeusler <franz.steinhaeusler@gmx.at> wrote:

On Thu, 1 Feb 2007 15:40:05 +0800, limodou <limodou@gmail.com> wrote:

>If you can get correct filename, try:
>
> >>> a=u'i\xdc\xfc\xe4k.txt'
> >>> file(a)
> <open file u'i\xdc\xfc\xe4k.txt', mode 'r' at 0x00E54530>

Hi limodou,

I want to display the file name in a list, you perhaps now, Dan from DrPython
has created a user written file open dialog, and this files cannot be displayed.

If you use unicode I think it'll ok. for example:

import os
os.listdir(u'.')

if the path is unicode, then python will return files encoded in
unicode. so you should use unicode version of wxPython to show the
list I think.

--

Franz Steinhaeusler

6

Not directly related, but I would be intersted, if you try your ulipad,
create on a windows xp partition a file for example äöü.txt (latin-1 or cp1252),
can ulipad show this file on linux correctly in your file open dialog?

Do you have otherwise a function or plugin dealing with showing filenames
in wxpython self? This would be very interesting for me! :slight_smile:

For filename, I did nothing, and I only use wxPython to open a file.
So it's not problem as long as wxPython can find it.

And if you said what about the file content, ulipad can auto detect
utf-8 encoding, if it failed, it'll use locale.getdefaultlocale() to
get the default locale encoding, if the local encoding still
none-exist, it'll use sys.getfilesystemencoding. But if the detected
encoding is not same as file encoding, ulipad will fail to open the
file. But it support custom encoding selection, so you can try to
specify the encoding name, then try to open the file.

···

--
I like python!
UliPad <<The Python Editor>>: UliPad - Woodpecker Wiki for CPUG
My Blog: http://www.donews.net/limodou

7

Thank you for your explanation!

···

On Thu, 1 Feb 2007 16:23:32 +0800, limodou <limodou@gmail.com> wrote:

Not directly related, but I would be intersted, if you try your ulipad,
create on a windows xp partition a file for example äöü.txt (latin-1 or cp1252),
can ulipad show this file on linux correctly in your file open dialog?

Do you have otherwise a function or plugin dealing with showing filenames
in wxpython self? This would be very interesting for me! :slight_smile:

For filename, I did nothing, and I only use wxPython to open a file.
So it's not problem as long as wxPython can find it.

And if you said what about the file content, ulipad can auto detect
utf-8 encoding, if it failed, it'll use locale.getdefaultlocale() to
get the default locale encoding, if the local encoding still
none-exist, it'll use sys.getfilesystemencoding. But if the detected
encoding is not same as file encoding, ulipad will fail to open the
file. But it support custom encoding selection, so you can try to
specify the encoding name, then try to open the file.

--

Franz Steinhaeusler