problem with (unicode)strings

wxPython Users
1

Hello group.

I’m trying to get a list of the network adapters of the computer and list them in a frame with StaticText(). Some of my adapters have german chars (äöü) in their names. When i print() them the names are shown correct, but with StaticText() they are wrong. Example:

Name of the adapter is “Täst”.

print() shows ‘Täst’

StaticText() shows ‘T„st’

I tried unicode(), encode() and decode() with different codecs, but wasn’t able to correctly show the name with StaticText().

Where do i miss something?

ListAdapters.pyw (1.33 KB)

2

What is you wxPython version?
Try with unicode build of wxPython.

Niki

···

On 16.07.2014 19:28, Torsten wrote:

Hello group.

I'm trying to get a list of the network adapters of the computer and
list them in a frame with StaticText(). Some of my adapters have german
chars (äöü) in their names. When i print() them the names are shown
correct, but with StaticText() they are wrong. Example:

    Name of the adapter is "Täst".

    print() shows 'Täst'

    StaticText() shows 'T„st'

3

I tried it with Python 2.7.8 / wxPython 2.9.5.0 msw (classic) and Python 3.4.1 / wxPython 3.0.2.dev76938 msw (phoenix). Neither i were able to get the correct result.

4

Hi Torsten,

I tried it with Python 2.7.8 / wxPython 2.9.5.0 msw (classic) and Python 3.4.1 / wxPython 3.0.2.dev76938 msw (phoenix). Neither i were able to get the correct result.

Hhm, can understand it on Py2.7 but on Py3.4 it should work without doing much.

I tried to run your script on Win8.1, Py3.4 and Phoenix, but to get it to run I had to change it, see attached.

It would run forever if I kept 'shell=True'.

Main change is that in Py3 the proc.stdout.readline() returns bytes so had to decode.

I also added '/nh' to the command, to suppress the headers.

BTW, how do you get "Umlaute" into these connection names?

Werner

getmac.py (1.24 KB)

···

On 7/17/2014 10:35, Torsten wrote:

5

Hi Torsten,

I tried it with Python 2.7.8 / wxPython 2.9.5.0 msw (classic) and
Python 3.4.1 / wxPython 3.0.2.dev76938 msw (phoenix). Neither i were
able to get the correct result.

Hhm, can understand it on Py2.7 but on Py3.4 it should work without
doing much.

I tried to run your script on Win8.1, Py3.4 and Phoenix, but to get it
to run I had to change it, see attached.

Yes, i also did some changes to make it work with 3.4 (it was my first time using Python 3). Sorry, should have mention that and attach the file.

It would run forever if I kept ‘shell=True’.

Main change is that in Py3 the proc.stdout.readline() returns bytes so
had to decode.

I also added ‘/nh’ to the command, to suppress the headers.

When i run your script with Python 3 i get a UnicodeDecodeError:

Traceback (most recent call last):

File “D:\Downloads\getmac.py”, line 42, in 

win = MainFrame()

File “D:\Downloads\getmac.py”, line 10, in init

adapters = self.GetAdapters()

File “D:\Downloads\getmac.py”, line 31, in GetAdapters

infos = line.decode('utf-8').split('","')

UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0x84 in position 17: invalid start byte

BTW, how do you get “Umlaute” into these connection names?

In Windows in the Network settings i just renamed one adapter.

···

Am Donnerstag, 17. Juli 2014 11:58:48 UTC+2 schrieb werner:

On 7/17/2014 10:35, Torsten wrote:

6

Hi Torsten,

    Hi Torsten,

    > I tried it with Python 2.7.8 / wxPython 2.9.5.0 msw (classic) and
    > Python 3.4.1 / wxPython 3.0.2.dev76938 msw (phoenix). Neither i
    were
    > able to get the correct result.
    Hhm, can understand it on Py2.7 but on Py3.4 it should work without
    doing much.

    I tried to run your script on Win8.1, Py3.4 and Phoenix, but to
    get it
    to run I had to change it, see attached.

Yes, i also did some changes to make it work with 3.4 (it was my first time using Python 3). Sorry, should have mention that and attach the file.

    It would run forever if I kept 'shell=True'.

    Main change is that in Py3 the proc.stdout.readline() returns
    bytes so
    had to decode.

    I also added '/nh' to the command, to suppress the headers.

When i run your script with Python 3 i get a UnicodeDecodeError:

    Traceback (most recent call last):
      File "D:\Downloads\getmac.py", line 42, in <module>
        win = MainFrame()
      File "D:\Downloads\getmac.py", line 10, in __init__
        adapters = self.GetAdapters()
      File "D:\Downloads\getmac.py", line 31, in GetAdapters
        infos = line.decode('utf-8').split('","')
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x84 in
    position 17: invalid start byte

O.K. after changing my network name I see this too.

So, no idea what stdout uses as encoding, tried a few others such as "ISO-8859-1", "cp1252" but no luck.

Doing a bit of googling I came across:
http://bugs.python.org/issue6135

But can't find if this has been resolved/introduced in Py3.4.

Werner

···

On 7/17/2014 12:15, Torsten wrote:

Am Donnerstag, 17. Juli 2014 11:58:48 UTC+2 schrieb werner:
    On 7/17/2014 10:35, Torsten wrote:

7

Did you try the work around attached to issue 6135 http://bugs.python.org/file28760/subProcessTest.py ?

···

On 17/07/2014 12:11, Werner wrote:

Hi Torsten,

On 7/17/2014 12:15, Torsten wrote:

Am Donnerstag, 17. Juli 2014 11:58:48 UTC+2 schrieb werner:

    Hi Torsten,

    On 7/17/2014 10:35, Torsten wrote:
    > I tried it with Python 2.7.8 / wxPython 2.9.5.0 msw (classic) and
    > Python 3.4.1 / wxPython 3.0.2.dev76938 msw (phoenix). Neither i
    were
    > able to get the correct result.
    Hhm, can understand it on Py2.7 but on Py3.4 it should work without
    doing much.

    I tried to run your script on Win8.1, Py3.4 and Phoenix, but to
    get it
    to run I had to change it, see attached.

Yes, i also did some changes to make it work with 3.4 (it was my first
time using Python 3). Sorry, should have mention that and attach the file.

    It would run forever if I kept 'shell=True'.

    Main change is that in Py3 the proc.stdout.readline() returns
    bytes so
    had to decode.

    I also added '/nh' to the command, to suppress the headers.

When i run your script with Python 3 i get a UnicodeDecodeError:

    Traceback (most recent call last):
      File "D:\Downloads\getmac.py", line 42, in <module>
        win = MainFrame()
      File "D:\Downloads\getmac.py", line 10, in __init__
        adapters = self.GetAdapters()
      File "D:\Downloads\getmac.py", line 31, in GetAdapters
        infos = line.decode('utf-8').split('","')
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x84 in
    position 17: invalid start byte

O.K. after changing my network name I see this too.

So, no idea what stdout uses as encoding, tried a few others such as
"ISO-8859-1", "cp1252" but no luck.

Doing a bit of googling I came across:
http://bugs.python.org/issue6135

But can't find if this has been resolved/introduced in Py3.4.

Werner

--
My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.

8

Hi,

When i run your script with Python 3 i get a UnicodeDecodeError:

    Traceback (most recent call last):
      File "D:\Downloads\getmac.py", line 42, in <module>
        win = MainFrame()
      File "D:\Downloads\getmac.py", line 10, in __init__
        adapters = self.GetAdapters()
      File "D:\Downloads\getmac.py", line 31, in GetAdapters
        infos = line.decode('utf-8').split('","')
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x84 in
    position 17: invalid start byte

O.K. after changing my network name I see this too.

So, no idea what stdout uses as encoding, tried a few others such as
"ISO-8859-1", "cp1252" but no luck.

Doing a bit of googling I came across:
http://bugs.python.org/issue6135

But can't find if this has been resolved/introduced in Py3.4.

Under Windows, you need to decode subprocess input and output with the
console input and output encodings. The only way to reliably get that
(which I know of) is to use Python for Windows extensions (pywin32). E.g.:

    import subprocess
    from encodings.aliases import aliases
    from win32console import GetConsoleOutputCP

    # Get console output encoding
    enc = aliases.get(str(GetConsoleOutputCP()))

    p = subprocess.Popen(r'getmac /V /FO CSV',
                         stdout=sp.PIPE, stderr=sp.PIPE)
    stdout, stderr = [v.decode(enc) for v in p.communicate()]

(shell=True is only necessary for shell commands by the way, e.g. things
like 'dir' etc.)

This works under Python 2.x and 3.x.

If you're using Python 2.x, one caveat applies if you localize your
application (i.e. if you use locale.setlocale and derivatives):

In that case, the encoding of stdio changes, and you need to use the
encoding specific to the locale being used, i.e. locale.getlocale()[1].

···

Am 17.07.2014 13:11, schrieb Werner:

--
Florian Höch

9

Hi Torsten,

...

Under Windows, you need to decode subprocess input and output with the
console input and output encodings. The only way to reliably get that
(which I know of) is to use Python for Windows extensions (pywin32). E.g.:

     import subprocess
     from encodings.aliases import aliases
     from win32console import GetConsoleOutputCP

     # Get console output encoding
     enc = aliases.get(str(GetConsoleOutputCP()))

     p = subprocess.Popen(r'getmac /V /FO CSV',
                          stdout=sp.PIPE, stderr=sp.PIPE)
     stdout, stderr = [v.decode(enc) for v in p.communicate()]

(shell=True is only necessary for shell commands by the way, e.g. things
like 'dir' etc.)

This works under Python 2.x and 3.x.

If you're using Python 2.x, one caveat applies if you localize your
application (i.e. if you use locale.setlocale and derivatives):

In that case, the encoding of stdio changes, and you need to use the
encoding specific to the locale being used, i.e. locale.getlocale()[1].

Adapted your script with Florian's suggestion and that works for me on Py3.4.

On my system the encoding is 'cp850'

getmac.py (1.38 KB)

···

On 7/17/2014 17:29, Florian Höch wrote:

10

One thing I forgot to say, if you run with the windowed python
executable (pythonw.exe) you have to use GetOEMCP instead of
GetConsoleOutputCP because the latter will always return 0 in that case.

Also, you can use ctypes instead of pywin32, e.g.

import ctypes
ctypes.windll.kernel32.GetOEMCP()

···

Am 17.07.2014 21:02, schrieb Werner:

Adapted your script with Florian's suggestion and that works for me on
Py3.4.

On my system the encoding is 'cp850'

--
Florian Höch