Hello,
Since I updated my linux box to the latest Mageia version (Mageia 5), one of the program I developed with wxpython is broken, and the cause seems very strange to me.
Shortly, I’m feeding a ListCtrl with unicode strings and, when these strings contains special chars (like a french é), the list box insert a blank line instead.
A reproduced the bug in a short program (joined file).
Note that there is a commented line
wx.SetDefaultPyEncoding(‘utf8’)
that I tried to use, but the bug is the same with or without this line…
my version of python is
Python 2.7.9 (default, Dec 14 2014, 10:12:16)
[GCC 4.9.2] on linux2
import wx
wx.VERSION
(3, 0, 1, 1, ‘’)
wx.GetDefaultPyEncoding()
‘ISO-8859-1’
and my Mageia system is configured to “french / france”
The same program used to work fine with me with the last Mageia release…
Does anybody can give me a clue on what I’m doing wrong ?
Many thanks in advance.
D. Guez
test_utf8_wxpython.py (1.01 KB)
The only way I got this working with Py2.x is using a bit of a hack, which is using "sys.setdefaultencoding('utf-8')", see the enclosed changed file.
Maybe you had used sitecustomize.py before?
Werner
test_utf8_wxpython.py (1.6 KB)
···
On 6/26/2015 10:47, david guez wrote:
Hello,
Since I updated my linux box to the latest Mageia version (Mageia 5), one of the program I developed with wxpython is broken, and the cause seems very strange to me.
Shortly, I'm feeding a ListCtrl with unicode strings and, when these strings contains special chars (like a french é), the list box insert a blank line instead.
A reproduced the bug in a short program (joined file).
Note that there is a commented line
# wx.SetDefaultPyEncoding('utf8')
that I tried to use, but the bug is the same with or without this line...
my version of python is
Python 2.7.9 (default, Dec 14 2014, 10:12:16)
[GCC 4.9.2] on linux2
>>> import wx
>>> wx.VERSION
(3, 0, 1, 1, '')
>>> wx.GetDefaultPyEncoding()
'ISO-8859-1'
and my Mageia system is configured to "french / france"
The same program used to work fine with me with the last Mageia release....
Does anybody can give me a clue on what I'm doing wrong ?
Many thanks in advance.
Thank you Werner… I tried it and it does not help, nor adding the lines
import sys
sys.setdefaultencoding(“utf-8”)import sys
sys.setdefaultencoding(“utf-8”)
to the python27\lib\site-packages\sitecustomize.py file…
I’m a bit lost!
Thank you anyway…
Cheers,
D. Guez
···
Le vendredi 26 juin 2015 17:23:58 UTC+3, werner a écrit :
On 6/26/2015 10:47, david guez wrote:
Hello,
Since I updated my linux box to the latest Mageia version (Mageia 5),
one of the program I developed with wxpython is broken, and the cause
seems very strange to me.
Shortly, I’m feeding a ListCtrl with unicode strings and, when these
strings contains special chars (like a french é), the list box insert
a blank line instead.
A reproduced the bug in a short program (joined file).
Note that there is a commented line
wx.SetDefaultPyEncoding(‘utf8’)
that I tried to use, but the bug is the same with or without this line…
my version of python is
Python 2.7.9 (default, Dec 14 2014, 10:12:16)
[GCC 4.9.2] on linux2
import wx
wx.VERSION
(3, 0, 1, 1, ‘’)
wx.GetDefaultPyEncoding()
‘ISO-8859-1’
and my Mageia system is configured to “french / france”
The same program used to work fine with me with the last Mageia
release…
Does anybody can give me a clue on what I’m doing wrong ?
Many thanks in advance.
The only way I got this working with Py2.x is using a bit of a hack,
which is using “sys.setdefaultencoding(‘utf-8’)”, see the enclosed
changed file.
Maybe you had used sitecustomize.py before?
Werner
Hi,
sorry, if I am missing something obvious, but I don't understand the
purpose of the explici decoding at line 18
r=row.decode('utf8')
Normally the communication of text content within the gui, I'd expect
to just pass the unicode objects directly.
If I disable this by using r=row instead and suply the expected
unicode literals for python 2 u"..." at lines 31-33, especially
self.frame.add_line(u'test éé')
everything seems to be working, but this is on windows 7, I
understand, there might be platform differences.
hth,
vbr
···
2015-06-26 10:47 GMT+02:00 david guez <guezdav@gmail.com>:
Hello,
Since I updated my linux box to the latest Mageia version (Mageia 5), one of
the program I developed with wxpython is broken, and the cause seems very
strange to me.
Shortly, I'm feeding a ListCtrl with unicode strings and, when these strings
contains special chars (like a french é), the list box insert a blank line
instead.
A reproduced the bug in a short program (joined file).
Note that there is a commented line
# wx.SetDefaultPyEncoding('utf8')
that I tried to use, but the bug is the same with or without this line...
my version of python is
Python 2.7.9 (default, Dec 14 2014, 10:12:16)
[GCC 4.9.2] on linux2
import wx
wx.VERSION
(3, 0, 1, 1, '')
wx.GetDefaultPyEncoding()
'ISO-8859-1'
and my Mageia system is configured to "french / france"
The same program used to work fine with me with the last Mageia release....
Does anybody can give me a clue on what I'm doing wrong ?
Many thanks in advance.
D. Guez
--
Hi,
Actually, the real code get the data from a sqlite database and the result of the query command can be str or unicode according to the string content. This is the reason why I explicitly changed the format to utf8.
But I tried a modified version ( r=row instead of r=row.decode(‘utf8’) and self.frame.add_line(u’test éé ') and the problem is the same.
Do you mean that for you, the first version of my code reproduce the bug where the second does not ? If so, it could give me a clue on what’s happen…
Thank you for your help,
david
···
Le dimanche 28 juin 2015 16:54:10 UTC+3, vbr a écrit :
2015-06-26 10:47 GMT+02:00 david guez gue...@gmail.com:
Hello,
Since I updated my linux box to the latest Mageia version (Mageia 5), one of
the program I developed with wxpython is broken, and the cause seems very
strange to me.
Shortly, I’m feeding a ListCtrl with unicode strings and, when these strings
contains special chars (like a french é), the list box insert a blank line
instead.
A reproduced the bug in a short program (joined file).
Note that there is a commented line
wx.SetDefaultPyEncoding(‘utf8’)
that I tried to use, but the bug is the same with or without this line…
my version of python is
Python 2.7.9 (default, Dec 14 2014, 10:12:16)
[GCC 4.9.2] on linux2
import wx
wx.VERSION
(3, 0, 1, 1, ‘’)
wx.GetDefaultPyEncoding()
‘ISO-8859-1’
and my Mageia system is configured to “french / france”
The same program used to work fine with me with the last Mageia release…
Does anybody can give me a clue on what I’m doing wrong ?
Many thanks in advance.
D. Guez
–
Hi,
sorry, if I am missing something obvious, but I don’t understand the
purpose of the explici decoding at line 18
r=row.decode(‘utf8’)
Normally the communication of text content within the gui, I’d expect
to just pass the unicode objects directly.
If I disable this by using r=row instead and suply the expected
unicode literals for python 2 u"…" at lines 31-33, especially
self.frame.add_line(u'test éé')
everything seems to be working, but this is on windows 7, I
understand, there might be platform differences.
hth,
vbr
Hi,
sorry for the confusing notice - no, both versions work for me (they
appear to be equivalent), I just thought, that is would be more
convenient to work with unicode objects rather than encoded strings,
but your notice about a db as the original data source explains it.
I hope, someone will have a better explanation of the problem.
regards,
vbr
···
2015-06-28 22:28 GMT+02:00 david guez <guezdav@gmail.com>:
Hi,
Actually, the real code get the data from a sqlite database and the result
of the query command can be str or unicode according to the string content.
This is the reason why I explicitly changed the format to utf8.
But I tried a modified version ( r=row instead of r=row.decode('utf8') and
self.frame.add_line(u'test éé ') and the problem is the same.
Do you mean that for you, the first version of my code reproduce the bug
where the second does not ? If so, it could give me a clue on what's
happen...
Thank you for your help,
david
Le dimanche 28 juin 2015 16:54:10 UTC+3, vbr a écrit :
2015-06-26 10:47 GMT+02:00 david guez <gue...@gmail.com>:
> Hello,
> Since I updated my linux box to the latest Mageia version (Mageia 5),
> one of
> the program I developed with wxpython is broken, and the cause seems
> very
> strange to me.
>
> Shortly, I'm feeding a ListCtrl with unicode strings and, when these
> strings
> contains special chars (like a french é), the list box insert a blank
> line
> instead.
> A reproduced the bug in a short program (joined file).
> Note that there is a commented line
> # wx.SetDefaultPyEncoding('utf8')
> that I tried to use, but the bug is the same with or without this
> line...
>
> my version of python is
> Python 2.7.9 (default, Dec 14 2014, 10:12:16)
> [GCC 4.9.2] on linux2
>
>>>> import wx
>>>> wx.VERSION
> (3, 0, 1, 1, '')
>
>>>> wx.GetDefaultPyEncoding()
> 'ISO-8859-1'
>
> and my Mageia system is configured to "french / france"
>
> The same program used to work fine with me with the last Mageia
> release....
>
> Does anybody can give me a clue on what I'm doing wrong ?
>
> Many thanks in advance.
>
> D. Guez
>
> --
Hi,
sorry, if I am missing something obvious, but I don't understand the
purpose of the explici decoding at line 18
r=row.decode('utf8')
Normally the communication of text content within the gui, I'd expect
to just pass the unicode objects directly.
If I disable this by using r=row instead and suply the expected
unicode literals for python 2 u"..." at lines 31-33, especially
self.frame.add_line(u'test éé')
everything seems to be working, but this is on windows 7, I
understand, there might be platform differences.
hth,
vbr
--
Hi,
Thank you Werner... I tried it and it does not help, nor adding the lines
import sys
sys.setdefaultencoding("utf-8")import sys
sys.setdefaultencoding("utf-8")
to the python27\lib\site-packages\sitecustomize.py file...
I'm a bit lost!
Thank you anyway...
I just tried again the changed sample I attached last time and it does work for me on Windows 7 with Py2.7 and either wxPython 2.8.12.1 Unicode or wxPython 3.x Phoenix.
I can make it work either with the setdefaultencoding trick or by adapting the change Vlastimil suggested (not using decode line).
Did you check that you use a Unicode build of wxPython? As of 2.9 there are only Unicode builds, but with 2.8 there where ANSI and Unicode builds.
How are you getting the data out of sqlite? Directly or are you using e.g. SQLAlchemy?
Werner
···
On 6/28/2015 0:33, david guez wrote:
Hi,
Tell, my wx version is 3.0.0.1, so I guess it should be a unicode version, and I’m getting the data directly from the sqlite database, which returns me str or unicode data according to the string content.
Does the unchanged version (those I sent) fails as it does for me ?
Anyway, the line
print type(r), r , "=>",itm.GetText()
shows me that I’m feeding the ListCtrl with unicode string which is well formated but not well taken into account by ListIm.
I realize that I didn’t sent the output of my program. As you can see on the console windows, the second line is feed with well formatted data (unicode string, well printed on the screen) but is emptied when passing through the SetText method
(see the output line in the console window :
<type ‘unicode’> test éé =>
showing that the r variable is a unicode string corresponding to the “test éé” value but, after I gave it to the ListItm, the result is an empty sring.
···
Le mardi 30 juin 2015 15:55:58 UTC+3, werner a écrit :
Hi,
I just tried again the changed sample I attached last time and it does
work for me on Windows 7 with Py2.7 and either wxPython 2.8.12.1 Unicode
or wxPython 3.x Phoenix.
I can make it work either with the setdefaultencoding trick or by
adapting the change Vlastimil suggested (not using decode line).
Did you check that you use a Unicode build of wxPython? As of 2.9 there
are only Unicode builds, but with 2.8 there where ANSI and Unicode builds.
How are you getting the data out of sqlite? Directly or are you using
e.g. SQLAlchemy?
Werner
Just tried your original version and it works for me too with
wxPython 2.9.5 on Win7 and on Linux Mint 17 with wxPython 2.8.12
Unicode.
Note that your original version does not feed unicode but it feeds
strings, see lines 31-33, to the listctrl.
Are you sure you are using wxPython 3.x?
What about with the enclosed version?
Werner
test_utf8_wxpython.py (1.08 KB)
···
On 6/30/2015 15:24, david guez wrote:
Hi,
Tell, my wx version is 3.0.0.1, so I
guess it should be a unicode version, and I’m getting the data
directly from the sqlite database, which returns me str or
unicode data according to the string content.
Does the unchanged version (those I
sent) fails as it does for me ?
Hi Werner,
that’s right, the line 31-33 get str or unicode data, but this corresponds to the fact that sqlite returns me str / unicode content. The thing is that I’m ensuring that I’m feeding the ListCtrl with unicode content via the line
r=row.decode(‘utf8’)
and I’m sure that I do this okay because the output of my code for the print type(r) is indeed .
I’m also confident that the french page of the utf8 charset is present on my system since I can print the é char on a terminal configured with the utf8 encoding.
I tried your version of the code which also gives me white lines (see the attached screenshot).
thx…
D. Guez
···
Le mardi 30 juin 2015 17:33:53 UTC+3, werner a écrit :
Just tried your original version and it works for me too with
wxPython 2.9.5 on Win7 and on Linux Mint 17 with wxPython 2.8.12
Unicode.
Note that your original version does not feed unicode but it feeds
strings, see lines 31-33, to the listctrl.
Are you sure you are using wxPython 3.x?
What about with the enclosed version?
Werner
Hi,
Hi Werner,
that's right, the line 31-33 get str or unicode data, but this corresponds to the fact that sqlite returns me str / unicode content. The thing is that I'm ensuring that I'm feeding the ListCtrl with unicode content via the line
r=row.decode('utf8')
and I'm sure that I do this okay because the output of my code for the print type(r) is indeed <unicode>.
I'm also confident that the french page of the utf8 charset is present on my system since I can print the é char on a terminal configured with the utf8 encoding.
I tried your version of the code which also gives me white lines (see the attached screenshot).
Only difference I can see is with your 'locale', I have "en_US.UTF-8" but you have "fr_FR".
What do you get when you add:
print(wx.GetDefaultPyEncoding())
print(sys.getdefaultencoding())
On my Mint I get:
UTF-8
ascii
Werner
···
On 6/30/2015 22:54, david guez wrote:
hi,
same as you… many thanks for all the time you give to me by the way 
[david@localhost ~]$ python
Python 2.7.9 (default, Dec 14 2014, 10:12:16)
[GCC 4.9.2] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.
import wx,sys
print(wx.GetDefaultPyEncoding())
UTF-8
print(sys.getdefaultencoding())
ascii
···
2015-07-01 9:22 GMT+03:00 Werner wernerfbd@gmx.ch:
Hi Werner,
that’s right, the line 31-33 get str or unicode data, but this corresponds to the fact that sqlite returns me str / unicode content. The thing is that I’m ensuring that I’m feeding the ListCtrl with unicode content via the line
r=row.decode(‘utf8’)
and I’m sure that I do this okay because the output of my code for the print type(r) is indeed .
I’m also confident that the french page of the utf8 charset is present on my system since I can print the é char on a terminal configured with the utf8 encoding.
I tried your version of the code which also gives me white lines (see the attached screenshot).
Hi,
On 6/30/2015 22:54, david guez wrote:
Only difference I can see is with your ‘locale’, I have “en_US.UTF-8” but you have “fr_FR”.
What do you get when you add:
print(wx.GetDefaultPyEncoding())
print(sys.getdefaultencoding())
On my Mint I get:
UTF-8
ascii
Werner
–
You received this message because you are subscribed to a topic in the Google Groups “wxPython-users” group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wxpython-users/56SMg8t4mMA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wxpython-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
–
| bye ! |
David GUEZ guezdav@gmail.com
It is really strange, for me, that your ‘print’ line shows the
correct text for the variable ‘r’ but shows blank for
‘itm.GetText()’, if it is encoding issue it would normally show some
strange symbols.
Just to be sure, when you run the program you don’t get any error?
Werner
···
Hi,
On 7/1/2015 22:54, david guez wrote:
hi,
same as you... many
thanks for all the time you give to me by the way
[david@localhost ~]$ python
Python 2.7.9 (default, Dec 14 2014,
10:12:16)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits"
or “license” for more information.
import wx,sys
>>>
print(wx.GetDefaultPyEncoding())
UTF-8
>>>
print(sys.getdefaultencoding())
ascii
Actually, I think the process is that one :
1 - I get in r a well encoded utf8 string.
2 - then I transfer it to the ListItem via the SetText method
3 - the SetText method for an unknown reason try to convert or to validate the string and fails to do it, so it set actually the text to nothing
4 - I’m retrieving the text that the SetText method actually set, which is empty because of the validation above mentioned
my problem is : does this conversion / validation fails because my input is not the expected one or is it a bug of wxpython… I guess this is a bug but very hard to reproduce in other systems
···
It is really strange, for me, that your 'print' line shows the
correct text for the variable ‘r’ but shows blank for
‘itm.GetText()’, if it is encoding issue it would normally show some
strange symbols.
Just to be sure, when you run the program you don't get any error?
Werner
Just for a test, can you try this:
if isinstance(row, str):
r=row.decode('utf8')
else:
r = row
self.lbDocuments.InsertStringItem(0, r)
I.e. not using ListItem, to see if it is a ListItem issue or ...?
Werner
P.S.
Could you also try using wxPython Phoenix?
http://wiki.wxpython.org/How%20to%20install%20wxPython#Installing_wxPython-Phoenix_using_pip
···
On 7/2/2015 13:34, david guez wrote:
Actually, I think the process is that one :
1 - I get in r a well encoded utf8 string.
2 - then I transfer it to the ListItem via the SetText method
3 - the SetText method for an unknown reason try to convert or to validate the string and fails to do it, so it set actually the text to nothing
4 - I'm retrieving the text that the SetText method actually set, which is empty because of the validation above mentioned
my problem is : does this conversion / validation fails because my input is not the expected one or is it a bug of wxpython.... I guess this is a bug but very hard to reproduce in other systems
