[wxPython] Locale, Mysql and wx...

Hi,

I'm french, and so i use LC_ALL=fr_FR.
Mysql alone works fine,
wxPython alone works fine,

But together i cannot query a field of type float, it throw an exception :
File "/usr/lib/python2.1/site-packages/MySQLdb/cursors.py", line 136, in
_fetch_row return self._result.fetch_row(size, self._fetch_type)
ValueError: invalid literal for float(): 68.00

If I do os.putenv("LC_ALL","C") before, it works fine...

Is-it a bug or i have to do something else ?

thanks

···

--
William Dodé - Informaticien Indépendant
http://www.flibuste.net

It looks like a bug to me, but don't ask me where. The difficulty is clearly
coming from a floating-point value of "68.00", which in your locale should
really be "68,00". Does MySQL actually return "68,00" when operating the the
French locale?

regards
Steve

···

----- Original Message -----
From: "William Dode" <wilk@flibuste.net>
To: <wxpython-users@lists.wxwindows.org>
Sent: Friday, May 03, 2002 11:07 AM
Subject: [wxPython] Locale, Mysql and wx...

Hi,

I'm french, and so i use LC_ALL=fr_FR.
Mysql alone works fine,
wxPython alone works fine,

But together i cannot query a field of type float, it throw an exception :
File "/usr/lib/python2.1/site-packages/MySQLdb/cursors.py", line 136, in
_fetch_row return self._result.fetch_row(size, self._fetch_type)
ValueError: invalid literal for float(): 68.00

If I do os.putenv("LC_ALL","C") before, it works fine...

Is-it a bug or i have to do something else ?

--
Steve Holden: http://www.holdenweb.com/ ; Python Web Programming:
http://pydish.holdenweb.com/pwp/

Le Fri, 3 May 2002 12:22:57 -0400
"Steve Holden" <sholden@holdenweb.com> écrivait:

···

----- Original Message -----
From: "William Dode" <wilk@flibuste.net>
To: <wxpython-users@lists.wxwindows.org>
Sent: Friday, May 03, 2002 11:07 AM
Subject: [wxPython] Locale, Mysql and wx...

> Hi,
>
> I'm french, and so i use LC_ALL=fr_FR.
> Mysql alone works fine,
> wxPython alone works fine,
>
> But together i cannot query a field of type float, it throw an
> exception : File
> "/usr/lib/python2.1/site-packages/MySQLdb/cursors.py", line 136,
> in_fetch_row return self._result.fetch_row(size, self._fetch_type)
> ValueError: invalid literal for float(): 68.00
>
> If I do os.putenv("LC_ALL","C") before, it works fine...
>
> Is-it a bug or i have to do something else ?
>

It looks like a bug to me, but don't ask me where. The difficulty is
clearly coming from a floating-point value of "68.00", which in your
locale should really be "68,00". Does MySQL actually return "68,00" when
operating the the French locale?

No, it return every time 68.00.
But is very strange that it doesn't work only if i import a wxpython
class...

from wxPython.wx import wxPySimpleApp

I would like to know if it's a bug in wxpython on in mysql...

bye

--
William Dodé - Informaticien Indépendant
http://www.flibuste.net

But together i cannot query a field of type float, it throw an exception :
File "/usr/lib/python2.1/site-packages/MySQLdb/cursors.py", line 136, in
_fetch_row return self._result.fetch_row(size, self._fetch_type)
ValueError: invalid literal for float(): 68.00

If I do os.putenv("LC_ALL","C") before, it works fine...

In wxPython/wx.py there is this code that is probably affecting it somehow:

# wxGTK sets the locale when initialized. Doing this at the Python
# level should set it up to match what GTK is doing at the C level.
if wxPlatform == "__WXGTK__":
    try:
        import locale
        locale.setlocale(locale.LC_ALL, "")
    except:
        pass

···

--
Robin Dunn
Software Craftsman
robin@AllDunn.com Java give you jitters?
http://wxPython.org Relax with wxPython!

Le Fri, 3 May 2002 13:11:07 -0700
"Robin Dunn" <robin@alldunn.com> écrivait:

> But together i cannot query a field of type float, it throw an
> exception : File
> "/usr/lib/python2.1/site-packages/MySQLdb/cursors.py", line 136,
> in_fetch_row return self._result.fetch_row(size, self._fetch_type)
> ValueError: invalid literal for float(): 68.00
>
> If I do os.putenv("LC_ALL","C") before, it works fine...

In wxPython/wx.py there is this code that is probably affecting it
somehow:

# wxGTK sets the locale when initialized. Doing this at the Python
# level should set it up to match what GTK is doing at the C level.
if wxPlatform == "__WXGTK__":
    try:
        import locale
        locale.setlocale(locale.LC_ALL, "")
    except:
        pass

I did'nt find this code in my version 2.2.9.2 from debian woody
maybe it's the problem...
But if i write
locale.setlocale(locale.LC_ALL,"C") instead of os.putenv("LC_ALL","C") it
doesn't work...

if somebody could try with the most recent version, it's just to do
os.putenv("LC_ALL","fr_FR") before.

bye

···

--
Robin Dunn
Software Craftsman
robin@AllDunn.com Java give you jitters?
http://wxPython.org Relax with wxPython!

_______________________________________________
wxpython-users mailing list
wxpython-users@lists.wxwindows.org
http://lists.wxwindows.org/mailman/listinfo/wxpython-users

--
William Dodé - Informaticien Indépendant
http://www.flibuste.net

But if i write
locale.setlocale(locale.LC_ALL,"C") instead of os.putenv("LC_ALL","C") it
doesn't work...

But those two things aren't supposed to do nearly the same thing.
See OS and Python manuals for details.

Basically, os.putenv("LC_ALL", "fr_FR") or LC_ALL=fr_FR in a unix
shell will make the OS use the settings for France. This should
affect how the sorting is done if you list directory content with
an alphabetical sorting etc. Various programs might or might not
honour this settting.

Python honours OS locale setting if you do:
locale.setlocale(locale.LC_ALL,"") # Note empty string.

If you don't call locale.setlocale I think it defaults to 'C'.
This might depend on your Python configuration though. I guess
this is because there is a lot of Python code out there that
assumes that all the world is American...

By doing locale.setlocale(locale.LC_ALL,"sv_SE") etc you can make
your python program use another national setting than the OS is
set to, which might be what you want.

if somebody could try with the most recent version, it's just to do
os.putenv("LC_ALL","fr_FR") before.

What if you do both? (They ought to be in synch with each other.)
Or better locale.setlocale(locale.LC_ALL,"") after having set the
OS environment. (BTW, that should normally be set to the right
national setting before you start your program.)

You can use locale.getlocale() to see what your current Python
settings are, and locale.getdefaultlocale() to see what your OS
settings are, i.e. what locale.setlocale(locale.LC_ALL,"") would
set it to.

···

At 11:59 2002-05-04 +0200, William Dode wrote:

--
Magnus Lyckå, Thinkware AB
Älvans väg 99, SE-907 50 UMEÅ
tel: 070-582 80 65, fax: 070-612 80 65
http://www.thinkware.se/ mailto:magnus@thinkware.se

Le Sun, 05 May 2002 16:09:25 +0200
Magnus Lycka <magnus@thinkware.se> écrivait:

>But if i write
>locale.setlocale(locale.LC_ALL,"C") instead of os.putenv("LC_ALL","C")
>it doesn't work...

But those two things aren't supposed to do nearly the same thing.
See OS and Python manuals for details.

Basically, os.putenv("LC_ALL", "fr_FR") or LC_ALL=fr_FR in a unix
shell will make the OS use the settings for France. This should
affect how the sorting is done if you list directory content with
an alphabetical sorting etc. Various programs might or might not
honour this settting.

Python honours OS locale setting if you do:
locale.setlocale(locale.LC_ALL,"") # Note empty string.

If you don't call locale.setlocale I think it defaults to 'C'.
This might depend on your Python configuration though. I guess
this is because there is a lot of Python code out there that
assumes that all the world is American...

By doing locale.setlocale(locale.LC_ALL,"sv_SE") etc you can make
your python program use another national setting than the OS is
set to, which might be what you want.

>if somebody could try with the most recent version, it's just to do
>os.putenv("LC_ALL","fr_FR") before.

What if you do both? (They ought to be in synch with each other.)
Or better locale.setlocale(locale.LC_ALL,"") after having set the
OS environment. (BTW, that should normally be set to the right
national setting before you start your program.)

You can use locale.getlocale() to see what your current Python
settings are, and locale.getdefaultlocale() to see what your OS
settings are, i.e. what locale.setlocale(locale.LC_ALL,"") would
set it to.

I think my configuration is ok, locale.getlocale locale.getdefaultlocale
and env LC_ALL are the same when i start my program. If i do
locale.setlocale(locale.LC_ALL,"") it doesn't change anything (of course).

It can work only if i put env LC_ALL to C before. For me it's not a big
problem to do this, but i would like to know if it's a bug or if it's a
problem of me...

bye

···

At 11:59 2002-05-04 +0200, William Dode wrote:

--
Magnus Lyckå, Thinkware AB
Älvans väg 99, SE-907 50 UMEÅ
tel: 070-582 80 65, fax: 070-612 80 65
http://www.thinkware.se/ mailto:magnus@thinkware.se

_______________________________________________
wxpython-users mailing list
wxpython-users@lists.wxwindows.org
http://lists.wxwindows.org/mailman/listinfo/wxpython-users

--
William Dodé - Informaticien Indépendant
http://www.flibuste.net

I think my configuration is ok, locale.getlocale locale.getdefaultlocale
and env LC_ALL are the same when i start my program. If i do
locale.setlocale(locale.LC_ALL,"") it doesn't change anything (of course).

It can work only if i put env LC_ALL to C before. For me it's not a big
problem to do this, but i would like to know if it's a bug or if it's a
problem of me...

To me it sounds like MySQLdb or MySQL doesn't handle national settings for the
decimal separator, but always expects a ".". I think it's better if you ask
at the db-sig mailinglist. See DB-SIG Info Page

Obviously wxPython gets involved by doing 'locale.setlocale(locale.LC_ALL, "")',
but that's how your locale ought to be set. I can't imagine this has anything
more to do with wxPython. It should be possible to provoke this without
importing wxPython.

In Python, a floating point value can either be converted to a string with
str() which will always give 68.00, or with locale.str() which will honour
national settings. See below:

>>> import locale
>>> f=68.5
>>> locale.setlocale(locale.LC_ALL,'C')
'C'
>>> str(f)
'68.5'
>>> locale.str(f)
'68.5'
>>> locale.setlocale(locale.LC_ALL,'')
'Swedish_Sweden.1252'
>>> str(f)
'68.5'
>>> locale.str(f)
'68,5'
>>>

Format strings are the same--the standard uses only C setting, but
there is a corresponding function in the locale module:

>>> locale.setlocale(locale.LC_ALL,'C')
'C'
>>> locale.format("%f", 68.5)
'68.500000'
>>> "%f" % 68.5
'68.500000'
>>> locale.setlocale(locale.LC_ALL, '')
'Swedish_Sweden.1252'
>>> locale.format("%f", 68.5)
'68,500000'
>>> "%f" % 68.5
'68.500000'

Likewise, the opposite is true, the standard float() only expects
a '.', but locale.atof() expects a national setting (although it
seems to be able to live with both '.' and ',' if locale says ',':

>>> locale.setlocale(locale.LC_ALL,'')
'Swedish_Sweden.1252'
>>> float('68.5')
68.5
>>> float('68,5')
Traceback (most recent call last):
   File "<interactive input>", line 1, in ?
ValueError: invalid literal for float(): 68,5
>>> locale.atof('68.5')
68.5
>>> locale.atof('68,5')
68.5
>>> locale.setlocale(locale.LC_ALL,'C')
'C'
>>> float('68.5')
68.5
>>> float('68,5')
Traceback (most recent call last):
   File "<interactive input>", line 1, in ?
ValueError: invalid literal for float(): 68,5
>>> locale.atof('68.5')
68.5
>>> locale.atof('68,5')
Traceback (most recent call last):
   File "<interactive input>", line 1, in ?
   File "G:\Python21\lib\locale.py", line 175, in atof
     return func(str)
ValueError: invalid literal for float(): 68,5
>>>

At least this is how Python 2.1.1 behaves. Has it changed
for 2.2? (Is that what you use?)

I'm really puzzled by:
> ValueError: invalid literal for float(): 68.00

In my tests standard Python float() always seem to accept '.'

On the other hand, one might wonder why floats are cast to
or from strings at a Python level at all? I assume it would
be possible (well, better) to pass floats as floats from the
MySQL C API to the C code in MySQLdb and then be made into
python floats without going via strings...

Or is this not a result value? Is it part of a query string?

In that case it gets more tricky. In my opinion, 68,00 is an
end user presentation format. In C or Python source code you
obviously write "f = 68.0" regardless of locale. "f = 68,0"
would be a tuple. On the other hand, end user programs like
MS Excel use national settings. So what about an SQL string?
Is "SELECT * FROM TABLE WHERE X = 68.0" correct, or is
"SELECT * FROM TABLE WHERE X = 68,0" right in France and Sweden?
Well, that's a bit off topic here I guess...

···

At 22:27 2002-05-05 +0200, you wrote:

--
Magnus Lyckå, Thinkware AB
Älvans väg 99, SE-907 50 UMEÅ
tel: 070-582 80 65, fax: 070-612 80 65
http://www.thinkware.se/ mailto:magnus@thinkware.se

Le Mon, 06 May 2002 01:46:38 +0200
Magnus Lycka <magnus@thinkware.se> écrivait:

>I think my configuration is ok, locale.getlocale
>locale.getdefaultlocale and env LC_ALL are the same when i start my
>program. If i do locale.setlocale(locale.LC_ALL,"") it doesn't change
>anything (of course).
>
>It can work only if i put env LC_ALL to C before. For me it's not a big
>problem to do this, but i would like to know if it's a bug or if it's a
>problem of me...

To me it sounds like MySQLdb or MySQL doesn't handle national settings
for the decimal separator, but always expects a ".". I think it's better
if you ask at the db-sig mailinglist. See
DB-SIG Info Page

Obviously wxPython gets involved by doing
'locale.setlocale(locale.LC_ALL, "")',
but that's how your locale ought to be set. I can't imagine this has
anything more to do with wxPython. It should be possible to provoke this
without importing wxPython.

I cannot provoke it without importing wxPython... So it's difficult for me
to write about this on the db-sig list (but i will do it)...

If I do (or not) locale.setlocale(locale.LC_ALL,'') it works fine.
look at my example :

import locale
locale.setlocale(locale.LC_ALL,'') # LC_ALL = fr_FR
from wxPython.wx import wxPySimpleApp # if I remove this line it will work
db=MySQLdb.connect(db="flibuste",user="wilk",passwd="xxx",host="blakie",p
ort=3306) cursor=db.cursor()
cursor.execute("select livre.prix from livre") # livre.prix is a float
res=cursor.fetchall()
print res

I use python2.1.3-3

bye

···

At 22:27 2002-05-05 +0200, you wrote:

In Python, a floating point value can either be converted to a string
with str() which will always give 68.00, or with locale.str() which will
honour national settings. See below:

>>> import locale
>>> f=68.5
>>> locale.setlocale(locale.LC_ALL,'C')
'C'
>>> str(f)
'68.5'
>>> locale.str(f)
'68.5'
>>> locale.setlocale(locale.LC_ALL,'')
'Swedish_Sweden.1252'
>>> str(f)
'68.5'
>>> locale.str(f)
'68,5'
>>>

Format strings are the same--the standard uses only C setting, but
there is a corresponding function in the locale module:

>>> locale.setlocale(locale.LC_ALL,'C')
'C'
>>> locale.format("%f", 68.5)
'68.500000'
>>> "%f" % 68.5
'68.500000'
>>> locale.setlocale(locale.LC_ALL, '')
'Swedish_Sweden.1252'
>>> locale.format("%f", 68.5)
'68,500000'
>>> "%f" % 68.5
'68.500000'

Likewise, the opposite is true, the standard float() only expects
a '.', but locale.atof() expects a national setting (although it
seems to be able to live with both '.' and ',' if locale says ',':

>>> locale.setlocale(locale.LC_ALL,'')
'Swedish_Sweden.1252'
>>> float('68.5')
68.5
>>> float('68,5')
Traceback (most recent call last):
   File "<interactive input>", line 1, in ?
ValueError: invalid literal for float(): 68,5
>>> locale.atof('68.5')
68.5
>>> locale.atof('68,5')
68.5
>>> locale.setlocale(locale.LC_ALL,'C')
'C'
>>> float('68.5')
68.5
>>> float('68,5')
Traceback (most recent call last):
   File "<interactive input>", line 1, in ?
ValueError: invalid literal for float(): 68,5
>>> locale.atof('68.5')
68.5
>>> locale.atof('68,5')
Traceback (most recent call last):
   File "<interactive input>", line 1, in ?
   File "G:\Python21\lib\locale.py", line 175, in atof
     return func(str)
ValueError: invalid literal for float(): 68,5
>>>

At least this is how Python 2.1.1 behaves. Has it changed
for 2.2? (Is that what you use?)

I'm really puzzled by:
> ValueError: invalid literal for float(): 68.00

In my tests standard Python float() always seem to accept '.'

On the other hand, one might wonder why floats are cast to
or from strings at a Python level at all? I assume it would
be possible (well, better) to pass floats as floats from the
MySQL C API to the C code in MySQLdb and then be made into
python floats without going via strings...

Or is this not a result value? Is it part of a query string?

In that case it gets more tricky. In my opinion, 68,00 is an
end user presentation format. In C or Python source code you
obviously write "f = 68.0" regardless of locale. "f = 68,0"
would be a tuple. On the other hand, end user programs like
MS Excel use national settings. So what about an SQL string?
Is "SELECT * FROM TABLE WHERE X = 68.0" correct, or is
"SELECT * FROM TABLE WHERE X = 68,0" right in France and Sweden?
Well, that's a bit off topic here I guess...

--
Magnus Lyckå, Thinkware AB
Älvans väg 99, SE-907 50 UMEÅ
tel: 070-582 80 65, fax: 070-612 80 65
http://www.thinkware.se/ mailto:magnus@thinkware.se

_______________________________________________
wxpython-users mailing list
wxpython-users@lists.wxwindows.org
http://lists.wxwindows.org/mailman/listinfo/wxpython-users

--
William Dodé - Informaticien Indépendant
http://www.flibuste.net