Strange problem with python's built-in functions

Hello,

I'm writing here, not '-users' mailing list because this problem seems to be
some kind of bug.

After creating wxFrame object (or another widget), such python built-in
functions as 'eval' and 'float' seems to be broken. For example,
'eval("1.432")', which should produce 1.432, produces 1.0, and float("0.43")
raises ValueError ("invalid literal for float(): 0.43"), but should produce
0.43 float number. This happenѕ only with wxPython 2.5.

I've written an example program to watch what's going on:

   import wx

   print "Good eval:", eval("-5.323")
   print "Good float:", float("-3.324")
   
   class MyFrame(wx.Frame):
      def __init__(self, parent, title):
         wx.Frame.__init__(self, parent, -1, title)
         p = wx.Panel(self, -1)
         b = wx.Button(p, -1, "Do It", (10,10))
         self.Bind(wx.EVT_BUTTON, self.JustDoIt, b)

         print "Bad eval:", eval("-5.323")
         print "Bad float:", float("-3.324")

   app = wx.PySimpleApp()
   f = MyFrame(None, "What's up?")
   f.Show()
   app.MainLoop()

$ python wxpyapp.py
Good eval: -5.323
Good float: -3.324
Bad eval: -5.0
Traceback (most recent call last):
  File "wxpyapp.py", line 18, in ?
    f = MyFrame(None, "What's up?")
  File "wxpyapp.py", line 13, in __init__
    print "Bad float:", float("-3.324")
ValueError: invalid literal for float(): -3.324

I haven't tried another functions, just these. What could you say about that?

Martynas

···

--
The best way of learning is learning by doing.
http://mjoc.sig.lt

Please include the exact versions of python, wxpython and OS

I've tried your code on WinXP python 2.3.3 and wxpython 2.5.1.5u and it works

···

On Sun, 16 May 2004 10:39:01 +0300, Martynas Jocius <mjoc@akl.lt> wrote:

Hello,

I'm writing here, not '-users' mailing list because this problem seems to be
some kind of bug.

After creating wxFrame object (or another widget), such python built-in
functions as 'eval' and 'float' seems to be broken. For example,
'eval("1.432")', which should produce 1.432, produces 1.0, and float("0.43")
raises ValueError ("invalid literal for float(): 0.43"), but should produce
0.43 float number. This happenѕ only with wxPython 2.5.

--
Peter Damoc
Hacker Wannabe
http://www.sigmacore.net/

Please include the exact versions of python, wxpython and OS

I've tried your code on WinXP python 2.3.3 and wxpython 2.5.1.5u and it
works

Python 2.3.3 (#2, Feb 24 2004, 09:29:20)
[GCC 3.3.3 (Debian)] on linux2

>>> import wx
>>> wx.__version__
'2.5.1.1u'

Debian GNU/Linux unstable. wxPython 2.5.1.1 is the newest available version
for it.

Martynas

···

On Sun, May 16, 2004 at 11:04:17AM +0300, Peter Damoc wrote:
--
The best way of learning is learning by doing.
http://mjoc.sig.lt

Martynas Jocius wrote:

Hello,

I'm writing here, not '-users' mailing list because this problem seems to be
some kind of bug.

After creating wxFrame object (or another widget), such python built-in
functions as 'eval' and 'float' seems to be broken. For example,
'eval("1.432")', which should produce 1.432, produces 1.0, and float("0.43")
raises ValueError ("invalid literal for float(): 0.43"), but should produce
0.43 float number. This happenѕ only with wxPython 2.5.

wxGTK changes the locale to match the system's locale, and wxPython's startup code also changes the local so Python will match the C lib. So it is probably being changed to a locale where ',' is the decimal separator instead of '.'.

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

Robin Dunn wrote:

Martynas Jocius wrote:

Hello,

I'm writing here, not '-users' mailing list because this problem seems to be
some kind of bug.

After creating wxFrame object (or another widget), such python built-in
functions as 'eval' and 'float' seems to be broken. For example,
'eval("1.432")', which should produce 1.432, produces 1.0, and float("0.43")
raises ValueError ("invalid literal for float(): 0.43"), but should produce
0.43 float number. This happenѕ only with wxPython 2.5.

wxGTK changes the locale to match the system's locale, and wxPython's startup code also changes the local so Python will match the C lib. So it is probably being changed to a locale where ',' is the decimal separator instead of '.'.

I seem to recall a discussion on python-dev a while back that suggests this is a bad plan if the LC_NUMERIC setting ends up getting changed.

And thanks to Google, here it is, along with the relevant part of the Python docs:
http://mail.python.org/pipermail/python-dev/2003-August/037605.html
http://www.python.org/doc/current/lib/embedding-locale.html

So if wxPython's startup code is changing the LC_NUMERIC locale to something other than "C", correct operation of the Python interpreter does not seem to be guaranteed.

Regards,
Nick.

···

--
Nick Coghlan | Brisbane, Australia
Email: ncoghlan@email.com | Mobile: +61 409 573 268

Nick Coghlan wrote:

I seem to recall a discussion on python-dev a while back that suggests this is a bad plan if the LC_NUMERIC setting ends up getting changed.

And thanks to Google, here it is, along with the relevant part of the Python docs:
[Python-Dev] Be Honest about LC_NUMERIC [REPOST]
http://www.python.org/doc/current/lib/embedding-locale.html

So if wxPython's startup code is changing the LC_NUMERIC locale to something other than "C", correct operation of the Python interpreter does not seem to be guaranteed.

FWIW, the problem actually seems to be with a GTK/Python incompatibility. GTK needs the LC_NUMERIC locale to be set to the system locale to process user input and output correctly, while Python needs it to be set to "C" to process floats internally.

A patch was proposed on python-dev to allow Python to be locale agnostic regarding LC_NUMERIC, but the discussion seemed to fizzle out without any real resolution. I've posted a question to see if things actually went anywhere.

Regards,
Nick.

···

--
Nick Coghlan | Brisbane, Australia
Email: ncoghlan@email.com | Mobile: +61 409 573 268

[Cross-posting to wx-dev as I'm a bit out of my depth here. Please cross-post replies as well so everybody can stay involved in the discussion]

Nick Coghlan wrote:

Nick Coghlan wrote:

I seem to recall a discussion on python-dev a while back that suggests this is a bad plan if the LC_NUMERIC setting ends up getting changed.

And thanks to Google, here it is, along with the relevant part of the Python docs:
[Python-Dev] Be Honest about LC_NUMERIC [REPOST]
http://www.python.org/doc/current/lib/embedding-locale.html

So if wxPython's startup code is changing the LC_NUMERIC locale to something other than "C", correct operation of the Python interpreter does not seem to be guaranteed.

FWIW, the problem actually seems to be with a GTK/Python incompatibility. GTK needs the LC_NUMERIC locale to be set to the system locale to process user input and output correctly, while Python needs it to be set to "C" to process floats internally.

Thanks for the info on this Nick. I don't recall reading that bit in the Python docs about not setting the locale, it may be newer than this change to wxPython. I don't remember the exact test case that prompted me to put the setlocale call into wxPython but basically the issue is that gtk_init() calls the C setlocale(LC_ALL, "") so I assumed that the Python setlocale should be done to match it, but apparently the real problem is that Python relies on LC_NUMERIC being "C".

It looks like it will be possible to (optionally) prevent gtk from changing the locale, but the question is should we do it? Or should we just reset LC_NUMERIC back to "C"ompatibility for wxPython? Anybody know what the real impacts on GTK (1 & 2) either of these options would have? Anybody able and willing to build and help test CVS versions of wxPython with these changes, (hopefully on various system locales)?

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

Robin Dunn wrote:

FWIW, the problem actually seems to be with a GTK/Python incompatibility. GTK needs the LC_NUMERIC locale to be set to the system locale to process user input and output correctly, while Python needs it to be set to "C" to process floats internally.

Thanks for the info on this Nick. I don't recall reading that bit in the Python docs about not setting the locale, it may be newer than this change to wxPython. I don't remember the exact test case that prompted me to put the setlocale call into wxPython but basically the issue is that gtk_init() calls the C setlocale(LC_ALL, "") so I assumed that the Python setlocale should be done to match it, but apparently the real problem is that Python relies on LC_NUMERIC being "C".

It looks like it will be possible to (optionally) prevent gtk from changing the locale, but the question is should we do it? Or should we just reset LC_NUMERIC back to "C"ompatibility for wxPython? Anybody know what the real impacts on GTK (1 & 2) either of these options would have? Anybody able and willing to build and help test CVS versions of wxPython with these changes, (hopefully on various system locales)?

From the original py-dev discussion (which I spent some time going back over), if you are working on a system where the system locale uses C-style formatting for floats (mainly using '.' as a decimal separator), you won't have a compatibility problem when combining GTK with Python.

However, the way I read it, GTK wants the C library LC_NUMERIC locale to be the system locale, so that it gets proper handling of numeric values from the user (e.g. places that use ',' as the decimal separator). If we change that, then GTK's widgets will not honour the user's locale settings for number formatting.

However, Python is using the C library at the same time, and is relying on the LC_NUMERIC locale being "C" (or some other locale with identical formatting rules for floats)

So I don't think it is anything to do with the new call in wxPython - I think it is the call in gtk_init() that breaks things (certainly, it was a GTK related problem that triggered the discussion on python-dev, rather than anything specific to wxPython). But it only breaks for people who have properly configured the locale setting for their Linux box.

The impression I get is that py-dev are going to fix the issue for Python 2.4 (that is, Python will include it's own locale-agnostic float formatting to use internally, and not be reliant on the C libary's locale setting). The change is big enough that I can't see it getting backported to any of the maintenance branches, though.

I've also asked the poster who originally raised the problem on python-dev whether or not he found a way to work around the issue for older versions of Python.

Regards,
Nick.

···

--
Nick Coghlan | Brisbane, Australia
Email: ncoghlan@email.com | Mobile: +61 409 573 268

Nick Coghlan wrote:

Robin Dunn wrote:

It looks like it will be possible to (optionally) prevent gtk from changing the locale, but the question is should we do it? Or should we just reset LC_NUMERIC back to "C"ompatibility for wxPython? Anybody know what the real impacts on GTK (1 & 2) either of these options would have? Anybody able and willing to build and help test CVS versions of wxPython with these changes, (hopefully on various system locales)?

From the original py-dev discussion (which I spent some time going back over), if you are working on a system where the system locale uses C-style formatting for floats (mainly using '.' as a decimal separator), you won't have a compatibility problem when combining GTK with Python.

However, the way I read it, GTK wants the C library LC_NUMERIC locale to be the system locale, so that it gets proper handling of numeric values from the user (e.g. places that use ',' as the decimal separator). If we change that, then GTK's widgets will not honour the user's locale settings for number formatting.

I don't think that wxGTK is using anything from GTK that needs to know anything about decimal separators (please correct me if I'm wrong) so I'm guessing that if wxPython sets the LC_NUMERIC back to "C" after it has initialized wxGTK that there shouldn't be any problems. I'll be checking in that change a little later if anybody wants to build and try it out.

The impression I get is that py-dev are going to fix the issue for Python 2.4 (that is, Python will include it's own locale-agnostic float formatting to use internally, and not be reliant on the C libary's locale setting). The change is big enough that I can't see it getting backported to any of the maintenance branches, though.

Yeah and I need to still support 2.2 and 2.3 for a while so we'll need a fix that doesn't depend on Python version.

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

Martynas Jocius wrote:

I haven't tried another functions, just these. What could you say about that?

Could you tell me what your locale settings (especially LC_NUMERIC) are? I'd like to try to reproduce the problem on Mandrake (to confirm that it is indeed this GTK/Python LC_NUMERIC problem), then CVS up to see if Robin's change addresses it.

Regards,
Nick.

···

--
Nick Coghlan | Brisbane, Australia
Email: ncoghlan@email.com | Mobile: +61 409 573 268