Hang in wx.App.__init__ in Spyder5.0.0dev

wxPython Users
#1

I am trying to track down a problem using wx inside the Spyder debugger on MacOS 10.15. What I am finding is that if I invoke an external pythonw interpreter created with miniconda that contains a number of packages including matplotlib, then this two-line program is sufficient to hang the debugger:

import wx
application = wx.App(0)

If I remove matplotlib from the installation (conda remove matplotlib), then everything works as it should. This does not mean that matplotlib itself is causing the problem, since conda adds installs about two dozen other packages with matplotlib and changes versions of a few others, so trying to track down exactly where the conflict is found will not be so easy.

I wanted to see where the hang is within wx.App.__init__ and have tracked it down to the self._BootstrapApp() call. That gets started, but never returns. Am I correct that self._BootstrapApp() is a wrapped wxwidgets routine? I can’t find any other references to it.

Anyone reading this have any suggestions on tracking this down? I would really like to be able to debug wx code in Spyder, but without matplotlib and its friends, that is going to be tough.

#2

If you use a derived wx.App class with a OnInit method does it get called?

_BootstrapApp is a wxPython-specific function that adapts the usual wxWidgets startup sequence so it can be used from Python. It is located here:

The method attempts to acquire the GIL as needed for dealing with the sys.argv values, so if Spyder’s debugger or something has also acquired the GIL then it could be a deadlock.

However, when you add in the correlation of the problem with whether matplotlib is installed then it becomes much less clear. Perhaps Spyder is preloading matplotlib in the debug environment if it is installed, or something like that? If so then it could be that matplotlib is creating its own wx.App and it’s conflicting with the wx.App you create? Try printing the value returned by wx.App.Get() before you create your wx.App to see if that might be the case. If so, then see if Spyder has an option to turn that off.

Those are the only ideas I have at the moment, but I’ll give it some more thought.

#3

One more idea. Unlike the other platforms wxWidgets on OSX must be run in the main thread, so that is something else to check. I wouldn’t expect something like Spyder’s debugger to run the debuggee in a non-main thread, but it’s possible I guess.

assert threading.current_thread() is threading.main_thread()
#4

Great suggestions. I would not have come up with them. Thanks Robin.

So far I have determined the following:

  1. The hang occurs before OnInit is called;
  2. There is no wx.App created prior to starting the code
  3. The code seems to be running in the main frame

Any other ideas? Seems that my next step is going to have to be to compile a copy of wx locally to see exactly what is happening inside wxPyApp::_BootstrapApp() (but not today).

I am confused on one thing: how could it even be called if something else holds the GIL?

Brian

FWIW, my test code w/output follows:

import wx
import threading
class MyApp(wx.App):
    def OnInit(self, *args, **kwargs):
        print('OnInit',threading.current_thread(), threading.main_thread())
        return True
print('Before init, prev App=',wx.App.Get())
assert threading.current_thread() is threading.main_thread()
application = MyApp(0) # create the GUI framework
frame = wx.Frame(None)
wx.StaticText(frame,wx.ID_ANY,"hello world")
frame.Show(True)
application.MainLoop()
print('done')

in debugger:

IPdb [1]: !continue
> /Users/toby/Scratch/spy_wx_bug/testwx1.py(15)<module>()
     13         print('OnInit',threading.current_thread(), threading.main_thread())
     14         return True
3--> 15 print('Before init, prev App=',wx.App.Get())
     16 assert threading.current_thread() is threading.main_thread()
     17 application = MyApp(0) # create the GUI framework

IPdb [2]: !next
Before init, prev App= None

IPdb [2]: !next

IPdb [2]: !next (calling MyApp, hangs)

in debugger after removing matplotlib from Python packages:

IPdb [1]: !continue
> /Users/toby/Scratch/spy_wx_bug/testwx1.py(15)<module>()
     13         print('OnInit',threading.current_thread(), threading.main_thread())
     14         return True
3--> 15 print('Before init, prev App=',wx.App.Get())
     16 assert threading.current_thread() is threading.main_thread()
     17 application = MyApp(0) # create the GUI framework


IPdb [2]: !next
Before init, prev App= None

IPdb [2]: !next

IPdb [2]: !next
OnInit <_MainThread(MainThread, started 4527955392)> <_MainThread(MainThread, started 4527955392)>

IPdb [2]: !next

IPdb [2]: !next

IPdb [2]: !next

IPdb [2]: !next (Frame appears)
#5

Ordinarily it wouldn’t be able to. However the thought crossed my mind that if Spyder, or its debugger at least, is exerting a lot of control over the runtime environment then it could notice that control had moved into extension module code and is doing something in the background while it waits for it to return. Probably a very long longshot, but the thought occurred to me so I went ahead and mentioned the idea.

Here are some more ideas. It’s been a very long time since I used Spyder so my memory is pretty fuzzy and some of these may make no sense:

  • The debug session you showed above reminded me that Spyder’s debugger is really just a little extra functionality on top of IPython’s ipdb debugger. At least it was the last time I looked at it. Does Spyder have the option to debug code using a plain python runtime without going through the built-in ipython interactive console? Or at least to start a completely clean environment/console when debugging? If so then try that. If not then I expect you may be running into something like Spyder/IPython preloading the matplotlib magics or similar and putting things into a state that is not expected when wxWidgets is initialized (again?) from your own code. If there’s no way to do that then I think the best choice would be to switch to a new Python debugger that can run the debugee in a mostly[1] isolated environment, like VSCode, PyCharm, or similar.

  • It might be worth looking at sys.modules before the hang to see what might have been helpfully imported by Spyder without your knowledge.

  • Do you have a PYTHONSTARTUP setting in your environment, or does Spyder have some equivalent feature where some code is always executed before starting up a new interactive session? If so then look at that and see if it’s contributing to the conflict somehow.

[1] Other than whatever shim is needed for the debugger to talk to the debuggee.

#6

Or if you want something more lightweight and simple, (just install and go) it looks like the new winpdb is functional with Python3 and wxPython4 again. Currently you’ll want to get it from the master branch on github, not pypi.

#7

Problem solved: more obvious than I realized – in retrospect. Spyder has a backend that can intercept plots and send them to a window (like Jupyter). If MPL is present, the debugger’s interpreter must be importing it before starting code to be debugged.

Fix: the hang can be prevented by setting the Graphics setting to either “Inline” or set “activate support” to off in Preferences/iPython console.

@Robin the newest (pre-release) Spyder is quite impressive. I have not fully switched over from Emacs quite yet, but I really like the Outline code view and the “Online help” (which grabs all the doc strings it can find).

#8

Thanks for the update Toby. I figured there was something like that going on.