Hi,
as I mentioned specifically in my message:
The presence or absence of wx.SafeYield() makes no difference to the outcome.
I get an access violation even if I remove it. And I have 2 or 3 wx.CallAfter in the process, not 30. Not to mention that all of this did not crash in wxPython Classic.
I may understand a wx.wxAssertionError or RuntimeError, what is inexplicable is the whole Python process coming down because of this.
Do you have a SCCCE that reproduces the problem?
Missed that. Long post.
Your traceback does show recursion, though, and that’s a classic for triggering messy program-crashing C/C++ bugs. I’d get rid of that before anything else.
Your traceback shows core.py:34127 and Pulse thrice. The explanation that comes most readily to mind is that you have multiple CallAfter events pending, and somehow you ended up processing the next CallAfter recursively. You may have another explanation.
Hi,
I can try and put together one later. Hopefully I’ll be able to reproduce the issue.
As a side note, I have tried to put a guard inside the overridden Pulse method, like “just return and don’t do anything if less than 0.5 seconds have passed since your last Pulse()”.
While this helps a bit with the problem by delaying it, after a certain number of simulations (higher than without the guard) it still crashes with the same access violation error.
I think wx.ProgressDialog.Pulse is the culprit.
src\generic\progdlgg.cpp:
bool wxGenericProgressDialog::Pulse(const wxString& newmsg, bool *skip)
{
if ( !DoBeforeUpdate(skip) )
return false;
...
}
bool wxGenericProgressDialog::DoBeforeUpdate(bool *skip)
{
// we have to yield because not only we want to update the display but
// also to process the clicks on the cancel and skip buttons
// NOTE: using YieldFor() this call shouldn't give re-entrancy problems
// for event handlers not interested to UI/user-input events.
wxEventLoopBase::GetActive()->YieldFor(wxEVT_CATEGORY_UI|wxEVT_CATEGORY_USER_INPUT);
...
}
I bet the YieldFor is implicated in your problem. It’s there to collect keypresses/clicks when in the middle of a long-running event handler, but if you’re not actually using the progress dialog in a long-running event handler, then perhaps you should go around it.
If you replace the wx.ProgressDialog with a wx.Gauge on a regular frame, then you can just call Pulse on the gauge directly, without incurring any weird event-loopery.
Thank you for your answer. I seem to remember though that by default the Windows wx.ProgressDialog is not based on the generic version but it uses the native widget.
I do not know if that piece of code is shared between the implementations - usually the shared sources live in the common/ directory - but I’ll take a look.
I just wonder how the CallAfter gets in here? or am I on the wrong planet 
from threading import Thread
from time import sleep
import wx
class Gui(wx.Frame):
def __init__(self, parent):
print(f'{self.__class__} __init__')
super().__init__(parent, title='ProgressDialog')
wx.Button(self, label='left click me, please')
self.Bind(wx.EVT_BUTTON, lambda _: Action().start())
self.Centre()
self.Show()
Action().start()
class Action(Thread):
def __init__(self):
super().__init__()
self.dlg = wx.ProgressDialog(
'from thread', 'start',
style=wx.PD_CAN_ABORT|wx.PD_AUTO_HIDE)
# add 'wx.PD_APP_MODAL' for modal
self.dlg.Show()
def run(self):
pls = 1
while not self.dlg.WasCancelled():
sleep(2)
self.dlg.Pulse(newmsg=f'pulse {str(pls)}')
pls += 1
app = wx.App()
Gui(None)
app.MainLoop()
My understanding is that all UI interactions should happen in the main thread - I.e. the one that creates the application top window and the one where the app lives.
I am not sure how and why your example does not bomb out altogether - although I haven’t tried it myself - but we have always been told that calling UI things from other threads was a no-no to start with.
So am I, Andrea, and successfully.
During my porting Python 2 to 3 and Classic to Phoenix up to 4.2 (and lower versions), I encountered many wx related bugs. Not a fatal access violation, though.
So FWIW, I’m currently running a very stable combination, for me at least:
I wonder:
Cheers, Thom
I also believe that you should only update the UI from the main thread. However, instead of using using wx.CallAfter() I just send events using AddPendingEvent(). I have used this technique in several applications, in some of which the child thread can quickly generate a lot of events containing progress messages. The UI typically displays the messages in an STC and I haven’t yet had any problems with it keeping up with the child thread. Note: I have only tested this on Linux.
The code below shows a much simplified example which uses two custom events and an operation class derived from Thread. The class’s run() method sends the appropriate events. The operation class also contains a Queue object which provides a thread safe mechanism to send messages from the main thread to the child thread. I use the Queue object as a way to abort the child thread if the Cancel button on the ProgressDialog is pressed.
from threading import Thread
from queue import Queue, Empty
from time import sleep
import wx
ABORT = "abort"
evt_op_progress_type = wx.NewEventType()
EVT_OP_PROGRESS = wx.PyEventBinder(evt_op_progress_type, 1)
class OpProgressEvent(wx.PyCommandEvent):
def __init__(self, count):
wx.PyCommandEvent.__init__(self, evt_op_progress_type, 1)
self.count = count
evt_op_ended_type = wx.NewEventType()
EVT_OP_ENDED = wx.PyEventBinder(evt_op_ended_type, 1)
class OpEndedEvent(wx.PyCommandEvent):
def __init__(self, status):
wx.PyCommandEvent.__init__(self, evt_op_ended_type, 1)
self.status = status
class MyOperation(Thread):
def __init__(self, limit):
Thread.__init__(self)
self.wx_app = wx.GetApp()
self.queue = Queue()
self.limit = limit
def checkQueue(self):
try:
item = self.queue.get_nowait()
except Empty:
item = ""
return item
def run(self):
count = 0
while True:
if self.checkQueue() == ABORT:
evt = OpEndedEvent("aborted")
self.wx_app.AddPendingEvent(evt)
return
count += 1
evt = OpProgressEvent(count)
self.wx_app.AddPendingEvent(evt)
if count > self.limit:
evt = OpEndedEvent("completed")
self.wx_app.AddPendingEvent(evt)
return
sleep(.01)
class MyFrame(wx.Frame):
def __init__(self, parent):
wx.Frame.__init__(self, parent, title='Test ProgressDialog')
sizer = wx.BoxSizer(wx.VERTICAL)
self.start_button = wx.Button(self, label='Start')
sizer.Add(self.start_button)
self.Bind(wx.EVT_BUTTON, self.OnStart, self.start_button)
self.wx_app = wx.GetApp()
self.wx_app.Bind(EVT_OP_PROGRESS, self.OnProgress)
self.wx_app.Bind(EVT_OP_ENDED, self.OnOpEnded)
self.SetSizer(sizer)
self.Layout()
self.Center()
self.Show()
def OnOpEnded(self, evt):
self.thread.join()
self.dlg.Destroy()
print("Status = %s" % evt.status)
self.start_button.Enable()
def OnProgress(self, evt):
if self.dlg.WasCancelled():
self.thread.queue.put(ABORT)
msg = "Count = %d" % evt.count
self.dlg.Pulse(msg)
def OnStart(self, _evt):
self.start_button.Disable()
limit = 1000
title = "Count to %d" % limit
self.dlg = wx.ProgressDialog(title, 'Count = 0',
style=wx.PD_CAN_ABORT|wx.PD_AUTO_HIDE)
self.dlg.Show()
self.thread = MyOperation(limit)
self.thread.start()
app = wx.App()
MyFrame(None)
app.MainLoop()
Hi Thom,
Thank you for your answer. I’m currently on a standard Windows 10 64 bit laptop, rather ok per se (64 GB RAM, 8 cores).
I have installed Python 3.9.10 64 bit via WinPython (which gives me all the optimised scientific stack for free) and wxPython 4.2.0, from PyPI. Plus a gazillion of other libraries always from PyPI.
I haven’t tried any other combination, although I guess it might be worth a try downgrading wxPython to see if anything changes… you never know.
What I know for certain now is that this specific part of the code needed zero changes to work on Python 3. And going from a (potential) wx.PyAssertionError (the old wx.wxAssertionError) to a hard crash of the whole application is a large punch in the face. You would think that porting dozens of custom C/Fortran extensions from a 10 year old code to modern Cython/f2py plus numerically heavy Python codes may be one of the tallest orders, but no, the suffering is in a wx.ProgressDialog…
It is proving very difficult to reproduce the issue in a simple app, but I’ll do my best.
Hi Richard,
Thank you for the detailed example. I was actually on the same road as well, trying to replace my various wx.CallAfters with AddPendingEvent.
It was actually out of desperation, but seeing that there’s others using the same mechanism gives me some hope.
On a marginally-related note: on the exact same version of Windows, the appearance and behaviour of wx.ProgressDialog are different. I’m Python 2.7 + wxPython 2.9.4, the gauge pulses back and forth in any case (even though the text on the dialog may not have time to update), while on Python 3.9 + wxPython 4.2.0 it does not. The visuals are also different. I’ll post a picture later of what I see, just for posterity…
well, this ‘dialog’ works even without main loop 
, so in that case posting events may not be a good idea (I think 
)
from threading import Thread
from time import sleep
import wx
class Action(Thread):
def __init__(self):
super().__init__()
self.dlg = wx.ProgressDialog(
'from thread', 'start',
style=wx.PD_CAN_ABORT|wx.PD_AUTO_HIDE)
# add 'wx.PD_APP_MODAL' for modal
self.dlg.Show()
self.start()
def run(self):
pls = 1
while not self.dlg.WasCancelled():
sleep(2)
self.dlg.Pulse(newmsg=f'pulse {str(pls)}')
pls += 1
self.dlg.Destroy()
app = wx.App()
Action()
You’re right, but the structure is similar enough that it doesn’t make a difference. You just arrive at wxEventLoop::GetActive()->YieldFor via a different path.
wxPython 2.9.4 (released over 10 years ago!) -> wxPython 4.2.0 is a massive change. Specifically, there have been a large number of changes to wxProgressDialog on MSW:
Yes yes, no doubt many things have changed in 10 years, it was just a bit surprising (to me) how different they look and behave. But it’s an irrelevant detail.
For the record, I think I found where the issue was, and in preliminary tests I don’t see the access violation anymore - but it’s way too early to celebrate.
Many years ago, a younger (and smartass) me had decided to put this line buried somewhere in the numerically-heavy section of the code:
sys.setcheckinterval(8192)
My older self (less smartass but still smartass enough) simply googled the replacement for that function (since in Python 3.9 it does not exist anymore) and substituted it like this:
sys.setswitchinterval(8192)
Without checking that the behaviour was the same. It definitely was not, by a mile or more.
From the docs (emphasis mine):
sys.setcheckinterval(interval )
Set the interpreter’s “check interval”. This integer value determines how often the interpreter checks for periodic things such as thread switches and signal handlers. The default is 100, meaning the check is performed every 100 Python virtual instructions.
sys.setswitchinterval(interval )
Set the interpreter’s thread switch interval (in seconds)
Both functions attempt to play with thread scheduling timings for the interpreter, but thei inputs and behaviour are completely different.
I will soon see if I’m out of the woods or if there’s yet another bomb ticking behind a yet-undiscovered subtle change.
well, I don’t think CallAfter or Python’s Threads cause any problems with this ‘dialog’ 
from threading import Thread, Lock
from time import sleep
import wx
class Gui(wx.Frame):
def __init__(self, parent):
super().__init__(parent, title='ProgressDialog')
wx.Button(self, label='left click me, please')
self.Bind(wx.EVT_BUTTON, lambda _: self.action())
self.Bind(wx.EVT_CLOSE, self.evt_close)
self.Centre()
self.Show()
self.thrs = 0
self.lock = Lock()
self.action()
def action(self):
Action(self.thrs, self.thrs_ref)
def thrs_ref(self, plus):
with self.lock:
if plus:
self.thrs += 1
else:
self.thrs -= 1
def evt_close(self, _):
if self.thrs > 0:
wx.Bell()
else:
self.Destroy()
class Action(Thread):
def __init__(self, thrs, thrs_ref):
super().__init__()
thrs_ref(True)
self.thrs_ref = thrs_ref
self.dlg = wx.ProgressDialog(
f'from thread {thrs}', 'start',
style=wx.PD_CAN_ABORT|wx.PD_AUTO_HIDE)
self.dlg.Show()
self.start()
def run(self):
pls = 1
while not self.dlg.WasCancelled():
sleep(0.01)
wx.CallAfter(self.pulse, pls)
if pls > 500:
break
pls += 1
self.thrs_ref(False)
def pulse(self, pls):
self.dlg.Pulse(newmsg=f'pulse {str(pls)}')
app = wx.App()
Gui(None)
app.MainLoop()
Very late to the party, but I actually found out that it’s not the use of set_switchinterval() creating a problem.
It’s matplotlib.
After months of chasing this stuff - it’s a rather infrequent and difficult to reproduce bug - I went to look for what matplotlib draw_idle() and flush_events() are doing.
Surprise surprise, in matplotlib/backends/backend_wx.py:
This hidden wx.Yield() called in a wx.CallAfter() bombs out the entire Python process in our application.
And it also magically bombs with a RecursionError the piece of code attached (!), even though the use of flush_events() is explicitly recommended in their page on blitting:
https://matplotlib.org/stable/users/explain/animations/blitting.html
Running that code on my machine results in this:
Traceback (most recent call last):
File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\wx\core.py", line 3427, in <lambda>
lambda event: event.callable(*event.args, **event.kw) )
File "C:\Users\j0514162\MyProjects\UAE\progress_dialog.py", line 120, in DoLongRunningTask
self.ax.draw_artist(self.line)
File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\axes\_base.py", line 3098, in draw_artist
a.draw(self.figure._cachedRenderer)
File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\artist.py", line 50, in draw_wrapper
return draw(artist, renderer)
File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\lines.py", line 732, in draw
self.recache()
File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\lines.py", line 651, in recache
xconv = self.convert_xunits(self._xorig)
File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\artist.py", line 252, in convert_xunits
return ax.xaxis.convert_units(x)
File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\axis.py", line 1497, in convert_units
if munits._is_natively_supported(x):
File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\units.py", line 67, in _is_natively_supported
return isinstance(thisx, Number) and not isinstance(thisx, Decimal)
File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\abc.py", line 119, in __instancecheck__
return _abc_instancecheck(cls, instance)
RecursionError: maximum recursion depth exceeded in comparison
Of course, if I remove the call to flush_events() everything works all right.
progress_dialog.py (3.5 KB)
Hi Andrea,
In your code, I fixed L103 as follows:
- wx.CallAfter(self.DoLongRunningTask, i)
+ self.DoLongRunningTask(i)
Then, everything works fine.
I’m using wxPython 4.2.1, Python 3.11.4.
The matplotlib version should be 3.7.2 ~ 3.8.0/WXAgg recently released, where a wx-event-related bug fix has been applied: https://github.com/matplotlib/matplotlib/pull/25559.
Thank you for your answer. The issue is, the matplotlib rendering has to happen in the main thread - not in a separate one. That’s why I used wx.CallAfter.
The code I posted is a huge simplification of what we actually have in our application. That said, it is my understanding that flush_events() invokes wx.Yield(), which should not be run in a different thread other than the main thread.
I’m not entirely sure what canvas.restore_region() and blit() in matplotlib do, but if they invoke wx functions/methods they have to be run in the main thread.