Windows fatal exception: access violation

Andrea_Gavana · December 13, 2022, 8:11am

Hello wxPython Users,

I am almost done in porting a very large, legacy Python 2 application to Python 3. Everything works all right except I am getting access violation error messages that bring down the whole Python process. I can trace it back to wx.ProgressDialog.Pulse() using the Python faulthandler environment variable.

I have a very simple subclass of wx.ProgressDialog, which does not do anything different from the standard wx.ProgressDialog beside setting an icon for it in the __init__ method. This is my overloaded Pulse method:

def Pulse(self, *args):

    print('I am in thread:', threading.current_thread())
    print('I am:', self)
    print('Deleted:', self.IsBeingDeleted(), 'Nonzero:', self.__nonzero__())
    print()
    if not self or self.IsBeingDeleted():
        return

    wx.ProgressDialog.Pulse(self, *args)
    wx.SafeYield()

The Pulse() method is called from another thread but always using wx.CallAfter. Just so we are clear, there is no instance in which a separate thread accesses GUI elements.

The presence or absence of wx.SafeYield() makes no difference to the outcome.

I get the following after a few seconds:

I am in thread: <_MainThread(MainThread, started 22296)>
I am: <mdp_widgets.PhaserProgressDialog object at 0x000001FA9D227670>
Deleted: False Nonzero: True

... more prints, another wx.ProgressDialog is created ...

I am in thread: <_MainThread(MainThread, started 22296)>
I am: <mdp_widgets.PhaserProgressDialog object at 0x000001FABE3A0550>
Deleted: False Nonzero: True

Windows fatal exception: access violation

Current thread 0x00005718 (most recent call first):
  File "C:\Users\J0514162\MyProjects\Phaser\refactoring_github\py3_working\mdp_widgets.py", line 11344 in Pulse
  File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\wx\core.py", line 3427 in <lambda>
  File "C:\Users\J0514162\MyProjects\Phaser\refactoring_github\py3_working\mdp_widgets.py", line 11344 in Pulse
  File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\wx\core.py", line 3427 in <lambda>
  File "C:\Users\J0514162\MyProjects\Phaser\refactoring_github\py3_working\mdp_widgets.py", line 11344 in Pulse
  File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\wx\core.py", line 3427 in <lambda>
  File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\wx\core.py", line 2262 in MainLoop
  File "C:\Users\J0514162\MyProjects\Phaser\refactoring_github\py3_working\Phaser.py", line 57 in phaser_main
  File "C:\Users\J0514162\MyProjects\Phaser\refactoring_github\py3_working\Phaser.py", line 69 in <module>

You may notice that line 3427 in core.py (at least on wxPython 4.2) is the implementation of wx.CallAfter:

This is so fantastically annoying that I have no words to describe the frustration. It has taken me longer to diagnose this issue and trying to find a workaround (which I cannot not find) than it took me to port 150,000 lines of code from Python 2 to Python 3, and from Classic to Phoenix.

Any suggestion on how to address this issue - workarounds, hacks, anything - would be most welcome.

Thank you in advance.

Andrea.

AndersMunch · December 13, 2022, 1:44pm

You are issuing CallAfter’s faster than they can be executed, causing you to have multiple pending Pulse calls queued.

wx.SafeYield causes CallAfter events to be processed, which means your Pulse method is called recursively, and then wx.SafeYield is called recursively. I don’t know if it SafeYield is supposed to be able to handle that – apparently it doesn’t, hence your crash.

There are several ways to rewrite this and avoid the recursion, but what I’m wondering is, why is there a SafeYield call there in the first place? The code returns to the wx event loop immediately after, so why is there a need to process events here? Perhaps Pulse is also called from blocking code and in this case it is needed, but when called using CallAfter, I don’t believe it is.

Andrea_Gavana · December 13, 2022, 3:12pm

Hi,

as I mentioned specifically in my message:

The presence or absence of wx.SafeYield() makes no difference to the outcome.

I get an access violation even if I remove it. And I have 2 or 3 wx.CallAfter in the process, not 30. Not to mention that all of this did not crash in wxPython Classic.

I may understand a wx.wxAssertionError or RuntimeError, what is inexplicable is the whole Python process coming down because of this.

swt2c · December 13, 2022, 3:38pm

Do you have a SCCCE that reproduces the problem?

AndersMunch · December 13, 2022, 4:01pm

Missed that. Long post.

Your traceback does show recursion, though, and that’s a classic for triggering messy program-crashing C/C++ bugs. I’d get rid of that before anything else.

Your traceback shows core.py:34127 and Pulse thrice. The explanation that comes most readily to mind is that you have multiple CallAfter events pending, and somehow you ended up processing the next CallAfter recursively. You may have another explanation.

Andrea_Gavana · December 13, 2022, 3:46pm

Hi,

I can try and put together one later. Hopefully I’ll be able to reproduce the issue.

As a side note, I have tried to put a guard inside the overridden Pulse method, like “just return and don’t do anything if less than 0.5 seconds have passed since your last Pulse()”.

While this helps a bit with the problem by delaying it, after a certain number of simulations (higher than without the guard) it still crashes with the same access violation error.

AndersMunch · December 13, 2022, 4:28pm

I think wx.ProgressDialog.Pulse is the culprit.

src\generic\progdlgg.cpp:

bool wxGenericProgressDialog::Pulse(const wxString& newmsg, bool *skip)
{
    if ( !DoBeforeUpdate(skip) )
        return false;

   ...
}

bool wxGenericProgressDialog::DoBeforeUpdate(bool *skip)
{
    // we have to yield because not only we want to update the display but
    // also to process the clicks on the cancel and skip buttons
    // NOTE: using YieldFor() this call shouldn't give re-entrancy problems
    //       for event handlers not interested to UI/user-input events.
    wxEventLoopBase::GetActive()->YieldFor(wxEVT_CATEGORY_UI|wxEVT_CATEGORY_USER_INPUT);
   ...
}

I bet the YieldFor is implicated in your problem. It’s there to collect keypresses/clicks when in the middle of a long-running event handler, but if you’re not actually using the progress dialog in a long-running event handler, then perhaps you should go around it.

If you replace the wx.ProgressDialog with a wx.Gauge on a regular frame, then you can just call Pulse on the gauge directly, without incurring any weird event-loopery.

Andrea_Gavana · December 13, 2022, 5:14pm

Thank you for your answer. I seem to remember though that by default the Windows wx.ProgressDialog is not based on the generic version but it uses the native widget.

I do not know if that piece of code is shared between the implementations - usually the shared sources live in the common/ directory - but I’ll take a look.

da-dada · December 13, 2022, 10:17pm

I just wonder how the CallAfter gets in here? or am I on the wrong planet

from threading import Thread
from time import sleep
import wx

class Gui(wx.Frame):

    def __init__(self, parent):
        print(f'{self.__class__} __init__')
        super().__init__(parent, title='ProgressDialog')

        wx.Button(self, label='left click me, please')
        self.Bind(wx.EVT_BUTTON, lambda _: Action().start())
        self.Centre()
        self.Show()
        Action().start()

class Action(Thread):

    def __init__(self):
        super().__init__()
        self.dlg = wx.ProgressDialog(
                'from thread', 'start',
                style=wx.PD_CAN_ABORT|wx.PD_AUTO_HIDE)
                # add 'wx.PD_APP_MODAL' for modal
        self.dlg.Show()

    def run(self):
        pls = 1
        while not self.dlg.WasCancelled():
            sleep(2)
            self.dlg.Pulse(newmsg=f'pulse {str(pls)}')
            pls += 1

app = wx.App()
Gui(None)
app.MainLoop()

Andrea_Gavana · December 13, 2022, 10:34pm

My understanding is that all UI interactions should happen in the main thread - I.e. the one that creates the application top window and the one where the app lives.

I am not sure how and why your example does not bomb out altogether - although I haven’t tried it myself - but we have always been told that calling UI things from other threads was a no-no to start with.

Thom · December 14, 2022, 5:09pm

So am I, Andrea, and successfully.

During my porting Python 2 to 3 and Classic to Phoenix up to 4.2 (and lower versions), I encountered many wx related bugs. Not a fatal access violation, though.

So FWIW, I’m currently running a very stable combination, for me at least:

Python 3.8.12 (heads/master:bd5e0bb, Aug 31 2021, 10:21:45) [MSC v.1929 64 bit (AMD64)] on win32
4.0.7.post2 msw (phoenix) wxWidgets 3.0.5

I wonder:

what version combination do you currently use?
have you tried other version combinations?

Cheers, Thom

RichardT · December 14, 2022, 5:42pm

I also believe that you should only update the UI from the main thread. However, instead of using using wx.CallAfter() I just send events using AddPendingEvent(). I have used this technique in several applications, in some of which the child thread can quickly generate a lot of events containing progress messages. The UI typically displays the messages in an STC and I haven’t yet had any problems with it keeping up with the child thread. Note: I have only tested this on Linux.

The code below shows a much simplified example which uses two custom events and an operation class derived from Thread. The class’s run() method sends the appropriate events. The operation class also contains a Queue object which provides a thread safe mechanism to send messages from the main thread to the child thread. I use the Queue object as a way to abort the child thread if the Cancel button on the ProgressDialog is pressed.

from threading import Thread
from queue import Queue, Empty
from time import sleep
import wx

ABORT = "abort"

evt_op_progress_type = wx.NewEventType()
EVT_OP_PROGRESS = wx.PyEventBinder(evt_op_progress_type, 1)

class OpProgressEvent(wx.PyCommandEvent):
    def __init__(self, count):
        wx.PyCommandEvent.__init__(self, evt_op_progress_type, 1)
        self.count = count

evt_op_ended_type = wx.NewEventType()
EVT_OP_ENDED = wx.PyEventBinder(evt_op_ended_type, 1)

class OpEndedEvent(wx.PyCommandEvent):
    def __init__(self, status):
        wx.PyCommandEvent.__init__(self, evt_op_ended_type, 1)
        self.status = status

class MyOperation(Thread):
    def __init__(self, limit):
        Thread.__init__(self)
        self.wx_app = wx.GetApp()
        self.queue  = Queue()
        self.limit = limit

    def checkQueue(self):
        try:
            item = self.queue.get_nowait()
        except Empty:
            item = ""
        return item

    def run(self):
        count = 0
        while True:
            if self.checkQueue() == ABORT:
                evt = OpEndedEvent("aborted")
                self.wx_app.AddPendingEvent(evt)
                return
            count += 1
            evt = OpProgressEvent(count)
            self.wx_app.AddPendingEvent(evt)
            if count > self.limit:
                evt = OpEndedEvent("completed")
                self.wx_app.AddPendingEvent(evt)
                return
            sleep(.01)


class MyFrame(wx.Frame):
    def __init__(self, parent):
        wx.Frame.__init__(self, parent, title='Test ProgressDialog')
        sizer = wx.BoxSizer(wx.VERTICAL)
        self.start_button = wx.Button(self, label='Start')
        sizer.Add(self.start_button)
        self.Bind(wx.EVT_BUTTON, self.OnStart, self.start_button)
        self.wx_app = wx.GetApp()
        self.wx_app.Bind(EVT_OP_PROGRESS, self.OnProgress)
        self.wx_app.Bind(EVT_OP_ENDED, self.OnOpEnded)
        self.SetSizer(sizer)
        self.Layout()
        self.Center()
        self.Show()


    def OnOpEnded(self, evt):
        self.thread.join()
        self.dlg.Destroy()
        print("Status = %s" % evt.status)
        self.start_button.Enable()


    def OnProgress(self, evt):
        if self.dlg.WasCancelled():
            self.thread.queue.put(ABORT)
        msg = "Count = %d" % evt.count
        self.dlg.Pulse(msg)


    def OnStart(self, _evt):
        self.start_button.Disable()
        limit = 1000
        title = "Count to %d" % limit
        self.dlg = wx.ProgressDialog(title, 'Count = 0',
                                     style=wx.PD_CAN_ABORT|wx.PD_AUTO_HIDE)
        self.dlg.Show()

        self.thread = MyOperation(limit)
        self.thread.start()


app = wx.App()
MyFrame(None)
app.MainLoop()

Andrea_Gavana · December 14, 2022, 5:44pm

Hi Thom,

Thank you for your answer. I’m currently on a standard Windows 10 64 bit laptop, rather ok per se (64 GB RAM, 8 cores).

I have installed Python 3.9.10 64 bit via WinPython (which gives me all the optimised scientific stack for free) and wxPython 4.2.0, from PyPI. Plus a gazillion of other libraries always from PyPI.

I haven’t tried any other combination, although I guess it might be worth a try downgrading wxPython to see if anything changes… you never know.

What I know for certain now is that this specific part of the code needed zero changes to work on Python 3. And going from a (potential) wx.PyAssertionError (the old wx.wxAssertionError) to a hard crash of the whole application is a large punch in the face. You would think that porting dozens of custom C/Fortran extensions from a 10 year old code to modern Cython/f2py plus numerically heavy Python codes may be one of the tallest orders, but no, the suffering is in a wx.ProgressDialog…

It is proving very difficult to reproduce the issue in a simple app, but I’ll do my best.

Andrea_Gavana · December 14, 2022, 5:50pm

Hi Richard,

Thank you for the detailed example. I was actually on the same road as well, trying to replace my various wx.CallAfters with AddPendingEvent.

It was actually out of desperation, but seeing that there’s others using the same mechanism gives me some hope.

On a marginally-related note: on the exact same version of Windows, the appearance and behaviour of wx.ProgressDialog are different. I’m Python 2.7 + wxPython 2.9.4, the gauge pulses back and forth in any case (even though the text on the dialog may not have time to update), while on Python 3.9 + wxPython 4.2.0 it does not. The visuals are also different. I’ll post a picture later of what I see, just for posterity…

da-dada · December 14, 2022, 6:27pm

well, this ‘dialog’ works even without main loop , so in that case posting events may not be a good idea (I think )

from threading import Thread
from time import sleep
import wx

class Action(Thread):

    def __init__(self):
        super().__init__()
        self.dlg = wx.ProgressDialog(
                'from thread', 'start',
                style=wx.PD_CAN_ABORT|wx.PD_AUTO_HIDE)
                # add 'wx.PD_APP_MODAL' for modal
        self.dlg.Show()
        self.start()

    def run(self):
        pls = 1
        while not self.dlg.WasCancelled():
            sleep(2)
            self.dlg.Pulse(newmsg=f'pulse {str(pls)}')
            pls += 1
        self.dlg.Destroy()

app = wx.App()
Action()

AndersMunch · December 15, 2022, 3:55pm

You’re right, but the structure is similar enough that it doesn’t make a difference. You just arrive at wxEventLoop::GetActive()->YieldFor via a different path.

swt2c · December 15, 2022, 5:39pm

wxPython 2.9.4 (released over 10 years ago!) -> wxPython 4.2.0 is a massive change. Specifically, there have been a large number of changes to wxProgressDialog on MSW:

Andrea_Gavana · December 15, 2022, 7:25pm

Yes yes, no doubt many things have changed in 10 years, it was just a bit surprising (to me) how different they look and behave. But it’s an irrelevant detail.

For the record, I think I found where the issue was, and in preliminary tests I don’t see the access violation anymore - but it’s way too early to celebrate.

Many years ago, a younger (and smartass) me had decided to put this line buried somewhere in the numerically-heavy section of the code:

sys.setcheckinterval(8192)

My older self (less smartass but still smartass enough) simply googled the replacement for that function (since in Python 3.9 it does not exist anymore) and substituted it like this:

sys.setswitchinterval(8192)

Without checking that the behaviour was the same. It definitely was not, by a mile or more.

From the docs (emphasis mine):

sys.setcheckinterval(interval )

Set the interpreter’s “check interval”. This integer value determines how often the interpreter checks for periodic things such as thread switches and signal handlers. The default is 100, meaning the check is performed every 100 Python virtual instructions.

sys.setswitchinterval(interval )

Set the interpreter’s thread switch interval (in seconds)

Both functions attempt to play with thread scheduling timings for the interpreter, but thei inputs and behaviour are completely different.

I will soon see if I’m out of the woods or if there’s yet another bomb ticking behind a yet-undiscovered subtle change.

da-dada · December 22, 2022, 10:26pm

well, I don’t think CallAfter or Python’s Threads cause any problems with this ‘dialog’

from threading import Thread, Lock
from time import sleep
import wx

class Gui(wx.Frame):

    def __init__(self, parent):
        super().__init__(parent, title='ProgressDialog')

        wx.Button(self, label='left click me, please')
        self.Bind(wx.EVT_BUTTON, lambda _: self.action())
        self.Bind(wx.EVT_CLOSE, self.evt_close)
        self.Centre()
        self.Show()
        self.thrs = 0
        self.lock = Lock()
        self.action()

    def action(self):
        Action(self.thrs, self.thrs_ref)

    def thrs_ref(self, plus):
        with self.lock:
            if plus:
                self.thrs += 1
            else:
                self.thrs -= 1

    def evt_close(self, _):
        if self.thrs > 0:
            wx.Bell()
        else:
            self.Destroy()

class Action(Thread):

    def __init__(self, thrs, thrs_ref):
        super().__init__()
        thrs_ref(True)
        self.thrs_ref = thrs_ref
        self.dlg = wx.ProgressDialog(
                f'from thread {thrs}', 'start',
                style=wx.PD_CAN_ABORT|wx.PD_AUTO_HIDE)
        self.dlg.Show()
        self.start()

    def run(self):
        pls = 1
        while not self.dlg.WasCancelled():
            sleep(0.01)
            wx.CallAfter(self.pulse, pls)
            if pls > 500:
                break
            pls += 1
        self.thrs_ref(False)

    def pulse(self, pls):
        self.dlg.Pulse(newmsg=f'pulse {str(pls)}')

app = wx.App()
Gui(None)
app.MainLoop()

Andrea_Gavana · September 18, 2023, 1:43pm

Very late to the party, but I actually found out that it’s not the use of set_switchinterval() creating a problem.

It’s matplotlib.

After months of chasing this stuff - it’s a rather infrequent and difficult to reproduce bug - I went to look for what matplotlib draw_idle() and flush_events() are doing.

Surprise surprise, in matplotlib/backends/backend_wx.py:

This hidden wx.Yield() called in a wx.CallAfter() bombs out the entire Python process in our application.

And it also magically bombs with a RecursionError the piece of code attached (!), even though the use of flush_events() is explicitly recommended in their page on blitting:

https://matplotlib.org/stable/users/explain/animations/blitting.html

Running that code on my machine results in this:

Traceback (most recent call last):
  File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\wx\core.py", line 3427, in <lambda>
    lambda event: event.callable(*event.args, **event.kw) )
  File "C:\Users\j0514162\MyProjects\UAE\progress_dialog.py", line 120, in DoLongRunningTask
    self.ax.draw_artist(self.line)
  File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\axes\_base.py", line 3098, in draw_artist
    a.draw(self.figure._cachedRenderer)
  File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\artist.py", line 50, in draw_wrapper
    return draw(artist, renderer)
  File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\lines.py", line 732, in draw
    self.recache()
  File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\lines.py", line 651, in recache
    xconv = self.convert_xunits(self._xorig)
  File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\artist.py", line 252, in convert_xunits
    return ax.xaxis.convert_units(x)
  File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\axis.py", line 1497, in convert_units
    if munits._is_natively_supported(x):
  File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\site-packages\matplotlib\units.py", line 67, in _is_natively_supported
    return isinstance(thisx, Number) and not isinstance(thisx, Decimal)
  File "C:\Users\J0514162\WinPython39\WPy64-39100\python-3.9.10.amd64\lib\abc.py", line 119, in __instancecheck__
    return _abc_instancecheck(cls, instance)
RecursionError: maximum recursion depth exceeded in comparison

Of course, if I remove the call to flush_events() everything works all right.

progress_dialog.py (3.5 KB)