[wxPython] wxPython Threading problem - possible solution

Robin,
Do you have any feedback on these ideas and experiments?

I've send off some queries about these suggestions to the guy who
implemented the threading support in wxWindows to see what he thinks.

Solution 1 - have wxWindows manage its own thread:

Have wxWindows create a thread of its only that all
wxWindows work is executed on. Any call from wxPython
is first marshalled to the wxWindows thread and executed
there, results are mashelled back.

So the calling thread gets blocked until the gui thread gets around to
processing the request? What about the normal case where the calling thread
is the gui thread? I would hate to add the overhead of checking the thread
for the normal 98% case just to make things easier for the 2%.

Call backs out of wxWindows to the python code would run
on the wxWindows thread, python will need to be told
about the wxWindows thread or python will assert.

Solution 2 - have mainloop choose the wx thread

Change mainloop() so that it sets gs_idMainThread when it
starts and clears gs_idMainThread before it returns.

This seems possible and I've asked Vadim about it. I expect that there
would be some negative impacts on general wxWindows C++ apps though. Also,
since MainLoop doesn't happen until after OnInit does, there could be some
potential problems waiting to explode there...

solution 3 - signal errors:

In every wxPython wrapper function call a function to
check the preconditions.

Precondition 1: Check that the thread is the wxWindows
thread. If its wrong raise a RuntimeError exception.

The following works for mainloop(). In wx.cpp line 1330
in _wrap_wxPyApp_MainLoop:

if( !wxThread::IsMain() )
{
PyErr_SetString(PyExc_RuntimeError,"wxPyApp_MainLoop must be called on the

wxWindows main thread.");

return NULL;
}

Again, I worry about the overhead for the normal case, although this would
be easier to implement that #1.

solution 4 - set the main thread on first use:

Building on solution 3 and setting gs_idMainThread on first use.

In thread.cpp at line 1100: comment out setting gs_idMainThread.
Add to IsMain() setting gs_idMainThread:

bool wxThread::IsMain()
{
    if( gs_idMainThread == 0 )
{
gs_idMainThread = ::GetCurrentThreadId();
return 1;
}

    return ::GetCurrentThreadId() == gs_idMainThread;
}

I suppose that as long nothing else in the thread module needed
gs_idMainThread before the first time IsMain is called this might work okay,
but it still wouldn't let you use different threads later in the process's
lifetime, only make it easier to let wxPython start up on a non main thread
and make it be the main gui thread.

For safty add an assert to wxWakeUpMainThread at line 1210:

    wxASSERT_MSG( wxThread::IsMain(), _T("should only be called from the

main thread") );

Actually, the reverse is true. wxWakeUpMainThread should be called by non
main threads to stimulate the main thread if it is waiting on an event, but
it wouldn't hurt if it was called by the main thread.

BTW, I found a couple functions in wxWindows that will let you make GUI
calls from non gui threads. They use a mutex to serialize access to the gui
between the main thread and the others. Normally the main thread holds the
mutex, but it releases it when there are no events pending. I'll add them
to wxPython.

    wxMutexGuiEnter()
    # do gui stuff
    wxMutexGuiLeave()

If there is an exception in the middle then the thread will never release
the mutex and deadlocks will probably happen, so there is a class that can
be instantiated that will aquire the mutex on creation and release it when
the object is deleted.

    locker = wxMutexGuiLocker()
    # do gui stuff

Hopefully these will help the situation eventhough they won't give you all
you need for running wxPython stuff from Barry's Emacs.

ยทยทยท

--
Robin Dunn
Software Craftsman
robin@AllDunn.com Java give you jitters?
http://wxPython.org Relax with wxPython!

> Solution 1 - have wxWindows manage its own thread:
>
> Have wxWindows create a thread of its only that all
> wxWindows work is executed on. Any call from wxPython
> is first marshalled to the wxWindows thread and executed
> there, results are mashelled back.

So the calling thread gets blocked until the gui thread gets around to
processing the request? What about the normal case where the calling thread
is the gui thread? I would hate to add the overhead of checking the thread
for the normal 98% case just to make things easier for the 2%.

What would the overhead be? WOuld an end user be able to detect the change
in overhead? I suspect the end user would not.

> Solution 2 - have mainloop choose the wx thread
>
> Change mainloop() so that it sets gs_idMainThread when it
> starts and clears gs_idMainThread before it returns.
>

This seems possible and I've asked Vadim about it. I expect that there
would be some negative impacts on general wxWindows C++ apps though. Also,
since MainLoop doesn't happen until after OnInit does, there could be some
potential problems waiting to explode there...

I'm optimistic that the potential problems would be small and easy to solve.

> solution 3 - signal errors:
>
> In every wxPython wrapper function call a function to
> check the preconditions.
>
> Precondition 1: Check that the thread is the wxWindows
> thread. If its wrong raise a RuntimeError exception.
>
> The following works for mainloop(). In wx.cpp line 1330
> in _wrap_wxPyApp_MainLoop:
>
> if( !wxThread::IsMain() )
> {
> PyErr_SetString(PyExc_RuntimeError,"wxPyApp_MainLoop must be called on the
wxWindows main thread.");
> return NULL;
> }

Again, I worry about the overhead for the normal case, although this would
be easier to implement that #1.

Here is the output of a benchmark for calling ismain on my 800Mhz
100MB FSB Windows 2000 machine.

wxThread::IsMain tester!
10000000 iterations averaged 0.104us/call

I think this is a very small overhead.

> solution 4 - set the main thread on first use:
>
> Building on solution 3 and setting gs_idMainThread on first use.
>
> In thread.cpp at line 1100: comment out setting gs_idMainThread.
> Add to IsMain() setting gs_idMainThread:
>
> bool wxThread::IsMain()
> {
> if( gs_idMainThread == 0 )
> {
> gs_idMainThread = ::GetCurrentThreadId();
> return 1;
> }
>
> return ::GetCurrentThreadId() == gs_idMainThread;
> }

I suppose that as long nothing else in the thread module needed
gs_idMainThread before the first time IsMain is called this might work okay,
but it still wouldn't let you use different threads later in the process's
lifetime, only make it easier to let wxPython start up on a non main thread
and make it be the main gui thread.

I'd not use IsMAin in the production implementation. I used IsMain as
a quick way to the test the idea out. I'd add methods to wxThread to
support the semantics required.

> For safty add an assert to wxWakeUpMainThread at line 1210:
>
> wxASSERT_MSG( wxThread::IsMain(), _T("should only be called from the
main thread") );

Actually, the reverse is true. wxWakeUpMainThread should be called by non
main threads to stimulate the main thread if it is waiting on an event, but
it wouldn't hurt if it was called by the main thread.

  True. Opss...

BTW, I found a couple functions in wxWindows that will let you make GUI
calls from non gui threads. They use a mutex to serialize access to the gui
between the main thread and the others. Normally the main thread holds the
mutex, but it releases it when there are no events pending. I'll add them
to wxPython.

    wxMutexGuiEnter()
    # do gui stuff
    wxMutexGuiLeave()

If there is an exception in the middle then the thread will never release
the mutex and deadlocks will probably happen, so there is a class that can
be instantiated that will aquire the mutex on creation and release it when
the object is deleted.

    locker = wxMutexGuiLocker()
    # do gui stuff

Hopefully these will help the situation eventhough they won't give you all
you need for running wxPython stuff from Barry's Emacs.

  This is promising. I'd need the error signalling as well.

    BArry