wx.CallAfter behavior

wxPython Users
1

Hello all,

My platform is Solaris 10 (x86), Python 2.4.3, and wxPython 2.6.3.3.

I’m using wx.CallAfter from a python sub-thread to make a call back to the main thread, always to the same method. For the most part this works very nicely. However today I discovered odd behavior in a case where the sub-thread makes two rapid successive
wx.CallAfters to the same ‘mainThreadMethod’. It is confusing to explain. To help illustrate, what I see are the following actions in the following relative time sequence.

Thread 5 waits for input, gets msgA, then calls

wx.CallAfter(mainThreadMethod, msgA)

Thread 5 again waits for input, immediately gets msgB, then calls

wx.CallAfter(mainThreadMethod, msgB)

Then in the main thread,

Enter --> mainThreadMethod(msgA)

Enter --> mainThreadMethod(msgB)

Return from <-- mainThreadMethod(msgB)

Return from <-- mainThreadMethod(msgA)

It is a bit confusing to me how this can happen. What I am expecting (and see most of the time) is:

Enter --> mainThreadMethod(msgA)

Return from <-- mainThreadMethod(msgA)

Enter --> mainThreadMethod(msgB)

Return from <-- mainThreadMethod(msgB)

The odd behavior seems to occur when there is more GUI activity within mainThreadMethod, such as displaying an image and updating alot of widgets.

I tried to serialize the calls with python locks as well as the python mutex class, so far without success. However, I don’t yet fully understand these mechanisms in python, and may not be using them correctly.

So, …

Is this correct wx.CallAfter behavior?

Is there a way to guarantee that the methods passed in wx.CallAfter are executed in the main thread serially, so that each instance of each call starts and finishes before the next?

Thanks in advance for any help or suggestions.

  • Chris Botos
2

wx.CallAfter uses standard wxPython events to handle all method calls in
the GUI thread. Because there is no guarantee that events will be
processed in any particular order, message B can be processed before
message A, as you have seen.

If you want to guarantee that one wx.CallAfter call will happen before
another, you can use the following.

import wx
import Queue

q = Queue.Queue()

def _process():
    fcn, args, kwargs = q.get()
    fcn(*args, **kwargs)

def CallAfter(fcn, *args, **kwargs):
    q.put((fcn, args, kwargs))
    wx.CallAfter(_process)

Just call the above CallAfter function the same as you did before, and
it will work the way you want.

- Josiah

···

"chris botos" <chris.botos@gmail.com> wrote:

My platform is Solaris 10 (x86), Python 2.4.3, and wxPython 2.6.3.3.

I'm using wx.CallAfter from a python sub-thread to make a call back to the
main thread, always to the same method. For the most part this works very
nicely. However today I discovered odd behavior in a case where the
sub-thread makes two rapid successive wx.CallAfters to the same
'mainThreadMethod'. It is confusing to explain. To help illustrate, what I
see are the following actions in the following relative time sequence.

Thread 5 waits for input, gets msgA, then calls
    wx.CallAfter(mainThreadMethod, msgA)

Thread 5 again waits for input, immediately gets msgB, then calls
    wx.CallAfter(mainThreadMethod, msgB)

Then in the main thread,
  Enter --> mainThreadMethod(msgA)
  Enter --> mainThreadMethod(msgB)
   Return from <-- mainThreadMethod(msgB)
  Return from <-- mainThreadMethod(msgA)

3

Sorry if you get this twice. I don’t think I sent it to the grop the first time.

CB

My platform is Solaris 10 (x86), Python 2.4.3, and wxPython 2.6.3.3.

I’m using wx.CallAfter from a python sub-thread to make a call back to the
main thread, always to the same method. For the most part this works very

nicely. However today I discovered odd behavior in a case where the
sub-thread makes two rapid successive wx.CallAfters to the same
‘mainThreadMethod’. It is confusing to explain. To help illustrate, what I

see are the following actions in the following relative time sequence.

Thread 5 waits for input, gets msgA, then calls
wx.CallAfter(mainThreadMethod, msgA)

Thread 5 again waits for input, immediately gets msgB, then calls

wx.CallAfter(mainThreadMethod, msgB)

Then in the main thread,
Enter → mainThreadMethod(msgA)
Enter → mainThreadMethod(msgB)
Return from ← mainThreadMethod(msgB)

Return from ← mainThreadMethod(msgA)

wx.CallAfter uses standard wxPython events to handle all method calls in
the GUI thread. Because there is no guarantee that events will be
processed in any particular order, message B can be processed before

message A, as you have seen.

If you want to guarantee that one wx.CallAfter call will happen before
another, you can use the following.

import wx
import Queue

q = Queue.Queue()

def _process():

fcn, args, kwargs = q.get()
fcn(*args, **kwargs)

def CallAfter(fcn, *args, **kwargs):
q.put((fcn, args, kwargs))
wx.CallAfter(_process)

Just call the above CallAfter function the same as you did before, and

it will work the way you want.

  • Josiah

Thanks, Josiah. I will give this a try (my work internet is down, so I have to do all my communicating when I’m home). I see that this guarantees the order that fcn is called. However does it guarantee that each call to _process will be atomic? What I saw might be described as “nested” fcn calls (almost like recursion?). Based on what I saw before I’m concerned the behavior will just be shifted. Extrapolating, I’m imagining that the following nesting might happen as before:

In sub-thread:

CallAfter(fcn, msgA)

CallAfter(fcn, msgB)

In main thread:

_process() # first call

get fcn from queue

fcn(msgA)

start msgA processing

_process() # second call interrupts msgA processing

get fcn from queue

fcn(msgB)

do all msgB processing

return from fcn(msgB)

return from second _process()

resume and finish msgA processing

return from fcn(msgA)

return from first _process()

As in my first example, where the execution of mainThreadMethod(msgA) was not “atomic”, here the first call to _process() is not “atomic”. Of course this second example is speculation because I haven’t tried it yet.

Thanks again. I’ll let you know later how this works.

-Chris

···

On 5/18/07, Josiah Carlson jcarlson@uci.edu wrote:

“chris botos” chris.botos@gmail.com wrote:

4

chris botos wrote:

To help illustrate, what I see are the following actions in the following relative time sequence.
Thread 5 waits for input, gets msgA, then calls
    wx.CallAfter(mainThreadMethod, msgA)
Thread 5 again waits for input, immediately gets msgB, then calls
    wx.CallAfter(mainThreadMethod, msgB)
Then in the main thread,
  Enter --> mainThreadMethod(msgA)
  Enter --> mainThreadMethod(msgB)
  Return from <-- mainThreadMethod(msgB)
  Return from <-- mainThreadMethod(msgA)

Does your mainThreadMethod do anything that could allow events to be delivered? For example, calling wx.Yield or a dialog's ShowModal. Since wx.CallAfter utilizes wx events, the functions are processed when idle events are delivered to the app. If there are multiple events waiting then the situation you describe can easily happen if something else pulls events from the queue before the first function call completes.

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

5

[snip]

> > Then in the main thread,
> > Enter --> mainThreadMethod(msgA)
> > Enter --> mainThreadMethod(msgB)
> > Return from <-- mainThreadMethod(msgB)
> > Return from <-- mainThreadMethod(msgA)
>
> wx.CallAfter uses standard wxPython events to handle all method calls in
> the GUI thread. Because there is no guarantee that events will be
> processed in any particular order, message B can be processed before
> message A, as you have seen.
>
> If you want to guarantee that one wx.CallAfter call will happen before
> another, you can use the following.
>
> import wx
> import Queue
>
> q = Queue.Queue()
>
> def _process():
> fcn, args, kwargs = q.get()
> fcn(*args, **kwargs)
>
> def CallAfter(fcn, *args, **kwargs):
> q.put((fcn, args, kwargs))
> wx.CallAfter(_process)
>
>
> Just call the above CallAfter function the same as you did before, and
> it will work the way you want.
>
> - Josiah

Thanks, Josiah. I will give this a try (my work internet is down, so I
have to do all my communicating when I'm home). I see that this guarantees
the order that fcn is called. However does it guarantee that each call to
_process will be atomic? What I saw might be described as "nested" fcn
calls (almost like recursion?). Based on what I saw before I'm concerned
the behavior will just be shifted. Extrapolating, I'm imagining that the
following nesting might happen as before:

In sub-thread:
CallAfter(fcn, msgA)
CallAfter(fcn, msgB)

In main thread:
_process() # first call
    get fcn from queue
    fcn(msgA)
        start msgA processing
        _process() # second call interrupts msgA processing
            get fcn from queue
            fcn(msgB)
                do all msgB processing
                return from fcn(msgB)
            return from second _process()
        resume and finish msgA processing
        return from fcn(msgA)
    return from first _process()

As in my first example, where the execution of mainThreadMethod(msgA) was
not "atomic", here the first call to _process() is not "atomic". Of course
this second example is speculation because I haven't tried it yet.

Here's the thing: all calls to _process() are the same. If I use
wx.CallAfter(_process) to handle message A, then immediately call
wx.CallAfter(_process) to handle message B, whichever of those calls
gets to run first will pull the function/arguments off to handle message
A, and the second will handle message B.

As long as you *never* call _process directly from your code (why would
you do that, it's an internal implementation detail?), don't call
wx.Yield() inside the functions called by _process(), and don't
inadvertantly create another event processing loop (like
dialog.ShowModal()), it will order your desired function calls in the
order in which they were tossed into the Queue.

For example...

Running the following...

    def foo(arg):
        print "foo", arg
        CallAfter(foo2, arg)

    def foo2(arg):
        print "foo2", arg

    CallAfter(foo, 1)
    CallAfter(foo2, 2)

Will produce:
    foo 1
    foo2 1
    foo2 2

Why? Because foo's call to CallAfter(foo2, arg) happens before the
CallAfter(foo2, 2) call.

- Josiah

···

"chris botos" <chris.botos@gmail.com> wrote:

On 5/18/07, Josiah Carlson <jcarlson@uci.edu> wrote:
> "chris botos" <chris.botos@gmail.com> wrote:

6

I’m not calling wx.Yield that I know of, and there are no dialogs at this point in the processing.

mainThreadMethod is a state machine (one of the reasons I need its execution to be synchronous) whose inputs are the messages, actually message objects. The message objects are C++ classes wrapped by SWIG that contain model results. They are instantiated by C++ methods running within C-threads that feed a Solaris queue. A C++ method, also wrapped in SWIG, is called from my python “thread 5” run method and waits on the queue. When it returns the message object, the thread calls wx.CallAfter.

mainThreadMethod extracts data from the message objects and makes GUI updates based on its state and the message contents. The GUI updates are initiated fron mainThreadMethod using a number of Publisher().sendMessage(…) calls. There are some try/except blocks along the way.

In the first mainThreadMethod call, one of the Publisher().sendMessage calls results in an image being displayed: in a wx.ScrolledWindow subclass, a wx.Image is created using wx.ImageFromBuffer and converted to a wx.Bitmap
. Then I call Refresh().

In the OnPaint handler for EVT_PAINT, I create a wx.PaintDC and call DrawBitmap.

In at least one instance of my tracking the problem (and maybe the others as well), the second call to mainThreadMethod occurs about a millisecond after the OnPaint handler exits. Could something associated with the Refresh and EVT_PAINT generation affect this?

BTW, I have been trying to reproduce this behavior in a stripped-down example, but so far have not been able to.

Chris

···

On 5/18/07, Robin Dunn robin@alldunn.com wrote:

chris botos wrote:

To help
illustrate, what I see are the following actions in the following

relative time sequence.

Thread 5 waits for input, gets msgA, then calls
wx.CallAfter(mainThreadMethod, msgA)

Thread 5 again waits for input, immediately gets msgB, then calls

wx.CallAfter(mainThreadMethod, msgB)

Then in the main thread,
Enter → mainThreadMethod(msgA)
Enter → mainThreadMethod(msgB)
Return from ← mainThreadMethod(msgB)

Return from ← mainThreadMethod(msgA)

Does your mainThreadMethod do anything that could allow events to be
delivered? For example, calling wx.Yield or a dialog’s ShowModal.
Since wx.CallAfter
utilizes wx events, the functions are processed when
idle events are delivered to the app. If there are multiple events
waiting then the situation you describe can easily happen if something
else pulls events from the queue before the first function call completes.

7

[snip]

Then in the main thread,
Enter → mainThreadMethod(msgA)
Enter → mainThreadMethod(msgB)
Return from ← mainThreadMethod(msgB)

Return from ← mainThreadMethod(msgA)

wx.CallAfter uses standard wxPython events to handle all method calls in
the GUI thread. Because there is no guarantee that events will be

processed in any particular order, message B can be processed before
message A, as you have seen.

If you want to guarantee that one wx.CallAfter call will happen before

another, you can use the following.

import wx
import Queue

q = Queue.Queue()

def _process():
fcn, args, kwargs = q.get()
fcn(*args, **kwargs)

def CallAfter(fcn, *args, **kwargs):
q.put((fcn, args, kwargs))
wx.CallAfter(_process)

Just call the above CallAfter function the same as you did before, and

it will work the way you want.

  • Josiah

Thanks, Josiah. I will give this a try (my work internet is down, so I
have to do all my communicating when I’m home). I see that this guarantees

the order that fcn is called. However does it guarantee that each call to
_process will be atomic? What I saw might be described as “nested” fcn
calls (almost like recursion?). Based on what I saw before I’m concerned

the behavior will just be shifted. Extrapolating, I’m imagining that the
following nesting might happen as before:

In sub-thread:
CallAfter(fcn, msgA)
CallAfter(fcn, msgB)

In main thread:
_process() # first call
get fcn from queue
fcn(msgA)
start msgA processing
_process() # second call interrupts msgA processing

        get fcn from queue
        fcn(msgB)
            do all msgB processing
            return from fcn(msgB)
        return from second _process()
    resume and finish msgA processing

    return from fcn(msgA)
return from first _process()

As in my first example, where the execution of mainThreadMethod(msgA) was
not “atomic”, here the first call to _process() is not “atomic”. Of course

this second example is speculation because I haven’t tried it yet.

I did try it, and this is indeed what happens.

Here’s the thing: all calls to _process() are the same. If I use
wx.CallAfter(_process) to handle message A, then immediately call

wx.CallAfter(_process) to handle message B, whichever of those calls
gets to run first will pull the function/arguments off to handle message
A, and the second will handle message B.

Yes, I understand this, and this is what happens, and this is what I see. Perhaps I’m not explaining myself well. I have not had a problem with the order in which the calls run, but with the “atomicicity” of their execution (coining a new word probably doesn’t help either). The call to _process(messageA) that runs first processes the correct message A. The call to _process(messageB) that runs second processes the correct message B. However the call to _process(messageA) does not finish before _process(messageB) starts running.

As long as you never call _process directly from your code (why would
you do that, it’s an internal implementation detail?),

I don’t, but I do call the function (fcn in your example code) that _process takes from the queue in response to user input. I’ll take a closer look at this, but in my problem use case the user is not involved.

don’t call
wx.Yield() inside the functions called by _process(),

I don’t.

and don’t
inadvertantly create another event processing loop (like
dialog.ShowModal()),

I don’t, but please see my response to Robin.

it will order your desired function calls in the
order in which they were tossed into the Queue.

For example…

Running the following…

def foo(arg):
print “foo”, arg
CallAfter(foo2, arg)

def foo2(arg):
print “foo2”, arg

CallAfter(foo, 1)

CallAfter(foo2, 2)

Will produce:
foo 1
foo2 1
foo2 2

Why? Because foo’s call to CallAfter(foo2, arg) happens before the
CallAfter(foo2, 2) call.

Yes, but the problematic output I am seeing would be equivalent to your example producing:

foo 1
foo2 2
foo2 1

because foo2(2) would have bugun executing after foo’s print statement, but before foo’s CallAfter(foo2, arg) statement.

-Chris

···

On 5/18/07, Josiah Carlson jcarlson@uci.edu wrote:

“chris botos” chris.botos@gmail.com wrote:

On 5/18/07, Josiah Carlson jcarlson@uci.edu wrote:

“chris botos” chris.botos@gmail.com wrote:

8

Ok, here's a version that shouldn't have the issues that you are running
into.

import wx
import Queue
import threading

q = Queue.Queue()
p = 0

def _process():
    global p
    if not p:
        p = 1
        try:
            fcn, args, kwargs = q.get()
            fcn(*args, **kwargs)
        finally:
            p = 0
    else:
        wx.CallAfter(_process)

def CallAfter(fcn, *args, **kwargs):
    q.put((fcn, args, kwargs))
    wx.CallAfter(_process)

That will keep as many pending _process calls in the event queue of the
wx.App instance as necessary until all of the functions have been called.
It will also make sure not to call any new functions until the last one
has completed. There is no need for a lock here because _process should
always be running in the main thread, AND if we were to use a lock, and
it were to re-enter itself, we could run into a deadlock (unless we used
an RLock, in which case two functions would be called at the same time).

If you are running into a situation where subsequent function calls are
being called too fast (sometimes the wxPython event loop works quite
well), you can replace the wx.CallAfter calls with wx.FutureCall() .
They seem to clean up after themselves all right. You could also use a
wx.Timer instance to poll the queue for any new functions to call
(basically use _process but remove the else clause above, checking for
q.empty()).

- Josiah

···

"chris botos" <chris.botos@gmail.com> wrote:

Yes, but the problematic output I am seeing would be equivalent to your
example producing:
   foo 1
   foo2 2
   foo2 1
because foo2(2) would have bugun executing after foo's print statement, but
before foo's CallAfter(foo2, arg) statement.

9

Josiah,

I can see how this will greatly reduce the probability of the second call to _process interrupting the first, but I believe it doesn’t go to zero. Please see my comment in the code…

Ok, here’s a version that shouldn’t have the issues that you are running
into.

import wx

import Queue
import threading

q = Queue.Queue()
p = 0

def _process():
global p
if not p:

Please correct me if I’m wrong, but I think there is a short window of time here, as p is being evaluated by the first _process, where the second _process can start executing. It will see p==0 and get the next fcn from q. But since the first _process didn’t get that far it will get the first fcn, not the second. …OHHHHHHHHH. But that doesn’t matter, does it? The first fcn still gets executed first, and it still finishes because it is still protected by p. When the second _process returns, the first _process still sees p==0, gets the second fcn from the q and executes it to completion. Tricky!

p = 1
try:
fcn, args, kwargs = q.get()
fcn(*args, **kwargs)

   finally:
       p = 0

else:
wx.CallAfter(_process)

def CallAfter(fcn, *args, **kwargs):
q.put((fcn, args, kwargs))
wx.CallAfter(_process)

That will keep as many pending _process calls in the event queue of the

wx.App instance as necessary until all of the functions have been called.
It will also make sure not to call any new functions until the last one
has completed. There is no need for a lock here because _process should

always be running in the main thread, AND if we were to use a lock, and
it were to re-enter itself, we could run into a deadlock (unless we used
an RLock, in which case two functions would be called at the same time).

I’m not sure I follow this.

If you are running into a situation where subsequent function calls are
being called too fast (sometimes the wxPython event loop works quite

well), you can replace the wx.CallAfter calls with wx.FutureCall() .
They seem to clean up after themselves all right. You could also use a
wx.Timer instance to poll the queue for any new functions to call
(basically use _process but remove the else clause above, checking for

q.empty()).

  • Josiah

Thanks for these suggestions! I’ll give it another try on Monday.

I would still like to understand what in my code I’m doing to make this happen (see my response to Robin yesterday).

Have a great weekend!

-Chris

···

On 5/19/07, Josiah Carlson jcarlson@uci.edu wrote:

10

chris botos wrote:

In at least one instance of my tracking the problem (and maybe the others as well), the second call to mainThreadMethod occurs about a millisecond after the OnPaint handler exits. Could something associated with the Refresh and EVT_PAINT generation affect this?

Paint events are low priority messages, and so are usually sent as the last message in the event queue, which would place them right before the idle events, which is where the wx.CallAfter functions are called from. If you're getting a paint event before the first function call completes then something is definitely causing a nested event loop to happen. What is the last thing in your mainThreadMethod that executes before the paint event and the nested mainThreadMethod is called?

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

11

chris botos wrote:

Josiah,
I can see how this will greatly reduce the probability of the second call to _process interrupting the first, but I believe it doesn't go to zero. Please see my comment in the code...

    Ok, here's a version that shouldn't have the issues that you are running
    into.

    import wx
    import Queue
    import threading

    q = Queue.Queue()
    p = 0

    def _process():
       global p
       if not p:

Please correct me if I'm wrong, but I think there is a short window of time here, as p is being evaluated by the first _process, where the second _process can start executing.

No, CallAfter calls can only happen when the app's idle events are processed, and that can't happen when just doing simple python statements like this in the gui thread.

···

On 5/19/07, *Josiah Carlson* <jcarlson@uci.edu > <mailto:jcarlson@uci.edu>> wrote:

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

12

Josiah,
I can see how this will greatly reduce the probability of the second call to
_process interrupting the first, but I believe it doesn't go to zero.
Please see my comment in the code...

>
>
> Ok, here's a version that shouldn't have the issues that you are running
> into.
>
> import wx
> import Queue
> import threading
>
> q = Queue.Queue()
> p = 0
>
> def _process():
> global p
> if not p:

Please correct me if I'm wrong, but I think there is a short window of time
here, as p is being evaluated by the first _process, where the second
_process can start executing. It will see p==0 and get the next fcn from q.
But since the first _process didn't get that far it will get the first fcn,
not the second. ...OHHHHHHHHH. But that doesn't matter, does it? The first
fcn still gets executed first, and it still finishes because it is still
protected by p. When the second _process returns, the first _process still
sees p==0, gets the second fcn from the q and executes it to completion.
Tricky!

Since _process should only be called in the GUI thread's event handling
rutines (again, never call _process() directly), and because it never
calls wx.Yield() or otherwise results in the main event handling rutine,
other threads _cannot_ be started, and there _cannot_ be a race
condition on p (as long as _process() is only ever called from a
wx.CallAfter or wx.FutureCall call).

> p = 1
> try:
> fcn, args, kwargs = q.get()
> fcn(*args, **kwargs)
> finally:
> p = 0
> else:
> wx.CallAfter(_process)
>
> def CallAfter(fcn, *args, **kwargs):
> q.put((fcn, args, kwargs))
> wx.CallAfter(_process)
>
>
> That will keep as many pending _process calls in the event queue of the
> wx.App instance as necessary until all of the functions have been called.
> It will also make sure not to call any new functions until the last one
> has completed. There is no need for a lock here because _process should
> always be running in the main thread, AND if we were to use a lock, and
> it were to re-enter itself, we could run into a deadlock (unless we used
> an RLock, in which case two functions would be called at the same time).

I'm not sure I follow this.

If we use a lock rather than a counter, and the lock is only accessed
from _process(), then if _process() gets called recursively (the
function that _process() calls runs the GUI MainLoop), then it will try
to acquire the lock again and deadlock. If we use an RLock instead,
then the GUI thread will just keep re-locking the lock, resulting in
unordered processing.

If we use a lock merely as synchronization around the counter, it would
guarantee that there isn't a race condition on p, but because _process
() is only ever run from a single thread (the GUI thread), a race
condition cannot happen anyways (again, as long as _process() is only
ever called via wx.CallAfter() or wx.FutureCall() ).

Thanks for these suggestions! I'll give it another try on Monday.

No problem. Thinking about threading isn't all that easy to do. The
only reason I can make any sense of it is because I've been doing it for
a good 7-8 years now.

I would still like to understand what in my code I'm doing to make this
happen (see my response to Robin yesterday).

Robin addressed this already.

- Josiah

···

"chris botos" <chris.botos@gmail.com> wrote:

On 5/19/07, Josiah Carlson <jcarlson@uci.edu> wrote:

13

Josiah Carlson wrote:

Since _process should only be called in the GUI thread's
event handling rutines (again, never call _process()
directly), and because it never calls wx.Yield() or otherwise
results in the main event handling rutine, other threads
_cannot_ be started, and there _cannot_ be a race condition
on p (as long as _process() is only ever called from a
wx.CallAfter or wx.FutureCall call).

Just a reminder that Robin recently gave us wx.CallLater as a synonym for
wx.FutureCall - it has a pleasing symmetry with wx.CallAfter.

In fact, according to help(), CallLater is now the primary name, and
FutureCall is an alias.

Frank Millman

14

Robin, I’ve tracked this down to a wx.TreeCtrl.ScrollTo(item) call. I get a paint event in mainThreadMethod about a millisecond after making this call. If I use my original code, and don’t make this call, then the wx.CallAfter functions behave nicely. One of the GUI updates in mainThreadMethod is sometimes adding an item to the tree, and I require the last item added to show in the window, so I must scroll if I add an item. Should I assume that wx.TreeCtrl.ScrollTo(item) generates the “offending” idle event?

I also implemented Josiah’s suggestion for serializing the mainThreadMethod calls, while keeping the ScrollTo call . I was stuck for awhile because the GUI kept hanging and becoming unresponsive. I added some tracing code as shown below and found the following:

q = Queue.Queue()
p = 0

def _process():
global p
if not p:
p = 1
try:
print “_process.try”

       fcn, args, kwargs = q.get()
       fcn(*args, **kwargs)

finally:

print “_process.finally”
p = 0
else:

print “_process.else”
wx.CallAfter(_process)

This behaved as expected until the else clause was entered. At that point it spun in a tight loop calling _process with p==1 without letting other code execute. The first call to _process that was interrupted never had a chance to continue, so p was never set back to 0. I fixed this by changing the else clause wx.CallAfter to wx.FutureCall(100). Does the time delay need to be long enough to let the first mainThreadMethod (fcn in the above code) complete, or would a shorter time that just breaks the loop be OK? How short could it be?

Finally, a general question. It seems that it is part of the overall design to allow events and their handlers to be “nestable”. In cases like mine, where real-time messages from another thread drive a state machine that in turn updates a variety of widgets, this became problematic. While Josiah’s code cleverly overcomes this, I’m wondering if my choice of design is flawed, or if there are other design patterns that avoid these pitfalls. I’d welcome any thoughts or suggestions.

  • Chris
···

On 5/20/07, Robin Dunn robin@alldunn.com wrote:

Paint events are low priority messages, and so are usually sent as the
last message in the event queue, which would place them right before the

idle events, which is where the wx.CallAfter functions are called from.
If you’re getting a paint event before the first function call
completes then something is definitely causing a nested event loop to

happen. What is the last thing in your mainThreadMethod that executes
before the paint event and the nested mainThreadMethod is called?

15

> Paint events are low priority messages, and so are usually sent as the
> last message in the event queue, which would place them right before the
> idle events, which is where the wx.CallAfter functions are called from.
> If you're getting a paint event before the first function call
> completes then something is definitely causing a nested event loop to
> happen. What is the last thing in your mainThreadMethod that executes
> before the paint event and the nested mainThreadMethod is called?

Robin, I've tracked this down to a wx.TreeCtrl.ScrollTo(item) call. I get a
paint event in mainThreadMethod about a millisecond after making this call.
If I use my original code, and don't make this call, then the
wx.CallAfterfunctions behave nicely. One of the GUI updates in
mainThreadMethod is
sometimes adding an item to the tree, and I require the last item added to
show in the window, so I must scroll if I add an item. Should I assume that
wx.TreeCtrl.ScrollTo(item) generates the "offending" idle event?

You may want to try doing wx.CallAfter(tree.ScrollTo, item). That would
delay the painting until the next run of the event loop, and would let
the _process() function to finish.

[snip]

This behaved as expected until the else clause was entered. At that point
it spun in a tight loop calling _process with p==1 without letting other
code execute. The first call to _process that was interrupted never had a
chance to continue, so p was never set back to 0. I fixed this by changing
the else clause wx.CallAfter to wx.FutureCall(100). Does the time delay
need to be long enough to let the first mainThreadMethod (fcn in the above
code) complete, or would a shorter time that just breaks the loop be OK?
How short could it be?

The underlying platform timers are generally less precise than the 1ms
precision that wxTimers (that wx.FutureCall uses) claim. I would start
halving the times on the futurecall until it either stopped working or
it got to 1.

Finally, a general question. It seems that it is part of the overall design
to allow events and their handlers to be "nestable". In cases like mine,
where real-time messages from another thread drive a state machine that in
turn updates a variety of widgets, this became problematic. While Josiah's
code cleverly overcomes this, I'm wondering if my choice of design is
flawed, or if there are other design patterns that avoid these pitfalls.
I'd welcome any thoughts or suggestions.

I've never run into a situation like you are experiencing, but it seems
as though you have managed to work around it. In terms of design
patterns, about the only one that I can think of that would help here is
the MVC pattern. Right now, it seems as though your model and view are
combined. If you separated the model and the view, then you could just
toss events at the view whenever the model changed (using the custom
events tutorial or wx.CallAfter or wx.FutureCall), and scan your model
for changes. Heck, you could do things like...

    while not q.empty():
        #process event

Where you handle any outstanding updates.

- Josiah

···

"chris botos" <chris.botos@gmail.com> wrote:

On 5/20/07, Robin Dunn <robin@alldunn.com> wrote:

16

chris botos wrote:

    Paint events are low priority messages, and so are usually sent as the
    last message in the event queue, which would place them right before
    the
    idle events, which is where the wx.CallAfter functions are called from.
    If you're getting a paint event before the first function call
    completes then something is definitely causing a nested event loop to
    happen. What is the last thing in your mainThreadMethod that executes
    before the paint event and the nested mainThreadMethod is called?

Robin, I've tracked this down to a wx.TreeCtrl.ScrollTo(item) call. I get a paint event in mainThreadMethod about a millisecond after making this call. If I use my original code, and don't make this call, then the wx.CallAfter functions behave nicely. One of the GUI updates in mainThreadMethod is sometimes adding an item to the tree, and I require the last item added to show in the window, so I must scroll if I add an item. Should I assume that wx.TreeCtrl.ScrollTo(item) generates the "offending" idle event?

Yep, in 2.6.3.3 it called wxYieldIfNeeded. That has been changed since then.

···

On 5/20/07, *Robin Dunn* <robin@alldunn.com <mailto:robin@alldunn.com>> > wrote:

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

17

Josiah, I had gone to some lengths to try to implement an MVP-like design, to the extent I grasp it. My “mainThreadMethod” actually is part of my “Presenter” class, and itself does not talk directly to widgets, except through an interface layer, and by issuing Publisher.sendMessage calls. It’s within one of the listeners that tree.ScrollTo is called. I will be rethinking this. Thanks again for your suggestions and help.

-Chris

···

On 5/22/07, Josiah Carlson jcarlson@uci.edu wrote:

You may want to try doing wx.CallAfter(tree.ScrollTo, item). That would
delay the painting until the next run of the event loop, and would let

the _process() function to finish.

The underlying platform timers are generally less precise than the 1ms
precision that wxTimers (that wx.FutureCall
uses) claim. I would start
halving the times on the futurecall until it either stopped working or
it got to 1.

I’ve never run into a situation like you are experiencing, but it seems
as though you have managed to work around it. In terms of design

patterns, about the only one that I can think of that would help here is
the MVC pattern. Right now, it seems as though your model and view are
combined. If you separated the model and the view, then you could just

toss events at the view whenever the model changed (using the custom
events tutorial or wx.CallAfter or wx.FutureCall), and scan your model
for changes.

18

I guess I’d better think about upgrading. Thanks for the help!

-Chris

···

On 5/22/07, Robin Dunn robin@alldunn.com wrote:

chris botos wrote:

Should I assume that wx.TreeCtrl.ScrollTo(item) generates the
“offending” idle event?

Yep, in 2.6.3.3 it called wxYieldIfNeeded. That has been changed since
then.