"Background" thread freezing wx.ProgressDialog

wxPython Users
1

Hi folks,

I'm working on a wxPython project that runs background threads behind
ProgressDialogs.

Previously, only one such dialog existed in the project. The code to
handle the whole process was very ad-hoc and ugly, but it worked.

I am now adding several more tasks that display progress dialogs, so I
decided to DRY/clean up my approach. I thought a generator that
yielded the next message to display on the ProgressDialog could do the
thread's real work, and a subclass of threading.Thread could handle
the bookkeeping and communication with the dialog.

Having implemented the approach, I have found that it almost works. A
dialog is displayed, progress is made, and the 'success' code written
by the user is run once the task is finished.

However, my code appears to freeze the main thread somehow - if you
try to click the Cancel button during processing, no events seem to be
triggered, and eventually the 'processing' cursor will be displayed.

I thought I had used wx.CallAfter() everywhere I update GUI components
from the background thread, and that this would be sufficient to keep
the UI responsive.

I've spent some time poring over my code and trawling the web, trying
to piece together why my old code worked and my new approach does not.
I have not been able to work it out thus far, and now seek advice from
those wiser than me.

Obviously, I've misunderstood something. What, I'm not sure.

I should note that I found running even a short time.sleep() inside
the ProgressThread.do_work() generator makes this problem 'go away'. I
doubt that's relevant, but I thought it was worth mentioning.

Here's a gist with the sample file. This was tested on Mac OS X
10.6.8, with wxPython 2.8.10.1 (Unicode version), and Python 2.6.5.

https://gist.github.com/2577876

You should be able to download it, unpack it, make progress.py
executable, and do './progress.py' on a *nix system.

(The old code is very entangled with app-specific functionality and
does not extract to an example at all, so I have not included it.)

Can anyone explain what I'm doing wrong?

Thanks for reading.

-Nate

2

It is relevant. It indicates that the worker thread is overloading the gui thread in some way, either by stealing all resources so it is unable to get any run time, or by flooding the input queue of the gui so it is not able to process any events. Glancing at your code leads me to think that it is the latter. Try not calling Update (via wx.CallAfter) unless the value has changed and the progress bar and/or the message actually need to be updated.

···

On 5/2/12 9:42 AM, NateEag wrote:

Hi folks,

I'm working on a wxPython project that runs background threads behind
ProgressDialogs.

Previously, only one such dialog existed in the project. The code to
handle the whole process was very ad-hoc and ugly, but it worked.

I am now adding several more tasks that display progress dialogs, so I
decided to DRY/clean up my approach. I thought a generator that
yielded the next message to display on the ProgressDialog could do the
thread's real work, and a subclass of threading.Thread could handle
the bookkeeping and communication with the dialog.

Having implemented the approach, I have found that it almost works. A
dialog is displayed, progress is made, and the 'success' code written
by the user is run once the task is finished.

However, my code appears to freeze the main thread somehow - if you
try to click the Cancel button during processing, no events seem to be
triggered, and eventually the 'processing' cursor will be displayed.

I thought I had used wx.CallAfter() everywhere I update GUI components
from the background thread, and that this would be sufficient to keep
the UI responsive.

I've spent some time poring over my code and trawling the web, trying
to piece together why my old code worked and my new approach does not.
I have not been able to work it out thus far, and now seek advice from
those wiser than me.

Obviously, I've misunderstood something. What, I'm not sure.

I should note that I found running even a short time.sleep() inside
the ProgressThread.do_work() generator makes this problem 'go away'. I
doubt that's relevant, but I thought it was worth mentioning.

--
Robin Dunn
Software Craftsman

3

Thanks for the quick response!

I'd wondered if I might be feeding it too many events too quickly, but since my previous approach had been doing the same workload and updating the dialog at roughly the same rate, I didn't think that was likely to be it. I'll go over it again and see if I can catch anything related to that.

My actual use case is decompressing a tarball with the tarfile module. With my old code, calling Update() for each file in the tarball seems to work fine. With the new code, I ran into the "cannot click cancel" issue.

Do you have any thoughts on how to find the best rate for Update()-ing the dialog?

-Nate

···

On May 2, 2012, at 2:34 PM, Robin Dunn wrote:

On 5/2/12 9:42 AM, NateEag wrote:

I should note that I found running even a short time.sleep() inside
the ProgressThread.do_work() generator makes this problem 'go away'. I
doubt that's relevant, but I thought it was worth mentioning.

It is relevant. It indicates that the worker thread is overloading the gui thread in some way, either by stealing all resources so it is unable to get any run time, or by flooding the input queue of the gui so it is not able to process any events. Glancing at your code leads me to think that it is the latter. Try not calling Update (via wx.CallAfter) unless the value has changed and the progress bar and/or the message actually need to be updated.

4

In hopes of preventing the DenverCoder9 problem (http://xkcd.com/
979/), you can see the solution I eventually came to at the gist I
posted: A way to DRY wxPython progress dialogs for background threads. · GitHub

Summary:

Keep a timestamp of the last time you updated the dialog. Whenever you
have an opportunity to update the dialog, only do so if a 'safe'
amount of time has passed. I used 34 milliseconds, about a thirtieth
of a second, which looked good and seemed to work well.

Thanks again for your help, Robin. You got me thinking on the right
track, and after that it was pretty straightforward.

-Nate

···

On May 2, 3:19 pm, Nate Eagleson <nat...@gmail.com> wrote:

On May 2, 2012, at 2:34 PM, Robin Dunn wrote:

> It is relevant. It indicates that the worker thread is overloading the gui thread in some way, either by stealing all resources so it is unable to get any run time, or by flooding the input queue of the gui so it is not able to process any events. Glancing at your code leads me to think that it is the latter. Try not calling Update (via wx.CallAfter) unless the value has changed and the progress bar and/or the message actually need to be updated.

Thanks for the quick response!

I'd wondered if I might be feeding it too many events too quickly, but since my previous approach had been doing the same workload and updating the dialog at roughly the same rate, I didn't think that was likely to be it. I'll go over it again and see if I can catch anything related to that.

My actual use case is decompressing a tarball with the tarfile module. With my old code, calling Update() for each file in the tarball seems to work fine. With the new code, I ran into the "cannot click cancel" issue.

Do you have any thoughts on how to find the best rate for Update()-ing the dialog?

-Nate

5

That's basically what I was going to suggest as well. You may also want to think about the case where the last update may be changing the message, but comes less than 34 ms after the previous update. If you skip it then the user won't see that message...

···

On 5/2/12 5:46 PM, NateEag wrote:

On May 2, 3:19 pm, Nate Eagleson<nat...@gmail.com> wrote:

On May 2, 2012, at 2:34 PM, Robin Dunn wrote:

It is relevant. It indicates that the worker thread is overloading the gui thread in some way, either by stealing all resources so it is unable to get any run time, or by flooding the input queue of the gui so it is not able to process any events. Glancing at your code leads me to think that it is the latter. Try not calling Update (via wx.CallAfter) unless the value has changed and the progress bar and/or the message actually need to be updated.

Thanks for the quick response!

I'd wondered if I might be feeding it too many events too quickly, but since my previous approach had been doing the same workload and updating the dialog at roughly the same rate, I didn't think that was likely to be it. I'll go over it again and see if I can catch anything related to that.

My actual use case is decompressing a tarball with the tarfile module. With my old code, calling Update() for each file in the tarball seems to work fine. With the new code, I ran into the "cannot click cancel" issue.

Do you have any thoughts on how to find the best rate for Update()-ing the dialog?

-Nate

In hopes of preventing the DenverCoder9 problem (http://xkcd.com/
979/), you can see the solution I eventually came to at the gist I
posted: A way to DRY wxPython progress dialogs for background threads. · GitHub

Summary:

Keep a timestamp of the last time you updated the dialog. Whenever you
have an opportunity to update the dialog, only do so if a 'safe'
amount of time has passed. I used 34 milliseconds, about a thirtieth
of a second, which looked good and seemed to work well.

--
Robin Dunn
Software Craftsman