wxPython on windows. Using threading still 'non responsive' screen

I've build a GUI app that runs a bunch of external commands.

Therefore i've used CallQueue:
http://code.activestate.com/recipes/491281-callqueue-easy-inter-thread-communication/

In my code we'll find the following snippet that does the actual
threading and in the mean time keep my screen updated.

==== Start code ====
        cq=callqueue.CallQueue(max_default_consumer_threads=6)
        starttime = time.time()
        while harvy.commandsLeft() > 0:
            cmd_name, cmd = harvy.commandsPopNext()
            cqitem=cq.call_and_collect(harvy.commandsRun,
(cmd_name,cmd)) #schedule
        while not cq.is_done():
            for cqitem in cq.get_next_collected(): #harvest
                a,b = cqitem.get_return()
                results[a] = b
                cmd_counter += 1
                starttime = time.time()
                oldtime=0
                self.lbl_pbar.SetLabel("Geduld aub. %s/%s tests
voltooid."%(cmd_counter,cmd_total))
                self.lbl_pbar2.SetLabel("Verwerken data")
                if cmd_counter == cmd_total:
                    self.pbar.Value = max
                else:
                    self.pbar.Value = (max/cmd_total)*cmd_counter
                self.Refresh()
                wx.Yield()
            newtime = int(time.time() - starttime)
            if newtime != oldtime:
                #time for a screenupdate
                oldtime = newtime
                if cmd_total - cmd_counter == 1:
                    self.lbl_pbar2.SetLabel("Geduld aub. wachten op de
laatste test [%s seconden]"%(newtime))
                else:
                    self.lbl_pbar2.SetLabel("Geduld aub. wachten op de
%s resterende tests [%s seconden]"%((cmd_total-cmd_counter),newtime))
                self.Refresh()
            time.sleep(0.1)
            #self.Refresh()
        self.lbl_pbar.SetLabel("All %s/%s tests zijn gedraaid, we gaan
nu de informatie uploaden"%(cmd_counter,cmd_total))
        self.lbl_pbar2.SetLabel("")
        self.Refresh()

==== end code piece ====

This IS functional, the screen gets appropriate updates during the
whole process as long as i don't touch the screen. When i minimize it
and reopen it, the screen is white. When i move it if freezes to the
'current' state. The tool keeps running however and will do the last
update after all threads are finished.

I'm running wxPython 2.8.11.0 (msw-unicode) on windows (Python 2.7.1)

Anyone can give me some hints on how to keep the screen active? (tried
wx.CallAfter, gives same problem and doesn't suit my needs)

I'm not an expert on threads, but it looks like you're calling
wxPython methods from this "threading" recipe, which is a no-no. I'm
guessing that the thread is blocking wxPython's mainloop, which is why
it isn't refreshing properly. Have a look at the following for tips:

http://wiki.wxpython.org/LongRunningTasks

···

On Jan 24, 4:32 am, xychix <m...@bergman.nl> wrote:

I've build a GUI app that runs a bunch of external commands.

Therefore i've used CallQueue:http://code.activestate.com/recipes/491281-callqueue-easy-inter-threa

In my code we'll find the following snippet that does the actual
threading and in the mean time keep my screen updated.

==== Start code ====
cq=callqueue.CallQueue(max_default_consumer_threads=6)
starttime = time.time()
while harvy.commandsLeft() > 0:
cmd_name, cmd = harvy.commandsPopNext()
cqitem=cq.call_and_collect(harvy.commandsRun,
(cmd_name,cmd)) #schedule
while not cq.is_done():
for cqitem in cq.get_next_collected(): #harvest
a,b = cqitem.get_return()
results[a] = b
cmd_counter += 1
starttime = time.time()
oldtime=0
self.lbl_pbar.SetLabel("Geduld aub. %s/%s tests
voltooid."%(cmd_counter,cmd_total))
self.lbl_pbar2.SetLabel("Verwerken data")
if cmd_counter == cmd_total:
self.pbar.Value = max
else:
self.pbar.Value = (max/cmd_total)*cmd_counter
self.Refresh()
wx.Yield()
newtime = int(time.time() - starttime)
if newtime != oldtime:
#time for a screenupdate
oldtime = newtime
if cmd_total - cmd_counter == 1:
self.lbl_pbar2.SetLabel("Geduld aub. wachten op de
laatste test [%s seconden]"%(newtime))
else:
self.lbl_pbar2.SetLabel("Geduld aub. wachten op de
%s resterende tests [%s seconden]"%((cmd_total-cmd_counter),newtime))
self.Refresh()
time.sleep(0.1)
#self.Refresh()
self.lbl_pbar.SetLabel("All %s/%s tests zijn gedraaid, we gaan
nu de informatie uploaden"%(cmd_counter,cmd_total))
self.lbl_pbar2.SetLabel("")
self.Refresh()

==== end code piece ====

This IS functional, the screen gets appropriate updates during the
whole process as long as i don't touch the screen. When i minimize it
and reopen it, the screen is white. When i move it if freezes to the
'current' state. The tool keeps running however and will do the last
update after all threads are finished.

I'm running wxPython 2.8.11.0 (msw-unicode) on windows (Python 2.7.1)

Anyone can give me some hints on how to keep the screen active? (tried
wx.CallAfter, gives same problem and doesn't suit my needs)

-------------------
Mike Driscoll

Blog: http://blog.pythonlibrary.org

It looks to me like you are still blocking the return to the main event loop until the command queue is processed. (Or is that code executing in a worker thread? If so then see #3 below.) Using a wx.Yield call can help but IMO it should only be used when nothing else works as having nested event loops can lead to problems.

Keeping a responsive and updated GUI is actually fairly easy if you follow these simple rules:

1. Do not do anything that could block an event handler or other UI callback from returning to the main event loop for longer than a "noticeable" amount of time, other than calling things like a dialog's ShowModal or a window's PopupMenu methods. The duration of a "noticable" amount of time depends on the context and also what else is going on in the application, but a general guideline would be something like 200ms. For anything more than that either show a busy cursor or do something to move that code out of the event handler code, such as run chunks of the big task in EVT_IDLE or EVT_TIMER events or move it to a worker thread.

2. Don't use wx.Yield or similar to get around #1. It may give you a quick fix now but sooner or later it is probably going to be the source of some hard to diagnose problem.

3. If you have worker threads that need to update or manipulate the UI in some way do not attempt to do it directly in the worker thread. Instead post a custom event or use wx.CallAfter to invoke some methods in your UI classes that will tell them to update themselves.

···

On 1/24/11 2:32 AM, xychix wrote:

I've build a GUI app that runs a bunch of external commands.

Therefore i've used CallQueue:
CallQueue: Easy inter-thread communication « Python recipes « ActiveState Code

In my code we'll find the following snippet that does the actual
threading and in the mean time keep my screen updated.

This IS functional, the screen gets appropriate updates during the
whole process as long as i don't touch the screen. When i minimize it
and reopen it, the screen is white. When i move it if freezes to the
'current' state. The tool keeps running however and will do the last
update after all threads are finished.

I'm running wxPython 2.8.11.0 (msw-unicode) on windows (Python 2.7.1)

Anyone can give me some hints on how to keep the screen active? (tried
wx.CallAfter, gives same problem and doesn't suit my needs)

--
Robin Dunn
Software Craftsman

cqitem=cq.call_and_collect(harvy.commandsRun,
(cmd_name,cmd)) #schedule

Thats where the thread gets started.

harvy.commandsRun function contains no wx. calls. only does his own job. Returning a value which in return gets returned by

a,b = cqitem.get_return()

···


ing. Mark Bergman RE CISA CISSP
g: +31 (0)6 18113618
e: mark@bergman.nl

pgp-key fingerprint: 02A5 689B 8BCC 525B 72F1 C845 E09E FE1D 774E 4C8C

On Jan 24, 2011, at 8:29 PM, Mike Driscoll wrote:

I’ve build a GUI app that runs a bunch of external commands.

Therefore i’ve used CallQueue:http://code.activestate.com/recipes/491281-callqueue-easy-inter-threa

In my code we’ll find the following snippet that does the actual
threading and in the mean time keep my screen updated.

==== Start code ====
cq=callqueue.CallQueue(max_default_consumer_threads=6)
starttime = time.time()
while harvy.commandsLeft() > 0:
cmd_name, cmd = harvy.commandsPopNext()
cqitem=cq.call_and_collect(harvy.commandsRun,
(cmd_name,cmd)) #schedule
while not cq.is_done():
for cqitem in cq.get_next_collected(): #harvest
a,b = cqitem.get_return()
results[a] = b
cmd_counter += 1
starttime = time.time()
oldtime=0
self.lbl_pbar.SetLabel(“Geduld aub. %s/%s tests
voltooid.”%(cmd_counter,cmd_total))
self.lbl_pbar2.SetLabel(“Verwerken data”)
if cmd_counter == cmd_total:
self.pbar.Value = max
else:
self.pbar.Value = (max/cmd_total)*cmd_counter
self.Refresh()
wx.Yield()
newtime = int(time.time() - starttime)
if newtime != oldtime:
#time for a screenupdate
oldtime = newtime
if cmd_total - cmd_counter == 1:
self.lbl_pbar2.SetLabel(“Geduld aub. wachten op de
laatste test [%s seconden]”%(newtime))
else:
self.lbl_pbar2.SetLabel(“Geduld aub. wachten op de
%s resterende tests [%s seconden]”%((cmd_total-cmd_counter),newtime))
self.Refresh()
time.sleep(0.1)
#self.Refresh()
self.lbl_pbar.SetLabel(“All %s/%s tests zijn gedraaid, we gaan
nu de informatie uploaden”%(cmd_counter,cmd_total))
self.lbl_pbar2.SetLabel(“”)
self.Refresh()

==== end code piece ====

This IS functional, the screen gets appropriate updates during the
whole process as long as i don’t touch the screen. When i minimize it
and reopen it, the screen is white. When i move it if freezes to the
‘current’ state. The tool keeps running however and will do the last
update after all threads are finished.

I’m running wxPython 2.8.11.0 (msw-unicode) on windows (Python 2.7.1)

Anyone can give me some hints on how to keep the screen active? (tried
wx.CallAfter, gives same problem and doesn’t suit my needs)

On Jan 24, 4:32 am, xychix m...@bergman.nl wrote:

I’m not an expert on threads, but it looks like you’re calling
wxPython methods from this “threading” recipe, which is a no-no. I’m
guessing that the thread is blocking wxPython’s mainloop, which is why
it isn’t refreshing properly. Have a look at the following for tips:

http://wiki.wxpython.org/LongRunningTasks
http://www.blog.pythonlibrary.org/2010/05/22/wxpython-and-threads/


Mike Driscoll

Blog: http://blog.pythonlibrary.org


To unsubscribe, send email to wxPython-users+unsubscribe@googlegroups.com
or visit http://groups.google.com/group/wxPython-users?hl=en

This code is in de main loop.
What is blocking in it?

The problem occurs while the main thread is in:

while not cq.is_done():
for cqitem in cq.get_next_collected(): #harvest
a,b = cqitem.get_return()
results[a] = b
cmd_counter += 1
starttime = time.time()
oldtime=0
self.lbl_pbar.SetLabel(“Geduld aub. %s/%s tests
voltooid.”%(cmd_counter,cmd_total))
self.lbl_pbar2.SetLabel(“Verwerken data”)
if cmd_counter == cmd_total:
self.pbar.Value = max
else:
self.pbar.Value = (max/cmd_total)*cmd_counter
self.Refresh()
wx.Yield()
newtime = int(time.time() - starttime)
if newtime != oldtime:
#time for a screenupdate
oldtime = newtime
if cmd_total - cmd_counter == 1:
self.lbl_pbar2.SetLabel(“Geduld aub. wachten op de
laatste test [%s seconden]”%(newtime))
else:
self.lbl_pbar2.SetLabel(“Geduld aub. wachten op de
%s resterende tests [%s seconden]”%((cmd_total-cmd_counter),newtime))
self.Refresh()
time.sleep(0.1)
#self.Refresh()

Does anyone see blocking code in this loop? because that sounds like the cause of my problems. And shouldn’t the GUI unfreeze after a consuming function (The a,b = cqitem.get_return() function might take a little time on big result values (10 KB of text) is finished?

···


ing. Mark Bergman RE CISA CISSP
g: +31 (0)6 18113618
e: mark@bergman.nl

pgp-key fingerprint: 02A5 689B 8BCC 525B 72F1 C845 E09E FE1D 774E 4C8C

On Jan 24, 2011, at 10:28 PM, Robin Dunn wrote:

I’ve build a GUI app that runs a bunch of external commands.

Therefore i’ve used CallQueue:
http://code.activestate.com/recipes/491281-callqueue-easy-inter-thread-communication/

In my code we’ll find the following snippet that does the actual
threading and in the mean time keep my screen updated.

This IS functional, the screen gets appropriate updates during the
whole process as long as i don’t touch the screen. When i minimize it
and reopen it, the screen is white. When i move it if freezes to the
‘current’ state. The tool keeps running however and will do the last
update after all threads are finished.

I’m running wxPython 2.8.11.0 (msw-unicode) on windows (Python 2.7.1)

Anyone can give me some hints on how to keep the screen active? (tried
wx.CallAfter, gives same problem and doesn’t suit my needs)

On 1/24/11 2:32 AM, xychix wrote:

It looks to me like you are still blocking the return to the main event loop until the command queue is processed. (Or is that code executing in a worker thread? If so then see #3 below.) Using a wx.Yield call can help but IMO it should only be used when nothing else works as having nested event loops can lead to problems.

Keeping a responsive and updated GUI is actually fairly easy if you follow these simple rules:

  1. Do not do anything that could block an event handler or other UI callback from returning to the main event loop for longer than a “noticeable” amount of time, other than calling things like a dialog’s ShowModal or a window’s PopupMenu methods. The duration of a “noticable” amount of time depends on the context and also what else is going on in the application, but a general guideline would be something like 200ms. For anything more than that either show a busy cursor or do something to move that code out of the event handler code, such as run chunks of the big task in EVT_IDLE or EVT_TIMER events or move it to a worker thread.

  2. Don’t use wx.Yield or similar to get around #1. It may give you a quick fix now but sooner or later it is probably going to be the source of some hard to diagnose problem.

  3. If you have worker threads that need to update or manipulate the UI in some way do not attempt to do it directly in the worker thread. Instead post a custom event or use wx.CallAfter to invoke some methods in your UI classes that will tell them to update themselves.


Robin Dunn
Software Craftsman
http://wxPython.org


To unsubscribe, send email to wxPython-users+unsubscribe@googlegroups.com
or visit http://groups.google.com/group/wxPython-users?hl=en

The blocking code *is* the loop. If I'm reading it correctly you are waiting in the loop until everything in the call queue is finished before returning from this function, processing the results as they become available. That is preventing the current code from returning to the main loop until the worker thread(s) has finished and therefore is not allowing pending events to be processed normally until then. You need to let this thread continue without waiting and then periodically (such as with an IDLE or TIMER handler) come back and process any collected results that may be done at that point in time, again not blocking at that time either.

···

On 1/24/11 11:16 PM, Mark Bergman wrote:

This code is in de main loop.
What is blocking in it?

The problem occurs while the main thread is in:

while not cq.is_done():
for cqitem in cq.get_next_collected(): #harvest
a,b = cqitem.get_return()
results[a] = b
cmd_counter += 1
starttime = time.time()
oldtime=0
self.lbl_pbar.SetLabel("Geduld aub. %s/%s tests
voltooid."%(cmd_counter,cmd_total))
self.lbl_pbar2.SetLabel("Verwerken data")
if cmd_counter == cmd_total:
self.pbar.Value = max
else:
self.pbar.Value = (max/cmd_total)*cmd_counter
self.Refresh()
wx.Yield()
newtime = int(time.time() - starttime)
if newtime != oldtime:
#time for a screenupdate
oldtime = newtime
if cmd_total - cmd_counter == 1:
self.lbl_pbar2.SetLabel("Geduld aub. wachten op de
laatste test [%s seconden]"%(newtime))
else:
self.lbl_pbar2.SetLabel("Geduld aub. wachten op de
%s resterende tests [%s seconden]"%((cmd_total-cmd_counter),newtime))
self.Refresh()
time.sleep(0.1)
#self.Refresh()

Does anyone see blocking code in this loop?

--
Robin Dunn
Software Craftsman

tnx, got it working. Code is a mess right now but it works!

···


ing. Mark Bergman RE CISA CISSP
g: +31 (0)6 18113618
e: mark@bergman.nl

pgp-key fingerprint: 02A5 689B 8BCC 525B 72F1 C845 E09E FE1D 774E 4C8C

On Jan 25, 2011, at 7:18 PM, Robin Dunn wrote:

On 1/24/11 11:16 PM, Mark Bergman wrote:

This code is in de main loop.
What is blocking in it?

The problem occurs while the main thread is in:

while not cq.is_done():
for cqitem in cq.get_next_collected(): #harvest
a,b = cqitem.get_return()
results[a] = b
cmd_counter += 1
starttime = time.time()
oldtime=0
self.lbl_pbar.SetLabel(“Geduld aub. %s/%s tests
voltooid.”%(cmd_counter,cmd_total))
self.lbl_pbar2.SetLabel(“Verwerken data”)
if cmd_counter == cmd_total:
self.pbar.Value = max
else:
self.pbar.Value = (max/cmd_total)*cmd_counter
self.Refresh()
wx.Yield()
newtime = int(time.time() - starttime)
if newtime != oldtime:
#time for a screenupdate
oldtime = newtime
if cmd_total - cmd_counter == 1:
self.lbl_pbar2.SetLabel(“Geduld aub. wachten op de
laatste test [%s seconden]”%(newtime))
else:
self.lbl_pbar2.SetLabel(“Geduld aub. wachten op de
%s resterende tests [%s seconden]”%((cmd_total-cmd_counter),newtime))
self.Refresh()
time.sleep(0.1)
#self.Refresh()

Does anyone see blocking code in this loop?

The blocking code is the loop. If I’m reading it correctly you are waiting in the loop until everything in the call queue is finished before returning from this function, processing the results as they become available. That is preventing the current code from returning to the main loop until the worker thread(s) has finished and therefore is not allowing pending events to be processed normally until then. You need to let this thread continue without waiting and then periodically (such as with an IDLE or TIMER handler) come back and process any collected results that may be done at that point in time, again not blocking at that time either.


Robin Dunn
Software Craftsman
http://wxPython.org


To unsubscribe, send email to wxPython-users+unsubscribe@googlegroups.com
or visit http://groups.google.com/group/wxPython-users?hl=en