GUI is unresponsive during long running, recursive task

RedHotChiliPepper · October 29, 2013, 12:39pm

I have an app that inserts filenames into a sortable listctrl. When you drag and drop a folder, all the .mp3 files in that directory or any subdirectory are parsed.

However if there’s directory with many files, the GUI becomes unresponsive. I read the article on Long Running Tasks http://wiki.wxpython.org/LongRunningTasks and evaluated the different approaches but am still unsure of what the best approach is and what the correct syntax would be to implement it into my application so it doesn’t “Lock Up”

I’m attaching a copy of my code. Can someone provide guidance and some simple syntax to make it not “Lock Up”?

longRunningRecursiveTask.py (2.99 KB)

Mike_Driscoll0 · October 29, 2013, 2:56pm

I would take the function or functions that take a long time and put them into a thread. In your case, I would spin up the thread when OnDropFiles is called and do your loop in the thread. I would also put the DecomposeDirectory into the thread class. Then when you want to update the list control, you would use one of wxPython’s thread-safe methods, such as wx.CallAfter. So in your case, you would change

self.window.dropUpdate(full_listing)

to something like

wx.CallAfter(self.window.dropUpdate, full_listing)

You’ll probably have to tweak it a bit to get it to work right. Perhaps my tutorial on the subject would help: wxPython and Threads - Mouse Vs Python

I hope that all made sense.

Mike

···

On Tuesday, October 29, 2013 7:39:53 AM UTC-5, RedHotChiliPepper wrote:

I have an app that inserts filenames into a sortable listctrl. When you drag and drop a folder, all the .mp3 files in that directory or any subdirectory are parsed.

However if there’s directory with many files, the GUI becomes unresponsive. I read the article on Long Running Tasks http://wiki.wxpython.org/LongRunningTasks and evaluated the different approaches but am still unsure of what the best approach is and what the correct syntax would be to implement it into my application so it doesn’t “Lock Up”

I’m attaching a copy of my code. Can someone provide guidance and some simple syntax to make it not “Lock Up”?

RedHotChiliPepper · December 23, 2013, 6:59am

Hi Mike -

Sorry for the delayed response. I tried your solution, creating a Thread class that runs when the files are dropped and putting the functions that take a long time inside this class. I also use the wx.CallAfter method to update the list control.

However the GUI is still hanging when I drag and drop files. I’m attaching my attempt from the previous attachment which did not include the threaded class.

Would you, or anyone else be able to advise me on what I’m doing wrong?

longRunningRecursiveTask_after.py (3.3 KB)

longRunningRecursiveTask.py (2.99 KB)

···

On Tuesday, October 29, 2013 10:56:06 AM UTC-4, Mike Driscoll wrote:

On Tuesday, October 29, 2013 7:39:53 AM UTC-5, RedHotChiliPepper wrote:

I have an app that inserts filenames into a sortable listctrl. When you drag and drop a folder, all the .mp3 files in that directory or any subdirectory are parsed.

However if there’s directory with many files, the GUI becomes unresponsive. I read the article on Long Running Tasks http://wiki.wxpython.org/LongRunningTasks and evaluated the different approaches but am still unsure of what the best approach is and what the correct syntax would be to implement it into my application so it doesn’t “Lock Up”

I’m attaching a copy of my code. Can someone provide guidance and some simple syntax to make it not “Lock Up”?

I would take the function or functions that take a long time and put them into a thread. In your case, I would spin up the thread when OnDropFiles is called and do your loop in the thread. I would also put the DecomposeDirectory into the thread class. Then when you want to update the list control, you would use one of wxPython’s thread-safe methods, such as wx.CallAfter. So in your case, you would change

self.window.dropUpdate(full_listing)

to something like

wx.CallAfter(self.window.dropUpdate, full_listing)

You’ll probably have to tweak it a bit to get it to work right. Perhaps my tutorial on the subject would help: http://www.blog.pythonlibrary.org/2010/05/22/wxpython-and-threads/

I hope that all made sense.

Mike

Rufus · December 25, 2013, 1:45am

It has been a while since I've played with drag and drop, but I may be revisiting it soon.

Are you processing the files during the drop event? Or are you only saving the file name from the
drop during the event and queuing up a "start processing thread" call with wx.CallAfter?

···

On 12/23/2013 1:59 AM, RedHotChiliPepper wrote:

Hi Mike -

Sorry for the delayed response. I tried your solution, creating a Thread class that runs when the files are dropped and putting the functions that take a long time inside this class. I also use the wx.CallAfter method to update the list control.

However the GUI is still hanging when I drag and drop files. I'm attaching my attempt from the previous attachment which did not include the threaded class.

Would you, or anyone else be able to advise me on what I'm doing wrong?

On Tuesday, October 29, 2013 10:56:06 AM UTC-4, Mike Driscoll wrote:

    On Tuesday, October 29, 2013 7:39:53 AM UTC-5, RedHotChiliPepper > wrote:

        I have an app that inserts filenames into a sortable listctrl.
         When you drag and drop a folder, all the .mp3 files in that
        directory or any subdirectory are parsed.

        However if there's directory with many files, the GUI becomes
        unresponsive. I read the article on Long Running Tasks
        LongRunningTasks - wxPyWiki
        <http://wiki.wxpython.org/LongRunningTasks> and evaluated the
        different approaches but am still unsure of what the best
        approach is and what the correct syntax would be to implement
        it into my application so it doesn't "Lock Up"

        I'm attaching a copy of my code. Can someone provide guidance
        and some simple syntax to make it not "Lock Up"?

    I would take the function or functions that take a long time and
    put them into a thread. In your case, I would spin up the thread
    when OnDropFiles is called and do your loop in the thread. I would
    also put the DecomposeDirectory into the thread class. Then when
    you want to update the list control, you would use one of
    wxPython's thread-safe methods, such as wx.CallAfter. So in your
    case, you would change

    self.window.dropUpdate(full_listing)

    to something like

    wx.CallAfter(self.window.dropUpdate, full_listing)

    You'll probably have to tweak it a bit to get it to work right.
    Perhaps my tutorial on the subject would help:
    wxPython and Threads - Mouse Vs Python
    <http://www.blog.pythonlibrary.org/2010/05/22/wxpython-and-threads/>

    I hope that all made sense.

    Mike

--
You received this message because you are subscribed to the Google Groups "wxPython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wxpython-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

RedHotChiliPepper · December 25, 2013, 9:49am

Hi Rufus -

I believe the files are being processed during a drop event. Can you confirm this by taking a look at the longRunningRecursiveTask_after.py attachment?

longRunningRecursiveTask_after.py (3.3 KB)

longRunningRecursiveTask.py (2.99 KB)

···

On Tuesday, December 24, 2013 5:45:56 PM UTC-8, Rufus wrote:

  It has been a while since I've played

with drag and drop, but I may be revisiting it soon.

  Are you processing the files during the drop event?ï¿½ Or are you

only saving the file name from the

  drop during the event and queuing up a "start processing thread"

call with wx.CallAfter?

  On 12/23/2013 1:59 AM, RedHotChiliPepper wrote:

Hi Mike -

      Sorry for the delayed response. ï¿½I tried your solution,
creating a Thread class that runs when the files are dropped
and putting the functions that take a long time inside this
class. ï¿½I also use the wx.CallAfter method to update the list
control.

      However the GUI is still hanging when I drag and drop
files. ï¿½I’m attaching my attempt from the previous attachment
which did not include the threaded class.

      Would you, or anyone else be able to advise me on what I'm

doing wrong?

      On Tuesday, October 29, 2013 10:56:06 AM UTC-4, Mike Driscoll > > wrote:

          On Tuesday, October 29, 2013 7:39:53 AM UTC-5, > > > RedHotChiliPepper wrote:
              I have an app that inserts filenames into
a sortable listctrl. ï¿½When you drag and drop a folder,
all the .mp3 files in that directory or any
subdirectory are parsed.

                However if there's directory with many files, the
GUI becomes unresponsive. ï¿½I read the article on
Long Running Tasksï¿½http://wiki.wxpython.org/LongRunningTasks ï¿½and
evaluated the different approaches but am still
unsure of what the best approach is and what the
correct syntax would be to implement it into my
application so it doesn’t “Lock Up”

                I'm attaching a copy of my code. ï¿½Can someone
provide guidance and some simple syntax to make it
not “Lock Up”?

            I would take the function or functions that take a long
time and put them into a thread. In your case, I would
spin up the thread when OnDropFiles is called and do
your loop in the thread. I would also put the
DecomposeDirectory into the thread class. Then when you
want to update the list control, you would use one of
wxPython’s thread-safe methods, such as wx.CallAfter. So
in your case, you would change
            self.window.dropUpdate(full_listing)



            to something like



            wx.CallAfter(self.window.dropUpdate, full_listing)



            You'll probably have to tweak it a bit to get it to work
right. Perhaps my tutorial on the subject would help: http://www.blog.pythonlibrary.org/2010/05/22/wxpython-and-threads/
            I hope that all made sense.



            Mike

–

  You received this message because you are subscribed to the Google

Groups “wxPython-users” group.

  To unsubscribe from this group and stop receiving emails from it,

send an email to wxpython-user...@googlegroups.com.

  For more options, visit [https://groups.google.com/groups/opt_out](https://groups.google.com/groups/opt_out).

Ricardo_Pedroso1 · December 25, 2013, 10:49pm

You have to many wx.CallAfter's in the threaded version.

I attached a two files, a fix for your _after sample and another way
of doing with generators.

PS: Tested only in linux.

longRunningRecursiveTask_after_fix.py (3.47 KB)

longRunningRecursiveTask_generators.py (1.68 KB)

···

On Wed, Dec 25, 2013 at 9:49 AM, RedHotChiliPepper <sharrukinjosephson@gmail.com> wrote:

Hi Rufus -

I believe the files are being processed during a drop event. Can you
confirm this by taking a look at the longRunningRecursiveTask_after.py
attachment?

On Tuesday, December 24, 2013 5:45:56 PM UTC-8, Rufus wrote:

It has been a while since I've played with drag and drop, but I may be
revisiting it soon.

Are you processing the files during the drop event?ï¿½ Or are you only
saving the file name from the

drop during the event and queuing up a "start processing thread" call with
wx.CallAfter?

On 12/23/2013 1:59 AM, RedHotChiliPepper wrote:

Hi Mike -

Sorry for the delayed response. ï¿½I tried your solution, creating a
Thread class that runs when the files are dropped and putting the functions
that take a long time inside this class. ï¿½I also use the wx.CallAfter
method to update the list control.

However the GUI is still hanging when I drag and drop files. ï¿½I'm
attaching my attempt from the previous attachment which did not include the
threaded class.

Would you, or anyone else be able to advise me on what I'm doing wrong?

RedHotChiliPepper · December 29, 2013, 12:25pm

Hi Ricardo -

I’m running on a mac. In longRunningRecursiveTask_after_fix.py the files are processed yet the gui hangs (just like before.)

I really like the approach with generators in longRunningRecursiveTask_generators.py. I modified your code to accept files being dropped instead of just directories and also made the listCtrl to be sortable.

However the dependency to the wx.EVT_IDLE event causes the delay between files being processed to be quite long.

self.Bind(wx.EVT_IDLE, self.OnIdle)

On my Mac, if I’m idle, a wx.EVT_IDLE event is emitted about 2-3 times per second.

I tried to remove the dependency to wx.EVT_IDLE to increase the speed at which files were processed, however this caused a hang in the gui. I’m attaching two files

longRunningRecursiveTask_generators.py - allows the gui to run without hanging but only processes 2-3 files per second if macbook is idle

longRunningRecursiveTask_generators_noIdle.py - gui hangs but processes thousands of files per second

Can you please advice me on what modifications I need to make to either attachment to get as many files processed per second, yet prevent the gui from hanging? Also if I’m using generators why does the for loop inside the run function hang in longRunningRecursiveTask_generators_noIdle.py ?

longRunningRecursiveTask_generators.py (2.73 KB)

longRunningRecursiveTask_generators_noIdle.py (2.38 KB)

···

On Wednesday, December 25, 2013 2:49:20 PM UTC-8, Ricardo Pedroso wrote:

On Wed, Dec 25, 2013 at 9:49 AM, RedHotChiliPepper > > sharrukin...@gmail.com wrote:

Hi Rufus -

I believe the files are being processed during a drop event. Can you

confirm this by taking a look at the longRunningRecursiveTask_after.py

attachment?

On Tuesday, December 24, 2013 5:45:56 PM UTC-8, Rufus wrote:

It has been a while since I’ve played with drag and drop, but I may be

revisiting it soon.

Are you processing the files during the drop event?ï¿½ Or are you only

saving the file name from the

drop during the event and queuing up a “start processing thread” call with

wx.CallAfter?

On 12/23/2013 1:59 AM, RedHotChiliPepper wrote:

Hi Mike -

Sorry for the delayed response. ï¿½I tried your solution, creating a

Thread class that runs when the files are dropped and putting the functions

that take a long time inside this class. ï¿½I also use the wx.CallAfter

method to update the list control.

However the GUI is still hanging when I drag and drop files. ï¿½I’m

attaching my attempt from the previous attachment which did not include the

threaded class.

Would you, or anyone else be able to advise me on what I’m doing wrong?

You have to many wx.CallAfter’s in the threaded version.

I attached a two files, a fix for your _after sample and another way

of doing with generators.

PS: Tested only in linux.

Chris_Barker1 · December 30, 2013, 6:55pm

However the dependency to the wx.EVT_IDLE event causes the delay between
files being processed to be quite long.

self.Bind(wx.EVT_IDLE, self.OnIdle)

On my Mac, if I'm idle, a wx.EVT_IDLE event is emitted about 2-3 times
per second.

the Idle event is tricky (and personally, I"ve never foudn a good use for

it), but I think it's not quite what you expect:

An EVT_IDLE is emitted whenever the event stack becomes empty. So if you
are doing somethign that generates a lot of events, like moving the mouse
and, and they are very quickily processed, then the stack is constantly
getting events on it, then emptying, and emitting an EVT_IDLE frequently.

But if the app is siting there, then the event stack simply sits empty, and
no EVT_IDLE gets emitted. So it's not really a good way to have constant
processing going on.

A few options:

1) use wx.CallAfter() to call a function that processes each file, and call
it again each time.

2) call wx.WakeUpIdle() in there somewhere to generate another EVT_IDLE

3) call wx.App.SafeYield() each time you process one file

4) use a wx.Timer to to create a (very) small delay between each file that
gets processed.

Sorry -- no time to test any of these out in your code....

-Chris

···

On Sun, Dec 29, 2013 at 4:25 AM, RedHotChiliPepper < sharrukinjosephson@gmail.com> wrote:

I tried to remove the dependency to wx.EVT_IDLE to increase the speed at
which files were processed, however this caused a hang in the gui. I'm
attaching two files

*longRunningRecursiveTask_generators.py *- allows the gui to run without
hanging but only processes 2-3 files per second if macbook is idle
*longRunningRecursiveTask_generators_noIdle.py* - gui hangs but processes
thousands of files per second

Can you please advice me on what modifications I need to make to either
attachment to get as many files processed per second, yet prevent the gui
from hanging? Also if I'm using generators why does the for loop inside
the run function hang in *longRunningRecursiveTask_generators_noIdle.py* ?

On Wednesday, December 25, 2013 2:49:20 PM UTC-8, Ricardo Pedroso wrote:

On Wed, Dec 25, 2013 at 9:49 AM, RedHotChiliPepper >> <sharrukin...@gmail.com> wrote:
> Hi Rufus -
>
> I believe the files are being processed during a drop event. Can you
> confirm this by taking a look at the longRunningRecursiveTask_after.py
> attachment?
>
>
>
> On Tuesday, December 24, 2013 5:45:56 PM UTC-8, Rufus wrote:
>>
>> It has been a while since I've played with drag and drop, but I may be
>> revisiting it soon.
>>
>> Are you processing the files during the drop event?ï¿½ Or are you only
>> saving the file name from the
>>
>> drop during the event and queuing up a "start processing thread" call
with
>> wx.CallAfter?
>>
>> On 12/23/2013 1:59 AM, RedHotChiliPepper wrote:
>>
>> Hi Mike -
>>
>> Sorry for the delayed response. ï¿½I tried your solution, creating a
>> Thread class that runs when the files are dropped and putting the
functions
>> that take a long time inside this class. ï¿½I also use the
wx.CallAfter
>> method to update the list control.
>>
>> However the GUI is still hanging when I drag and drop files. ï¿½I'm
>> attaching my attempt from the previous attachment which did not
include the
>> threaded class.
>>
>> Would you, or anyone else be able to advise me on what I'm doing
wrong?

You have to many wx.CallAfter's in the threaded version.

I attached a two files, a fix for your _after sample and another way
of doing with generators.

PS: Tested only in linux.

--
You received this message because you are subscribed to the Google Groups
"wxPython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to wxpython-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

Robin · December 30, 2013, 8:31pm

Chris Barker wrote:

···

On Sun, Dec 29, 2013 at 4:25 AM, RedHotChiliPepper > <sharrukinjosephson@gmail.com <mailto:sharrukinjosephson@gmail.com>> wrote:

    However the dependency to the wx.EVT_IDLE event causes the delay
    between files being processed to be quite long.

    self.Bind(wx.EVT_IDLE, self.OnIdle)

    On my Mac, if I'm idle, awx.EVT_IDLE event is emitted about 2-3
    times per second.

the Idle event is tricky (and personally, I"ve never foudn a good use
for it), but I think it's not quite what you expect:

An EVT_IDLE is emitted whenever the event stack becomes empty. So if you
are doing somethign that generates a lot of events, like moving the
mouse and, and they are very quickily processed, then the stack is
constantly getting events on it, then emptying, and emitting an EVT_IDLE
frequently.

But if the app is siting there, then the event stack simply sits empty,
and no EVT_IDLE gets emitted. So it's not really a good way to have
constant processing going on.

A few options:

1) use wx.CallAfter() to call a function that processes each file, and
call it again each time.

2) call wx.WakeUpIdle() in there somewhere to generate another EVT_IDLE

3) call wx.App.SafeYield() each time you process one file

4) use a wx.Timer to to create a (very) small delay between each file
that gets processed.

5) in your EVT_IDLE handler call event.RequestMore() if there are more files to be processed. That tells the system that if there are still no new pending events then go ahead and send another EVT_IDLE immediately. If there are other events waiting to be processed then they will be and then another EVT_IDLE will happen naturally as the event queue becomes empty again.

--
Robin Dunn
Software Craftsman

RedHotChiliPepper · December 30, 2013, 9:14pm

Thanks for all the help throughout this entire thread.

The event.RequestMore() method Robin suggested did exactly what I needed.

···

On Tuesday, October 29, 2013 5:39:53 AM UTC-7, RedHotChiliPepper wrote:

I have an app that inserts filenames into a sortable listctrl. When you drag and drop a folder, all the .mp3 files in that directory or any subdirectory are parsed.

However if there’s directory with many files, the GUI becomes unresponsive. I read the article on Long Running Tasks http://wiki.wxpython.org/LongRunningTasks and evaluated the different approaches but am still unsure of what the best approach is and what the correct syntax would be to implement it into my application so it doesn’t “Lock Up”

I’m attaching a copy of my code. Can someone provide guidance and some simple syntax to make it not “Lock Up”?

Dev_Player · January 1, 2014, 1:53am

I didn’t look at your code but you might want to check you are not blasting the event queue with IDLE events -after- you finish listing all the files into the widget. In other words I hope in your event handler you used something like
if len(filesToAdd): event.RequestMore() # so event.RequestMore() is not called after you emptied the filesToAdd queue.