Yes, the cells contain strings, not integers. The calculation is not made
up of summing cell values.
That's sort-of what you wrote, though.
There are look-ups, conversions, additions etc.
Therefore the calculation takes long time.
Can you cache some (a lot) of the lookups or conversions you mention?
and only recalculate part of the whole dataset?
As for multiprocessing:
I don't see the need for an extra thread to manage this,
just have a wx.Timer pull your results from the processes.
Something like:
from multiprocessing import Process, Queue
from Queue import Empty
if __name__ == '__main__':
input=Queue()
output=Queue()
for x in xrange(0,num_procs):
p = Process(target=calculating_function, args=(input,output))
p.daemon=True
p.start()
def calculating_function(input,output):
run=True
while run:
data=input.get() # this waits until it can get something from the queue
result=...(data)
output.put(result)
def OnCalculate(self,event):
input.put(dataset[0])
input.put(dataset[1])
you can also do something like
input.put( ( 'sumup', dataset[0] ) )
so your processes can do different tasks.
and somewhere in your UI ( wx.Timer called function )
try:
result=output.get_nowait()
put_in_grid(result)
except Empty:
pass
Caveat:
Starting your processes will import/re-run your main application.
Take care of what you put inside and outside of __name__ == '__main__'.
···
On Sun, 20 Jul 2014 07:04:33 +0200, steve <oslocourse@gmail.com> wrote:
On Sunday, July 20, 2014 2:56:06 AM UTC+3, Michael Ross wrote:
On Sat, 19 Jul 2014 23:14:02 +0200, steve <osloc...@gmail.com >> <javascript:>> wrote:
> Hi,
>
> In my wxpython GUI application, I have a huge wx.grid object (1000 rows,
> 4000 columns = 4,000,000 cells), which contains values in each cell.
>
> I will do calculations on this wx.grid (say, sum the values in all
> cells).
>
> I also want the GUI responsive during grid calculation.
> Since there are 4,000,000 cells, the calculation takes too long time. I
> have 8 cores on my PC's processor, so why not take advantage of it?
>
> Therefore I decided to use threading and multiprocessing at the same
time
> (I don't know if it is possible or the best idea):
>
> 1- When user presses "CALCULATE" button, a new thread ("grid calculation
> thread") will do the calculation, and it will prevent the GUI becoming
> unresponsive.
> 2-" The grid calculation thread" will use python's multiprocessing
> module,
> it will create 2 processes. Each process will get 2,000,000 cells and do
> the calculation (sum)
> 3- Then, " The grid calculation thread" will get the results from these
2
> processes, sum them, and update the GUI (for example it will write the
> result to a StaticText)
>
> Is this possible? Or is there a better way to do this?
>
I do share C M's doubts about the usability of this, but:
It is certainly possible to do this multiprocessing-thing,
but I'd like to ask, why does it take "too long"?
I tried on my machine:
>>> from timeit import timeit
>>> def sumup():
s=0
for x in c:
s+=x
>>> c=
>>> for x in xrange(0,4000000):
c.append(123.456)
>>> timeit(sumup,number=1)
0.3921653333151198
That's 0.4 seconds for adding 4 million floats in a list.
With a dictionary:
>>> c={}
>>> for x in xrange(0,1000):
c={}
for y in xrange(0,4000):
c[y]=123.456
>>> c={}
>>> for x in xrange(0,1000):
c={}
for y in xrange(0,4000):
c[y]=123.456
>>> def sumup():
s=0
for x in c.iterkeys():
for y in c.itervalues():
s+=y
>>> timeit(sumup,number=1)
0.38582007965885623
Why "too long" on your end?
I'd try for improvment of the calculation time before doing complex
thread-multiprocess-setups.
... let me guess... Grids require values to be strings, right?
Do you convert string-to-float for every value when you do this
calculation?
Michael