wxPython and Multiprocessing issues

Hi Together,
I'm actually writing a commercial program with a wxPython GUI.

Here a short description of what the GUI does:
The Gui opens a wizard which collects different data from the user's
input and saves it into a dictionary. After finishing the Wizard the
Worker Class is called.

Now that's the point where I need your help. Actually only one working
Object is possible, but I would like to parellelize this using the
multiprocessing module of python (Threading is not senseful here -
tested). Unfortunately the multiprocessing module isn't as easy to
implement as i've hoped. As it's a commercial project I can't offer
you code, but I can give you here an insight how i'm calling the
process.

But this doesn't work here is the traceback:

    Traceback (most recent call last):
      File "D:\eclipse_workspace_peanuts\qwert\src\Gui\Aui
\mPaneLeft.py", line 89, in start
        self.worker.start()
      File "C:\Python26\lib\multiprocessing\process.py", line 104, in
start
        self._popen = Popen(self)
      File "C:\Python26\lib\multiprocessing\forking.py", line 239, in
__init__
        dump(process_obj, to_child, HIGHEST_PROTOCOL)
      File "C:\Python26\lib\multiprocessing\forking.py", line 162, in
dump
        ForkingPickler(file, protocol).dump(obj)
      File "C:\Python26\lib\pickle.py", line 224, in dump
        self.save(obj)
      File "C:\Python26\lib\pickle.py", line 331, in save
        self.save_reduce(obj=obj, *rv)
      File "C:\Python26\lib\pickle.py", line 419, in save_reduce
        save(state)
      File "C:\Python26\lib\pickle.py", line 286, in save
        f(self, obj) # Call unbound method with explicit self
      File "C:\Python26\lib\pickle.py", line 649, in save_dict
        self._batch_setitems(obj.iteritems())
      File "C:\Python26\lib\pickle.py", line 681, in _batch_setitems
        save(v)
      File "C:\Python26\lib\pickle.py", line 331, in save
        self.save_reduce(obj=obj, *rv)
      File "C:\Python26\lib\pickle.py", line 419, in save_reduce
        save(state)
      File "C:\Python26\lib\pickle.py", line 286, in save
        f(self, obj) # Call unbound method with explicit self
      File "C:\Python26\lib\pickle.py", line 649, in save_dict
        self._batch_setitems(obj.iteritems())
      File "C:\Python26\lib\pickle.py", line 681, in _batch_setitems
        save(v)
      File "C:\Python26\lib\pickle.py", line 331, in save
        self.save_reduce(obj=obj, *rv)
      File "C:\Python26\lib\pickle.py", line 419, in save_reduce
        save(state)
      File "C:\Python26\lib\pickle.py", line 286, in save
        f(self, obj) # Call unbound method with explicit self
      File "C:\Python26\lib\pickle.py", line 649, in save_dict
        self._batch_setitems(obj.iteritems())
      File "C:\Python26\lib\pickle.py", line 681, in _batch_setitems
        save(v)
      File "C:\Python26\lib\pickle.py", line 295, in save
        self.save_global(obj)
      File "C:\Python26\lib\pickle.py", line 748, in save_global
        (obj, module, name))
    pickle.PicklingError: Can't pickle <class 'ctypes._FuncPtr'>: it's
not found as ctypes._FuncPtr
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Python26\lib\multiprocessing\forking.py", line 342, in
main
        self = load(from_parent)
      File "C:\Python26\lib\pickle.py", line 1370, in load
        return Unpickler(file).load()
      File "C:\Python26\lib\pickle.py", line 858, in load
        dispatch[key](self)
      File "C:\Python26\lib\pickle.py", line 880, in load_eof
        raise EOFError
    EOFError

My first thought was that this error is throwed because i'm using
datatypes which can't be serialized. But i tried it with pickle.dump()
and it worked, so there has to be anything else.

I already talked to Dav1d and driscollis. Dav1d told me he already had
similar issues with wxpython + multiprocessing.

It would be awesome if you could give me a solution to this :slight_smile:

Here a short description of what the GUI does:
The Gui opens a wizard which collects different data from the user's
input and saves it into a dictionary. After finishing the Wizard the
Worker Class is called.
I already talked to Dav1d and driscollis. Dav1d told me he already had
similar issues with wxpython + multiprocessing.

Does this have anything to do with wx? Can you run the worker processes with no GUI at all?

If you are using wx classes to store/pass data to your workers, then wx could be an issue, but I'd suggest that you don't do that -- stick with simipler, ideally pure python classes for teh data you need to pass around.

It would be awesome if you could give me a solution to this :slight_smile:

if wx has nothing to do with your issues, then I'd try another list - comp.lang.python, for instance -- I don't know if there is a better list for multi-processing issues.

-Chris

···

On 9/13/11 7:22 AM, pyMe wrote:

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

Thanks for the Answer Chris.
Today I found the source of my error message. You're right it has
nothing to do with wx, so this topic can be closed.

Just for your and others interest:
I'm using a ctypes.CDLL Object in my worker function and this can't
be pickled. This is why i'm getting this annoying error message.
Hopefully i can solve this :wink:

Regards
PyMe

···

On 13 Sep., 20:30, "Chris.Barker" <Chris.Bar...@noaa.gov> wrote:

On 9/13/11 7:22 AM, pyMe wrote:

> Here a short description of what the GUI does:
> The Gui opens a wizard which collects different data from the user's
> input and saves it into a dictionary. After finishing the Wizard the
> Worker Class is called.
> I already talked to Dav1d and driscollis. Dav1d told me he already had
> similar issues with wxpython + multiprocessing.

Does this have anything to do with wx? Can you run the worker processes
with no GUI at all?

If you are using wx classes to store/pass data to your workers, then wx
could be an issue, but I'd suggest that you don't do that -- stick with
simipler, ideally pure python classes for teh data you need to pass around.

> It would be awesome if you could give me a solution to this :slight_smile:

if wx has nothing to do with your issues, then I'd try another list -
comp.lang.python, for instance -- I don't know if there is a better list
for multi-processing issues.

-Chris

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Bar...@noaa.gov