Weirdness with threads and .dll -> COM

wxPython Users
1

Hi Everyone,

I am experiencing a very weird problem with my wxPython application that I
believe is related to the internals of wxPython/wxWindows and the main
event thread. The situation is as follows (Windows 2000, Python 2.2.2,
wxPython 2.4.0.1):

My main application talks to a python-wrapped .dll which is a low-level
serial driver talking to a serial device. We have currently added a new
protocol which talks to a wireless device using a third-party .dll via COM.
Our .dll does a CoInitialize(NULL) to talk to the device in a
single-threaded apartment arrangement. Well, inside my application there
are basically two cases involved:

1) I need to do a long operation, so I fire off another thread to do the
serial operation, waiting in my main thread to be informed when it is
complete.

2) I need to execute a simple, serial command which will succeed or fail
immediately, so I just call it inline from the main wxApp thread, from
within a menu event handler, for instance.

Well, as it turns out, I am getting sporadic behaviour with my application
crashing sometimes but only when calling the simple functions from the main
thread. I am also seeing cases where I will call a simple command a few
times (from the main thread), and then try to execute a normal, purely
wxPython event handler and my application will simply vanish. No traceback,
no violent crash - it simply vanishes. So, for instance, I will execute a
quick serial operation a couple times (from the main thread), then goto my
File..Open.. menu (which simply brings up a wxFileDialog) - and the
application will disappear. Neither of these crashes happen very often,
and sometimes I have to try quite a few times to repeat the error. Also,
it only happens with the new device we have added which uses COM.

I cannot make the application crash with any of the commands which have
been wrapped to be executed in a separate thread. Similarly, if I add some
code to execute the simple commands in separate threads rather than the
main thread, the problem seems to go away as well. This is leading me to
believe that there is something funky going on with calling COM-based stuff
from the main thread of a wxApp.

Can anyone see why this may (or may not) be the case? I am not a COM
expert, but I have done quite a lot with wxPython, and this one is driving
me crazy. I am trying to minimize this to a simple case to confirm it is
not something weird in my app, but I am fairly confident there isn't.

BTW - I have tried making a multithreaded .dll using CoInitializeEx, then
importing pythoncom to force my application to advertise itself as being
multi-threaded as well, and the problem seems to go away in this case too.
I can then call the serial functions from the main thread, or from child
threads and I cannot make it crash. This further points to a threading
thing in my opinion...

Any help or thoughts would be appreciated.

Thanks,
Mark.

2

Mark Melvin wrote:

Can anyone see why this may (or may not) be the case? I am not a COM
expert, but I have done quite a lot with wxPython, and this one is driving
me crazy. I am trying to minimize this to a simple case to confirm it is
not something weird in my app, but I am fairly confident there isn't.

Sorry, but I am also fairly ignorant about the low levels of COM and possible interaction scenarios...

One possibility is that when you make the call to the "quick serial operation" from the main thread that pythoncom is not restoring the right Python thread state (or just creates a new one) and so when you try to go invoke another event handler the interpreter does a fatal exit because of an invalid tstate. Do you see any "tstate" messages if you run your app from a console window with python.exe? Usually they are printed to the real stderr, not sys.stderr.

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

3

This is interesting...I haven't yet been able to make it fail from the command line in order to catch any stderr messages (I have been trying this in the hopes of grabbing any useful output). Perhaps something I wasn't clear on was the fact that I am not actually using pythoncom (or win32all for that matter) at all in my program. The calls to COM are through the various functions in the C .dll - when I initiate a "connect()" from Python for instance. I did manage to fix the problem (it looks that way, anyway) by making a multithreaded version of the .dll, and importing pythoncom for the sole purpose of forcing my Python application to appear multithreaded as well. Otherwise, I get an exception inside my C .dll when connecting to the COM server that it can't change the "threading model".

Thanks for your help. I'll keep trying to simplify this.

···

On Sat, 30 Aug 2003 10:56:45 -0700, Robin Dunn <robin@alldunn.com> wrote:

Mark Melvin wrote:

Can anyone see why this may (or may not) be the case? I am not a COM
expert, but I have done quite a lot with wxPython, and this one is driving
me crazy. I am trying to minimize this to a simple case to confirm it is
not something weird in my app, but I am fairly confident there isn't.

Sorry, but I am also fairly ignorant about the low levels of COM and possible interaction scenarios...

One possibility is that when you make the call to the "quick serial operation" from the main thread that pythoncom is not restoring the right Python thread state (or just creates a new one) and so when you try to go invoke another event handler the interpreter does a fatal exit because of an invalid tstate. Do you see any "tstate" messages if you run your app from a console window with python.exe? Usually they are printed to the real stderr, not sys.stderr.

--
Mark

4

Can anyone see why this may (or may not) be the case? I am not a COM
expert, but I have done quite a lot with wxPython, and this one is driving
me crazy. I am trying to minimize this to a simple case to confirm it is
not something weird in my app, but I am fairly confident there isn't.

Sorry, but I am also fairly ignorant about the low levels of COM and possible interaction scenarios...

One possibility is that when you make the call to the "quick serial operation" from the main thread that pythoncom is not restoring the right Python thread state (or just creates a new one) and so when you try to go invoke another event handler the interpreter does a fatal exit because of an invalid tstate. Do you see any "tstate" messages if you run your app from a console window with python.exe? Usually they are printed to the real stderr, not sys.stderr.

This is interesting...I haven't yet been able to make it fail from the command line in order to catch any stderr messages (I have been trying this in the hopes of grabbing any useful output). Perhaps something I wasn't clear on was the fact that I am not actually using pythoncom (or win32all for that matter) at all in my program.

Ah, I hadn't realized that.

The calls to COM are through the various functions in the C .dll - when I initiate a "connect()" from Python for instance. I did manage to fix the problem (it looks that way, anyway) by making a multithreaded version of the .dll, and importing pythoncom for the sole purpose of forcing my Python application to appear multithreaded as well. Otherwise, I get an exception inside my C .dll when connecting to the COM server that it can't change the "threading model".

Okay.

Well - as it turns out we seem to have fixed this issue by re-coding the C .dll to access the COM server in a separate thread, removing the dependency on the threading model of the caller (in my case the wx App). I still do not have a reason for the crashing (being COM-ignorant) but I did find out that single threaded apartments in COM use Windows messages to communicate back to the caller, while multi-threaded apartments do not. I was able to see this behaviour with MSVC Spy++, but nothing out of the ordinary would happen at the time of the crash. I am still going to attempt to figure this out, but we seem to have found a solution.

Thanks for your help,

···

--
Mark