You might want to take a look at twisted (www.twistedmatrix.com).
It's a network server/client framework that does everything async.
I'm using it together with a wxPython client for a fairly large insurance
application and have no latency problems with 400+ users driven by a single
quad xeon server.
> > I have a question regarding threads & Queues and how these relate to or
> > influence the main wxPython GUI thread.
> >
> > I have several threads in my app - the main GUI thread, n (user
> > tunable) SQL worker threads that service a SQL job Queue, message
> > objects placed in the Queue has several attributes, a job (usually a
> > SQL string), originating window reference, dispatch function name etc.
> > The same scheme applies with a RAC (xml-rpc) server and associated
> > Queue and worker threads.
> >
> > The SQL/RAC worker thread that executes the job sends a WorkerResponse
> > event to the originating window referenced in the message object using
> > wx.PostEvent on job completion. The reult of the job travels with the
> > event. So far so good, it works beautifully and effectively severs the
> > GUI from network latency issues.
> >
> > I do, however, lack faith in this solution. If I pass a reference to a
> > object created and managed in one thread to another, doesn't it violate
> > the "separation" principle - wouldn't this potentially lead to some
> > subtle deadlock issues ? I thought I understood the issues involved but
> > lately I've been having some doubts - someone please educate me.
>
> This is a great example of the right way to do this - why the worries?
> If you're used to the FUD that Microsoft puts out about "threading
> models" and the problems that cross-thread access to objects might cause,
> don't worry about it. Most of the problems with COM and threading are a
> result of MS using hidden message loops (as Queues) to do thread
> synchronization and communication w/o the programmer being able to see
> what's going on. Since I don't know what separation principle you're
> thinking of, I assume this is it; the idea that objects are somehow
> 'locked' to a thread is a stupid one, promulgated by MS when they
> couldn't explain how to use threads to VB programmers w/o using scary
> words like 'synchronization'.In retrospect, I think my biggest concern is barking up the wrong tree.
I'll elaborate below - even if it serves no purpose other than a brief
case study for the weary.When I started development I didn't really appreciate the effect of
network latency on the application's usability. As features were added
and the application assumed more real-time monitoring responsibilities
the GUI response times worsened. Never in a big way, even if a lot were
happening in the background, a few threads (using their own DB/RAC
connections) 'firing' simultaneously, response were acceptable but
certainly noticeably slower, as can be expected. However, at the
slightest network glitch or server hick-up all went the porcelain way.I deployed elaborate early-warning systems and even a 'lets try again'
error handling mechanism. But I new early on that the only real solution
would be to completely divorce the GUI from the network.Now, of course, I have to replace all the 'synchronous'
res = SQLCon.fetchall("select fruit from basket")
calls with a equivalent 'asynchronous' event driven system:In the Universe.py module I have a derived wx.Panel containing, among
other things, a general workerresponse handler. All the app's panels
derive from it.EVT_WORKERRESPONSE(self, self.OnworkerResponse)
def OnworkerResponse(self, evt):
"""handle worker response messages """
response = evt.response
exec "f = self.%s" % response.dispatchname
result = f(response.result, response.data)(You'll notice that I construct callback functions from their names in a
rather hackish manner, In stead of passing a real live method object
reference kicking and screaming between threads - this might very well
change once I've done a few tests, the code above bugs me no end - see
below)The panel, in stead of calling the synchronous method, uses the foreman
object to place a message in the relevant Queue and then forgets about
it: e.g.def getStationList(self, window, dispatchname):
""" returns list of Full validated stations"""
table = self.table + '_stationvalidation'query = """
SELECT name
FROM %s
ORDER BY name""" % tablecommand = "[e[0].strip() for e in DB.fetchAll(%s)]"
FOREMAN.SQLPut(window, dispatchname, query, command)The window requesting the stationlist has a method
stationListArrived(self, data) and does the right thing when a event
with that dispatchname arrives.This is all good and well until one realizes the amount of work
'refactoring' all synchronous to event driven code. I then decided that
I need to reuse all the old sync code in the event driven system without
elaborated function construction, but by simply passing the method
objects to the thread. That is, pass the originalres = SQLCon.fetchall("select fruit from basket") method, from the
instantiated Foo module to the thread to be executed, the result (res)
will be returned via the resultevent.This is where I got cold feet - passing a method object from a
instantiated module from one thread to another and asking that thread to
execute it, seemed risky. It does seem to work fine though...This then is my real question, I think...
Thanks Chris, at least I now understand my own problem
- --
UC
- --
Open Source Solutions 4U, LLC 2570 Fleetwood Drive
Phone: +1 650 872 2425 San Bruno, CA 94066
Cell: +1 650 302 2405 United States
Fax: +1 650 872 2417
ยทยทยท
On Monday 05 April 2004 11:50 am, Thys Meintjes wrote:
On Mon, 2004-04-05 at 18:54, Chris Olds wrote:
> On Mon, 5 Apr 2004, Thys Meintjes wrote: