[xcb] Unknown request in queue while dequeuing

I’m trying to plot a dynamic histogram using wxPython and matplotlib. The data to be displayed by the histogram is a constant stream of laser range finder output. As such I believe I need to put the wx main loop in it’s own thread while I update the range finder data in another thread. To handle the multi-threading I basically copied the code found here: http://wiki.wxpython.org/MainLoopAsThread (Method 3)

When I run the test.py script I receive this error message more than 50% of the time.
[xcb] Unknown request in queue while dequeuing
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
[xcb] Aborting, sorry about that.
python: …/…/src/xcb_io.c:178: dequeue_pending_request: Assertion `!xcb_xlib_unknown_req_in_deq’ failed.
Aborted

This error occurs on both Fedora 16 and Ubuntu 11.10. Since this is my first time using wxPython, python threads, and non-static matplotlib graphs I’m a little lost on who’s responsible for dealing with xbc’s XinitThreads call.
Also, I would be much obliged if someone could take a look at the “terrible terrible hack” section of the code in dynplt.py and give hints towards fixing it.

Thanks,
-DarkAnt

dynplt.py (6.9 KB)

test.py (305 Bytes)

I figured it out. I thought having a mutex protecting the data in used
in draw calls would prevent the race condition. This proved not to be
the case. The proper way to handle this is to create a dummy set_data
function and have the dummy function use wx.callafter() on the real
set_data function.

···

On Feb 13, 4:34 pm, DarkAnt <dark...@gmail.com> wrote:

I'm trying to plot a dynamic histogram using wxPython and matplotlib. The
data to be displayed by the histogram is a constant stream of laser range
finder output. As such I believe I need to put the wx main loop in it's own
thread while I update the range finder data in another thread. To handle
the multi-threading I basically copied the code found here:http://wiki.wxpython.org/MainLoopAsThread(Method 3)

When I run the test.py script I receive this error message more than 50% of
the time.
[xcb] Unknown request in queue while dequeuing
[xcb] Most likely this is a multi-threaded client and XInitThreads has not
been called
[xcb] Aborting, sorry about that.
python: ../../src/xcb_io.c:178: dequeue_pending_request: Assertion
`!xcb_xlib_unknown_req_in_deq' failed.
Aborted

This error occurs on both Fedora 16 and Ubuntu 11.10. Since this is my
first time using wxPython, python threads, and non-static matplotlib graphs
I'm a little lost on who's responsible for dealing with xbc's XinitThreads
call.
Also, I would be much obliged if someone could take a look at the "terrible
terrible hack" section of the code in dynplt.py and give hints towards
fixing it.

Thanks,
-DarkAnt

dynplt.py
6KViewDownload

test.py
< 1KViewDownload

Something somewhere is trying to do something with the UI or UI objects from the wrong thread. You'll need to track that down to be able to fix it. Try creating an application with similar structure but that doesn't used matplotlib, for example, to help you narrow down the problem or to eliminate that as the source of the problem. Then narrow down further and try again.

Ah, I see in another message that you've solved it. Good job.

···

On 2/13/12 1:34 PM, DarkAnt wrote:

I'm trying to plot a dynamic histogram using wxPython and matplotlib.
The data to be displayed by the histogram is a constant stream of laser
range finder output. As such I believe I need to put the wx main loop in
it's own thread while I update the range finder data in another thread.
To handle the multi-threading I basically copied the code found here:
http://wiki.wxpython.org/MainLoopAsThread (Method 3)

When I run the test.py script I receive this error message more than 50%
of the time.
[xcb] Unknown request in queue while dequeuing
[xcb] Most likely this is a multi-threaded client and XInitThreads has
not been called
[xcb] Aborting, sorry about that.
python: ../../src/xcb_io.c:178: dequeue_pending_request: Assertion
`!xcb_xlib_unknown_req_in_deq' failed.
Aborted

This error occurs on both Fedora 16 and Ubuntu 11.10. Since this is my
first time using wxPython, python threads, and non-static matplotlib
graphs I'm a little lost on who's responsible for dealing with xbc's
XinitThreads call.
Also, I would be much obliged if someone could take a look at the
"terrible terrible hack" section of the code in dynplt.py and give hints
towards fixing it.

--
Robin Dunn
Software Craftsman
http://wxPython.org

    Something somewhere is trying to do something with the UI or UI objects
    from the wrong thread. You'll need to track that down to be able to
    fix
    it.

Is this completely out of the question?

The different platforms have different issues with doing GUI manipulations from the non-GUI thread.

For example, on Windows the UI elements belong to the thread that created them, and their messages (events) will be delivered to that thread's message queue, and since the thread doesn't have one (no MainLoop) then things break.

Similarly on X-Windows systems the low level XLib code gets confused if it sees responses coming back for messages it did not send to the X server. I think there is a way to marshal the responses to alternate input queues, but the higher level libraries built on top of Xlib don't do that, at least not by default, and from what I've read most of the implementations of the XLib code isn't very thread-safe anyway so it would still likely have problems.

And we've discovered recently that the OSX-cocoa must be running the UI code in the MAIN thread or it won't work. (On the other platforms the UI thread could usually be a non-main thread if the wx.App was created there and it calls MainLoop on that thread. But not on OSX.)

I've written a wxPython GUI that
uses embedded python scripts to let the user set up, run and make plots
of a simulation. I was hoping to run these scripts in a separate thread
or process but I've run into the same problem: creating plots (more
precisely, calling the matplotlib.show() function) from a different
thread or process causes anything from this error to a segmentation
fault. Since it's the user's scripts that create the plots and call
show() I can't simply move the data back to the original thread before
displaying it.

Using a separate process would work, as long as it is not a multiprocess.Process that is trying to share the same wx objects as the parent process. It would have to be a new invocation of Python[1] with its own wx.App object and MainLoop.

[1] Or I think that multiprocess.Process could work as long as the child processes were created before the wx.App was created (and maybe even before wx was imported the first time) in the parent process. The children would still need to create their own wx.App and run an event loop though.

Or you could train your users to separate their scripts into a computational or data gathering portion, and a GUI portion, and then provide a way for them to run the GUI portion in the GUI thread. For example:

···

On 8/8/12 12:22 AM, Michael Clerx wrote:

On Tuesday, February 14, 2012 10:05:07 PM UTC+1, Robin Dunn wrote:

----------------------------------------------------------
def loadDatabase():
     pass

def crunchNumbers(data):
    pass

def showPlots(plotData):
    pass

data = loadDatabase()
plotData = crunchNumbers(data)
wx.CallAfter(showPlots, plotData)
------------------------------------------------------------

Although you may want to give wx.CallAfter a different name that would make more sense for your users, like RunInUIThread or something like that.

Or probably even better you could include the decorator from wxAnyThread (see http://pypi.python.org/pypi/wxAnyThread/) in the script's namespace, and then your users could do it in a more natural way, like this:

----------------------------------------------------------
def loadDatabase():
     pass

def crunchNumbers(data):
    pass

@anythread
def showPlots(plotData):
    pass

data = loadDatabase()
plotData = crunchNumbers(data)
showPlots(plotData)
------------------------------------------------------------

--
Robin Dunn
Software Craftsman
http://wxPython.org

Michael Clerx wrote:

Is this completely out of the question? I've written a wxPython GUI
that uses embedded python scripts to let the user set up, run and make
plots of a simulation. I was hoping to run these scripts in a separate
thread or process but I've run into the same problem: creating plots
(more precisely, calling the matplotlib.show() function) from a
different thread or process causes anything from this error to a
segmentation fault. Since it's the user's scripts that create the
plots and call show() I can't simply move the data back to the
original thread before displaying it.

All UI actions for a given window must be done by the thread that
created the window. This is because all of the window messages get
routed to the creating thread, and there is no protective interlocking.

So, if a secondary thread creates a window, that thread can draw to it.
But if the window is created by your main thread, then it has to be
drawn from the main thread.

···

--
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.