wx.TextCtrl segfaults when AppendText is called in a thread.

wxPython Users
1

Hi,

I am trying to write a program that runs a command shell
and shows its output in a wx.TextCtrl.

To monitor the shell output, I am starting a thread
that calls AppendText for every character received.

Almost every time I run this, it crashes - with
various error messages ranging from python runtime
errors to Gtk errors and even segfaults.

Is the wx.TextCtrl not thread-safe?

Is there a better way to do this?

Here is my code:

#!/usr/bin/env python
"""
   Run a shell.
"""

__author__ = "Carsten Koch"
__date__ = "2008-10-27"
__version__ = "$Id: run_shell.py $"
__revision__ = "$Revision: 58918 $"

import os
import subprocess
import sys
import threading
import wx

class MainWindow(wx.Frame):
   """
      Main window.
   """

   def __init__(self):
      wx.Frame.__init__(self, None, title=' '.join(sys.argv), size=(800, 600))
      panel = wx.Panel(self, -1)
      vertical_sizer = wx.BoxSizer(wx.VERTICAL)
      self.text_area = wx.TextCtrl(panel, -1, style=wx.TE_MULTILINE)

      vertical_sizer.Add(self.text_area, 1, wx.EXPAND)

      panel.SetSizer(vertical_sizer)
      panel.Layout()
      self.Show()

      self.shell = subprocess.Popen(os.environ["SHELL"], bufsize=0,
                                    stdin=subprocess.PIPE, stdout=subprocess.PIPE,
                                    stderr=subprocess.STDOUT)
      self.shell_thread = threading.Thread(target=self.read_shell_output)
      self.shell_thread.start()
      self.last_line_index = 0

      self.shell.stdin.write("ls -lR /tmp\n")

   def read_shell_output(self):
      """
         Read output from shell character by character and display it in text_area.
      """
      while True:
         output = self.shell.stdout.read(1)
         if output == '':
            break
         self.text_area.AppendText(output)

if __name__ == "__main__":
   app = wx.PySimpleApp()
   MainWindow()
   app.MainLoop()

2

Hello,

Carsten Koch wrote:

Hi,

I am trying to write a program that runs a command shell
and shows its output in a wx.TextCtrl.

To monitor the shell output, I am starting a thread
that calls AppendText for every character received.

Almost every time I run this, it crashes - with
various error messages ranging from python runtime
errors to Gtk errors and even segfaults.

Is the wx.TextCtrl not thread-safe?

You should not try to update any widget from outside the main thread. So you should use something like CallAfter to update your TextCtrl.
You can check the wiki for long running tasks LongRunningTasks - wxPyWiki at the bottom of the page.

Mathias

···

Is there a better way to do this?

Here is my code:

#!/usr/bin/env python
"""
   Run a shell.
"""

__author__ = "Carsten Koch"
__date__ = "2008-10-27"
__version__ = "$Id: run_shell.py $"
__revision__ = "$Revision: 58918 $"

import os
import subprocess
import sys
import threading
import wx

class MainWindow(wx.Frame):
   """
      Main window.
   """

   def __init__(self):
      wx.Frame.__init__(self, None, title=' '.join(sys.argv), size=(800, 600))
      panel = wx.Panel(self, -1)
      vertical_sizer = wx.BoxSizer(wx.VERTICAL)
      self.text_area = wx.TextCtrl(panel, -1, style=wx.TE_MULTILINE)

      vertical_sizer.Add(self.text_area, 1, wx.EXPAND)

      panel.SetSizer(vertical_sizer)
      panel.Layout()
      self.Show()

      self.shell = subprocess.Popen(os.environ["SHELL"], bufsize=0,
                                    stdin=subprocess.PIPE, stdout=subprocess.PIPE,
                                    stderr=subprocess.STDOUT)
      self.shell_thread = threading.Thread(target=self.read_shell_output)
      self.shell_thread.start()
      self.last_line_index = 0

      self.shell.stdin.write("ls -lR /tmp\n")

   def read_shell_output(self):
      """
         Read output from shell character by character and display it in text_area.
      """
      while True:
         output = self.shell.stdout.read(1)
         if output == '':
            break
         self.text_area.AppendText(output)

if __name__ == "__main__":
   app = wx.PySimpleApp()
   MainWindow()
   app.MainLoop()
_______________________________________________
wxpython-users mailing list
wxpython-users@lists.wxwidgets.org
http://lists.wxwidgets.org/mailman/listinfo/wxpython-users
---------------------------------------------------------------------------------------
Orange vous informe que cet e-mail a ete controle par l'anti-virus mail. Aucun virus connu a ce jour par nos services n'a ete detecte.

3

Hi Mathias,

Mathias Lorente wrote:
...

You should not try to update any widget from outside the main thread. So
you should use something like CallAfter to update your TextCtrl.
You can check the wiki for long running tasks
LongRunningTasks - wxPyWiki at the bottom of the page.

Thanks for the excellent hint!

I changed my code to this:

#!/usr/bin/env python
"""
   Run a shell.
"""

__author__ = "Carsten Koch"
__date__ = "2008-10-27"
__version__ = "$Id: run_shell.py $"
__revision__ = "$Revision: $"

import os
import subprocess
import sys
import threading
import wx

EVT_CHARACTER_ID = wx.NewId()

class CharacterEvent(wx.PyEvent):
   """
      Simple event to carry a character.
   """
   def __init__(self, character):
      """
         Set up character event.
      """
      wx.PyEvent.__init__(self)
      self.SetEventType(EVT_CHARACTER_ID)
      self.character = character

class MainWindow(wx.Frame):
   """
      Main window.
   """

   def __init__(self):
      wx.Frame.__init__(self, None, title=' '.join(sys.argv), size=(800, 600))
      panel = wx.Panel(self, -1)
      vertical_sizer = wx.BoxSizer(wx.VERTICAL)
      self.text_area = wx.TextCtrl(panel, -1, style=wx.TE_MULTILINE)

      vertical_sizer.Add(self.text_area, 1, wx.EXPAND)

      panel.SetSizer(vertical_sizer)
      panel.Layout()
      self.Show()

      self.shell = subprocess.Popen(os.environ["SHELL"], bufsize=0,
         stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
      self.shell_thread = threading.Thread(target=self.read_shell_output)
      self.shell_thread.start()

      self.Connect(-1, -1, EVT_CHARACTER_ID, self.on_result)

      self.shell.stdin.write("ls -lR /tmp\n")

   def read_shell_output(self):
      """
         Read output from shell character by character and display it in text_area.
      """
      while True:
         output = self.shell.stdout.read(1)
         if output == '':
            break
         wx.PostEvent(self, CharacterEvent(output))

   def on_result(self, event):
      self.text_area.AppendText(event.character)

if __name__ == "__main__":
   app = wx.PySimpleApp()
   MainWindow()
   app.MainLoop()

Which works fine, but very slow.
I guess I'll experiment with blocking/nonblocking
to speed it up.

Thanks again,
Carsten.

4

Carsten,

Carsten Koch wrote:

Hi Mathias,

Mathias Lorente wrote:
...
  

You should not try to update any widget from outside the main thread. So
you should use something like CallAfter to update your TextCtrl.
You can check the wiki for long running tasks
LongRunningTasks - wxPyWiki at the bottom of the page.
    
Thanks for the excellent hint!

I changed my code to this:

#!/usr/bin/env python
"""
   Run a shell.
"""

__author__ = "Carsten Koch"
__date__ = "2008-10-27"
__version__ = "$Id: run_shell.py $"
__revision__ = "$Revision: $"

import os
import subprocess
import sys
import threading
import wx

EVT_CHARACTER_ID = wx.NewId()

class CharacterEvent(wx.PyEvent):
   """
      Simple event to carry a character.
   """
   def __init__(self, character):
      """
         Set up character event.
      """
      wx.PyEvent.__init__(self)
      self.SetEventType(EVT_CHARACTER_ID)
      self.character = character

class MainWindow(wx.Frame):
   """
      Main window.
   """

   def __init__(self):
      wx.Frame.__init__(self, None, title=' '.join(sys.argv), size=(800, 600))
      panel = wx.Panel(self, -1)
      vertical_sizer = wx.BoxSizer(wx.VERTICAL)
      self.text_area = wx.TextCtrl(panel, -1, style=wx.TE_MULTILINE)

      vertical_sizer.Add(self.text_area, 1, wx.EXPAND)

      panel.SetSizer(vertical_sizer)
      panel.Layout()
      self.Show()

      self.shell = subprocess.Popen(os.environ["SHELL"], bufsize=0,
         stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
      self.shell_thread = threading.Thread(target=self.read_shell_output)
      self.shell_thread.start()

      self.Connect(-1, -1, EVT_CHARACTER_ID, self.on_result)

      self.shell.stdin.write("ls -lR /tmp\n")

   def read_shell_output(self):
      """
         Read output from shell character by character and display it in text_area.
      """
      while True:
         output = self.shell.stdout.read(1)
         if output == '':
            break
         wx.PostEvent(self, CharacterEvent(output))

   def on_result(self, event):
      self.text_area.AppendText(event.character)

if __name__ == "__main__":
   app = wx.PySimpleApp()
   MainWindow()
   app.MainLoop()

Which works fine, but very slow.
I guess I'll experiment with blocking/nonblocking
to speed it up.

Thanks again,
Carsten.
  

I would suggest you not to send one character at a time but as much characters as you can.
And you should replace your infinite loop by something else so that you can terminate your thread and avoid overloading your CPU.

Mathias

5

Hi Mathias,

Mathias Lorente wrote:
...

I would suggest you not to send one character at a time but as much
characters as you can.

Yes, that would be ideal.
I have experimented with that a little.
There seems to be no portable way of doing it.
fcntl and select for pipes are available under UNIX only.
When I read more than one character in blocking mode,
the loop cannot work properly, since buffers will never
be delivered partially. So I will not get the last block
before it is full and it will not become full if there
currently is no output.

There is a solution here
    Module to allow Asynchronous subprocess use on Windows and Posix platforms « Python recipes « ActiveState Code
which is much longer than my entire program.
I wonder if a simple solution exists.

And you should replace your infinite loop by something else so that you
can terminate your thread and avoid overloading your CPU.

Please note that the infinite loop blocks in the
read(1) call when there is no data.
So this does not consume any CPU cycles while
the shell is not producing output.

Carsten.

6

Carsten Koch wrote:

Hi Mathias,

Mathias Lorente wrote:
...
  

I would suggest you not to send one character at a time but as much
characters as you can.
    
Yes, that would be ideal.
I have experimented with that a little.
There seems to be no portable way of doing it.
fcntl and select for pipes are available under UNIX only.
When I read more than one character in blocking mode,
the loop cannot work properly, since buffers will never
be delivered partially. So I will not get the last block
before it is full and it will not become full if there
currently is no output.

There is a solution here
    Module to allow Asynchronous subprocess use on Windows and Posix platforms « Python recipes « ActiveState Code
which is much longer than my entire program.
I wonder if a simple solution exists.

There is the new multiprocessing module in Python 2.6 which I assume is cross-platform. Looks like there's even a backport to 2.4/2.5:

I'm not sure if this will help you or not, but you might look into it.

Mike

7

Hello,

Hi Mathias,

Mathias Lorente wrote:

…

I would suggest you not to send one character at a time but as much

characters as you can.

Yes, that would be ideal.

I have experimented with that a little.

There seems to be no portable way of doing it.

fcntl and select for pipes are available under UNIX only.

When I read more than one character in blocking mode,

the loop cannot work properly, since buffers will never

be delivered partially. So I will not get the last block

before it is full and it will not become full if there

currently is no output.

There is a solution here

[http://code.activestate.com/recipes/440554](http://code.activestate.com/recipes/440554)

which is much longer than my entire program.

I wonder if a simple solution exists.

Here is one way to do a non blocking pipe read on Windows. It requires ctypes so this will only work python2.5 and higher. This is just some code extracted from one of my modules in Editra but it should give you the idea (self._proc is the return value of subprocess.Popen).

Windows nonblocking pipe read implementation

import msvcrt

import ctypes

read = u’’

try:

handle = msvcrt.get_osfhandle(self._proc.stdout.fileno())

avail = ctypes.c_long()

ctypes.windll.kernel32.PeekNamedPipe(handle, None, 0, 0,

ctypes.byref(avail), None)

if avail.value > 0:

read = self._proc.stdout.read(avail.value)

Do something with the new text…

else:

if self._proc.poll() is None:

time.sleep(1) # Pause for a second

return True

else:

Process has Exited

return False

except ValueError:

return False

except (subprocess.pywintypes.error, Exception), msg:

if msg[0] in (109, errno.ESHUTDOWN):

return False

Cody

···

On Tue, Oct 28, 2008 at 3:04 AM, Carsten Koch Carsten.Koch@icem.com wrote: