wx.Execute(self.cmd... polling for stdout

I have an app that runs a list of long running commands and shows each
commands stdout in it's own TextCtrl. It mostly works, but I am not
happy with the implementation.

1. I hardly understand how it works, aka why it doesn't not work.

Like I don't see what is keeping objects from being destroyed - maybe
they are, but I have a stale pointer that happens to work, or maybe
something is keeping a reference count that I don't understand. I did
change my code around to something I think should work, but the fact
that my previous version worked and I don't know how isn't good. The
new version is still on the edge of my comprehension.

And trying to follow it when I am debugging still gives me a
headache. Mainly when the timer calls my .ShowIO().

Below is my code trimmed down to just what is needed to see what I am
working on. The full version is
https://github.com/CarlFK/dvsmon/blob/master/dvs-mon.py

foos is kinda redundant - it is basically the set of panels I add to
the empty frame. Isn't there a list of objects that have been added
to a container?

2. I am using a timer to poll for output every second. What I would
really like is to hear there is an event I can trap when the command
issues output. That seems more efficient, and likely much cleaner
code. Is my polling really the best/only way?

btw, I tried to use python's subprocess - there doesn't seem to be
something like stream.CanRead() and so p.stdin.read() would wait for
data to be there, which isn't good at all.

Example code: uses ping as the long running commands. ping is
actually more chatty - the real commands only output every few
minutes, but I need to see the output right away.

#!/usr/bin/python

import random

import wx
import wx.lib.sized_controls as sc

class CommandRunner(object):

    ctr = 0

    def __init__(self, cmd, parent):

        self.cmd = cmd

        self.panel = sc.SizedPanel(parent)
        self.panel.SetSizerType('horizontal')
        self.panel.SetSizerProps(expand=True)

        self.txt_cmd = wx.TextCtrl(self.panel, value=self.cmd,
style=wx.TE_READONLY)
        self.txt_cmd.SetSizerProps(proportion=40, expand=True)

        self.panel2 = sc.SizedPanel(parent)
        self.panel2.SetSizerType('vertical')
        self.panel2.SetSizerProps(expand=True, proportion=2)
        self.stdout = wx.TextCtrl( self.panel2, style=wx.TE_READONLY|
wx.TE_MULTILINE)
        self.stdout.SetSizerProps(proportion=1, expand=True)

    def RunCmd(self):
        self.process = wx.Process(self.panel)
        self.process.Redirect()
        self.pid = wx.Execute(self.cmd, wx.EXEC_ASYNC, self.process)
        print 'Executed: %s' % (self.cmd)

    def ShowIO(self):

        # flip color to show when it hit
        c = [random.randrange(0,255) for i in range(4)]
        self.txt_cmd.SetForegroundColour(c)

        # move stdout to TextCtrl
        stream = self.process.GetInputStream()
        while stream.CanRead():
            line = stream.read()
            line = line.strip()
            # print "O", line.__repr__()
            self.stdout.AppendText("\n %d %s" % (self.ctr,line))
            self.ctr += 1

def main():

    app = wx.PySimpleApp()

    # add command blocks:
    frame = sc.SizedFrame(None, pos=(1,1), size=(500, 500))
    panel = frame.GetContentsPane()

    foos=[]
    for i in range(3):
        cmd = 'ping -i %s 127.0.0.1' % (i+1)
        cr = CommandRunner(cmd, panel)
        cr.RunCmd()
        foos.append(cr)

    frame.Show()

    # setup poling
    def OnTimer(evt):
        for foo in foos:
            foo.ShowIO()

    panel.Bind(wx.EVT_TIMER, OnTimer)
    timer = wx.Timer(panel)
    timer.Start(500)

    # wait in event loop:
    app.MainLoop()

if __name__ == '__main__':
    main()

I have an app that runs a list of long running commands and shows each
commands stdout in it's own TextCtrl. It mostly works, but I am not
happy with the implementation.

1. I hardly understand how it works, aka why it doesn't not work.

Like I don't see what is keeping objects from being destroyed

Which ones?

- maybe
they are, but I have a stale pointer that happens to work, or maybe
something is keeping a reference count that I don't understand. I did
change my code around to something I think should work, but the fact
that my previous version worked and I don't know how isn't good. The
new version is still on the edge of my comprehension.

And trying to follow it when I am debugging still gives me a
headache. Mainly when the timer calls my .ShowIO().

Below is my code trimmed down to just what is needed to see what I am
working on. The full version is
dvsmon/dvs-mon.py at master · CarlFK/dvsmon · GitHub

foos is kinda redundant - it is basically the set of panels I add to
the empty frame. Isn't there a list of objects that have been added
to a container?

window.GetChildren()

2. I am using a timer to poll for output every second. What I would
really like is to hear there is an event I can trap when the command
issues output. That seems more efficient, and likely much cleaner
code.

Unfortunately not.

Is my polling really the best/only way?

btw, I tried to use python's subprocess - there doesn't seem to be
something like stream.CanRead() and so p.stdin.read() would wait for
data to be there, which isn't good at all.

With subprocess you can use the select module to test for data availability on the streams because they have a fileno() method returning the native file handle.

···

On 10/26/11 4:20 PM, CarlFK wrote:

--
Robin Dunn
Software Craftsman

I have an app that runs a list of long running commands and shows each
commands stdout in it's own TextCtrl. It mostly works, but I am not
happy with the implementation.

1. I hardly understand how it works, aka why it doesn't not work.

Like I don't see what is keeping objects from being destroyed

Which ones?

In my real code (i guess I should have save a version of my sample
code before I changed it to something that made more sense)

#142 for cmd in COMMANDS:
        cr = CommandRunner(cmd, panel)

that calls .__init, which ends with:

# 79: self.timerCallbacks.append(self.ShowIO)

So if it loops 3 times, the first 2 instances of CommandRunner get
dropped, only the last is still in cr, so I would think they would be
destroyed. Does saving self.ShowIO keep them alive?

- maybe
they are, but I have a stale pointer that happens to work, or maybe
something is keeping a reference count that I don't understand. I did
change my code around to something I think should work, but the fact
that my previous version worked and I don't know how isn't good. The
new version is still on the edge of my comprehension.

And trying to follow it when I am debugging still gives me a
headache. Mainly when the timer calls my .ShowIO().

Below is my code trimmed down to just what is needed to see what I am
working on. The full version is
dvsmon/dvs-mon.py at master · CarlFK/dvsmon · GitHub

foos is kinda redundant - it is basically the set of panels I add to
the empty frame. Isn't there a list of objects that have been added
to a container?

window.GetChildren()

cool - thanks.

2. I am using a timer to poll for output every second. What I would
really like is to hear there is an event I can trap when the command
issues output. That seems more efficient, and likely much cleaner
code.

Unfortunately not.

Is my polling really the best/only way?

btw, I tried to use python's subprocess - there doesn't seem to be
something like stream.CanRead() and so p.stdin.read() would wait for
data to be there, which isn't good at all.

With subprocess you can use the select module to test for data availability
on the streams because they have a fileno() method returning the native file
handle.

Hmm... not sure this gets me anything that wx doesn't offer. Good to
know, maybe I should read the lib docs cover to cover.. yeah right,
much more fun to hack on code.

···

On Wed, Oct 26, 2011 at 11:45 PM, Robin Dunn <robin@alldunn.com> wrote:

On 10/26/11 4:20 PM, CarlFK wrote:

--
Carl K

I have an app that runs a list of long running commands and shows each
commands stdout in it's own TextCtrl. It mostly works, but I am not
happy with the implementation.

1. I hardly understand how it works, aka why it doesn't not work.

Like I don't see what is keeping objects from being destroyed

Which ones?

In my real code (i guess I should have save a version of my sample
code before I changed it to something that made more sense)

dvsmon/dvs-mon.py at master · CarlFK/dvsmon · GitHub

#142 for cmd in COMMANDS:
         cr = CommandRunner(cmd, panel)

that calls .__init, which ends with:

# 79: self.timerCallbacks.append(self.ShowIO)

So if it loops 3 times, the first 2 instances of CommandRunner get
dropped, only the last is still in cr, so I would think they would be
destroyed. Does saving self.ShowIO keep them alive?

Binding event handlers to bound methods of the CommandRunner will hold a reference to the method, and therefore to the instance as well.

With subprocess you can use the select module to test for data availability
on the streams because they have a fileno() method returning the native file
handle.

select — Waiting for I/O completion — Python 3.13.0 documentation

Hmm... not sure this gets me anything that wx doesn't offer. Good to
know, maybe I should read the lib docs cover to cover.. yeah right,
much more fun to hack on code.

I should have followed through on the rest of my thought... Select can wait for IO to become available on the file handles that you tell it to monitor, so you could have a thread that waits in select() and then when it returns you can find out which file handles are ready to read/write and then send a message to the UI thread. The message could be a custom event or it could be just a function called with wx.CallAfter.

···

On 10/26/11 10:52 PM, Carl Karsten wrote:

On Wed, Oct 26, 2011 at 11:45 PM, Robin Dunn<robin@alldunn.com> wrote:

On 10/26/11 4:20 PM, CarlFK wrote:

--
Robin Dunn
Software Craftsman

1. I hardly understand how it works, aka why it doesn't not work.

Like I don't see what is keeping objects from being destroyed - maybe
they are, but I have a stale pointer that happens to work, or maybe
something is keeping a reference count that I don't understand.

Just some thoughts:
cr = CommandRunner(cmd, panel) is not a wx object, yet it contains
references to wx instance objects.
I can imagine that "cr" will contain PyDeadObject references - if the
MainLoop() wasn't the last line in your main().
When main() exits after MainLoop() "cr" itself should be destroyed and
that should take care of those PyDeadObject references.

BUT you have a timing issue going on. You set your timer to exactly
500 milliseconds, and your "ping" command defaults to 1000
milliseconds. This means potentially you'll have a Timer event pending
in a queue for up to 1500 milliseconds if for some reason the ping and
timer event is posted even ONE millisecond off (from 500). AND the
millisecond timer is only accurate to 16 to 20 milliseconds on
Windows. So I am wondering if there is some wacky arrangement with the
actually deletion of the window amongst Python's GC (garbage
collection) after the MainLoop() on exit of main() and the processing
of the pending timer event and the return from the call to the new os
"environment" issued using the "ping" command.

I'd recommend setting the timer to about 415 milliseconds. If you try
it let me know if that does anything. I'd like to see if I had some
solid logic in my thought process.

Example code: uses ping as the long running commands. ping is
actually more chatty - the real commands only output every few
minutes, but I need to see the output right away.

Another thing about setting the timer to 500 milliseconds. You said
that you need your UI to up date right away, yet your messages only
finish ever frew minutes. Do you really need immediate response in the
UI -faster- then 1/2 of a second. I ask because if a human is
watching your UI and waiting a couple a minutes before she expects the
UI to update because she is waiting for a long running process to
finish (say 3 minutes), then I have a hard time imagining she will
notice the UI updating 1/2 a second or even 5 seconds later (due to
polling the streams with a timer) then when the actual process
finishes.

2. I am using a timer to poll for output every second. What I would
really like is to hear there is an event I can trap when the command
issues output. That seems more efficient, and likely much cleaner
code. Is my polling really the best/only way?

I am suprised PubSub wasn't mentioned.

btw, I tried to use python's subprocess - there doesn't seem to be
something like stream.CanRead() and so p.stdin.read() would wait for
data to be there, which isn't good at all.

Consider Popen.communicate(); There are pitfalls with it and stdin though.

def OnTimer(evt):
for foo in foos:
foo.ShowIO()

I wonder if this prevents garbage collection for some time after
main()'s line with app.MainLoop()

···

On Wed, Oct 26, 2011 at 7:20 PM, CarlFK <cfkarsten@gmail.com> wrote: