Questions about unit testing GUIs

wxPython Users
1

Hi,

I have been playing with wxPython for a while, and am about to start using it for serious development. I have only just subscribed to this list, so my apologies if this is an FAQ.

I have been heavily influenced by Extreme Programming, and would like use test driven development via the unittest module in the Python library. At the moment there does not seem to be a standard way of driving wxPython GUI objects from code, so I am considering writing a framework for it. This would emulate user interactions by creating fake wxCommandEvent objects and sending them to the relevant control.

My questions are:

1) has anyone done anything like this already? I would rather not have to reinvent the wheel.

2) is it feasible? The wxCommandEvent objects are normally generated by the system, so is it possible to generate them from within the python code? I did a search in the mailing list on the subject, and there was a discussion a while ago that said the eventType codes are dynamically generated, so how do I know what to set the eventType to? What about the other data fields - is there any documentation about what needs to be filled in, or do I have to go through the C++ source for each event? This is probably not much of an issue for simple events, but the more complex events such as wxTreeEvent look like they may be tricky.

3) is there an alternative way to drive the GUI programatically that I have missed? I have written a Python COM interface to ActiveAccessibility which I can use for driving MFC based programs, so when wxWindows supports ActiveAccessibility that is a possible alternative. Unfortunately the rights to that are held by the company I was working for at the time, so I can't make it open source. It is also limited to the Windows platform, while the wxCommandEvent solution would be cross platform.

         Regards,

Dave Kirby - The Developers' Coach
Helping software professionals and teams achieve peak performance

email: dave@thedeveloperscoach.com
UK Phone: 020 8376 2274
International Phone: (+44) 20 8376 2274
web: http://www.thedeveloperscoach.com - sign up for my new free eZine, The Agile Life

2

Dave Kirby wrote:

Hi,

I have been playing with wxPython for a while, and am about to start using it for serious development. I have only just subscribed to this list, so my apologies if this is an FAQ.

I have been heavily influenced by Extreme Programming, and would like use test driven development via the unittest module in the Python library. At the moment there does not seem to be a standard way of driving wxPython GUI objects from code, so I am considering writing a framework for it. This would emulate user interactions by creating fake wxCommandEvent objects and sending them to the relevant control.

My questions are:

1) has anyone done anything like this already? I would rather not have to reinvent the wheel.

2) is it feasible? The wxCommandEvent objects are normally generated by the system, so is it possible to generate them from within the python code?

Hi Dave,

I can't say whether this is feasible in the general, but it works for menu events. I have an application that can be scripted, and since I was lazy, one of the ways you can script it is to call menu items using wxCommandEvent. The code looks like so:

# aFrame.CallMenuItem("File", "Save") will attempt to execute the "Save" item in the "File" menu.

    def CallMenuItem(self, arg0, *args):
        item = tools.GetMenuItem(self, arg0, *args)
        event = wx.wxCommandEvent(wxEVT_COMMAND_MENU_SELECTED, item.GetId())
        wxPostEvent(self, event)

# And in tools.py

def GetMenuItem(obj, arg0, *args):
    """GetMenuItem(obj, arg0, *args) -> named menu item
       obj can be a Menu, MenuBar or Frame.
    arg0 specifies the first name to look for.
    args are names in successively deeper nested submenus.

    """
    if isinstance(obj, wx.wxMenuBar):
        for i in range(obj.GetMenuCount()):
            if arg0 == obj.GetLabelTop(i):
                item = obj.GetMenu(i)
                break
        else:
            raise IndexError("object does not have menu item '%s'" % arg0)
        if args:
            item = GetMenuItem(item, *args)
    elif isinstance(obj, wx.wxMenu):
        item = obj.FindItemById(obj.FindItem(arg0))
        if args:
            item = GetMenuItem(item.GetSubMenu(), *args)
    elif isinstance(obj, wx.wxFrame):
        return GetMenuItem(obj.GetMenuBar(), arg0, *args)
    else:
        raise ValueError("cannot get menu item from %s" % type(obj))
    return item

···

#------------------------------------------------------------------------

I'd say more but I'm off to catch a plane. However, if you do work on this, let me encourage you to not tie it too tightly to UnitTest -- it sounds like something that I would like to use, but I for one use doctest almost exclusively.

Good Luck,

-tim

I did a search in the mailing list on the subject, and there was a discussion a while ago that said the eventType codes are dynamically generated, so how do I know what to set the eventType to? What about the other data fields - is there any documentation about what needs to be filled in, or do I have to go through the C++ source for each event? This is probably not much of an issue for simple events, but the more complex events such as wxTreeEvent look like they may be tricky.

3) is there an alternative way to drive the GUI programatically that I have missed? I have written a Python COM interface to ActiveAccessibility which I can use for driving MFC based programs, so when wxWindows supports ActiveAccessibility that is a possible alternative. Unfortunately the rights to that are held by the company I was working for at the time, so I can't make it open source. It is also limited to the Windows platform, while the wxCommandEvent solution would be cross platform.

        Regards,

Dave Kirby - The Developers' Coach
Helping software professionals and teams achieve peak performance

email: dave@thedeveloperscoach.com
UK Phone: 020 8376 2274
International Phone: (+44) 20 8376 2274
web: http://www.thedeveloperscoach.com - sign up for my new free eZine, The Agile Life

------------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: wxPython-users-unsubscribe@lists.wxwindows.org
For additional commands, e-mail: wxPython-users-help@lists.wxwindows.org

3

Tim Hochberg wrote:

Dave Kirby wrote:

I have been heavily influenced by Extreme Programming, and would like use test driven development via the unittest module in the Python library. At the moment there does not seem to be a standard way of driving wxPython GUI objects from code, so I am considering writing a framework for it. This would emulate user interactions by creating fake wxCommandEvent objects and sending them to the relevant control.

I can't say whether this is feasible in the general, but it works for menu events. I have an application that can be scripted, and since I was lazy, one of the ways you can script it is to call menu items using wxCommandEvent. The code looks like so: [...]

As I understand it, it depends on the level of the events in question. You can simulate menu events and such, but you cannot effectively simulate, say, mouse events or keypresses. You can feed a value into a text control, say, by issuing an event that will SetText(), but you can't generate a series of wxKeyEvents because wxPython/wxWindows can't force the low-level events back into the underlying (native) control. Similarly, you can send a wxButton event but you can't send a wxEVT_LCLICK to the button.

Jeff Shannon
Technician/Programmer
Credit International

4

Dave Kirby wrote:

Hi,

I have been playing with wxPython for a while, and am about to start using it for serious development. I have only just subscribed to this list, so my apologies if this is an FAQ.

I have been heavily influenced by Extreme Programming, and would like use test driven development via the unittest module in the Python library. At the moment there does not seem to be a standard way of driving wxPython GUI objects from code, so I am considering writing a framework for it. This would emulate user interactions by creating fake wxCommandEvent objects and sending them to the relevant control.

Besides generating the events, there is another difficulty with this approach:

Event-driven programs spend most of their time sitting in an event loop waiting (or polling) for new events. Unit testing, at least where I've seen it applied, usually follows a sequential programming model: make a call, then check the program state, then make another call, etc. In an event-driven program, such a sequence would have to be done in an event handler called by the event loop in response to the receipt of an event. Normally, the handler would not return execution to the event loop until the sequence was finished. However, this doesn't work because the GUI must process the events sent by the handler before it can check the program state and proceed to the next step.

I'm not sure how to get around this dilemma. It is possible that you could use a wxEventIdle handler to check the results of the previous step and start the next one. I think that EVT_IDLE handlers are only supposed to be called when there are no other events pending, so that *might* be sufficient to ensure that the GUI's response to the previous step was complete. However, if that step caused the GUI to do something asynchronous, like posting another event, or waiting for data from a socket, that would not be true.

In principle, you could run the tests from a separate thread. However, this is hard to do in wxPython, because only the main thread is allowed to make GUI calls, so while you could post events from the second thread, you'd have to rely on inter-thread communication to check the GUI state. Also, while using a second thread allows you to perform a long sequence of steps without preventing the event loop from handling events, it doesn't help with the synchronization problem, and may hurt.

I'd be very interested to find out if you know or discover a solution to this dilemma. I'm currently working on a large project with an extensive test suite influenced by the XP and unit testing approach (though not using pyUnit). So far, we've just used tests which check the internal state of the program, and relied on manual testing for the GUI.

Another possibility, of course, is to use an external program to run the tests. However, in that case, I don't think you can generate artificial wxEvents (even if the tester is also a wxPython program). Also, you can't easily check the internal state of the program, so you have to rely on taking snapshots of the GUI windows. This is a pain and doesn't readily allow for minor differences in the appearance across platforms, or between versions of the application.

3) is there an alternative way to drive the GUI programatically that I have missed? I have written a Python COM interface to ActiveAccessibility which I can use for driving MFC based programs, so when wxWindows supports ActiveAccessibility that is a possible alternative. Unfortunately the rights to that are held by the company I was working for at the time, so I can't make it open source. It is also limited to the Windows platform, while the wxCommandEvent solution would be cross platform.

Too bad about that being closed-source. Is it available commercially from that company?

David

5

Jeff Shannon wrote:

As I understand it, it depends on the level of the events in question. You can simulate menu events and such, but you cannot effectively simulate, say, mouse events or keypresses. You can feed a value into a text control, say, by issuing an event that will SetText(), but you can't generate a series of wxKeyEvents because wxPython/wxWindows can't force the low-level events back into the underlying (native) control. Similarly, you can send a wxButton event but you can't send a wxEVT_LCLICK to the button.

You understand correctly.

One possible alternative is to write a framework around some third-party tool that is able to send the low-level events, and then write the unit tests such that they send events with that tool and then check the results in your program. It's been a long time since I looked at tools in this space but I seem to recall that there were some shareware tools for Windows available. There may be some free/Open ones by now. I think there are some for X too. They don't have to real sophisticated like the high cost commercial tools out there since you could do all the fancy scripting that they do from Python...

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!