Speaking of Extreme Programming, I highly recommend the article "The
Humble Dialog Box", by Michael Feathers. It may completely change how
you are considering testing GUI apps.
As I recall, he suggests designing the app such that all the actual logic
of the application is handled by abstract classes that don't rely on user
interface events. These abstract classes can be tested without a GUI:
easier by far than testing the GUI.
Matthew
···
-----Original Message-----
From: Dave Kirby [mailto:dkirby@blueyonder.co.uk]
Sent: Tuesday, June 24, 2003 12:42 AM
To: wxPython-users@lists.wxwindows.org
Subject: Re: [wxPython-users] Questions about unit testing GUIs
"David C. Fox" <davidcfox@post.harvard.edu> wrote:
Dave Kirby wrote:
[snip]
> I have been heavily influenced by Extreme Programming, and would like
> use test driven development via the unittest module in the Python
> library. At the moment there does not seem to be a standard way of
> driving wxPython GUI objects from code, so I am considering writing a
> framework for it. This would emulate user interactions by creating fake
> wxCommandEvent objects and sending them to the relevant control.
>Besides generating the events, there is another difficulty with this
approach:Event-driven programs spend most of their time sitting in an event loop
waiting (or polling) for new events. Unit testing, at least where I've
seen it applied, usually follows a sequential programming model: make a
call, then check the program state, then make another call, etc. In an
event-driven program, such a sequence would have to be done in an event
handler called by the event loop in response to the receipt of an event.
Normally, the handler would not return execution to the event loop
until the sequence was finished. However, this doesn't work because the
GUI must process the events sent by the handler before it can check the
program state and proceed to the next step.
I am aware of the need to run the tests from within an event loop, and have
considered several options, including the ones that you have described. I
am writing a wxPython version of the PyUnit GUI, so that the tests are run
within the context of the event loop.
However I had not considered the possibility of an event deadlock - I
assume that calling wxControl.Command with the fake event, followed by a
call to wxApp.Yield to allow the event to be processed, would be
sufficient. I could be wrong.
I'm not sure how to get around this dilemma. It is possible that you
could use a wxEventIdle handler to check the results of the previous
step and start the next one. I think that EVT_IDLE handlers are only
supposed to be called when there are no other events pending, so that
*might* be sufficient to ensure that the GUI's response to the previous
step was complete. However, if that step caused the GUI to do something
asynchronous, like posting another event, or waiting for data from a
socket, that would not be true.
That is another option that I have considered. I would need to add a flag
to ensure that the tests are only run on the first EVT_IDLE, but that would
not be difficult.
I'd be very interested to find out if you know or discover a solution to
this dilemma. I'm currently working on a large project with an
extensive test suite influenced by the XP and unit testing approach
(though not using pyUnit). So far, we've just used tests which check
the internal state of the program, and relied on manual testing for the GUI.
I will report back when I have something working. I am aiming to get a
'proof of concept' version working ASAP, and evolve it from there.
Another possibility, of course, is to use an external program to run the
tests. However, in that case, I don't think you can generate artificial
wxEvents (even if the tester is also a wxPython program). Also, you
can't easily check the internal state of the program, so you have to
rely on taking snapshots of the GUI windows. This is a pain and doesn't
readily allow for minor differences in the appearance across platforms,
or between versions of the application.
I don't like the screen snapshot method that some commercial test tools use
- it is far too flaky since even the tiniest change to your GUI will break
all your tests, making it far more trouble than it is worth.
> 3) is there an alternative way to drive the GUI programatically that I
> have missed? I have written a Python COM interface to
> ActiveAccessibility which I can use for driving MFC based programs, so
> when wxWindows supports ActiveAccessibility that is a possible
> alternative. Unfortunately the rights to that are held by the company I
> was working for at the time, so I can't make it open source. It is also
> limited to the Windows platform, while the wxCommandEvent solution would
> be cross platform.Too bad about that being closed-source. Is it available commercially
from that company?
Unfortunately not, since it was part of an in-house testing suite that I
developed. However when I have some spare time I would like to write a new
version from scratch in my own time and make it open source. However I
always seem to have far more ideas for projects than time to do them, so
don't hold your breath.
Regards,
Dave Kirby - The Developers' Coach
Helping software professionals and teams achieve peak performance
email: dave@thedeveloperscoach.com
UK Phone: 020 8376 2274
International Phone: (+44) 20 8376 2274
web: http://www.thedeveloperscoach.com - sign up for my new free
eZine, The Agile Life