SoC Unit Test Status

Hi all,

Now that the Unit Test SoC project has results to show, I wanted to inform the community and solicit feedback.

You can get the latest copy of the test suite from SVN by running:
svn checkout
https://svn.wxwidgets.org/svn/wx/wxPython/branches/SOC2007_UNITTEST/tests/unittest
unittest

From there, ‘runAllTests.py’ runs the whole test suite with default options, or use ‘runAllTests.py --help’ to check out the various options for configuration and data output.

I’ve updated the project page on the wxPyWiki, which is available at http://wiki.wxpython.org/GUI_Unit_Testing_Project
That should explain most of the unit test suite; be sure to leave comments or email me with questions if I omitted anything.

I’m going to need community support if this project is going to succeed. In effect, unit tests are executable documentation: both communicate the software’s expected behavior to developers and users, with unit tests being able to check the code’s adherence. In writing unit tests, I’ve found a lot of ambiguity and lack of documentation showing the expected behavior of many different classes and methods. Going forward, I plan on singling out these cases, in order to clarify what the code should do. I expect that, when these important questions and issues arise, the community will reach a consensus about what the code should do, and then it can be codified in the form of a test.

The goals here are to improve code quality, reduce bugs, clear up cross-platform issues, improve documentation. The major focus, of course, is on improving the user experience (and making development more enjoyable :). I’m looking forward to your feedback.

-Frank

Hi Frank,

Thanks again for all your hard work on the project! I think that we should go ahead and discuss the client and virtual size failures. For those who haven’t read through Frank’s wiki page, the direct link to the test results is here:

http://wiki.wxpython.org/Test_Suite_Results

First off, I wanted to point out to people that we run wx.Window method tests on all wx.Window subclasses, so that we can catch any inconsistencies between base methods and overridden implementations of base class methods in subclasses. So while the number of failures is rather large, many of those are failures trace back to failures in the base wx.Window methods here.

Looking at the test failures, I think a number of them are due to ambiguities, as Frank mentioned. For the ClientRect failure, my guess is that the client rect is adding the control’s border to the x and y positions. Or perhaps it’s giving it’s x,y relative to the parent? Also, will GetClientSize always return a value >= the MinSize?

As for the virtual size failures, I suspect the issue is that SetVirtualSize doesn’t allow the virtual size to go below the control’s current size? Is that correct? It makes sense, but if so, we should update the tests and the documentation to reflect that. Also, Robin, the Get/SetVirtualSizeWH APIs just convert tuples into wx.Size (and vice-versa) then call the wx.Size version of the method, right? If so, maybe a test that ensures GetVirtualSizeWH returns the same value as GetVirtualSize would be sufficient.

One last question for Frank. I noticed you have an assert for missing wx.Slider.GetRange(). AFAICT, that function is not part of the wx.Slider API. It probably SHOULD be, as we have SetRange(min, max), but right now you get the values by calling GetMin() and GetMax(). It was probably done this way because in C++ returning two values is difficult and they didn’t want to use wx.Size or wx.Point for it, so they just stuck with the GetMin/GetMax functions.

Thanks,

Kevin

···

On Jul 11, 2007, at 9:02 AM, Frank Tobia wrote:

Hi all,

Now that the Unit Test SoC project has results to show, I wanted to inform the community and solicit feedback.

You can get the latest copy of the test suite from SVN by running:
svn checkout https://svn.wxwidgets.org/svn/wx/wxPython/branches/SOC2007_UNITTEST/tests/unittest unittest

From there, ‘runAllTests.py’ runs the whole test suite with default options, or use ‘runAllTests.py --help’ to check out the various options for configuration and data output.

I’ve updated the project page on the wxPyWiki, which is available at http://wiki.wxpython.org/GUI_Unit_Testing_Project
That should explain most of the unit test suite; be sure to leave comments or email me with questions if I omitted anything.

I’m going to need community support if this project is going to succeed. In effect, unit tests are executable documentation: both communicate the software’s expected behavior to developers and users, with unit tests being able to check the code’s adherence. In writing unit tests, I’ve found a lot of ambiguity and lack of documentation showing the expected behavior of many different classes and methods. Going forward, I plan on singling out these cases, in order to clarify what the code should do. I expect that, when these important questions and issues arise, the community will reach a consensus about what the code should do, and then it can be codified in the form of a test.

The goals here are to improve code quality, reduce bugs, clear up cross-platform issues, improve documentation. The major focus, of course, is on improving the user experience (and making development more enjoyable :). I’m looking forward to your feedback.

-Frank

Kevin,

···

On 7/11/07, Kevin Ollivier kevino@theolliviers.com wrote:

One last question for Frank. I noticed you have an assert for missing wx.Slider.GetRange(). AFAICT, that function is not part of the wx.Slider API. It probably SHOULD be, as we have SetRange(min, max), but right now you get the values by calling GetMin() and GetMax(). It was probably done this way because in C++ returning two values is difficult and they didn’t want to use wx.Size or wx.Point for it, so they just stuck with the GetMin/GetMax functions.

I included that test because of pg.207 in “wxPython in Action”: Table 7.10 shows a GetRange function. Perhaps this should be an erratum?

This point raises a greater issue. In the absence of online docs, we can sometimes defer to “wxPython in Action.” In these cases, or in cases where docs and book conflict, which is to be considered canon? Should docs reflect code, and “wxPython in Action” reflect docs? Should the docs, and the book as well, specify a contract against which the code must conform? I feel this is an important issue which needs clarification.

-Frank

Kevin Ollivier wrote:

Looking at the test failures, I think a number of them are due to ambiguities, as Frank mentioned. For the ClientRect failure, my guess is that the client rect is adding the control's border to the x and y positions. Or perhaps it's giving it's x,y relative to the parent?

     wxRect GetClientRect() const
     {
         return wxRect(GetClientAreaOrigin(), GetClientSize());
     }

Both GetClientAreaOrigin and GetClientSize will call platform specific methods located in the derived classes, so they can (and do) vary across platforms and window types. IMO because of this potential variability the only valid test for GetClientRect is that it returns a rectangle positioned at GetClientAreaOrigin and with a size of GetClientSize.

Also, will GetClientSize always return a value >= the MinSize?

No. The min size is just a hint for sizers and in the case of top-level windows it's also a hint for the window manager. In most cases it's possible to explicitly set the size smaller than the minsize. Even if that is not the case, the min size is for the whole window, whereas the client size can be smaller than that depending on window type, platform and how borders and other decorations are accounted for.

As for the virtual size failures, I suspect the issue is that SetVirtualSize doesn't allow the virtual size to go below the control's current size? Is that correct?

It is limited based on what the virtual size hints are currently set to. Here is the common (platform independent) base class version:

void wxWindowBase::DoSetVirtualSize( int x, int y )
{
     if ( m_minVirtualWidth != wxDefaultCoord && m_minVirtualWidth > x )
         x = m_minVirtualWidth;
     if ( m_maxVirtualWidth != wxDefaultCoord && m_maxVirtualWidth < x )
         x = m_maxVirtualWidth;
     if ( m_minVirtualHeight != wxDefaultCoord && m_minVirtualHeight > y )
         y = m_minVirtualHeight;
     if ( m_maxVirtualHeight != wxDefaultCoord && m_maxVirtualHeight < y )
         y = m_maxVirtualHeight;

     m_virtualSize = wxSize(x, y);
}

Again this method can be overridden in the derived classes to have different behavior if needed for that class.

It makes sense, but if so, we should update the tests and the documentation to reflect that. Also, Robin, the Get/SetVirtualSizeWH APIs just convert tuples into wx.Size (and vice-versa) then call the wx.Size version of the method, right?

SWIG sees these declarations:

     void SetVirtualSize(const wxSize& size );
     %Rename(SetVirtualSizeWH, void, SetVirtualSize( int w, int h ));

so in addition to the wrapper for the regular SetVirtualSize it generates a Python wrapper method named SetVirtualSizeWH which calls the C++ SetVirtualSize which takes two parameters. On the C++ side we have these methods:

     void SetVirtualSize( const wxSize &size ) { DoSetVirtualSize( size.x, size.y ); }
     void SetVirtualSize( int x, int y ) { DoSetVirtualSize( x, y ); }

So, yes they do exactly the same thing, but just provide a way for the programmer to have more choices on how to call them. For the getters we have this for SWIG:

     wxSize GetVirtualSize() const;
     DocDeclAName(
         void, GetVirtualSize( int *OUTPUT, int *OUTPUT ) const,
         "GetVirtualSizeTuple() -> (width, height)",
         GetVirtualSizeTuple);

And this C++ code:

     wxSize GetVirtualSize() const { return DoGetVirtualSize(); }
     void GetVirtualSize( int *x, int *y ) const
     {
         wxSize s( DoGetVirtualSize() );

         if( x )
             *x = s.GetWidth();
         if( y )
             *y = s.GetHeight();
     }

This same pattern is fairly common in wx and wxPython.

If so, maybe a test that ensures GetVirtualSizeWH returns the same value as GetVirtualSize would be sufficient.

Also that setting the same value via the two setters results in the same values. Also via using the VirtualSize property.

One last question for Frank. I noticed you have an assert for missing wx.Slider.GetRange(). AFAICT, that function is not part of the wx.Slider API. It probably SHOULD be, as we have SetRange(min, max), but right now you get the values by calling GetMin() and GetMax(). It was probably done this way because in C++ returning two values is difficult and they didn't want to use wx.Size or wx.Point for it, so they just stuck with the GetMin/GetMax functions.

Actually it's good to point out things like that because it's super easy to add something like this in the declarations that SWIG sees:

     %pythoncode {
         def GetRange(self):
             return self.GetMin(), self.GetMax()
     }

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

Frank Tobia wrote:

Kevin,

    One last question for Frank. I noticed you have an assert for
    missing wx.Slider.GetRange(). AFAICT, that function is not part of
    the wx.Slider API. It probably SHOULD be, as we have SetRange(min,
    max), but right now you get the values by calling GetMin() and
    GetMax(). It was probably done this way because in C++ returning two
    values is difficult and they didn't want to use wx.Size or wx.Point
    for it, so they just stuck with the GetMin/GetMax functions.

I included that test because of pg.207 in "wxPython in Action": Table 7.10 shows a GetRange function. Perhaps this should be an erratum?

This point raises a greater issue. In the absence of online docs, we can sometimes defer to "wxPython in Action." In these cases, or in cases where docs and book conflict, which is to be considered canon? Should docs reflect code, and "wxPython in Action" reflect docs? Should the docs, and the book as well, specify a contract against which the code must conform? I feel this is an important issue which needs clarification.

I view the book as a (mostly) accurate snapshot of only a subset of wxPython functionality that was taken at a specific point in time. Therefore it is only "cannon" at that point in time, and only at a high level view. To me it's main purpose is to make the ramp-up time easier for newbies and to help more experienced folks get a better grasp of the more advanced general concepts like event processing, application design, or how to use the more complex things. However I don't think the nitty-gritty details can ever be considered "doctrine" because with the nature of the project and community these details would be like trying to document the details of a sandcastle in a windstorm. Since that snapshot point in time the library has continued to evolve and change, some of which affects things that were documented before (although that is usually avoided if possible.)

The C++ docs are closer to being accurate, but they can fall behind sometimes and of course there is the issue of not being clear about what is available in wxPython. The wxPython specific docs, while incomplete, are probably the closest to being totally accurate since they are generated from the actual classes (including the Python-specific things) with very little by-hand maintenance. If some function or method is listed there then you know that it actually exists in wxPython and has parameters named the same as in the docs. One of these days I should organize a sprint or something for working on filling in the gaps of this set of docs.

Of course the ultimate authority is the source code, which IMO is the best thing about Open Source Software.

···

On 7/11/07, *Kevin Ollivier* <kevino@theolliviers.com > <mailto:kevino@theolliviers.com>> wrote:

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

Robin,

As for the virtual size failures, I suspect the issue is that
SetVirtualSize doesn’t allow the virtual size to go below the control’s
current size? Is that correct?

It is limited based on what the virtual size hints are currently set to.

Here is the common (platform independent) base class version:

I’m still curious about this, and having problems. SetVirtualSize doesn’t look like it’s using those hints as minima (on Windows).

For example:
a.SetVirtualSizeHints(1,1)
a.SetVirtualSize((2,2))
a.GetVirtualSize() # does not return ‘wx.Size(2,2)’

Am I missing something? Should setting the VirtualSizeHints give me control over what range of VirtualSize is possible?

Also, there’s no corresponding “GetVirtualSizeHints”, which may be useful for tests like these.

-Frank

···

On 7/11/07, Robin Dunn robin@alldunn.com wrote:

Frank Tobia wrote:

Robin,

     >
     > As for the virtual size failures, I suspect the issue is that
     > SetVirtualSize doesn't allow the virtual size to go below the
    control's
     > current size? Is that correct?

    It is limited based on what the virtual size hints are currently set
    to.
      Here is the common (platform independent) base class version:

I'm still curious about this, and having problems. SetVirtualSize doesn't look like it's using those hints as minima (on Windows).
For example:
a.SetVirtualSizeHints(1,1)
a.SetVirtualSize((2,2))
a.GetVirtualSize() # does not return 'wx.Size(2,2)'

Am I missing something? Should setting the VirtualSizeHints give me control over what range of VirtualSize is possible?

The current client size is factored in.

wxSize wxWindowBase::DoGetVirtualSize() const
{
     // we should use the entire client area so if it is greater than our
     // virtual size, expand it to fit (otherwise if the window is big enough we
     // wouldn't be using parts of it)
     wxSize size = GetClientSize();
     if ( m_virtualSize.x > size.x )
         size.x = m_virtualSize.x;

     if ( m_virtualSize.y >= size.y )
         size.y = m_virtualSize.y;

     return size;
}

Also, there's no corresponding "GetVirtualSizeHints", which may be useful for tests like these.

Enter a bug report about this.

···

On 7/11/07, *Robin Dunn* <robin@alldunn.com <mailto:robin@alldunn.com>> > wrote:

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

Hi Frank,

It's great to have a test framework, hopefully it will help to provide a
consistent behavior across platforms for the the basic things like
window frame/client area sizes.

···

On Wed, 2007-07-11 at 12:02 -0400, Frank Tobia wrote:

Hi all,

Now that the Unit Test SoC project has results to show, I wanted to
inform the community and solicit feedback.

You can get the latest copy of the test suite from SVN by running:
svn checkout
https://svn.wxwidgets.org/svn/wx/wxPython/branches/SOC2007_UNITTEST/tests/unittest unittest

From there, 'runAllTests.py' runs the whole test suite with default
options, or use 'runAllTests.py --help' to check out the various
options for configuration and data output.

Your code seem to be using some Python2.5 features like mixing
"try/except" and "finally" in the same block, so it can't be run with
Python2.4. Did you decide not to support the previous versions?

Roman

Roman Rolinsky wrote:

Your code seem to be using some Python2.5 features like mixing
"try/except" and "finally" in the same block, so it can't be run with
Python2.4. Did you decide not to support the previous versions?

We should try to maintain compatibility with Python 2.4, at least until after Python 2.6 is released and is in use.

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

Roman,

···

On 7/13/07, Roman Rolinsky rolinsky@femagsoft.com wrote:

Your code seem to be using some Python2.5 features like mixing
“try/except” and “finally” in the same block, so it can’t be run with
Python2.4. Did you decide not to support the previous versions?

The test suite is meant to support Python 2.4 and above. That was an oversight on my part. Thanks for catching it, I’ve since applied a fix.

Have you noticed any other Python2.5-only
features I should clean up?

Thanks again.

-Frank