Hi Christopher,
Thanks for the detailed reply. Many comments are below.
Is that all inside the loop? I thought I put that outside, so that there
wouldn't be any overhead if you weren't using different Pens. If not,
then you could do that -- do switch on the two cases -- a Pen list
passed in, or not.
Looking at wxPyDrawXXXList in drawlisrt.c it seems that dc.SetPen and
dc.SetBrush are invoked each time round the loop, along with some
logic.
I would stress that my case of having 30,000 lines in one pen is
perhaps uncommon. In part this results form problems I have had where
if I blindly allocated 30,000 wx.Pens, one for each line then
something goes wrong and they all end up green. So now my Python code
groups all lines of one colour together before making a single pen for
them.
> I imaging I can get a significant speedup by writing my own function
> that:
> 1) Ommits the brushes and pens logic,
or moves it.
Indeed. Perhaps expanding the function to be a bit less generic with
separate loops for cases with one or multiple pens might help. Having
said that I think a seperate set of functions for numpy might be
better (not many people will need this level of specialisation...) As
you go on to say though, the main time is probably spent in the
sequence access and the actual drawing, not here...
> 2) Replaces the PySequence_GetItem() with direct access the numpy
> array
That could make a difference -- a big one perhaps. However, you might
want to do some careful profiling. I was generally disappointed with the
performance of the DrawXXXList methods -- it turns out that it takes a
while to draw things, so lowering the calling overhead didn't help much,
except for simple drawing: DrawPointList was a big improvement. Lines
are simple enough that I think you can improve it, but it's worth making
sure.
I have a feeling you're right, although as you say I will need to
profile things. Still even a 20% speedup would make me a happy man.
> 3) Inlines the actual line drawing.
I'm not entirely sure what you intend here -- but because of the above,
I doubt it will make too much of a difference.
Sorry; I should have been clearer. The actual call to the wxWidgets
drawing function does not occour within wxPyDrawXXXList but within a
helper function doDraw which is passed in to wxPyDrawXXXList as an
argument (from _gdi_wrap.cpp) which is one of wxPyDrawXXXPoints,
wxPyDrawXXXLine etc. within drawlist.cpp.
This is the calling line from wxPyDrawXXXList
// call the drawOp
bool success = doDraw(dc, coords);
and this is the code for drawing a line
bool wxPyDrawXXXLine(wxDC& dc, PyObject* coords)
{
int x1, y1, x2, y2;
if (! wxPy4int_seq_helper(coords, &x1, &y1, &x2, &y2)) {
PyErr_SetString(PyExc_TypeError, "Expected a sequence of
(x1,y1, x1,y2) sequences.");
return false;
}
dc.DrawLine(x1,y1, x2,y2);
return true;
}
So as well as the function call overhead there is a sequence unpacking
(more function calls) and error check going on for each line drawn
which could be omitted if running form a numpy.ndarray
One other option: you might try converitng your numpy arrays to lists
before passing it in to DrawXXXList. It's a bit counter-intuitive, but
numpy can convert to lists faster than PySequence_GetItem(), as it knows
that the array is homogenous. Then, in the wxPython code, there is
special case code for accessing lists and tuples, rather than generic
sequences. This won't b as good as it can get, though!
Thanks, I will try that tomorrow and measure some timings. Would
never have thought of that!
> I am guessing that code directly accessing numpy arrays (vs. using
> PySequence_GetItem() is not welcom in the wxPython core code as it
> requires numpy?
True, which is why it doesn't to it already! I now I would have done it
that way if I could.
However, it would be much better if you could enhance the wxPython code
rather than forking -- then we could all benefit.
> If that is the case I either need to maintain a fork
> of wx or ideally just build an extension module with this funciton.
not a bad option, either, but it would be nice if you could get this
into wxPython. My thoughts:
I don't know how familiar with numpy you are, but you don't really have
to use the numpy API to access the data -- the data itself is simply in
a char* pointer.
So what you do need to do it check the Python Object passed in to see if
it is is anumpy array of the type you can handle. There are utiity
functions in numpy for this (and for converting from anything to a numpy
array of the type you want), but maybe we could get less featured numpy
support by only supporting passing in the "right" numpy array: an NX2
C-contiguous array or int32, for instance (though I'd like to see float
support..)
This is a very good point - in some other code that I run on different
architectures I just pass the pointer to the numpy data into my
extension code as my limited patience expires when it comes to getting
all the headers in the right place for building against numpy. Numpy
is particularly good at exposing the data pointer within Python
compared to numeric/numarray with numpy.ndarray.ctypes.data
A couple options for doing this:
1) borrowing the code from numpy that checks for an array, and then
does a couple checks for array dytpe, shape, etc -- it's not that hard
to access the numpy struct to get what you need for the simple case.
This would involving including just a bit of numpy code with wxPython.
THe downside is that you would need to keep it in sync.
I have no experience of working on or contributing to a project the
size of wxPython but I can imaging keeping the borrowed numpy code in
sync and dealing with differnt numpy versions being a real pain. I
think all the numpy related code can be kept within the Python source
only. This would ensure the numpy array is of the correct format
before invoking C code.
This should abstract away the details of the numpy implementation. I
could create a Python function as part of wxPython called something
like
def drawLineList_numpy(dc, points):
assert NUMPY_IMPORTED_OKAY # Some check done when importing numpy
at module level
assert isinstance(points, numpy.ndarray)
n_points, n4 = points.shape
assert n4 == 4 # A line has four coords
points = numpy.ascontiguousarray(points) # doesn't do anything if
points is contig
dat = dat.astype(numpy.float32) # C counterpart expects float32
p_points = points.ctypes.data # pointer to array data of graphical
points.
dc.drawLineList_numpy(n_points, p_points)
As long as the calling code uses numpy.float32 and does't get the
'points' array from slicing the .astype and .ascontiguousarray calls
will be very fast as no new arrays are allocated as no changes are
needed. The corresponding C code is then an incredibly simple loop
calling dc.DrawLine.
I don't know how such a cavalier approach to having C code without any
error checking on the arguments and moving it all to Python sits with
the way wxPython is run? I quite like it as I would rather spend my
time doing error and compliance checking in Python - this especially
helps with numpy support as it means only the Python and not the C has
to know anything at all about numpy. I would put all these helper
functions (drawLineList_numpy, drawPointsList_numpy etc.) into a
separate module called something like 'PerformanceDraw.py' that only
imports if numpy is present. I think this would make a nice
supplement to drawXXXList.py for people who have extreme drawing
requirements and are willing to accept more restrictions.
2) use the buffer interface -- numpy exports the buffer interface, so
you could use standard buffer access -- this would blow up if the numpy
array was the wrong kind -- but user beware, as with every other use of
buffer!
3) use the new buffer interface - this is better, as it includes
things like type and shape of the data, so you could know that the data
pointer is what you expect. The problem here is that it is PY2.6 and up
only, and may not even be supported in numpy (though it should be in teh
latest version, anyway)
4) use the numpy array interface:
The array interface protocol — NumPy v2.1 Manual
It was designed specifically for this sort of thing, but as you can see
from that doc, is deprecated in favor of the not-yet-well supported new
buffer interface -- oh well, a tricky time to figure out what to do!
Arg. Several good options here. I favour the Python/C approach I
outlined above, but perhaps mainly on grounds of laziness.
Chris, thanks for your advice. I am going to try and get to a stage
where I can build wxPython myself and then work on implementing a
version of this. If I get it working well I am happy to contribute it
back to wxPython.
Let me know what you think.
Some background - I am working on a hybrid bitmap/vector drawing
library. I use this in many places, two of which really stress
drawing speed. One is a work project where I overlay vector
annotations on movies of biological data sets and the other is a hobby
project building a vector map viewer for OpenStreetMap data. This is
an excellent stress test for drawing code...
Regards,
Chris S.
···
Another option is to use Cython to write your code -- I wonder how hard
it would be to integrate it into the SWIGed wxPython code. It seems it
should be possible -- the Cython function would take a wx.DC, and then
call wx code. Once you linked to the same wx, it should work, if you can
figure out how to extract the SWIGified pointer from the wrapped wx.DC.
NOTE: if you come up with a way to do this that can be integrated into
wxPython, please try to do it as an enhancement to the wxPointListHelper
(Or something like that...) so that it can help all such functions!
HTH,
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chris.Bar...@noaa.gov