[ANN] wxPyPlot V1.3 Beta

Changes from 1.2
- font cache for Linux speed problems

Great! I hadn't known about this, but it's nice to see.

A couple of people on this list have been working on multiple
bitmaps for
the background (grid, labels, legend) and foreground plotting in
wxPyPlotbecause they felt that replotting the background for each
time was slowing
things down.

Well, that's not the only reason. At the moment, all I did was one extra
bitmap for the axis and all, and then the data itself can be draw
separatly. I did think that having a bitmap for the legen could be
nifty, as then it would be easy to drag it around the plot, but that
would only make sense if there was a level of interaction involved anyway.

I could not understand this because I felt that
backgrounddrawing was only a small percentage of the time for large
numbers of data
and this was not really an issue.

But I think the light just came on thanks to Eric Blossom!!
Differencesbetween Windows and Linux again!

FWIW, we're using wxPyPlot to plot real time data with a refresh rate
of 25 Hz. Without this patch, the X font server ends up sucking up
about 70% of the CPU on a 1.7 GHz pentium. With the patch applied,
the font server is out of the loop.

Hmm. I never noticed this per se, but I did test mostly on Linux, and it
was definately a problem to do this sort of thing. Caching the fonts
sounds like a great idea. however...

After some profile testing on *windows*, I confirmed that drawing the
background takes 8-10 % of the time. The rest of the 90%+ time is
takendrawing points and lines (25,000 squares, 25,000 lines). On
my 1GHz Athlon
this takes roughly .3 sec.

Here's the difference. When drawing 25,000 squares, you're probably
right that the axis drawing is not a big deal. However, I introduced the
extra bitmap when trying to get fast an smooth updates with a thousand
or so points with no markers, i.e. one (or maybe up to four) call of
DC.DrawLines() with 1000 points, which is MUCH faster than 25,000
DrawRectangle() calls. In this case, the time it takes to calculate and
draw the axis will probably be larger than drawign the data. I do admit
that I haven't carefuly profiled this yet. The problems I did see may
have been caused by the font issue, but I doubt that that's all of it.

Windows, but would like users to test it out under Linux to see if
there is any speed improvement.

I can't do that now, but I'll try soon, I hope. I'd like to see what
other folks find. Peter, I'd like to here how wxOcilliscope works with
this version. If anyone does try: see what kind of a frame rate you can
get with 1000 points or so.

If I have misunderstood the problem, please let me know.

Well, the only misunderstanding is that we're only concerned about
perfomance with LOTS of points.

Frankly, I'm not entirely sure that the extra buffer is worth it, but
that was easier than doing it the way I think is right: separating out
the "figure how to draw" code from the "draw" code.

wxPyPlot was clearly designed with the idea that it would be used to
creat a plot, then perhaps a whole new plot with totaly different data,
rather than a single plot with the data updated frequently, but the rest
of the plot staying the same. I think with a little re-structuring, it
could do both well.

While I admit I havn't profiled carefully, it just seems silly to
compeletly re-calculate and draw all the dimensions, etc of the axis,
labels, etc, every time you re-draw. If the whole thing could be
calculated, then stored, and only the actual drawing re-done, it coudl
be much more efficient. As a first step, all we'd need to do is save the
data about how the gridlines and/or tick marks are drawn. Then, instead
of using the extra buffer, the DrawData() method could just re-draw the
gidlines with a few DC calls and then ReDraw the data. This would remove
the need for the extra buffer, and be almost as fast as the blit.

However, what is wrong with the extra buffer? I think it's a pretty
elegant solution, easy to do, and in fact, done. Unneccessary
optimization to save memory is just as bad as unnecessary optimization
to save computation.

In any case, let's see how the speed thing plays out with the fixed font
code on Linux.

By the way, as a related note: Profiling drawing code on Linux is a
difficult, because X is asycronous: you time how long it takes the
program to tell X what to do, not how long it takes to do it. This can
be quite different for complicated drawings: I 've noticed it
particularly for large polygons.

I certainly rathre not forkl wxPyPlot: I'm sure we can find a solution
we all like.

-Chris

···

From: "Gordon Williams" <g_will@cyberus.ca>

I look forward to see some profiling data on Linux. I'm sure it will be
quite illuminating. Without this data, we are just shooting in the dark. I
now understand that there were speed issues with Linux that were not in
Windows. I'm hoping with Eric's patch, these have been resolved.

For small numbers of points (eg 1000), plotting speed is a moot issue
because it becomes a negligible part of the overall process. I know in
oscilloscope applications the problem quickly becomes IO bounded with
communication to the external device. Been there, done that. As well, once
you reach a certain screen refresh rate, a faster rate doesn't improve the
quality of the product. The law of diminishing returns.

I don't know how many points that Eric was plotting on his Linux box, but a
25 Hz refresh rate is not unreasonable. As I mentioned, 25K lines and
squares took .3 sec on my 1GHz Windows machine. If you only want to plot
lines, that will drop to .1 sec. (10 Hz refresh rate) - see the benchmark
data at the top of the wxPyPlot code. Now if you reduce that down to 1K
points, I would expect a 10 times increase in speed or something in the
range of a 50-100 Hz refresh rate. Also consider the monitor screen is
refreshed at 60-70 Hz and above 30 Hz, you don't get much visual
improvement.

The point that I am trying to make is that drawing the background separately
*appears* not a useful optimization because it is only a small fraction of
the process time for larger point sets and it doesn't really matter for
small point sets. Furthermore, and just as importantly, it adds complexity
and weight to the code, which is against the original intent for wxPyPlot.

I look forward to seeing some comparative results.

Regards,

Gordon Williams

Gordon Williams wrote:

I look forward to see some profiling data on Linux. I'm sure it will be
quite illuminating.

I've done some: see below.

I now understand that there were speed issues with Linux that were not in
Windows. I'm hoping with Eric's patch, these have been resolved.

I think so. It is MUCH faster with the font caching. I havn't reported
any numbers for the old one below, as no one is proposing using it.

For small numbers of points (eg 1000), plotting speed is a moot issue
because it becomes a negligible part of the overall process.

Not true. In fact, quite the opposite. The speed of plotting the points
themselves is negligable, so the speed of the computing and drawing the
axes becomes relatively more important.

oscilloscope applications the problem quickly becomes IO bounded with
communication to the external device. Been there, done that.

There are other reasons to do animated displays that have nothing to do
with IO.

As well, once
you reach a certain screen refresh rate, a faster rate doesn't improve the
quality of the product. The law of diminishing returns.

True.

As I mentioned, 25K lines and
squares took .3 sec on my 1GHz Windows machine. If you only want to plot
lines, that will drop to .1 sec. (10 Hz refresh rate) - see the benchmark
data at the top of the wxPyPlot code. Now if you reduce that down to 1K
points, I would expect a 10 times increase in speed or something in the
range of a 50-100 Hz refresh rate.

Not really, as the time to create the axes, etc becomes a factor.

Also consider the monitor screen is
refreshed at 60-70 Hz and above 30 Hz, you don't get much visual
improvement.

Good point

The point that I am trying to make is that drawing the background separately
*appears* not a useful optimization because it is only a small fraction of
the process time for larger point sets and it doesn't really matter for
small point sets.

Again, I disagree, it doesn't matter for large sets, but might for small
sets.

Furthermore, and just as importantly, it adds complexity
and weight to the code, which is against the original intent for wxPyPlot.

I can't say I've done it well, but I think a proper re-stucturing would
make the code clearer and cleaner. I had a hell of a time figuring out
where stuff was actually being drawn, etc. As I wrote it, it's a
pretty simple change anyway, mostly just moving a small chunk of code
from the Draw() method to a DrawData() method.

As for weight, an extra bitmap is not that big a deal these days. If I
get a
chance I'll post the memory numbers

Now the results:

I used a small test app (attached) adapted from something Peter had sent
me awhile
back when I started working on this. It's enclosed. Essentially, it
draws two sinusoids with n points, animating them. I report the max
frame rate possible with two versions of wxPyPlot: wxPyPlot1.4, the one
Gordon just released, and wxPyPlot2, my version with the extra buffer,
based on wxPyPlot 1.2, I think. If anyone wants, I could merge the
two to take advantage of Gordon's recent work.

Here we go:

# 100 points:
wxPyPlot1.4:
Frame Rate = 114.218845 fps

wxPyPlot2:
Frame Rate = 354.189966 fps

*So, as I predicted, the extra buffer is much faster, but as Gordon
predicted, the old version is plenty fast.

# 500 points:
wxPyPlot1.4:
Frame Rate = 91.168793 fps

wxPyPlot2:
Frame Rate = 266.505971 fps

*Pretty much same as above.

# 1000 points:
wxPyPlot1.4:
Frame Rate = 72.520699 fps

wxPyPlot2:
Frame Rate = 153.479360 fps

* getting closer to critical speed for the non-buffer version, but if
your machine is as fast as mine, and not doing much else, it's still
plenty fast.

# 10000 points:
wxPyPlot1.4:
Frame Rate = 15.343403 fps

wxPyPlot2:
Frame Rate = 17.318894 fps

* and here is where the time to draw the data itself is significant, and
it doesn't make much difference which way you do it.

I'm not sure where to go from here. I imagine Gordon will look at the
results and think his current code is plenty fast. It's hard to disagree
with that, but I have two concerns:

1) It just doesn't seem clean to me to do all that work every time you
just want to re-draw the data.

2) Someone very well may want to do something like this on a slower or
busier machine than mine (1Ghz P4, Linux, doing nothing else of
consequence) of course, my machine's getting kind of old, so maybe not.

Peter: have you tried Gordon's latest version with your oscilloscope
app?

However, I don't care enough about this to maintain a forked version, so
it's up to Gordon. Besides, what I'd really like to do is re-structure
the code more substantially. If I have a need for it, I may do that, and
then submit that for consideration.

In any case, I've learned something...and thanks to Gordon for
maintaining wxPyPlot

-Chris

pyplottest.py (3.47 KB)

···

--
Christopher Barker, Ph.D.
Oceanographer
                                        
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov