floatcanvas/navcanvas memory increasing with text object

wxPython Users
1

Hello all,

I’ve run into an issue with the NavCanvas/FloatCanvas. I have an application where I add a ScaledTextBox to the canvas. I then call canvas.Draw(Force=True) every clock tick and memory continues to increase as if there’s memory leak (based on Windows Task Manager). I have been unable to diagnose the issue further and would love some help. Attached is a simple python app that reproduces the issue.

I’m running Windows 7 x64 with wxpython version ‘2.8.12.1 (msw-unicode)’ and python 2.7.4.

Please let me know if you need further information or if there is somewhere else I should post this issue.

Thanks!

Bryan Lott

reproduce_canvas_memory_increase.py (810 Bytes)

2

Hi Bryan,

I recently realized that there is a circular reference in FC that might cause this… Though I. The last it’s shown up in more extreme situations. Though scaledTextBox does require a fair number of temp objects to draw, do maybe some of them aren’t getting cleaned up.

You might try a call to gc.collect() after the Draw.

I’ll try to take a look when I can.

Chris

···

On Aug 26, 2013, at 9:52 AM, Bryan Lott mystickphoenix@gmail.com wrote:

Hello all,

I’ve run into an issue with the NavCanvas/FloatCanvas. I have an application where I add a ScaledTextBox to the canvas. I then call canvas.Draw(Force=True) every clock tick and memory continues to increase as if there’s memory leak (based on Windows Task Manager). I have been unable to diagnose the issue further and would love some help. Attached is a simple python app that reproduces the issue.

I’m running Windows 7 x64 with wxpython version ‘2.8.12.1 (msw-unicode)’ and python 2.7.4.

Please let me know if you need further information or if there is somewhere else I should post this issue.

Thanks!

Bryan Lott

–

You received this message because you are subscribed to the Google Groups “wxPython-users” group.

To unsubscribe from this group and stop receiving emails from it, send an email to wxpython-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

<reproduce_canvas_memory_increase.py>

3

Thanks Chris!

Unfortunately adding the call to gc.collect() didn’t help with the memory issue.

If there’s anything I can do to help, please let me know.

Bryan Lott

···

On Monday, August 26, 2013 11:04:40 AM UTC-6, Chris Barker - NOAA Federal wrote:

Hi Bryan,

I recently realized that there is a circular reference in FC that might cause this… Though I. The last it’s shown up in more extreme situations. Though scaledTextBox does require a fair number of temp objects to draw, do maybe some of them aren’t getting cleaned up.

You might try a call to gc.collect() after the Draw.

I’ll try to take a look when I can.

Chris

On Aug 26, 2013, at 9:52 AM, Bryan Lott mystick...@gmail.com wrote:

Hello all,

I’ve run into an issue with the NavCanvas/FloatCanvas. I have an application where I add a ScaledTextBox to the canvas. I then call canvas.Draw(Force=True) every clock tick and memory continues to increase as if there’s memory leak (based on Windows Task Manager). I have been unable to diagnose the issue further and would love some help. Attached is a simple python app that reproduces the issue.

I’m running Windows 7 x64 with wxpython version ‘2.8.12.1 (msw-unicode)’ and python 2.7.4.

Please let me know if you need further information or if there is somewhere else I should post this issue.

Thanks!

Bryan Lott

–

You received this message because you are subscribed to the Google Groups “wxPython-users” group.

To unsubscribe from this group and stop receiving emails from it, send an email to wxpython-user...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

<reproduce_canvas_memory_increase.py>

4

Unfortunately adding the call to gc.collect() didn't help with the memory
issue.

Darn -- it turns out that some wrapped C++ objects dont get cleaned up
by gc.collect -- I still don't get why but Robin told me that, and I'm
sure he's right.

If there's anything I can do to help, please let me know.

Well, here's the trick -- I just ran your code on my machine:

wxPython 2.9.4.0 osx-cocoa (classic) (32 bit python 2.7.5)

And I see memory go strait to 29.2MB and just stay there -- and I even
cranked up the number of iterations by a factor of ten. So maybe there
is Windows font resource or something that is getting leaked.

I'll poke at it a bit more, but it'll be particularly difficult if I
can't reproduce it.

One possible semi-fix -- take a look at the ScaledTextBox code --
maybe you could add some code in there to cache the fonts and re-use
them if the scale hasn't changed. You could still get a leak if the
scale does change, but that will be less often, at least.

Or even explicitly delete some stuff at the end of the _draw method...

-CHB

···

On Mon, Aug 26, 2013 at 10:22 AM, Bryan Lott <mystickphoenix@gmail.com> wrote:

Bryan Lott

On Monday, August 26, 2013 11:04:40 AM UTC-6, Chris Barker - NOAA Federal > wrote:

Hi Bryan,

I recently realized that there is a circular reference in FC that might
cause this.... Though I. The last it's shown up in more extreme situations.
Though scaledTextBox does require a fair number of temp objects to draw, do
maybe some of them aren't getting cleaned up.

You might try a call to gc.collect() after the Draw.

I'll try to take a look when I can.

Chris

On Aug 26, 2013, at 9:52 AM, Bryan Lott <mystick...@gmail.com> wrote:

Hello all,

I've run into an issue with the NavCanvas/FloatCanvas. I have an
application where I add a ScaledTextBox to the canvas. I then call
canvas.Draw(Force=True) every clock tick and memory continues to increase as
if there's memory leak (based on Windows Task Manager). I have been unable
to diagnose the issue further and would love some help. Attached is a
simple python app that reproduces the issue.

I'm running Windows 7 x64 with wxpython version '2.8.12.1 (msw-unicode)'
and python 2.7.4.

Please let me know if you need further information or if there is
somewhere else I should post this issue.

Thanks!
Bryan Lott

--
You received this message because you are subscribed to the Google Groups
"wxPython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to wxpython-user...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

<reproduce_canvas_memory_increase.py>

--
You received this message because you are subscribed to the Google Groups
"wxPython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to wxpython-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

5

Well, here's the trick -- I just ran your code on my machine:

wxPython 2.9.4.0 osx-cocoa (classic) (32 bit python 2.7.5)

And I see memory go strait to 29.2MB and just stay there -- and I even
cranked up the number of iterations by a factor of ten. So maybe there
is Windows font resource or something that is getting leaked.

I figured that maybe the issue didn't show because there wasn't
anything actually being draw to the screen i nthis example. Presumably
in your real use-case, you do want to display something. So I edited
your test case to Show() teh Frame, and call the MianLoop, and then
call Draw in a Timer event. It's slower to iterate, but it's actually
doing something!

and now I see the memory leak:

start: 32.7MB
2000 iterations: 33MB
10,000 iterations: 34.3MB
20,000 iterations: 36.2MB

so about 0.2k per iteration. -- not huge with today's memory
requirements, but not good either!

I'll be spending some time on a plane today, maybe I'll try to see
what's going on.

Anyone know how to see what objects python has allocated and not
cleaned up? Can it be done?

-Chris

···

On Mon, Aug 26, 2013 at 11:08 AM, Chris Barker - NOAA Federal <chris.barker@noaa.gov> wrote:

I'll poke at it a bit more, but it'll be particularly difficult if I
can't reproduce it.

One possible semi-fix -- take a look at the ScaledTextBox code --
maybe you could add some code in there to cache the fonts and re-use
them if the scale hasn't changed. You could still get a leak if the
scale does change, but that will be less often, at least.

Or even explicitly delete some stuff at the end of the _draw method...

-CHB

Bryan Lott

On Monday, August 26, 2013 11:04:40 AM UTC-6, Chris Barker - NOAA Federal >> wrote:

Hi Bryan,

I recently realized that there is a circular reference in FC that might
cause this.... Though I. The last it's shown up in more extreme situations.
Though scaledTextBox does require a fair number of temp objects to draw, do
maybe some of them aren't getting cleaned up.

You might try a call to gc.collect() after the Draw.

I'll try to take a look when I can.

Chris

On Aug 26, 2013, at 9:52 AM, Bryan Lott <mystick...@gmail.com> wrote:

Hello all,

I've run into an issue with the NavCanvas/FloatCanvas. I have an
application where I add a ScaledTextBox to the canvas. I then call
canvas.Draw(Force=True) every clock tick and memory continues to increase as
if there's memory leak (based on Windows Task Manager). I have been unable
to diagnose the issue further and would love some help. Attached is a
simple python app that reproduces the issue.

I'm running Windows 7 x64 with wxpython version '2.8.12.1 (msw-unicode)'
and python 2.7.4.

Please let me know if you need further information or if there is
somewhere else I should post this issue.

Thanks!
Bryan Lott

--
You received this message because you are subscribed to the Google Groups
"wxPython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to wxpython-user...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

<reproduce_canvas_memory_increase.py>

--
You received this message because you are subscribed to the Google Groups
"wxPython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to wxpython-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

6

Interesting that in your case unless you show the frame, memory doesn’t increase. On mine it increases whether I show it or not. In any case, glad you were able to reproduce the error. And agreed, it’s not a lot in today’s memory requirements but I’m adding a number of text objects to a canvas for an app that should be able to be kept open while the user is working. In an hour I’ve had it end up using close to a half gig of memory.

When I thought the issue was in my code, I was using Pympler and Objgraph to do various memory profiling, looking for allocated objects that gc didn’t collect, etc (and I was able to find a bug in my own code that happened to be leaking memory as well).

Pympler (muppy): http://pythonhosted.org/Pympler/muppy.html

Objgraph: http://mg.pov.lt/objgraph/

-Bryan

···

On Monday, August 26, 2013 12:41:10 PM UTC-6, Chris Barker - NOAA Federal wrote:

On Mon, Aug 26, 2013 at 11:08 AM, Chris Barker - NOAA Federal > > chris....@noaa.gov wrote:

Well, here’s the trick – I just ran your code on my machine:

wxPython 2.9.4.0 osx-cocoa (classic) (32 bit python 2.7.5)

And I see memory go strait to 29.2MB and just stay there – and I even

cranked up the number of iterations by a factor of ten. So maybe there

is Windows font resource or something that is getting leaked.

I figured that maybe the issue didn’t show because there wasn’t

anything actually being draw to the screen i nthis example. Presumably

in your real use-case, you do want to display something. So I edited

your test case to Show() teh Frame, and call the MianLoop, and then

call Draw in a Timer event. It’s slower to iterate, but it’s actually

doing something!

and now I see the memory leak:

start: 32.7MB

2000 iterations: 33MB

10,000 iterations: 34.3MB

20,000 iterations: 36.2MB

so about 0.2k per iteration. – not huge with today’s memory

requirements, but not good either!

I’ll be spending some time on a plane today, maybe I’ll try to see

what’s going on.

Anyone know how to see what objects python has allocated and not

cleaned up? Can it be done?

-Chris

I’ll poke at it a bit more, but it’ll be particularly difficult if I

can’t reproduce it.

One possible semi-fix – take a look at the ScaledTextBox code –

maybe you could add some code in there to cache the fonts and re-use

them if the scale hasn’t changed. You could still get a leak if the

scale does change, but that will be less often, at least.

Or even explicitly delete some stuff at the end of the _draw method…

-CHB

Bryan Lott

On Monday, August 26, 2013 11:04:40 AM UTC-6, Chris Barker - NOAA Federal > > >> wrote:

Hi Bryan,

I recently realized that there is a circular reference in FC that might

cause this… Though I. The last it’s shown up in more extreme situations.

Though scaledTextBox does require a fair number of temp objects to draw, do

maybe some of them aren’t getting cleaned up.

You might try a call to gc.collect() after the Draw.

I’ll try to take a look when I can.

Chris

On Aug 26, 2013, at 9:52 AM, Bryan Lott mystick...@gmail.com wrote:

Hello all,

I’ve run into an issue with the NavCanvas/FloatCanvas. I have an

application where I add a ScaledTextBox to the canvas. I then call

canvas.Draw(Force=True) every clock tick and memory continues to increase as

if there’s memory leak (based on Windows Task Manager). I have been unable

to diagnose the issue further and would love some help. Attached is a

simple python app that reproduces the issue.

I’m running Windows 7 x64 with wxpython version ‘2.8.12.1 (msw-unicode)’

and python 2.7.4.

Please let me know if you need further information or if there is

somewhere else I should post this issue.

Thanks!

Bryan Lott

–

You received this message because you are subscribed to the Google Groups

“wxPython-users” group.

To unsubscribe from this group and stop receiving emails from it, send an

email to wxpython-user...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

<reproduce_canvas_memory_increase.py>

–

You received this message because you are subscribed to the Google Groups

“wxPython-users” group.

To unsubscribe from this group and stop receiving emails from it, send an

email to wxpython-user...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

–

Christopher Barker, Ph.D.

Oceanographer

Emergency Response Division

NOAA/NOS/OR&R (206) 526-6959 voice

7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

–

Christopher Barker, Ph.D.

Oceanographer

Emergency Response Division

NOAA/NOS/OR&R (206) 526-6959 voice

7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

7

OK,

I did some more testing and looking at the code, and can't see where
therewould be a circular reference here. The code does create and delete a
wxFont on every iteration -- perhaps that's the issue?

Here's how the memory grows for me with the enclosed sample:

With ScaledTextBox:

start 32.6MB
5000 iterations: 33.6
10000 iterations: 34.4
15000 itertaions: 35.1

And with drawing a polygon, but no text:

With Polygon:

start 32.5MB
5000 iterations: 32.6
10000 iterations: 32.6
15000 itertaions: 32.6

so that looks pretty solid.

So I tried doing a similar thing with just a DC -- no floatcanvas code:

With DrawTextList
Start; 30.0
10,000 iterations: 30.5
20,000 iterations: 30.6
30,000 iterations: 30.6
50,000 iterations: 30.8
100,000 iterations: 31.2

So still some memory growth -- but not as much.

And I use a raw DC with a Polygon and no Text:

With DrawPolygon:
Start: 29.8
20,000 iterations: 30.4
30,000 iterations: 30.4
50,000 iterations: 30.6
100,000 iterations: 31.1

pretty similar.

So I'm not sure at this point.

Bryan -- does your memory growth look similar? If so, the it would be a
LONG time before there was any real issue (not that it's not a bug) Perhaps
you could re-factor your code not to re-draw as frequently -- why does it
re-draw so often?

-Chris

···

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

8

I see similar memory growth with a single object. Unfortunately, I’m drawing more than just a single object. Do you think there’s a way the code could preserve the wxFont object instead of creating and deleting it? It may be that there’s a bit of something that the gc can’t collect from the wxFont.

The frequent redraw is to get animation of (labeled) nodes in a graph. I’m using a force based layout to draw a dynamic, interactive graph of connections between objects in a database.

If this is bending the FC/DC framework too far, do you have a recommendation for another framework? I’ve tried Kivy but ran into issues with its display of tabular data.

Thanks!

Bryan

···

On Monday, September 2, 2013 6:16:00 PM UTC-6, Chris Barker - NOAA Federal wrote:

OK,

I did some more testing and looking at the code, and can’t see where therewould be a circular reference here. The code does create and delete a wxFont on every iteration – perhaps that’s the issue?

Here’s how the memory grows for me with the enclosed sample:

With ScaledTextBox:

start 32.6MB

5000 iterations: 33.6

10000 iterations: 34.4

15000 itertaions: 35.1

And with drawing a polygon, but no text:

With Polygon:

start 32.5MB

5000 iterations: 32.6

10000 iterations: 32.6

15000 itertaions: 32.6

so that looks pretty solid.

So I tried doing a similar thing with just a DC – no floatcanvas code:

With DrawTextList

Start; 30.0

10,000 iterations: 30.5

20,000 iterations: 30.6

30,000 iterations: 30.6

50,000 iterations: 30.8

100,000 iterations: 31.2

So still some memory growth – but not as much.

And I use a raw DC with a Polygon and no Text:

With DrawPolygon:

Start: 29.8

20,000 iterations: 30.4

30,000 iterations: 30.4

50,000 iterations: 30.6

100,000 iterations: 31.1

pretty similar.

So I’m not sure at this point.

Bryan – does your memory growth look similar? If so, the it would be a LONG time before there was any real issue (not that it’s not a bug) Perhaps you could re-factor your code not to re-draw as frequently – why does it re-draw so often?

-Chris

–

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice

7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

9

I see similar memory growth with a single object. Unfortunately, I’m drawing more than just a single object.

So you’re seeing memory growth proportional to how many text objects you have?

Do you think there’s a way the code could preserve the wxFont object instead of creating and deleting it?

Actually, it does do that, if I can believe my own comments. At least within the draw loop. So if you have a lot of text objects that are all the same, it should create only one wxFont object per draw loop.

It does clear the font cache for each draw, as I didn’t want to keep a whole pile of them around that might never get used again.

It may be that there’s a bit of something that the gc can’t collect from the wxFont.

I guess I could try turning off the cache clearing and see what happens.

The frequent redraw is to get animation of (labeled) nodes in a graph. I’m using a force based layout to draw a dynamic, interactive graph of connections between objects in a database.

If this is bending the FC/DC framework too far,

It shouldn’t be–that’s exactly the kind of thing it’s intended to do.

I’m still not sure the root of the problem, though. It does appear that raw use of DCs for text does make memory grow, but it is pretty darn small. But I don’t see a circular reference issue here, either, so not sure what’s going on.

By the way, in you investigations, did you confirm that it is extra wxFonts being kept around? Or are still not sure about that?

-Chris

I’ll poke at it some more.

···

On Sep 3, 2013, at 7:11 AM, Bryan Lott mystickphoenix@gmail.com wrote:

do you have a recommendation for another framework? I’ve tried Kivy but ran into issues with its display of tabular data.

Thanks!

Bryan

On Monday, September 2, 2013 6:16:00 PM UTC-6, Chris Barker - NOAA Federal wrote:

OK,

I did some more testing and looking at the code, and can’t see where therewould be a circular reference here. The code does create and delete a wxFont on every iteration – perhaps that’s the issue?

Here’s how the memory grows for me with the enclosed sample:

With ScaledTextBox:

start 32.6MB

5000 iterations: 33.6

10000 iterations: 34.4

15000 itertaions: 35.1

And with drawing a polygon, but no text:

With Polygon:

start 32.5MB

5000 iterations: 32.6

10000 iterations: 32.6

15000 itertaions: 32.6

so that looks pretty solid.

So I tried doing a similar thing with just a DC – no floatcanvas code:

With DrawTextList

Start; 30.0

10,000 iterations: 30.5

20,000 iterations: 30.6

30,000 iterations: 30.6

50,000 iterations: 30.8

100,000 iterations: 31.2

So still some memory growth – but not as much.

And I use a raw DC with a Polygon and no Text:

With DrawPolygon:

Start: 29.8

20,000 iterations: 30.4

30,000 iterations: 30.4

50,000 iterations: 30.6

100,000 iterations: 31.1

pretty similar.

So I’m not sure at this point.

Bryan – does your memory growth look similar? If so, the it would be a LONG time before there was any real issue (not that it’s not a bug) Perhaps you could re-factor your code not to re-draw as frequently – why does it re-draw so often?

-Chris

–

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice

7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

–

You received this message because you are subscribed to the Google Groups “wxPython-users” group.

To unsubscribe from this group and stop receiving emails from it, send an email to wxpython-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

10

Memory seems to grow quite a bit faster with more text objects:

1 object -

0 - 30.292

5000 - 31.944

10000 - 33.492

15000 - 34.956

20000 - 36.428

20 objects (a good number for the amount on-screen in my real app at a given point) -

0 - 30.340

5000 - 60.476

10000 - 90.028

15000 - 119.856

20000 - 149.724

Sounds like you’re running into the same “memory is being used but I can’t figure out where” issue that I was running into. I was never able to confirm that anything was being kept around that shouldn’t (which was the frustrating part).

Is it possible that it’s the underlying c libraries that aren’t letting go of something? (No evidence to back this up, kinda grasping at straws at the moment…)

I’ve attached my test code for reference.

-Bryan

reproduce_canvas_memory_increase.py (1004 Bytes)

···

On Tuesday, September 3, 2013 1:13:32 PM UTC-6, Chris Barker - NOAA Federal wrote:

On Sep 3, 2013, at 7:11 AM, Bryan Lott mystick...@gmail.com wrote:

I see similar memory growth with a single object. Unfortunately, I’m drawing more than just a single object.

So you’re seeing memory growth proportional to how many text objects you have?

Do you think there’s a way the code could preserve the wxFont object instead of creating and deleting it?

Actually, it does do that, if I can believe my own comments. At least within the draw loop. So if you have a lot of text objects that are all the same, it should create only one wxFont object per draw loop.

It does clear the font cache for each draw, as I didn’t want to keep a whole pile of them around that might never get used again.

It may be that there’s a bit of something that the gc can’t collect from the wxFont.

I guess I could try turning off the cache clearing and see what happens.

The frequent redraw is to get animation of (labeled) nodes in a graph. I’m using a force based layout to draw a dynamic, interactive graph of connections between objects in a database.

If this is bending the FC/DC framework too far,

It shouldn’t be–that’s exactly the kind of thing it’s intended to do.

I’m still not sure the root of the problem, though. It does appear that raw use of DCs for text does make memory grow, but it is pretty darn small. But I don’t see a circular reference issue here, either, so not sure what’s going on.

By the way, in you investigations, did you confirm that it is extra wxFonts being kept around? Or are still not sure about that?

-Chris

I’ll poke at it some more.

do you have a recommendation for another framework? I’ve tried Kivy but ran into issues with its display of tabular data.

Thanks!

Bryan

On Monday, September 2, 2013 6:16:00 PM UTC-6, Chris Barker - NOAA Federal wrote:

OK,

I did some more testing and looking at the code, and can’t see where therewould be a circular reference here. The code does create and delete a wxFont on every iteration – perhaps that’s the issue?

Here’s how the memory grows for me with the enclosed sample:

With ScaledTextBox:

start 32.6MB

5000 iterations: 33.6

10000 iterations: 34.4

15000 itertaions: 35.1

And with drawing a polygon, but no text:

With Polygon:

start 32.5MB

5000 iterations: 32.6

10000 iterations: 32.6

15000 itertaions: 32.6

so that looks pretty solid.

So I tried doing a similar thing with just a DC – no floatcanvas code:

With DrawTextList

Start; 30.0

10,000 iterations: 30.5

20,000 iterations: 30.6

30,000 iterations: 30.6

50,000 iterations: 30.8

100,000 iterations: 31.2

So still some memory growth – but not as much.

And I use a raw DC with a Polygon and no Text:

With DrawPolygon:

Start: 29.8

20,000 iterations: 30.4

30,000 iterations: 30.4

50,000 iterations: 30.6

100,000 iterations: 31.1

pretty similar.

So I’m not sure at this point.

Bryan – does your memory growth look similar? If so, the it would be a LONG time before there was any real issue (not that it’s not a bug) Perhaps you could re-factor your code not to re-draw as frequently – why does it re-draw so often?

-Chris

–

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice

7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

–

You received this message because you are subscribed to the Google Groups “wxPython-users” group.

To unsubscribe from this group and stop receiving emails from it, send an email to wxpython-user...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

11

Memory seems to grow quite a bit faster with more text objects:

And 20 is not a lot–but it appears more or less linear with number of objects and number of iterations. I should test with more objects…

Still no solution, but some more info:

If I turn off the font cache clearing, it still happens. And I don’t see the problem with a regular Text object–without the scaling, no memory growth. So it doesn’t look like a wxFont issue.

Can you use a regular Text object? Sounds like it might be better for your app anyway.

Still no idea what the problem could be though… What makes a scaledText object different than any other? Just some computation and DC calls…

Mysterious!

CHB

···

On Sep 3, 2013, at 2:49 PM, Bryan Lott mystickphoenix@gmail.com wrote:

1 object -

0 - 30.292

5000 - 31.944

10000 - 33.492

15000 - 34.956

20000 - 36.428

20 objects (a good number for the amount on-screen in my real app at a given point) -

0 - 30.340

5000 - 60.476

10000 - 90.028

15000 - 119.856

20000 - 149.724

Sounds like you’re running into the same “memory is being used but I can’t figure out where” issue that I was running into. I was never able to confirm that anything was being kept around that shouldn’t (which was the frustrating part).

Is it possible that it’s the underlying c libraries that aren’t letting go of something? (No evidence to back this up, kinda grasping at straws at the moment…)

I’ve attached my test code for reference.

-Bryan

On Tuesday, September 3, 2013 1:13:32 PM UTC-6, Chris Barker - NOAA Federal wrote:

On Sep 3, 2013, at 7:11 AM, Bryan Lott mystick...@gmail.com wrote:

I see similar memory growth with a single object. Unfortunately, I’m drawing more than just a single object.

So you’re seeing memory growth proportional to how many text objects you have?

Do you think there’s a way the code could preserve the wxFont object instead of creating and deleting it?

Actually, it does do that, if I can believe my own comments. At least within the draw loop. So if you have a lot of text objects that are all the same, it should create only one wxFont object per draw loop.

It does clear the font cache for each draw, as I didn’t want to keep a whole pile of them around that might never get used again.

It may be that there’s a bit of something that the gc can’t collect from the wxFont.

I guess I could try turning off the cache clearing and see what happens.

The frequent redraw is to get animation of (labeled) nodes in a graph. I’m using a force based layout to draw a dynamic, interactive graph of connections between objects in a database.

If this is bending the FC/DC framework too far,

It shouldn’t be–that’s exactly the kind of thing it’s intended to do.

I’m still not sure the root of the problem, though. It does appear that raw use of DCs for text does make memory grow, but it is pretty darn small. But I don’t see a circular reference issue here, either, so not sure what’s going on.

By the way, in you investigations, did you confirm that it is extra wxFonts being kept around? Or are still not sure about that?

-Chris

I’ll poke at it some more.

do you have a recommendation for another framework? I’ve tried Kivy but ran into issues with its display of tabular data.

Thanks!

Bryan

On Monday, September 2, 2013 6:16:00 PM UTC-6, Chris Barker - NOAA Federal wrote:

OK,

I did some more testing and looking at the code, and can’t see where therewould be a circular reference here. The code does create and delete a wxFont on every iteration – perhaps that’s the issue?

Here’s how the memory grows for me with the enclosed sample:

With ScaledTextBox:

start 32.6MB

5000 iterations: 33.6

10000 iterations: 34.4

15000 itertaions: 35.1

And with drawing a polygon, but no text:

With Polygon:

start 32.5MB

5000 iterations: 32.6

10000 iterations: 32.6

15000 itertaions: 32.6

so that looks pretty solid.

So I tried doing a similar thing with just a DC – no floatcanvas code:

With DrawTextList

Start; 30.0

10,000 iterations: 30.5

20,000 iterations: 30.6

30,000 iterations: 30.6

50,000 iterations: 30.8

100,000 iterations: 31.2

So still some memory growth – but not as much.

And I use a raw DC with a Polygon and no Text:

With DrawPolygon:

Start: 29.8

20,000 iterations: 30.4

30,000 iterations: 30.4

50,000 iterations: 30.6

100,000 iterations: 31.1

pretty similar.

So I’m not sure at this point.

Bryan – does your memory growth look similar? If so, the it would be a LONG time before there was any real issue (not that it’s not a bug) Perhaps you could re-factor your code not to re-draw as frequently – why does it re-draw so often?

-Chris

–

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice

7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

–

You received this message because you are subscribed to the Google Groups “wxPython-users” group.

To unsubscribe from this group and stop receiving emails from it, send an email to wxpython-user...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

–

You received this message because you are subscribed to the Google Groups “wxPython-users” group.

To unsubscribe from this group and stop receiving emails from it, send an email to wxpython-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

<reproduce_canvas_memory_increase.py>

12

So, interesting bit of info… I swapped the ScaledTextBox to ScaledText and the memory leak went away… memory use stayed static or just bumped up a couple of bytes every now and again. The question now becomes what’s the difference between a ScaledText object and a ScaledTextBox?

ScaledText doesn’t do exactly what I want (multi-line text) but I should be able to shoehorn it in without too many issues.

Maybe this will give you what you need to find the bug/difference?

I’d use a regular text object but I do want the user to have zooming abilities as the DB relationship chains can get extremely long in our (convoluted as hell) model.

Bryan

···

On Tuesday, September 3, 2013 6:28:43 PM UTC-6, Chris Barker - NOAA Federal wrote:

On Sep 3, 2013, at 2:49 PM, Bryan Lott mystick...@gmail.com wrote:

Memory seems to grow quite a bit faster with more text objects:

And 20 is not a lot–but it appears more or less linear with number of objects and number of iterations. I should test with more objects…

Still no solution, but some more info:

If I turn off the font cache clearing, it still happens. And I don’t see the problem with a regular Text object–without the scaling, no memory growth. So it doesn’t look like a wxFont issue.

Can you use a regular Text object? Sounds like it might be better for your app anyway.

Still no idea what the problem could be though… What makes a scaledText object different than any other? Just some computation and DC calls…

Mysterious!

CHB

1 object -

0 - 30.292

5000 - 31.944

10000 - 33.492

15000 - 34.956

20000 - 36.428

20 objects (a good number for the amount on-screen in my real app at a given point) -

0 - 30.340

5000 - 60.476

10000 - 90.028

15000 - 119.856

20000 - 149.724

Sounds like you’re running into the same “memory is being used but I can’t figure out where” issue that I was running into. I was never able to confirm that anything was being kept around that shouldn’t (which was the frustrating part).

Is it possible that it’s the underlying c libraries that aren’t letting go of something? (No evidence to back this up, kinda grasping at straws at the moment…)

I’ve attached my test code for reference.

-Bryan

On Tuesday, September 3, 2013 1:13:32 PM UTC-6, Chris Barker - NOAA Federal wrote:

On Sep 3, 2013, at 7:11 AM, Bryan Lott mystick...@gmail.com wrote:

I see similar memory growth with a single object. Unfortunately, I’m drawing more than just a single object.

So you’re seeing memory growth proportional to how many text objects you have?

Do you think there’s a way the code could preserve the wxFont object instead of creating and deleting it?

Actually, it does do that, if I can believe my own comments. At least within the draw loop. So if you have a lot of text objects that are all the same, it should create only one wxFont object per draw loop.

It does clear the font cache for each draw, as I didn’t want to keep a whole pile of them around that might never get used again.

It may be that there’s a bit of something that the gc can’t collect from the wxFont.

I guess I could try turning off the cache clearing and see what happens.

The frequent redraw is to get animation of (labeled) nodes in a graph. I’m using a force based layout to draw a dynamic, interactive graph of connections between objects in a database.

If this is bending the FC/DC framework too far,

It shouldn’t be–that’s exactly the kind of thing it’s intended to do.

I’m still not sure the root of the problem, though. It does appear that raw use of DCs for text does make memory grow, but it is pretty darn small. But I don’t see a circular reference issue here, either, so not sure what’s going on.

By the way, in you investigations, did you confirm that it is extra wxFonts being kept around? Or are still not sure about that?

-Chris

I’ll poke at it some more.

do you have a recommendation for another framework? I’ve tried Kivy but ran into issues with its display of tabular data.

Thanks!

Bryan

On Monday, September 2, 2013 6:16:00 PM UTC-6, Chris Barker - NOAA Federal wrote:

OK,

I did some more testing and looking at the code, and can’t see where therewould be a circular reference here. The code does create and delete a wxFont on every iteration – perhaps that’s the issue?

Here’s how the memory grows for me with the enclosed sample:

With ScaledTextBox:

start 32.6MB

5000 iterations: 33.6

10000 iterations: 34.4

15000 itertaions: 35.1

And with drawing a polygon, but no text:

With Polygon:

start 32.5MB

5000 iterations: 32.6

10000 iterations: 32.6

15000 itertaions: 32.6

so that looks pretty solid.

So I tried doing a similar thing with just a DC – no floatcanvas code:

With DrawTextList

Start; 30.0

10,000 iterations: 30.5

20,000 iterations: 30.6

30,000 iterations: 30.6

50,000 iterations: 30.8

100,000 iterations: 31.2

So still some memory growth – but not as much.

And I use a raw DC with a Polygon and no Text:

With DrawPolygon:

Start: 29.8

20,000 iterations: 30.4

30,000 iterations: 30.4

50,000 iterations: 30.6

100,000 iterations: 31.1

pretty similar.

So I’m not sure at this point.

Bryan – does your memory growth look similar? If so, the it would be a LONG time before there was any real issue (not that it’s not a bug) Perhaps you could re-factor your code not to re-draw as frequently – why does it re-draw so often?

-Chris

–

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice

7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

–

You received this message because you are subscribed to the Google Groups “wxPython-users” group.

To unsubscribe from this group and stop receiving emails from it, send an email to wxpython-user...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

–

You received this message because you are subscribed to the Google Groups “wxPython-users” group.

To unsubscribe from this group and stop receiving emails from it, send an email to wxpython-user...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

<reproduce_canvas_memory_increase.py>

13

So, interesting bit of info... I swapped the ScaledTextBox to ScaledText
and the memory leak went away...

It turns out I jsut did the same thing -- with the same results -- so far
382,000 iterations, with 100 objects, memory up a bit, but pretty stable.
Glad to hear you got the same results.

memory use stayed static or just bumped up a couple of bytes every now and

again. The question now becomes what's the difference between a ScaledText
object and a ScaledTextBox?

well, as you say, multi-line text -- and it pre-computes the relative
position of all the words, so there's a bit more going on there, though
still not sure what. But I'll poke at ti a bit more...

ScaledText doesn't do exactly what I want (multi-line text) but I should
be able to shoehorn it in without too many issues.

Probably the best option for now.

I'd use a regular text object but I do want the user to have zooming
abilities as the DB relationship chains can get extremely long in our
(convoluted as hell) model.

sure, but the reason regular text is there (and was there first) is because
you can only read text so small, and if it gets large, you're more likely
to have text overlapping. But I dont really know your use-case...

Thanks for the update -- we're narrowing it down...
-Chris

···

On Wed, Sep 4, 2013 at 7:43 AM, Bryan Lott <mystickphoenix@gmail.com> wrote:

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

14

BINGO!

Found it – the leak is in DC.DrawTextList() – if I repalce that call with a loop:

(around line 1859 of FloatCanvas.py)

NOTE: DrawTextList seems to have a memory leak

– calling it thousands of times causes a problem

If the leak gets found and fixed, it may be worth putting the call back

#dc.DrawTextList(self.Words, Points)

for word, point in zip(self.Words, Points):

dc.DrawTextPoint(word, point)

The leak seems to have gone away.

so far, 20,000 iterations with 100 objects, and still pretty steady memory usage.

The weird particularly frustrating part is that I thought I’d tested that with a raw DC, and didn’t see a problem. Oh well.

So I suggest you patch your copy of FloatCanvas with the above, and I’ll submit an update to SVN, and also look more into what may be wrong with DrawTextList (I did write that code originally…and yes, it’s C++…)

Thanks for finding and reporting and testing this bug!

-Chris

···

On Wed, Sep 4, 2013 at 11:40 AM, Chris Barker - NOAA Federal chris.barker@noaa.gov wrote:

On Wed, Sep 4, 2013 at 7:43 AM, Bryan Lott mystickphoenix@gmail.com wrote:

–

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

So, interesting bit of info… I swapped the ScaledTextBox to ScaledText and the memory leak went away…

It turns out I jsut did the same thing – with the same results – so far 382,000 iterations, with 100 objects, memory up a bit, but pretty stable. Glad to hear you got the same results.

memory use stayed static or just bumped up a couple of bytes every now and again. The question now becomes what’s the difference between a ScaledText object and a ScaledTextBox?

well, as you say, multi-line text – and it pre-computes the relative position of all the words, so there’s a bit more going on there, though still not sure what. But I’ll poke at ti a bit more…

ScaledText doesn’t do exactly what I want (multi-line text) but I should be able to shoehorn it in without too many issues.

Probably the best option for now.

I’d use a regular text object but I do want the user to have zooming abilities as the DB relationship chains can get extremely long in our (convoluted as hell) model.

sure, but the reason regular text is there (and was there first) is because you can only read text so small, and if it gets large, you’re more likely to have text overlapping. But I dont really know your use-case…

Thanks for the update – we’re narrowing it down…

-Chris

–

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division

NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

15

You sir? Are a ROCKSTAR!

That fixed it both in my test app and in the real app.

Thank you!!!

Bryan

···

On Wednesday, September 4, 2013 1:08:42 PM UTC-6, Chris Barker - NOAA Federal wrote:

BINGO!

Found it – the leak is in DC.DrawTextList() – if I repalce that call with a loop:

(around line 1859 of FloatCanvas.py)

NOTE: DrawTextList seems to have a memory leak

– calling it thousands of times causes a problem

If the leak gets found and fixed, it may be worth putting the call back

#dc.DrawTextList(self.Words, Points)

for word, point in zip(self.Words, Points):

dc.DrawTextPoint(word, point)

The leak seems to have gone away.

so far, 20,000 iterations with 100 objects, and still pretty steady memory usage.

The weird particularly frustrating part is that I thought I’d tested that with a raw DC, and didn’t see a problem. Oh well.

So I suggest you patch your copy of FloatCanvas with the above, and I’ll submit an update to SVN, and also look more into what may be wrong with DrawTextList (I did write that code originally…and yes, it’s C++…)

Thanks for finding and reporting and testing this bug!

-Chris

On Wed, Sep 4, 2013 at 11:40 AM, Chris Barker - NOAA Federal chris....@noaa.gov wrote:

On Wed, Sep 4, 2013 at 7:43 AM, Bryan Lott mystick...@gmail.com wrote:

So, interesting bit of info… I swapped the ScaledTextBox to ScaledText and the memory leak went away…

It turns out I jsut did the same thing – with the same results – so far 382,000 iterations, with 100 objects, memory up a bit, but pretty stable. Glad to hear you got the same results.

memory use stayed static or just bumped up a couple of bytes every now and again. The question now becomes what’s the difference between a ScaledText object and a ScaledTextBox?

well, as you say, multi-line text – and it pre-computes the relative position of all the words, so there’s a bit more going on there, though still not sure what. But I’ll poke at ti a bit more…

ScaledText doesn’t do exactly what I want (multi-line text) but I should be able to shoehorn it in without too many issues.

Probably the best option for now.

I’d use a regular text object but I do want the user to have zooming abilities as the DB relationship chains can get extremely long in our (convoluted as hell) model.

sure, but the reason regular text is there (and was there first) is because you can only read text so small, and if it gets large, you’re more likely to have text overlapping. But I dont really know your use-case…

Thanks for the update – we’re narrowing it down…

-Chris

–

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division

NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

–

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

16

You sir? Are a ROCKSTAR!

Well, thanks, though it may have been my bug in the first place....

I did some more testing, it turns out that it's not DrawTextList per se,
but DrawTextList when passed a numpy array, so this also works:

dc.DrawTextList(self.Words, Points.tolist())

i.e. let numpy convert to a list, and let wxPython work with a list. That's
the fix I'll commit to SVN. It should be a bit faster for large blocks of
text.

By the way, do send us some screen shots, or a link to your app when it's
on good shape -- always nice to see how stuff is used.

-Chris

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

···

On Wed, Sep 4, 2013 at 12:34 PM, Bryan Lott <mystickphoenix@gmail.com>wrote:

17

Chris Barker - NOAA Federal wrote:

fix-drawlist-mem-leak.patch (585 Bytes)

···

On Wed, Sep 4, 2013 at 12:34 PM, Bryan Lott <mystickphoenix@gmail.com > <mailto:mystickphoenix@gmail.com>> wrote:

    You sir? Are a ROCKSTAR!

Well, thanks, though it may have been my bug in the first place....

I did some more testing, it turns out that it's not DrawTextList per se,
but DrawTextList when passed a numpy array, so this also works:

dc.DrawTextList(self.Words, Points.tolist())

i.e. let numpy convert to a list, and let wxPython work with a list.
That's the fix I'll commit to SVN. It should be a bit faster for large
blocks of text.

By the way, do send us some screen shots, or a link to your app when
it's on good shape -- always nice to see how stuff is used.

Thanks for narrowing it down. I think the attached patch will fix the problem. If you're set up to do a build then please test this for me and let me know.

--
Robin Dunn
Software Craftsman

18

Thanks for narrowing it down. I think the attached patch will fix the
problem.

whoa! before I even got a chance to send a note to wxpython-dev about
this... and yes, that does look right.

If you're set up to do a build then please test this for me and let me
know.

well, I'm not set up at this point -- maybe I'll get set up, but not sure

when. In the meantime, here is a script that triggers the problem if you
want to try it out.

Thanks,
   -Chris

DC_memory_check.py (1.37 KB)

···

On Wed, Sep 4, 2013 at 1:38 PM, Robin Dunn <robin@alldunn.com> wrote:

--
Robin Dunn
Software Craftsman
http://wxPython.org

--
You received this message because you are subscribed to the Google Groups
"wxPython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to wxpython-users+unsubscribe@**googlegroups.com<wxpython-users%2Bunsubscribe@googlegroups.com>
.
For more options, visit https://groups.google.com/**groups/opt_out&lt;https://groups.google.com/groups/opt_out&gt;
.

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

19

Chris Barker - NOAA Federal wrote:

···

On Wed, Sep 4, 2013 at 1:38 PM, Robin Dunn <robin@alldunn.com > <mailto:robin@alldunn.com>> wrote:

    Thanks for narrowing it down. I think the attached patch will fix
    the problem.

whoa! before I even got a chance to send a note to wxpython-dev about
this... and yes, that does look right.

      If you're set up to do a build then please test this for me and
    let me know.

well, I'm not set up at this point -- maybe I'll get set up, but not
sure when. In the meantime, here is a script that triggers the problem
if you want to try it out.

Actually I didn't look far enough. There is a bit of code down below the DrawText that will take care of that decref. (I'll refactor that a bit so I don't make that mistake again.) Not sure when I'll be able to look at this further so if you want to try to diagnose it please do.

--
Robin Dunn
Software Craftsman