David Woods wrote:
Hi Robin,
The line of code that fails is:
# Load the image
self.bgImage = wx.Image(self.obj.image_filename)
Application transferred too few scanlines
(I have no idea where this message comes from) and self.bgImage.IsOk()
returns false.
It looks like it is coming from libjpeg. See
wxWidgets/src/jpeg/jerror.h line #116. It's used in the functions
jpeg_finish_compress and jpeg_finish_decompress, with code like this:
if (cinfo->next_scanline< cinfo->image_height)
ERREXIT(cinfo, JERR_TOO_LITTLE_DATA);
My guess is that it's probably a memory or resource issue, and that the
actual problem is happening some time before the lines of code above,
but that it is manifesting as not enough image data being sent to those
functions so that is where it is caught.
Thanks for all of your help. I've finally resolved the issue.
It turns out that my wx.Image call was the problem. I wasn't Destroying
it properly. Since it was defined in the context of a wx.Frame, I
assumed it would be destroyed when the frame was destroyed, but
Was it holding a reference to the image, something like "self.img =
wx.Image(...)" or was it just used as a local variable like "img =
wx.Image(...)" ?
It was (and is) a self.img reference, as I reuse the image in a couple
of methods in the object.
Ok.
For context, this part of the application is for doing analytic mark-up
of still images using a FloatCanvas. If the user chooses to clear or
temporarily hide some of their mark-up, I need to redraw, and I reuse
self.img during that process.
For the reports, I open the markup window without showing it, load the
image from the file and the mark-up from the database, extract the
marked-up image to put into the report, and close the markup window
without ever showing it.
In terms of avoiding memory leaks, am I better off retaining the
self.img reference, or am I better off just remembering the file name
and creating a new local img when I need one within the markup window?
No it's best to keep the image object, loading it from disk and reconverting that data to a wx.Image each time you need it is overhead that is easily avoidable. The issue here is mainly making sure that everything will get released when you don't need it any more.
apparently that isn't the way it works. It wasn't a child of the frame,
and the image object was persisting after the frame was destroyed. As I
built my report, 25 or 30 images would build up in memory until there
was no room for additional images. It's not clear to me why one image
failing caused later images to be able to work, but that's what was
happening. I guess something in wx.Python/wxWidgets was freeing up the
image memory at that point that my attempts at explicit garbage
collection weren't freeing up.
The solution was to add an explicit Image.Destroy() call in the
EVT_CLOSE handler for the Frame object. No more crashes, even with
significantly more images.
IIUC the garbage collector is not able to do anything with objects
that are implemented as extension module types (unless they implement
the proper protocols) or with classes that have a __del__ method. The
wrapped wx classes in Classic fall into the latter category, and their
.this attribute (a SwigObject) falls into the first category. So
object cleanup of wxPython objects relies solely on old-fashioned
reference counting only.
In the case of wx.Image there is no other object that the ownership of
the C++ object will be transferred to, like is done with widgets that
have a parent window, etc. so the C++ image object should always be
owned by the Python proxy object and will be destroyed when the
proxy's refcount reaches zero. Calling Destroy will force the C++
object to be deleted and will tell the proxy that it no longer owns
the C++ object, but the proxy will continue to exist until it's
refcount reaches zero and if any of those references tries to use the
image then there will be an exception or a crash.
So it sounds to me like perhaps there are some extra references to the
image object, or perhaps a reference cycle. Since the GC can't wipe
those out you may want to figure out where they are and break the
cycles yourself. Using Destroy will release the bulk of the memory
needed for each image object, but if the proxies are hanging around
then you still have a leak.
>>> img = wx.Image('/Users/robind/Desktop/Snap015.png')
>>> sys.getrefcount(img)
2
>>> img.Destroy()
>>> sys.getrefcount(img)
2
This shows that Destroy does not reduce the refcount of the proxy
object (so the C++ object is not holding a reference to its proxy like
we do with widget objects and a few others.) The count of 2 is
expected in this case, one for the img variable and one for passing it
as a parameter to getrefcount.
Okay.... I mostly understand what you're saying, I think. Maybe. But I
haven't got a clue how to DO this "breaking the cycles yourself" of
which you speak.
A cycle in this case means a set of objects that have references to each other such that you can start at one object and follow some path of references from object to object and eventually get back to the original object. For example, object A has a reference to object B, which has a reference to object C, which has a reference to object A. This is the type of things that Python's garbage collector was designed to take care of, but as I mentioned before it's not able to deal with objects that have a __del__ method, or those that are extension module types that don't have the necessary protocols implemented. It's not just the A, B, and C objects that can get stuck in memory, but anything else that they have references to. For example, if B has a reference to your image then it will be stuck in memory too until that reference is released.
"Breaking the cycle yourself" means providing some way to disconnect one of the references that is causing the reference graph to have a cycle in it. For example if object A has a method like this:
def forgetB(self):
self.B = None
Then if that enables B's reference count to drop to zero then it will be destroyed and the references it had to the image and to C to be decremented. If their reference count also drops to zero then they will be destroyed, and so on.
It's also possible to assist the garbage collector after it has detected unreachable and uncollectable objects by examining the contents of the gc.garbage list and dealing with your objects that are there. See the docs for the gc module for more info.
Basically, I open the SnapshotWindow (a wx.Frame that houses some
toolbars and a FloatCanvas object named canvas, and which loads an image
from a file) without showing it, draw the markup (using _DrawObjects),
and copy the image to tmpBMP, then I close and destroy the Snapshot Window.
Here's the critical code:
# Open a HIDDEN Snapshot Window
tmpSnapshotWindow = SnapshotWindow.SnapshotWindow(menuWindow, -1,
tmpObj.id, tmpObj, showWindow=False)
# Reset the Bounding Box to avoid NaN problems
tmpSnapshotWindow.canvas._ResetBoundingBox()
# Get the image's Bounding Box
box = tmpSnapshotWindow.canvas.BoundingBox
# Create an empty Bitmap the size of the image
tmpBMP = wx.EmptyBitmap(tmpSnapshotWindow.canvas.GetSize()[0],
tmpSnapshotWindow.canvas.GetSize()[1])
# Get the MemoryDC for the empty bitmap
tempDC = wx.MemoryDC(tmpBMP)
# Set the bitmap's background colour to WHITE
tempDC.SetBackground(wx.Brush("white"))
tempDC.Clear()
# Create a ClientDC for the hidden Shapshot Window
tempDC2 = wx.ClientDC(tmpSnapshotWindow)
# Create another MemoryDC (although I'm not sure why!)
tempDC3 = wx.MemoryDC()
# Get the image from the hidden Snapshot Window's Canvas and
# put it in the Device Contexts we just created
tmpSnapshotWindow.canvas._DrawObjects(tempDC,
tmpSnapshotWindow.canvas._DrawList,
tempDC2, box, tempDC3)
# Close the hidden Snapshot Window
tmpSnapshotWindow.Close()
# Explicitly Delete the Temporary Snapshot Window
tmpSnapshotWindow.Destroy()
To the extent that I understand this, I think I've made a copy of image
drawn on the SnapshotWindow's canvas and don't think I retain any
reference to that canvas or to the wx.Image object used inside the
SnapshotWindow object to populate the FloatCanvas object. But to be
honest, I'm a bit fuzzy about device contexts and exactly how the image
I need ends up in tmpBMP here.
The Blit()'s being done in the _DrawObjects method are essentially just moving pixel data from one DC to the other. So yes, at that point you are done with the image object and canvas and etc. However, just because you don't have a reference to those object from the code above (other than the local variables that will go away when this code returns to its caller) that doesn't mean that there isn't a cycle involving your SnapShot window class, or perhaps something in floatcanvas.
I think I would start by trying to reorganize and simplify the above. For example it seems to me that getting a bitmap from the snapshot window would be the job of the snapshot window and not code that is external to it. So I would give it a method that could create and return the bitmap. Secondly, creating a window that will never be shown just to get it to create a bitmap smells really really bad. So I would probably create a new class that can create and manage the collection of FC draw objects, and use that class both from this code and also from SnapshotWindow
I have no idea why 3 device contexts seem
to come into play, for example.
It looks like the 3rd DC is used to maintain the hit testing bitmap inside of the floatcanvas code, so since you are never showing this window and never interacting with it then I expect that you can just let it default to None.
So if Destroy() isn't going to do what I need here, what do I do? Do I
need to explicitly call del(tmpBMP) or del(tmpSnapshotWindow)?
No, local variables are always disposed of at the end of the scope and that will decrement the reference count of the objects they were referring to.
Is there
something more I need to do?
A. Look for object references that are saved in some other object, and see if there is some other path back to the original object.
B. Some time after the code in question has run (so you expect the objects to all have been destroyed naturally by that time) run gc.collect() and then look at gc.garbage to see if there are any objects there that you think should have been disposed of but were not.
···
At the moment, I get a console message that says:
--
Robin Dunn
Software Craftsman
