Caveats for implementing your own filesystem handlers

For BitPim I have implemented my own filesystem handler. They are
used to supply images for various displays which are rendered using
wxHTML. This lets the html look like <img src="bpimage:watermark.jpg">
and the watermark.jpg will be found in the program resources directory.
I also use it to lookup images users have downloaded from their phone
such as <img src="bpuserimage:mycat.bmp">. Finally I include parameters
as part of the name to do scaling (preserving the aspect ratio). For
thumbnails that last piece may be
<img src="bpuserimagemycat.bmp;width=32;height=32">

Since handlers get invoked every single time there is one of
these tags, things can get slow, so I implemented a cache which
is when all hell broke loose.

If you return a wx.FSFile object you have returned before then
wxPython will crash/coredump. As far as I can tell there are
reference counting errors for wx.InputStream (passed as constructor
to wx.FSFile) and for wx.FSFile itself.

This can be worked around by recreating the wx.FSFile every
time, and you also need to recreate the wx.InputStream parameter
each time as well.

Lastly you need to ensure that the actual file object passed
to the wx.InputStream constructor is seeked back to the
begining each time otherwise you get non-fatal dialogs
popping up telling you that there are no handlers for the
image format.

Roger

Roger Binns wrote:

Since handlers get invoked every single time there is one of
these tags, things can get slow, so I implemented a cache which
is when all hell broke loose.

If you return a wx.FSFile object you have returned before then
wxPython will crash/coredump. As far as I can tell there are
reference counting errors for wx.InputStream (passed as constructor
to wx.FSFile) and for wx.FSFile itself.

Yes I struggled with this when doing the wrappers. On the C++ side the wxFSFile is assumed to be deleted by the caller of the wxFileSystem::OpenFile method. It doesn't manage its own life-cycle so the wxPython wrapper for it can't either. Consequently you must always return a new instance of wx.FSFile from OpenFile and not touch it again after you have done so.

The tricky part was making it so when you create a wx.FSFile and return it that the C++ object would not be destroyed when the proxy is garbage collected, but if you call OpenFile yourself from Python then it is destroyed when you are done with the proxy. I'm fairly sure that this is working correctly in the current version.

I'm less sure about reference counting of the wx.InputStream objects, as it's been a while since I've worked much on those classes. If you have a simple example that shows the leak then it will help me take a closer look.

This can be worked around by recreating the wx.FSFile every
time, and you also need to recreate the wx.InputStream parameter
each time as well.

Lastly you need to ensure that the actual file object passed
to the wx.InputStream constructor is seeked back to the
begining each time otherwise you get non-fatal dialogs
popping up telling you that there are no handlers for the
image format.

Yep. No assumptions are made about the Python file-like object given to the wx.InputStream, so if you are reusing them this is expected. The HtmlWindow will pass the stream as-is to the wx.Image which will try to find an wx.ImageHandler that recognizes the first few bytes on the stream.

BTW, I don't know if it fully meets you needs, but using the wx.MemoryFSHandler for preloading your images might work for you. You can put a wx.Bitmap, wx.Image or just raw data into it using the wx.MemoryFSHandler.AddFile static method (docstring below) and access it from the html using the "memory:" protocol in the uri. You would lose the on the fly resize-info-embedded-in-the-name functionality that you are using currently but the <img> tag's width and height attributes should also do it for you.

def MemoryFSHandler_AddFile(filename, dataItem, imgType=-1):
     """
     Add 'file' to the memory filesystem. The dataItem parameter can
     either be a `wx.Bitmap`, `wx.Image` or a string that can contain
     arbitrary data. If a bitmap or image is used then the imgType
     parameter should specify what kind of image file it should be
     written as, wx.BITMAP_TYPE_PNG, etc.
     """

...

AddFile = staticmethod(MemoryFSHandler_AddFile)

ยทยทยท

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

I'm less sure about reference counting of the wx.InputStream objects, as it's been a while since I've worked much on those
classes. If you have a simple example that shows the leak then it will help me take a closer look.

It wasn't a leak but a crash. I reported some bugs in the past which
I think you said you fixed post 2.5.3.1. I don't have any simple examples.

BTW, I don't know if it fully meets you needs, but using the wx.MemoryFSHandler for preloading your images might work for you.

It may have in the past, but I am doing a new widget that is
far more dynamic. The number of images multiplied by the various
sizes and configurations they would appear in would result in
huge amounts of memory being consumed. Currently I have a
cache (which is how I found the reuse issues :slight_smile: The cache
is up to 100 items, and roughly only needs to cover what is
visible onscreen.

Roger