using wxImage in C++ python extension

wxPython Users
1

I've writen (and built) a wee C++ extension that allows me to load and
manipulate some custom image types from Python.

To make the class offered by my extension more useful in a wxPython image
display app that I use, I'd now like to add a method that returns a reference to
a instance of a wxImage containing a representation of the member image data.

My problem lies in determining the simplest/most elegant way to add the ability
to use a wxImage type in my extension.

I know that I require a header file to include in my C++ (to define wxImage) and
a library file (that includes a wxImage constructor) to link against. My
initial investigations pointed to use of the wxWidgets wx/image.h header. Alas
my attempts to use this seem to indicate that I'm going to need to make
wxWidgets from source on my machine. I'm doing this in the background - but it
seems like overkill.

Am I barking up the wrong tree? Is there header supplied in the wxPython source
distribution I can use and a pre-compiled library in the wxPython binary
distribution that I can link to instead?

Or do compiler/name-mangling difference mean that I'm really better to build
everything myself?

thanks in advance for any tips...

Andrew :wink:

ps. I'm using MING which I know may complicate things as wxPython seems to be
build using MSVC

2

Andrew Murray wrote:

I've writen (and built) a wee C++ extension that allows me to load and
manipulate some custom image types from Python.

To make the class offered by my extension more useful in a wxPython image
display app that I use, I'd now like to add a method that returns a reference to
a instance of a wxImage containing a representation of the member image data.

My problem lies in determining the simplest/most elegant way to add the ability
to use a wxImage type in my extension.

I know that I require a header file to include in my C++ (to define wxImage) and
a library file (that includes a wxImage constructor) to link against. My
initial investigations pointed to use of the wxWidgets wx/image.h header. Alas
my attempts to use this seem to indicate that I'm going to need to make wxWidgets from source on my machine. I'm doing this in the background - but it
seems like overkill.

Am I barking up the wrong tree? Is there header supplied in the wxPython source
distribution I can use and a pre-compiled library in the wxPython binary
distribution that I can link to instead?

Or do compiler/name-mangling difference mean that I'm really better to build
everything myself?

thanks in advance for any tips...

Andrew :wink:

ps. I'm using MING which I know may complicate things as wxPython seems to be
build using MSVC

Correct. Mingw32 uses a different name-mangling scheme than MSVC, so yes, you'll have to build everything yourself if you want to use wxImage from C++.

OTOH, if you don't mind an extra data copy then your extension could just provide the image data as a string or RGB bytes and then you can create the wx.Image from Python using that string and the wx.ImageFromData constructor.

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

3

Andrew Murray wrote:

To make the class offered by my extension more useful in a wxPython image
display app that I use, I'd now like to add a method that returns a reference to
a instance of a wxImage containing a representation of the member image data.

Perhaps the easiest way to do this is to have your code create a Buffer Object. Then, in Python code, you can call wx.Image.SetData.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

4

Robin Dunn wrote:

OTOH, if you don't mind an extra data copy then your extension could just provide the image data as a string or RGB bytes and then you can create the wx.Image from Python using that string and the wx.ImageFromData constructor.

Why does that have to create a copy? It looks like the C++ wx.ImageSetData() method is expecting a pointer to the data. Couldn't the wxPython version pass the pointer through from the buffer object? Or better yet, from a nd-array (from the numpy array protocol).

I partly ask this because there is a thread on the matplotlib list about how to make the wx back-end pure Python, rather than having them depend on the right wx-devel stuff being installed, and after building, being as version specific as they are. Being able to pass data into a wxImage (or better yet, a wxBitmap) without copying would be a great help.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

5

Christopher Barker wrote:

Robin Dunn wrote:

OTOH, if you don't mind an extra data copy then your extension could just provide the image data as a string or RGB bytes and then you can create the wx.Image from Python using that string and the wx.ImageFromData constructor.

Why does that have to create a copy?

Actually, I was thinking of the copy made when making the PyString. I wasn't thinking of making a buffer object, that would definitely be the way to go.

It looks like the C++ wx.ImageSetData() method is expecting a pointer to the data. Couldn't the wxPython version pass the pointer through from the buffer object? Or better yet, from a nd-array (from the numpy array protocol).

I partly ask this because there is a thread on the matplotlib list about how to make the wx back-end pure Python, rather than having them depend on the right wx-devel stuff being installed, and after building, being as version specific as they are. Being able to pass data into a wxImage (or better yet, a wxBitmap) without copying would be a great help.

wx.Image.SetData makes a copy and gives ownership of the copy to the image. This allows it to work even when the source buffer doesn't live as long as the image does. There is also the wx.Image.SetDataBuffer method, which will have the wxImage use the buffer passed in without making a copy, but if the buffer doesn't live as long as the image does it will likely cause a crash.

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

6

Robin Dunn wrote:

There is also the wx.Image.SetDataBuffer method, which will have the wxImage use the buffer passed in without making a copy, but if the buffer doesn't live as long as the image does it will likely cause a crash.

Could the reference count of the Buffer object be Py_INCREF() when SetDataBuffer() is called, then Py_DECREF() when the wx.Image is destroyed?

-CHB

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

7

Robin Dunn wrote:

There is also the wx.Image.SetDataBuffer method, which will have the wxImage use the buffer passed in without making a copy, but if the buffer doesn't live as long as the image does it will likely cause a crash.

Another thought -- could I subclass wx.Image with one that holds a reference to the Buffer that was passed in? Then they'd get destroyed (or DECREF'd, anyway) together.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

8

Depending on the implementation of the object that the buffer points to,
that may not matter.

import array
a = array.array('B', 'a')
b = buffer(a)
ia = a.buffer_info()[0]
while ia == a.buffer_info()[0]:

... a.extend(a)
...

c = buffer(a)
b

<read-only buffer for 0x0077A158, ptr 0x007B1DD8, size 1 at 0x007DAAC0>

c

<read-only buffer for 0x0077A158, ptr 0x008135C0, size 32 at 0x007DAD40>

len(a)

32

···

Christopher Barker <Chris.Barker@noaa.gov> wrote:

Robin Dunn wrote:
> There is also the wx.Image.SetDataBuffer
> method, which will have the wxImage use the buffer passed in without
> making a copy, but if the buffer doesn't live as long as the image does
> it will likely cause a crash.

Could the reference count of the Buffer object be Py_INCREF() when
SetDataBuffer() is called, then Py_DECREF() when the wx.Image is destroyed?

-CHB

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

---------------------------------------------------------------------
To unsubscribe, e-mail: wxPython-users-unsubscribe@lists.wxwidgets.org
For additional commands, e-mail: wxPython-users-help@lists.wxwidgets.org

9

Robin,

Sorry to keep peppering you with little messages about this, but...

A couple months ago, you mentioned that you were trying to see if you could expose some of the methods for manipulating wx.Bitmap data directly. This would be a great application of that -- if we could just pass a pointer from the Agg buffer straight into a wxBitmap, then there would be no overhead at all. Of course, that would only work if the data format was right, but if we're lucky, that might be the common use case.

Also, it would be great if that could also be accomplished through the numpy array interface. Travis recently donated a patch to PIL that does this:

http://mail.python.org/pipermail/image-sig/2006-July/003999.html

I imagine the code for wxBitmap and wxImage would look similar.

I'd like to get the array interface to be accepted by PointListHelper and friends too. I have it in mind to write that code, but who knows when I can get to it.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

10

I should probably clarify a bit. The pointer that buffer b holds points
to memory that is no longer associated with any array object. It may or
may not have been freed, which may or may not induce a segfault on
access.

- Josiah

···

Josiah Carlson <jcarlson@uci.edu> wrote:

Depending on the implementation of the object that the buffer points to,
that may not matter.

>>> import array
>>> a = array.array('B', 'a')
>>> b = buffer(a)
>>> ia = a.buffer_info()[0]
>>> while ia == a.buffer_info()[0]:
... a.extend(a)
...
>>> c = buffer(a)
>>> b
<read-only buffer for 0x0077A158, ptr 0x007B1DD8, size 1 at 0x007DAAC0>
>>> c
<read-only buffer for 0x0077A158, ptr 0x008135C0, size 32 at 0x007DAD40>
>>> len(a)
32
>>>

Christopher Barker <Chris.Barker@noaa.gov> wrote:
> Robin Dunn wrote:
> > There is also the wx.Image.SetDataBuffer
> > method, which will have the wxImage use the buffer passed in without
> > making a copy, but if the buffer doesn't live as long as the image does
> > it will likely cause a crash.
>
> Could the reference count of the Buffer object be Py_INCREF() when
> SetDataBuffer() is called, then Py_DECREF() when the wx.Image is destroyed?
>
> -CHB
>
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> NOAA/OR&R/HAZMAT (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
>
> Chris.Barker@noaa.gov
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: wxPython-users-unsubscribe@lists.wxwidgets.org
> For additional commands, e-mail: wxPython-users-help@lists.wxwidgets.org

---------------------------------------------------------------------
To unsubscribe, e-mail: wxPython-users-unsubscribe@lists.wxwidgets.org
For additional commands, e-mail: wxPython-users-help@lists.wxwidgets.org

11

Josiah Carlson wrote:

I should probably clarify a bit.

Thanks. I really hadn't followed that.

The pointer that buffer b holds points
to memory that is no longer associated with any array object. It may or
may not have been freed, which may or may not induce a segfault on
access.

I understand that better now.

However, if if there is still a reference to the buffer object, is that memory still good? It looks like it is:

>>> b[0]
'a'
>>> b[5]
'a'

That makes it look to me like the buffer object is responsible for that memory, and if it's reference count > 0, then is will keep it around.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

12

Josiah Carlson wrote:
> I should probably clarify a bit.

Thanks. I really hadn't followed that.

> The pointer that buffer b holds points
> to memory that is no longer associated with any array object. It may or
> may not have been freed, which may or may not induce a segfault on
> access.

I understand that better now.

However, if if there is still a reference to the buffer object, is that
memory still good? It looks like it is:

>>> b[0]
'a'
>>> b[5]
'a'

That makes it look to me like the buffer object is responsible for that
memory, and if it's reference count > 0, then is will keep it around.

No. The current implementation of buffer keeps a refcounted pointer to
the object it has a buffer of, as well as a second pointer to the data
(acquired via PyObject_AsBuffer() ). The data pointer within the object
pointed to by buffer changes, and no buffers pointed to the object are
updated (to update them would violate various assumptions with how
buffers work). This is a long-known issue, and is a major problem with
how Python buffers currently work. The only solution to this requires
having a realloc that warns you if it is going to allocate a new block,
or strictly over-allocating initially, only ever using malloc, and
creating a new data buffer object that holds memory used by array, etc.

The fact that the memory still has what you expect *now*, doesn't mean
it will in the future, nor does it mean that you will always be able to
access the memory without segfault. Here's a sample that doesn't
segfault, but whose memory is not the same:

import array
a = array.array('B', 1024*'a')
b = buffer(a)
ia = a.buffer_info()[0]
while ia == a.buffer_info()[0]:

... a.extend(a)
...

len(a)

2048

b[:10]

' p~\x00@p~\x00`p'

- Josiah

···

Christopher Barker <Chris.Barker@noaa.gov> wrote:

13

Josiah Carlson wrote:

No. The current implementation of buffer keeps a refcounted pointer to
the object it has a buffer of, as well as a second pointer to the data
(acquired via PyObject_AsBuffer() ). The data pointer within the object
pointed to by buffer changes, and no buffers pointed to the object are
updated (to update them would violate various assumptions with how
buffers work). This is a long-known issue, and is a major problem with
how Python buffers currently work. The only solution to this requires
having a realloc that warns you if it is going to allocate a new block,
or strictly over-allocating initially, only ever using malloc, and
creating a new data buffer object that holds memory used by array, etc.

The fact that the memory still has what you expect *now*, doesn't mean
it will in the future, nor does it mean that you will always be able to
access the memory without segfault.

And just to tie this explanation back to the original question: When the buffer is given to the wx.Image then it pulls out the pointer and gives it to the wx.Image. At that point all the wx.Image knows about is that pointer so if the original Python object moves its buffer to another location in memory then the pointer that the wx.Image currently has will be invalid and it will not be aware that it moved.

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

14

Ultimately, if a user wants to use a buffer without copying, one should
be sure that the buffer cannot be moved in memory by design (string) or
by convention. Note that Python strings can have their contents (not
size) mutated by C code, and will work just fine, as long as one doesn't
explicitly or implicitly use hash() before and after such mutations
(don't let them be keys in a dictionary). But realize that such
behavior is unsupported.

- Josiah

···

Robin Dunn <robin@alldunn.com> wrote:

Josiah Carlson wrote:

> No. The current implementation of buffer keeps a refcounted pointer to
> the object it has a buffer of, as well as a second pointer to the data
> (acquired via PyObject_AsBuffer() ). The data pointer within the object
> pointed to by buffer changes, and no buffers pointed to the object are
> updated (to update them would violate various assumptions with how
> buffers work). This is a long-known issue, and is a major problem with
> how Python buffers currently work. The only solution to this requires
> having a realloc that warns you if it is going to allocate a new block,
> or strictly over-allocating initially, only ever using malloc, and
> creating a new data buffer object that holds memory used by array, etc.
>
> The fact that the memory still has what you expect *now*, doesn't mean
> it will in the future, nor does it mean that you will always be able to
> access the memory without segfault.

And just to tie this explanation back to the original question: When
the buffer is given to the wx.Image then it pulls out the pointer and
gives it to the wx.Image. At that point all the wx.Image knows about is
that pointer so if the original Python object moves its buffer to
another location in memory then the pointer that the wx.Image currently
has will be invalid and it will not be aware that it moved.

15

Josiah Carlson wrote:

Ultimately, if a user wants to use a buffer without copying, one should
be sure that the buffer cannot be moved in memory by design (string) or
by convention.

To wrap this up even more, and to make sure I get it... (is the OP even paying attention anymore?).

The buffer object itself does not manage memory, it is a container for a pointer. Which ever object first allocates the memory that the buffer is using is responsible for deleting it.

In the OP's case, I think the goal was to create the data for a wxImage in a C/C++ extension, then have a wxImage use it. One option is to use the wxWidgets C++ API to create a wxImage, then use the wxPython Swig stuff to wrap it. This means that the extension would have to be compiled against the right version of wxWidgets, and use all the right SWIG stuff.

The alternative is for the OP to create a buffer object that points to the data, and just pass that in to wx.Image.SetDataBuffer, all in Python, thus making sure that the object that manges the data stays around as long as the wx.Image does.

This is also relevant to the Matplotlib wx wrappers, which now need to be compiled against wx to get optimized passing of data form the Agg buffer to wxImage. I think we could use this technique instead, and then it's be easier to build MPL, and have it work with multiple versions of wx.

I've also used this to create a wx.Image from the data from a GDAL (geo-referenced raster maps) image. I had been creating a string from the data, then a wx.Image from the string, which really was one copy too many. Now I populate the wx.Image with SetDataBuffer, and make sure I keep a reference to the string around.

Which brings me to the next point:

I've written a tiny little factory function that creates a wx.Image with the data from a buffer, with a reference to the buffer. That way, the buffer object won't get deleted until after the wx.Image is deleted. Of course, as Josiah has made clear, that doesn't guarantee anything if the object you pass it doesn't keep that memory around, but it's a start, and should work at least for the common case of a simple Python string.

I first thought to subclass wx.Image, and make a version that is identical, except that it holds a reference to the buffer object that is used to create it. However, I couldn't' figure out how to do it, because it seems wxImages are all created by different factory functions, so I can't just override the __init__. Can I do this with some kind of judicious calling of wx.Image.__new__?

I've enclosed a simple test of my ImageFromBuffer function. It seems to work. If I don't put the reference to the string in there, I get some garbage. If I do, the image looks fine. would this be a worthwhile small addition to wxPython?

-Chris

ImageBuffer.py (2.39 KB)

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

16

Christopher Barker wrote:

I've written a tiny little factory function that creates a wx.Image with the data from a buffer, with a reference to the buffer. That way, the buffer object won't get deleted until after the wx.Image is deleted. Of course, as Josiah has made clear, that doesn't guarantee anything if the object you pass it doesn't keep that memory around, but it's a start, and should work at least for the common case of a simple Python string.

I first thought to subclass wx.Image, and make a version that is identical, except that it holds a reference to the buffer object that is used to create it. However, I couldn't' figure out how to do it, because it seems wxImages are all created by different factory functions, so I can't just override the __init__. Can I do this with some kind of judicious calling of wx.Image.__new__?

I've enclosed a simple test of my ImageFromBuffer function. It seems to work. If I don't put the reference to the string in there, I get some garbage. If I do, the image looks fine. would this be a worthwhile small addition to wxPython?

I've added it, but changed the API to be more like the other wx.Image factories and also added the ability to specify a buffer for the alpha channel. Does this work for you?

def ImageFromBuffer(width, height, dataBuffer, alphaBuffer=None):
     """
     Creates a `wx.Image` from the data in dataBuffer. The dataBuffer
     parameter must be a Python object that implements the buffer
     interface, such as a string, array, etc. The dataBuffer object is
     expected to contain a series of RGB bytes and be width*height*3
     bytes long. A buffer object can optionally be supplied for the
     image's alpha channel data, and it is expected to be width*height
     bytes long.

     A reference to the data and alpha buffer objects are kept with the
     wx.Image, so that they won't get deleted until after the wx.Image
     is deleted. However please be aware that it is not guaranteed that
     an object won't move its memory buffer to a new location when it
     needs to resize its contents. If that happens then the wx.Image
     will end up referring to an invalid memory location and could cause
     the application to crash. Therefore care should be taken to not
     manipulate the objects used for the data and alpha buffers in a
     way that would cause them to change size.
     """
     image = wx.EmptyImage(width, height)
     image.SetDataBuffer(dataBuffer)
     if alphaBuffer is not None:
         image.SetAlphaBuffer(alphaBuffer)
     image._buffer = dataBuffer
     image._alpha = alphaBuffer
     return image

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

17

Robin Dunn wrote:

I've added it, but changed the API to be more like the other wx.Image factories and also added the ability to specify a buffer for the alpha channel. Does this work for you?

Thanks Robin. It seems to work just great, and is, or course better than my version.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

18

Hi all :slight_smile:

I've followed the combined advice of Robin and Christopher in using a python
buffer object to instance a wx.Image in the python wrapper layer. It all seems
to be working - and I don't think there are any 'data copies' going on :wink:

To keep my underlying C++ python-free, I introduced the buffer (and creation of
the wx.Image) via my .i Swig file. In case anyone who is reading is keen trying
to solve a similar problem, I've included the code that I added to my .i file in
order to implement the change below. (I'd appreciate any comments regarding
errors or shortcomings that anyone spots in my implementation.)

Sorry I didn't reply to the thread earlier. A combination of being tied up with
more boring work, moving house and background research into using python buffers
with Swig means that I've only just got this far.

One remaining question I have is: Does the call to wx.EmptyImage that I make
cause any memory to be allocated? I see from Robin's recent post that in my
case this call is now redundant (as I will soon be using ImageFromBuffer) - but
I'm curious to know the answer anyway :wink:

Thanks for all the help. The more I use wxPython the more I like it...

Andrew :wink:

/* the RawImage C++ class that we are wrapping offers a method that
   returns a pointer to an internal 'unsigned char' buffer of display
   data. To make the python class generated by SWIG a bit more
   friendly to the rest of our wxPython code we instruct SWIG to make
   two modifications during the wrapping process. First we add a
   method to the C++ class that will return a version of the internal
   display buffer wrapped up as a 'python buffer object' (performing
   this in the C++ wrapper class keeps the wrapped C++ python-free)...
*/
%extend RawImage {
  PyObject* RawImage::getDisplayBuffer(void) {
    // return a new 'python buffer object' that intialised using
    // the memory address and length of our display buffer...
    void* pvBufferData = (void*)self->pcGetDisplayData();
    int iBufferSize = self->getDisplayBufferSize();
    return PyBuffer_FromMemory(pvBufferData, iBufferSize);
  }
}
/* ... we also add a method to the python wrapper class that will use
   the buffer offered by our new C++ method to create a wx.Image
   (neither of these methods actually copies the image data)...
*/
%extend RawImage {
%pythoncode %{
def getImage(self):
    # create an unintialised wx.Image of the correct size...
    wximage = wx.EmptyImage(self.getWidth(), self.getHeight(), clear=False)
    # then define the data content of the image...
    wximage.SetDataBuffer(self.getDisplayBuffer())
    return wximage
%}
}

19

Andrew Murray wrote:

In case anyone who is reading is keen trying
to solve a similar problem, I've included the code that I added to my .i file in
order to implement the change below.

Thanks. I just might need this some day.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

20

Andrew Murray wrote:

One remaining question I have is: Does the call to wx.EmptyImage that I make
cause any memory to be allocated?

Hmmm... Yes it does. I guess should make a real constructor wrapper for this to be able to avoid the initial memory allocation.

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!