What is a Python and wx "buffer" object ?

wxPython Users
1

I’m confused about the difference (or sameness) on the documentation:

ImageFromBuffer(width, height, dataBuffer, alphaBuffer=None)

“Creates a wx.Image from the data in dataBuffer. The dataBuffer
parameter must be a Python object that implements the buffer interface,
such as a string, array, etc.”

So far, so good. Now:

ImageFromDataWithAlpha(width, height, data, alpha)

“Construct an Image from a buffer of RGB bytes with an Alpha channel.
Accepts either a string or a buffer object holding the data…”

OK, ImageFromDataWithAlpha() accepts only these mysterious “buffers”:
“Python object that implements the buffer interface, such as a string,
array, etc.”

What’s the difference ? Are the 2 functions really the same ? What are
the RGB organizations ?

What’s a Python “buffer” ? I can’t find documentation for this.

Ray Pasco

2

Hi,

I'm confused about the difference (or sameness) on the documentation:

ImageFromBuffer(width, height, dataBuffer, alphaBuffer=None)

"Creates a wx.Image from the data in dataBuffer. The dataBuffer parameter
must be a Python object that implements the buffer interface, such as a
string, array, etc."

So far, so good. Now:

ImageFromDataWithAlpha(width, height, data, alpha)

"Construct an Image from a buffer of RGB bytes with an Alpha channel.
Accepts either a string or a buffer object holding the data..."

OK, ImageFromDataWithAlpha() accepts only these mysterious "buffers":
"Python object that implements the buffer interface, such as a string,
array, etc."

What's the difference ? Are the 2 functions really the same ? What are the
RGB organizations ?

What's a Python "buffer" ? I can't find documentation for this.

First google result: Buffer Protocol — Python 3.13.0 documentation
(http://lmgtfy.com/?q=python+buffer+interface) :wink:

From the wx documentation it appears that those two functions do
essentially the same thing except that the latter requires the alpha
argument instead of specifying it as a keyword, but I could missing
something.

Cody

···

On Tue, Jun 22, 2010 at 12:02 PM, Ray Pasco <pascor@verizon.net> wrote:

3

Ray Pasco wrote:

I'm confused about the difference (or sameness) on the documentation:

ImageFromBuffer(width, height, dataBuffer, alphaBuffer=None)

"Creates a wx.Image from the data in dataBuffer. The dataBuffer parameter must be a Python object that implements the buffer interface, such as a string, array, etc."

So far, so good. Now:

ImageFromDataWithAlpha(width, height, data, alpha)

"Construct an Image from a buffer of RGB bytes with an Alpha channel. Accepts either a string or a buffer object holding the data..."

OK, ImageFromDataWithAlpha() accepts only these mysterious "buffers": "Python object that implements the buffer interface, such as a string, array, etc."

What's the difference ?

The idea of a Buffer object is that C/C++ code can access the data pointer directly, rather than making a copy. The trick with that is that you need to make sure the data isn't deleted yourself.

So I guess that ImageFromBuffer doesn't copy the data, and ImageFromData...() does.

Th point of adding ImageFromBuffer...() is that you can grab data from ANY buffer object (lke a numpy array) without knowing anything else about the object, of doing any uneccesary copying. Before Robin added that, you need to copy a numpy array to a string, then the string to the Image, so two copies, rather than potentially zero copies.

NOTE that I'm not entirely sure ImageFromBuffer doesn't copy at all == I've tested it, but don't remember the results.

If it doesn't, then you can do cool things like create a wxImage from a numpy array, then change the array, and the wxImage will have changed as well.

I know wxBitmapFromBuffer does copy.

What are the RGB organizations ?

RGBRGBRGBRGB

for a 3x2 image (all in a row, though)

What's a Python "buffer" ? I can't find documentation for this.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

4

Ray Pasco wrote:

I'm confused about the difference (or sameness) on the documentation:

ImageFromBuffer(width, height, dataBuffer, alphaBuffer=None)

Here's some test code I wrote a while back -- though I haven't even tried to run it lately. It might give you some ideas.

-Chris

BitmapFromBufferTest.py (2.73 KB)

BitmapFromPILTest.py (896 Bytes)

···

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

5

So I guess that ImageFromBuffer doesn't copy the data, and
ImageFromData...() does.

Correct. That is mentioned in the next paragraphs of the ImageFromBuffer docstring:

"""
The wx.Image will be created with its data and alpha pointers initialized to the memory address pointed to by the buffer objects, thus saving the time needed to copy the image data from the buffer object to the wx.Image. While this has advantages, it also has the shoot-yourself-in-the-foot risks associated with sharing a C pointer between two objects.

To help alleviate the risk a reference to the data and alpha buffer objects are kept with the wx.Image, so that they won't get deleted until after the wx.Image is deleted. However please be aware that it is not guaranteed that an object won't move its memory buffer to a new location when it needs to resize its contents. If that happens then the wx.Image will end up referring to an invalid memory location and could cause the application to crash. Therefore care should be taken to not manipulate the objects used for the data and alpha buffers in a way that would cause them to change size.
"""

I know wxBitmapFromBuffer does copy.

Correct. Because the in-memory representation of the pixel data varies between platforms and image bit-depths, so there is no way to make it share the pixel buffer in a platform independent way.

···

On 6/22/10 11:34 AM, Christopher Barker wrote:

--
Robin Dunn
Software Craftsman

6

I think I understand now: From a "pure Python" standpoint a buffer is
an internal C implementation, so there is no direct creation or
access. To use a buffer is simply a matter of creating or using a byte
string or an array of multi-byte values. That these 2 object types
(and others ?) meet the requirements of a Python buffer is completely
transparent in pure Python code. Buffers are only a concern in C
extensions to Python (and the implementations of Python and its add-on
packages).

It seems to me the wxPthon docs should eliminate the use of the term
"Python buffer" and substitute "byte string", "data array" or similar
in its place because a Python coder can't directly access a buffer.
All the other Python docs I've seen are from a "pure Python" point of
view. Perhaps docs concerning C extension's Python objects' access
should be separate.

Thanks,
Ray Pasco

···

On Jun 22, 5:44 pm, Robin Dunn <ro...@alldunn.com> wrote:

On 6/22/10 11:34 AM, Christopher Barker wrote:

> So I guess that ImageFromBuffer doesn't copy the data, and
> ImageFromData...() does.

Correct. That is mentioned in the next paragraphs of the
ImageFromBuffer docstring:

"""
The wx.Image will be created with its data and alpha pointers
initialized to the memory address pointed to by the buffer objects, thus
saving the time needed to copy the image data from the buffer object to
the wx.Image. While this has advantages, it also has the
shoot-yourself-in-the-foot risks associated with sharing a C pointer
between two objects.

To help alleviate the risk a reference to the data and alpha buffer
objects are kept with the wx.Image, so that they won't get deleted until
after the wx.Image is deleted. However please be aware that it is not
guaranteed that an object won't move its memory buffer to a new location
when it needs to resize its contents. If that happens then the wx.Image
will end up referring to an invalid memory location and could cause the
application to crash. Therefore care should be taken to not manipulate
the objects used for the data and alpha buffers in a way that would
cause them to change size.
"""

> I know wxBitmapFromBuffer does copy.

Correct. Because the in-memory representation of the pixel data varies
between platforms and image bit-depths, so there is no way to make it
share the pixel buffer in a platform independent way.

--
Robin Dunn
Software Craftsmanhttp://wxPython.org

7

WinCrazy wrote:

I think I understand now: From a "pure Python" standpoint a buffer is
an internal C implementation, so there is no direct creation or
access. To use a buffer is simply a matter of creating or using a byte
string or an array of multi-byte values.

or any other class that supports the buffer protocol, numpy arrays, for instance.

That these 2 object types
(and others ?) meet the requirements of a Python buffer is completely
transparent in pure Python code. Buffers are only a concern in C
extensions to Python (and the implementations of Python and its add-on
packages).

It seems to me the wxPthon docs should eliminate the use of the term
"Python buffer" and substitute "byte string", "data array" or similar
in its place because a Python coder can't directly access a buffer.

except that there is a difference between a class that supports the buffer protocol and one that doesn't. So even if you are only writing "pure python", you need to select an appropriate class for your use.

While you can't make a new class that supports the buffer protocol in pure python, you can use one. In fact, the point of the buffer protocol (which has been improved for py3) is to allow different C extensions to efficiently exchange data without having to know about each other. So the wxPython ImageFromBuffer is specifically designed to take ANY class that supports the protocol --not just the ones we know about at the time.

(I wonder if PIL supports it, for instance)

Not that the docs couldn't, be and should be, more clear, but it's key to mention the buffer interface -- that's the whole point of that constructor.

-Chris

···

All the other Python docs I've seen are from a "pure Python" point of
view. Perhaps docs concerning C extension's Python objects' access
should be separate.

Thanks,
Ray Pasco

On Jun 22, 5:44 pm, Robin Dunn <ro...@alldunn.com> wrote:

On 6/22/10 11:34 AM, Christopher Barker wrote:

So I guess that ImageFromBuffer doesn't copy the data, and
ImageFromData...() does.

Correct. That is mentioned in the next paragraphs of the
ImageFromBuffer docstring:

"""
The wx.Image will be created with its data and alpha pointers
initialized to the memory address pointed to by the buffer objects, thus
saving the time needed to copy the image data from the buffer object to
the wx.Image. While this has advantages, it also has the
shoot-yourself-in-the-foot risks associated with sharing a C pointer
between two objects.

To help alleviate the risk a reference to the data and alpha buffer
objects are kept with the wx.Image, so that they won't get deleted until
after the wx.Image is deleted. However please be aware that it is not
guaranteed that an object won't move its memory buffer to a new location
when it needs to resize its contents. If that happens then the wx.Image
will end up referring to an invalid memory location and could cause the
application to crash. Therefore care should be taken to not manipulate
the objects used for the data and alpha buffers in a way that would
cause them to change size.
"""

I know wxBitmapFromBuffer does copy.

Correct. Because the in-memory representation of the pixel data varies
between platforms and image bit-depths, so there is no way to make it
share the pixel buffer in a platform independent way.

--
Robin Dunn
Software Craftsmanhttp://wxPython.org

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

8

I think I understand now: From a "pure Python" standpoint a buffer is
an internal C implementation, so there is no direct creation or
access.

Not quite true. "buffer" is a type object accessible in Python and you can create instances of it:

  >>> buffer
  <type 'buffer'>
  >>> print buffer.__doc__
  buffer(object [, offset[, size]])

  Create a new buffer object which references the given object.
  The buffer will reference a slice of the target object from the
  start of the object (or at the specified offset). The slice will
  extend to the end of the target object (or with the specified size).
  >>>
  >>> b = buffer("01234")
  >>> b
  <read-only buffer for 0x19282880, size -1, offset 0 at 0x19282840>

To use a buffer is simply a matter of creating or using a byte
string or an array of multi-byte values. That these 2 object types
(and others ?) meet the requirements of a Python buffer is completely
transparent in pure Python code. Buffers are only a concern in C
extensions to Python (and the implementations of Python and its add-on
packages).

It seems to me the wxPthon docs should eliminate the use of the term
"Python buffer" and substitute "byte string", "data array" or similar
in its place because a Python coder can't directly access a buffer.

I don't think that gives enough information either. The wxPython functions that can accept buffers can accept anything that implements the "buffer protocol", which can, in theory, be much more than just strings and arrays.

···

On 6/23/10 2:36 PM, WinCrazy wrote:

--
Robin Dunn
Software Craftsman

9

Why would anyone want to directly create a buffer like this :

    buf = buffer

What Python-code buffer operations are available ? How can it be
modified ?

A Python-created buffer seems have the same creation characteristics
of a dc:
"The wx.DC class is abstract and can not be instantiated, you must use
one of the derived classes instead."

It seems pointless to expose the buffer object at all in Python for
exactly the same reasons as a dc: It can not be directly used in
Python code.

Are there any other *base classes* that implement the buffer
interface ?

···

On Jun 24, 3:02 pm, Robin Dunn <ro...@alldunn.com> wrote:

On 6/23/10 2:36 PM, WinCrazy wrote:

> I think I understand now: From a "pure Python" standpoint a buffer is
> an internal C implementation, so there is no direct creation or
> access.

Not quite true. "buffer" is a type object accessible in Python and you
can create instances of it:

>>> buffer
<type 'buffer'>
>>> print buffer.__doc__
buffer(object [, offset[, size]])

Create a new buffer object which references the given object.
The buffer will reference a slice of the target object from the
start of the object (or at the specified offset). The slice will
extend to the end of the target object (or with the specified size).
>>>
>>> b = buffer("01234")
>>> b
<read-only buffer for 0x19282880, size -1, offset 0 at 0x19282840>

> To use a buffer is simply a matter of creating or using a byte
> string or an array of multi-byte values. That these 2 object types
> (and others ?) meet the requirements of a Python buffer is completely
> transparent in pure Python code. Buffers are only a concern in C
> extensions to Python (and the implementations of Python and its add-on
> packages).

> It seems to me the wxPthon docs should eliminate the use of the term
> "Python buffer" and substitute "byte string", "data array" or similar
> in its place because a Python coder can't directly access a buffer.

I don't think that gives enough information either. The wxPython
functions that can accept buffers can accept anything that implements
the "buffer protocol", which can, in theory, be much more than just
strings and arrays.

--
Robin Dunn
Software Craftsmanhttp://wxPython.org

10

WinCrazy wrote:

Why would anyone want to directly create a buffer like this :

    buf = buffer

well, not like that -- you need to pass in an existing buffer object:

buf = buffer(a_string)

What Python-code buffer operations are available ? How can it be
modified ?

It looks like it's read-only.

The entire point of a buffer object is to be able to pass a data pointer to another function -- no other reason to use it. So why not just use a_string directly in the example above?

The only reason I can think of is that you can take a sub-slice when you create a new buffer from another buffer:

In [61]: string
Out[61]: 'an example string'

In [62]: buf = buffer(string, 3, 4)

In [63]: str(buf)
Out[63]: 'exam'

So you've got a sub-slice, but without making a copy of the data. This could be useful when you have a string holding a huge array of data.

Anyway, it's pretty low level stuff, but useful when you need it.

Are there any other *base classes* that implement the buffer
interface ?

The ones I know about are:

string
array.array
bytes (which is a string in py2)
numpy.ndarray

and who knows what other objects in other packages, including wxPython -- wx.Image maybe?

-Chris

···

On Jun 24, 3:02 pm, Robin Dunn <ro...@alldunn.com> wrote:

On 6/23/10 2:36 PM, WinCrazy wrote:

I think I understand now: From a "pure Python" standpoint a buffer is
an internal C implementation, so there is no direct creation or
access.

Not quite true. "buffer" is a type object accessible in Python and you
can create instances of it:

  >>> buffer
  <type 'buffer'>
  >>> print buffer.__doc__
  buffer(object [, offset[, size]])

  Create a new buffer object which references the given object.
  The buffer will reference a slice of the target object from the
  start of the object (or at the specified offset). The slice will
  extend to the end of the target object (or with the specified size).
  >>>
  >>> b = buffer("01234")
  >>> b
  <read-only buffer for 0x19282880, size -1, offset 0 at 0x19282840>

To use a buffer is simply a matter of creating or using a byte
string or an array of multi-byte values. That these 2 object types
(and others ?) meet the requirements of a Python buffer is completely
transparent in pure Python code. Buffers are only a concern in C
extensions to Python (and the implementations of Python and its add-on
packages).
It seems to me the wxPthon docs should eliminate the use of the term
"Python buffer" and substitute "byte string", "data array" or similar
in its place because a Python coder can't directly access a buffer.

I don't think that gives enough information either. The wxPython
functions that can accept buffers can accept anything that implements
the "buffer protocol", which can, in theory, be much more than just
strings and arrays.

--
Robin Dunn
Software Craftsmanhttp://wxPython.org

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

11

Are there any other *base classes* that implement the buffer
interface ?

The ones I know about are:

string
array.array
bytes (which is a string in py2)
numpy.ndarray

ISTR that there are some other stock types that implement the buffer protocol, but my mind is blank at the moment... Perhaps mmap files?

In Python 2.7 and 3.x there is the memoryview type which IIUC can allow easier access from Python to the data in a buffer object, and buffers can expose some more information about themselves such as the size and format of the items in the buffer, (e.g., ints vs. bytes, etc.) dimensions/shape of the array, etc.

and who knows what other objects in other packages, including wxPython
-- wx.Image maybe?

You can already get a buffer object from a wx.Image with the GetDataBuffer method, but it might be interesting to be able to directly use the wx.Image instance itself as a buffer. I'll look in to that.

···

On 6/25/10 10:12 AM, Christopher Barker wrote:

--
Robin Dunn
Software Craftsman