dealing with graphics and matrices...

wxPython Users
1

Good morning!
I'm using a wx.bitmap and wx.MemoryDC() to store screen pixel values
and to operate with them (I retrieve them with pixelRedVal function),
as you can see in my code (just look at first ten lines):

[code]
def StarTrack(self, dcScreen, actualX, actualY,iter):
    # centers the star and finds fwhm
        #create a copy of screen image in memory
        screenSizeX, screenSizeY = wx.DisplaySize()
        bitmap=wx.EmptyBitmap(screenSizeX ,screenSizeY,-1)
        memory=wx.MemoryDC()
        memory.SelectObject(bitmap)
        memory.Blit(0,0,screenSizeX,screenSizeY,dcScreen,0,0)

        def pixelRedVal(x,y): #returns red value of the pixel x y
            return memory.GetPixel(x,y)[0]
...
...[operations with pixel red value]
...
        memory.SelectObject(wx.NullBitmap) #erase memory copy of
screen image
        return actualXr, actualYr
[/code]

Now, I'd like to modify this code in order to operate with the sum of
pixel value during the time, i.e. I'd like to store in a matrix
something like:

[code]
every tot millesecs:
    read actual_pixel_value(x,y)
    pixel_value = 0.9*(pixel_value(x,y))+0.1*(actual_pixel_value(x,y))
[/code]

How can I do this in a efficient way?

2

Could you maybe use something equivalent to an accumulation buffer in OpenGL?

You would make a memory dc and consecutively draw the bitmaps grabbed from the screen to it with wx.GraphicsContext using alpha blending.

Different ways of initializing the alpha channel on those bitmaps could be used to tweak the exact method for summing the recent pixel values.

···

On Wed, 02 Nov 2011 09:20:41 +0100, andreaconsole <andreaconsole@gmail.com> wrote:

Good morning!
I'm using a wx.bitmap and wx.MemoryDC() to store screen pixel values
and to operate with them (I retrieve them with pixelRedVal function),
as you can see in my code (just look at first ten lines):

[code]
def StarTrack(self, dcScreen, actualX, actualY,iter):
    # centers the star and finds fwhm
        #create a copy of screen image in memory
        screenSizeX, screenSizeY = wx.DisplaySize()
        bitmap=wx.EmptyBitmap(screenSizeX ,screenSizeY,-1)
        memory=wx.MemoryDC()
        memory.SelectObject(bitmap)
        memory.Blit(0,0,screenSizeX,screenSizeY,dcScreen,0,0)

        def pixelRedVal(x,y): #returns red value of the pixel x y
            return memory.GetPixel(x,y)[0]
...
...[operations with pixel red value]
...
        memory.SelectObject(wx.NullBitmap) #erase memory copy of
screen image
        return actualXr, actualYr
[/code]

Now, I'd like to modify this code in order to operate with the sum of
pixel value during the time, i.e. I'd like to store in a matrix
something like:

[code]
every tot millesecs:
    read actual_pixel_value(x,y)
    pixel_value = 0.9*(pixel_value(x,y))+0.1*(actual_pixel_value(x,y))
[/code]

How can I do this in a efficient way?

3

Are you saying I just have to modify this line:

memory.Blit(0,0,screenSizeX,screenSizeY,dcScreen,0,0)

with something involving this wx.GraphicsContext and alphablending, right?
I must suppose that there is a way to copy dcScreen in the bitmap by
adding the values and not simply by substiting them, right?

I think I have to find in-depth documentation about wx.GraphicsContext, thanks!

···

2011/11/2 Toni Ruža <gmr.gaf@gmail.com>:

On Wed, 02 Nov 2011 09:20:41 +0100, andreaconsole <andreaconsole@gmail.com> > wrote:

Good morning!
I'm using a wx.bitmap and wx.MemoryDC() to store screen pixel values
and to operate with them (I retrieve them with pixelRedVal function),

something like:

[code]
every tot millesecs:
   read actual_pixel_value(x,y)
   pixel_value = 0.9*(pixel_value(x,y))+0.1*(actual_pixel_value(x,y))
[/code]

How can I do this in a efficient way?

Could you maybe use something equivalent to an accumulation buffer in
OpenGL?

You would make a memory dc and consecutively draw the bitmaps grabbed from
the screen to it with wx.GraphicsContext using alpha blending.

Different ways of initializing the alpha channel on those bitmaps could be
used to tweak the exact method for summing the recent pixel values.

4

Good morning!
I'm using a wx.bitmap and wx.MemoryDC() to store screen pixel values
and to operate with them (I retrieve them with pixelRedVal function),

something like:

[code]
every tot millesecs:
   read actual_pixel_value(x,y)
   pixel_value = 0.9*(pixel_value(x,y))+0.1*(actual_pixel_value(x,y))
[/code]

How can I do this in a efficient way?

Could you maybe use something equivalent to an accumulation buffer in
OpenGL?

You would make a memory dc and consecutively draw the bitmaps grabbed from
the screen to it with wx.GraphicsContext using alpha blending.

Different ways of initializing the alpha channel on those bitmaps could be
used to tweak the exact method for summing the recent pixel values.

Are you saying I just have to modify this line:

memory.Blit(0,0,screenSizeX,screenSizeY,dcScreen,0,0)

with something involving this wx.GraphicsContext and alphablending, right?
I must suppose that there is a way to copy dcScreen in the bitmap by
adding the values and not simply by substiting them, right?

You would also have to add an alpha channel to the bitmap you grabbed from the screen, converting the bitmap to wxImage might be the easiest route to do this.

In the GraphicsContext demo DrawBitmap is used to draw bitmaps with alpha blending.

I think I have to find in-depth documentation about wx.GraphicsContext, thanks!

Glad to help.

···

On Wed, 02 Nov 2011 10:31:59 +0100, andrea console <andreaconsole@gmail.com> wrote:

2011/11/2 Toni Ruža <gmr.gaf@gmail.com>:

On Wed, 02 Nov 2011 09:20:41 +0100, andreaconsole <andreaconsole@gmail.com> >> wrote:

5

You might like to take a look at some of the methods in numpy - I am not
an expert but I think that there are efficient functions
to do something like storing the displayable values in a [x, y, 3]
matrix, getting the red value as a [x, y, 1] matrix and adding a scaled
version together before converting it to a bitmap.

Gadget/Steve

···

On 02/11/2011 8:20 AM, andreaconsole wrote:

Good morning!
I'm using a wx.bitmap and wx.MemoryDC() to store screen pixel values
and to operate with them (I retrieve them with pixelRedVal function),
as you can see in my code (just look at first ten lines):

[code]
def StarTrack(self, dcScreen, actualX, actualY,iter):
    # centers the star and finds fwhm
        #create a copy of screen image in memory
        screenSizeX, screenSizeY = wx.DisplaySize()
        bitmap=wx.EmptyBitmap(screenSizeX ,screenSizeY,-1)
        memory=wx.MemoryDC()
        memory.SelectObject(bitmap)
        memory.Blit(0,0,screenSizeX,screenSizeY,dcScreen,0,0)

        def pixelRedVal(x,y): #returns red value of the pixel x y
            return memory.GetPixel(x,y)[0]
...
...[operations with pixel red value]
...
        memory.SelectObject(wx.NullBitmap) #erase memory copy of
screen image
        return actualXr, actualYr
[/code]

Now, I'd like to modify this code in order to operate with the sum of
pixel value during the time, i.e. I'd like to store in a matrix
something like:

[code]
every tot millesecs:
    read actual_pixel_value(x,y)
    pixel_value = 0.9*(pixel_value(x,y))+0.1*(actual_pixel_value(x,y))
[/code]

How can I do this in a efficient way?

6

Hi,
I think that storing values from a bitmap to a matrix could be
time-consuming even for numpy. Do you know about any way to do it
fast?

···

2011/11/2 Gadget/Steve <GadgetSteve@live.co.uk>:

On 02/11/2011 8:20 AM, andreaconsole wrote:

Good morning!
I'm using a wx.bitmap and wx.MemoryDC() to store screen pixel values
and to operate with them (I retrieve them with pixelRedVal function),
as you can see in my code (just look at first ten lines):

[code]
def StarTrack(self, dcScreen, actualX, actualY,iter):
# centers the star and finds fwhm
#create a copy of screen image in memory
screenSizeX, screenSizeY = wx.DisplaySize()
bitmap=wx.EmptyBitmap(screenSizeX ,screenSizeY,-1)
memory=wx.MemoryDC()
memory.SelectObject(bitmap)
memory.Blit(0,0,screenSizeX,screenSizeY,dcScreen,0,0)

    def pixelRedVal\(x,y\): \#returns red value of the pixel x y
        return memory\.GetPixel\(x,y\)\[0\]

...
...[operations with pixel red value]
...
memory.SelectObject(wx.NullBitmap) #erase memory copy of
screen image
return actualXr, actualYr
[/code]

Now, I'd like to modify this code in order to operate with the sum of
pixel value during the time, i.e. I'd like to store in a matrix
something like:

[code]
every tot millesecs:
read actual_pixel_value(x,y)
pixel_value = 0.9*(pixel_value(x,y))+0.1*(actual_pixel_value(x,y))
[/code]

How can I do this in a efficient way?

You might like to take a look at some of the methods in numpy - I am not
an expert but I think that there are efficient functions
to do something like storing the displayable values in a [x, y, 3]
matrix, getting the red value as a [x, y, 1] matrix and adding a scaled
version together before converting it to a bitmap.

Gadget/Steve

7

Numpy implements the Python buffer protocol, so it can be the source or target of the buffer methods grafted on to wx.Image and wx.Bitmap like wx.Bitmap.CopyToBuffer, wx.Bitmap.CopyFromBuffer, the 'Data' methods in wx.Image, etc. See also the RawBitmapAccess sample in the demo.

···

On 11/2/11 7:47 AM, andrea console wrote:

2011/11/2 Gadget/Steve<GadgetSteve@live.co.uk>:

On 02/11/2011 8:20 AM, andreaconsole wrote:

How can I do this in a efficient way?

You might like to take a look at some of the methods in numpy - I am not
an expert but I think that there are efficient functions
to do something like storing the displayable values in a [x, y, 3]
matrix, getting the red value as a [x, y, 1] matrix and adding a scaled
version together before converting it to a bitmap.

Gadget/Steve

Hi,
I think that storing values from a bitmap to a matrix could be
time-consuming even for numpy. Do you know about any way to do it
fast?

--
Robin Dunn
Software Craftsman

8

trying something like this, but prints strange results...

        screenSizeX, screenSizeY = wx.DisplaySize()
        bitmap=wx.EmptyBitmap(screenSizeX ,screenSizeY,-1)
        memory=wx.MemoryDC()
        memory.SelectObject(bitmap)
        memory.Blit(0,0,screenSizeX,screenSizeY,dcScreen,0,0)
        mybuffer = numpy.zeros((screenSizeX,screenSizeY,3))
        bitmap.CopyToBuffer(mybuffer, wx.BitmapBufferFormat_RGB)
        print mybuffer[1,1,1]

···

2011/11/2 Robin Dunn <robin@alldunn.com>:

On 11/2/11 7:47 AM, andrea console wrote:

2011/11/2 Gadget/Steve<GadgetSteve@live.co.uk>:

On 02/11/2011 8:20 AM, andreaconsole wrote:

How can I do this in a efficient way?

You might like to take a look at some of the methods in numpy - I am not
an expert but I think that there are efficient functions
to do something like storing the displayable values in a [x, y, 3]
matrix, getting the red value as a [x, y, 1] matrix and adding a scaled
version together before converting it to a bitmap.

Gadget/Steve

Hi,
I think that storing values from a bitmap to a matrix could be
time-consuming even for numpy. Do you know about any way to do it
fast?

Numpy implements the Python buffer protocol, so it can be the source or
target of the buffer methods grafted on to wx.Image and wx.Bitmap like
wx.Bitmap.CopyToBuffer, wx.Bitmap.CopyFromBuffer, the 'Data' methods in
wx.Image, etc. See also the RawBitmapAccess sample in the demo.

9

Numpy implements the Python buffer protocol, so it can be the source or
target of the buffer methods grafted on to wx.Image and wx.Bitmap like
wx.Bitmap.CopyToBuffer, wx.Bitmap.CopyFromBuffer, the 'Data' methods in
wx.Image, etc. See also the RawBitmapAccess sample in the demo.

trying something like this, but prints strange results...

         screenSizeX, screenSizeY = wx.DisplaySize()
         bitmap=wx.EmptyBitmap(screenSizeX ,screenSizeY,-1)
         memory=wx.MemoryDC()
         memory.SelectObject(bitmap)
         memory.Blit(0,0,screenSizeX,screenSizeY,dcScreen,0,0)
         mybuffer = numpy.zeros((screenSizeX,screenSizeY,3))

you need to be careful about data types in numpy -- zeros() defaults to double precision floats. Try:

mybuffer = numpy.zeros((screenSizeX,screenSizeY,3), dtype=numpy.uint8)

         bitmap.CopyToBuffer(mybuffer, wx.BitmapBufferFormat_RGB)
         print mybuffer[1,1,1]

Note that there is a new buffer protocol, in which the buffer includes information about the shape and type of the data block -- if we can get wxPython to use that, then you would have gotten a meaningful error here.

-Chris

···

On 11/4/11 1:29 AM, andrea console wrote:

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

10

I plan on looking into that for Phoenix as I'll be rewriting the buffer related methods for wx.Bitmap and wx.Image anyway.

···

On 11/4/11 10:47 AM, Chris Barker wrote:

bitmap.CopyToBuffer(mybuffer, wx.BitmapBufferFormat_RGB)
print mybuffer[1,1,1]

Note that there is a new buffer protocol, in which the buffer includes
information about the shape and type of the data block -- if we can get
wxPython to use that, then you would have gotten a meaningful error here.

--
Robin Dunn
Software Craftsman

11

Thank you!
Finally this is the code that do exactly what I want:

        screenSizeX, screenSizeY = wx.DisplaySize()
        bitmap=wx.EmptyBitmap(screenSizeX ,screenSizeY,-1)
        memory=wx.MemoryDC()
        memory.SelectObject(bitmap)
        memory.Blit(0,0,screenSizeX,screenSizeY,dcScreen,0,0)
        mybuffer = numpy.zeros((screenSizeX*screenSizeY*3), dtype=numpy.uint8)
        bitmap.CopyToBuffer(mybuffer, wx.BitmapBufferFormat_RGB)
        print mybuffer[3*(actualX+screenSizeX*actualY)],
mybuffer[3*(actualX+screenSizeX*actualY)+1],
mybuffer[3*(actualX+screenSizeX*actualY)+2]

Just a final question: is this the most efficient option? I mean: do I
really need to "Blit" to a bitmap or I can use a more straightforward
way?

···

2011/11/4 Chris Barker <Chris.Barker@noaa.gov>:

On 11/4/11 1:29 AM, andrea console wrote:

Numpy implements the Python buffer protocol, so it can be the source or
target of the buffer methods grafted on to wx.Image and wx.Bitmap like
wx.Bitmap.CopyToBuffer, wx.Bitmap.CopyFromBuffer, the 'Data' methods in
wx.Image, etc. See also the RawBitmapAccess sample in the demo.

trying something like this, but prints strange results...

    screenSizeX, screenSizeY = wx\.DisplaySize\(\)
    bitmap=wx\.EmptyBitmap\(screenSizeX ,screenSizeY,\-1\)
    memory=wx\.MemoryDC\(\)
    memory\.SelectObject\(bitmap\)
    memory\.Blit\(0,0,screenSizeX,screenSizeY,dcScreen,0,0\)
    mybuffer = numpy\.zeros\(\(screenSizeX,screenSizeY,3\)\)

you need to be careful about data types in numpy -- zeros() defaults to
double precision floats. Try:

mybuffer = numpy.zeros((screenSizeX,screenSizeY,3), dtype=numpy.uint8)

    bitmap\.CopyToBuffer\(mybuffer, wx\.BitmapBufferFormat\_RGB\)
    print mybuffer\[1,1,1\]

Note that there is a new buffer protocol, in which the buffer includes
information about the shape and type of the data block -- if we can get
wxPython to use that, then you would have gotten a meaningful error here.

-Chris

12

That would be great -- and yes, those are good places to start.

I'm not sure it would be helpful with how you are doing Phoenix, but Cython has a nice implementation of the buffer protocol -- it generates C code that indexes, etc a buffer -- there may be some example code in there worth borrowing.

Once it's there for Images, etc, it would be great to use it for things like passing in arrays of coordinates to drawing functions, etc -- hopefully I"ll get a chance to look at that some day.

-Chris

···

On 11/4/11 1:13 PM, Robin Dunn wrote:

Note that there is a new buffer protocol, in which the buffer includes
information about the shape and type of the data block -- if we can get
wxPython to use that, then you would have gotten a meaningful error here.

I plan on looking into that for Phoenix as I'll be rewriting the buffer
related methods for wx.Bitmap and wx.Image anyway.

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

13

yes, I think you do need to Blit from the ScreenDC to a bitmap -- you need to get it into regular memory some how -- I don't think the screen buffer is in regular memory that you can use for a buffer, etc.

However, how did the image get on the screen in the first place? Maybe you can draw to an off-screen buffer instead, and draw that to the screen to display, but keep it around for other purposes (see double buffering)

you can improve a little bit the numpy code:

mybuffer = numpy.empty((screenSizeX*screenSizeY*3), dtype=numpy.uint8)

# no need to initialize to zeros -- though I doubt you'll see the difference. but you may want to keep the numpy array a shape that makes sense:

mybuffer = numpy.empty((screenSizeX,screenSizeY,3), dtype=numpy.uint8)

now you can get at certain pixels this way:

mybuffer[x_pixel, y_pixel, :]

or get a row with:

mybuffer[:, y_pixel, :]

IMPORTANT NOTE: I think numpy's indexing convention may be (height, width) rather than (width, height) -- so do some experiments to be sure.

More numpy info:

Another nifty feature of numpy is "views" these allow you to make multiple numpy arrays that all share the same data buffer:

In [48]: # first some arbitrary data:

In [49]: a = np.arange(w*h*3, dtype=np.uint8)

In [50]: a_2d = a.reshape((w,h,3))

In [51]: # set a pixel:

In [52]: a_2d[0,0,:] = (0,0,0)

In [53]: # the original array is also changed:

In [54]: a[0]
Out[54]: 0

In [55]: a[0:3]
Out[55]: array([0, 0, 0], dtype=uint8)

In [27]: # set a row white:

In [28]: a_2d[:, 3, :] = (255,255,255)

In [29]: # create a "custom dtype" to represent an RGB pixel:

In [35]: rgb_dt = np.dtype([('R',np.uint8),('G',np.uint8),('B',np.uint8),])

In [36]: rgb_dt
Out[36]: dtype([('R', '|u1'), ('G', '|u1'), ('B', '|u1')])

# create a view with that dtype:

In [40]: a_rgb = a.view(dtype = rgb_dt).reshape((w,h))

In [41]: a_rgb.shape
Out[41]: (5, 7)

In [42]: # now set a pixel white:

In [43]: a_rgb[3,4] = (255, 255, 255)

In [44]: a_rgb
Out[44]:
array([[(0, 0, 0), (3, 4, 5), (6, 7, 8), (255, 255, 255), (12, 13, 14),
         (15, 16, 17), (18, 19, 20)],
        [(21, 22, 23), (24, 25, 26), (27, 28, 29), (255, 255, 255),
         (33, 34, 35), (36, 37, 38), (39, 40, 41)],
        [(42, 43, 44), (45, 46, 47), (48, 49, 50), (255, 255, 255),
         (54, 55, 56), (57, 58, 59), (60, 61, 62)],
        [(63, 64, 65), (66, 67, 68), (69, 70, 71), (255, 255, 255),
         (255, 255, 255), (78, 79, 80), (81, 82, 83)],
        [(84, 85, 86), (87, 88, 89), (90, 91, 92), (255, 255, 255),
         (96, 97, 98), (99, 100, 101), (102, 103, 104)]],
       dtype=[('R', '|u1'), ('G', '|u1'), ('B', '|u1')])

In [46]: # get just the green:

In [47]: a_rgb['G']
Out[47]:
array([[ 0, 4, 7, 255, 13, 16, 19],
        [ 22, 25, 28, 255, 34, 37, 40],
        [ 43, 46, 49, 255, 55, 58, 61],
        [ 64, 67, 70, 255, 255, 79, 82],
        [ 85, 88, 91, 255, 97, 100, 103]], dtype=uint8)

# And lots more nifty operations!

Sorry to get carried away -- I am a bit of a numpy evangelist!

-Chris

···

On 11/5/11 7:49 AM, andrea console wrote:

Finally this is the code that do exactly what I want:

         screenSizeX, screenSizeY = wx.DisplaySize()
         bitmap=wx.EmptyBitmap(screenSizeX ,screenSizeY,-1)
         memory=wx.MemoryDC()
         memory.SelectObject(bitmap)
         memory.Blit(0,0,screenSizeX,screenSizeY,dcScreen,0,0)
         mybuffer = numpy.zeros((screenSizeX*screenSizeY*3), dtype=numpy.uint8)
         bitmap.CopyToBuffer(mybuffer, wx.BitmapBufferFormat_RGB)
         print mybuffer[3*(actualX+screenSizeX*actualY)],
mybuffer[3*(actualX+screenSizeX*actualY)+1],
mybuffer[3*(actualX+screenSizeX*actualY)+2]

Just a final question: is this the most efficient option? I mean: do I
really need to "Blit" to a bitmap or I can use a more straightforward
way?

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov