Embedded images using img2py: memory usage

wxPython Users
1

Hi everyone. I'm designing an interface that contains quite a few images (png
format) which I would like to embed in a .py module. I will be generating
the .py files using img2py. I've noticed a few odd things that I would
appreciate some clarification on.

I designed a small test script to display 1) an image string from the .py
file generated by img2py, and 2) the same png but imported as an image.
Here's what I've noticed. Using process explorer to monitor memory usage,
I've noticed that the GUI with the img2py image string uses FAR less memory
in the working set than directly importing the png. Here's some stats I
pulled:

GUI with image stream from img2py:
Private bytes: 15,660K, Working set: 2,396K (GUI maximized on screen)
Private bytes: 15,660K, Working set: 888K (GUI minimized)

GUI with image as png:
Private bytes: 15,576K, Working set: 21,300K (GUI maximized on screen)
Private bytes: 15,576K, Working set: 21,288 K (GUI minimized)

So my question is, why the differences in memory usage? Will I see an
improvement in start-up times using the img2py approach? Any downsides to
embedding images? Thanks

···

--
View this message in context: http://wxpython-users.1045709.n5.nabble.com/Embedded-images-using-img2py-memory-usage-tp4413728p4413728.html
Sent from the wxPython-users mailing list archive at Nabble.com.

2

Hi everyone. I'm designing an interface that contains quite a few images (png
format) which I would like to embed in a .py module. I will be generating
the .py files using img2py. I've noticed a few odd things that I would
appreciate some clarification on.

I designed a small test script to display 1) an image string from the .py
file generated by img2py, and 2) the same png but imported as an image.
Here's what I've noticed. Using process explorer to monitor memory usage,
I've noticed that the GUI with the img2py image string uses FAR less memory
in the working set than directly importing the png. Here's some stats I
pulled:

GUI with image stream from img2py:
Private bytes: 15,660K, Working set: 2,396K (GUI maximized on screen)
Private bytes: 15,660K, Working set: 888K (GUI minimized)

GUI with image as png:
Private bytes: 15,576K, Working set: 21,300K (GUI maximized on screen)
Private bytes: 15,576K, Working set: 21,288 K (GUI minimized)

So my question is, why the differences in memory usage?

Interesting... I would have expected the opposite since the Python string containing the image's bytes will remain in memory, at least until the module containing them is unloaded.

Perhaps something about loading the images from files is causing the app to retain the memory without releasing it back to the OS even though it is not in active use. IOW, the process is holding on to some of the unused memory in its heap for reuse later.

Will I see an
improvement in start-up times using the img2py approach?

That depends on a number of factors and AFAIK nobody has actually tested it. My gut feel is that in there can be some improvement depending on the nature of the images and how they are used. For example if you have a few dozen icons all img2py'd into a single python module then importing the module takes one access to the disk to load the image data into memory, but reading them from individual files means that there are a few dozen files being opened and read.

Any downsides to
embedding images? Thanks

The only one that's bugged me is when editing the images they have to run through img2py again and sometimes I forget. Related to that sometimes I get a contributed module with some embedded images and the contributor does not send the source images along with it.

···

On 5/20/11 3:58 PM, convoluted wrote:

--
Robin Dunn
Software Craftsman

3

Thanks Robin. I edited my original post if anyone is wondering, sorry if this
causes any confusion. The reason for the edit is that upon closer inspection
only process explorer is giving me the strange working set numbers. If I use
task manager I get very similar numbers for both MEM Usage and VM Size in
both scenarios, and from what I've read MEM Usage is analogous to the
Working Set and VM Size to Private Bytes.

Robin I'm hoping you could provide your thoughts on the slightly higher CPU
usage value for img2py scenario. Here's the context in which I'm using:

This is a small code fragment found within "Test.py"

From Images import *

def on_paint(self, event):
    self.dc.DrawBitmap(LoadImage("Frame"), self.x, self.y, True)

This is a small subset of the code in "Images.py"

from wx.lib.embeddedimage import PyEmbeddedImage

def LoadImage(name):
    bmp = eval(name).Bitmap
    return bmp

Frame = PyEmbeddedImage(
   
"iVBORw0KGgoAAAANSUhEUgAAACMAAAAjCAYAAAAe2bNZAAAACXBIWXMAAHUwAAB1MAHdM3LN"
   
"AAANHElEQVRYhZ1Ya5BcxXX+Tvft+9iZ2d157GMksbtaaVlEJEsruSJhEIEgITALJYydxFCg"
    ...
    ...

The way I've tested processor usage is to continually redraw a panel with
one larger image at around 10hz. If I use the png image then the processor
usage is around 5%, where using the image byte stream consumes around 10%
(using an older PC). Not a huge deal but I would have thought that decoding
the png then redrawing it to the panel would be more laborious. If I store
the call to LoadImage in self.storedImage first, then it decreases CPU
usage:

self.storedImage = LoadImage("Frame")
def on_paint(self, event):
    self.dc.DrawBitmap(self.storedImage, self.x, self.y, True)

As always, thanks for the help.

···

--
View this message in context: http://wxpython-users.1045709.n5.nabble.com/Embedded-images-using-img2py-practical-tp4413728p4416389.html
Sent from the wxPython-users mailing list archive at Nabble.com.

4

The way I've tested processor usage is to continually redraw a panel with
one larger image at around 10hz. If I use the png image then the processor
usage is around 5%, where using the image byte stream consumes around 10%
(using an older PC). Not a huge deal but I would have thought that decoding
the png then redrawing it to the panel would be more laborious.

The bytestream stored in the PyEmbeddedImage object is in PNG format, so it will still need to do that part of the conversion in either case. In addition the embedded image data will need to be basse64 decoded, although I would not have thought that would have made that big of a difference, but perhaps image file case is so much better because by the time you come around to load it the 2nd time it will be in your system's disk cache and so there is no disk overhead for any of the subsequent loads.

If I store
the call to LoadImage in self.storedImage first, then it decreases CPU
usage:

Yep, that is to be expected in either case.

···

On 5/22/11 1:43 AM, convoluted wrote:

--
Robin Dunn
Software Craftsman

5

Hi Robin. So I decided to run a few more tests to test the direct comparison
between implementations. I've built two dedicated scripts each which clears
the panel and redraw the image to the screen every 10ms using a wxTimer.
Strangely enough, with these test scripts I can't seem to raise the
processor usage above 0% in either case, even if I set the refresh rate to
1ms! So there must be something strange that is using CPU cycles in my other
interface that I'm designing, I'll have to look into that a bit more. I'm
attaching the two scripts, complete with the png and the img2py version of
the same png (it's the wxPython logo by the way).

So at this point I can't really find any major downside to using the img2py
implementation, other than the hassle of replacing the images as you've
mentioned. But the ability to secure the image files outweighs the
inconvenience for me at least.

Thanks again for the help

http://wxpython-users.1045709.n5.nabble.com/file/n4417807/using_stream.py
using_stream.py
http://wxpython-users.1045709.n5.nabble.com/file/n4417807/using_png.py
using_png.py
http://wxpython-users.1045709.n5.nabble.com/file/n4417807/logo.png logo.png
http://wxpython-users.1045709.n5.nabble.com/file/n4417807/images.py
images.py

···

--
View this message in context: http://wxpython-users.1045709.n5.nabble.com/Embedded-images-using-img2py-practical-tp4413728p4417807.html
Sent from the wxPython-users mailing list archive at Nabble.com.

6

Trying to get the OS to do just about anything at a 10 ms rate, much less at 1 ms, isn’t going to happen as fast as that.Refreshes are going to get ignored. Also, all code is going to be running out of the OS cache RAM, and certainly not your hard drive.