wx.Refresh create a lot of threads on Mac

I have an application which runs on Mac ever since 10.6. But recently I found a strange thing.
python2.7 +wxpython 3.0
My application calls wx.Refresh(Erasebackground=False) , based on a timer (50ms), and this function kind of cause problem. (Timer doesn’t matter here, because I adjusted timer from 50ms to 1s and no improvement on anything.)

I ran the app on 4 different MacBooks,
on two of them(call them good Mac), wx.Refresh() generate 3-4 extra threads in addition to some basic threads while wx.Refresh() is disabled intentionally. App runs very well.
on the other two Mac (call them bad Mac), wx.Refresh() generates 12-13 extra threads. Without doubt, my app runs badlly on these two Mac. And on these mac, CPU usage of my app goes up to as high as 125%.

The sample information from activity monitor shows the 12-13 extra threads are generated by api from wxpython: wxWindow::MacDoRedraw(long)

  • 2062 NSDisplayCycleObserverInvoke (in AppKit) + 155 [0x7fff37ee7cc8]
  • 2061 __NSWindowGetDisplayCycleObserverForDisplay_block_invoke (in AppKit) + 646 [0x7fff37eec1ef]
  • ! 2061 -[NSWindow displayIfNeeded] (in AppKit) + 261 [0x7fff37eec39b]
  • ! 2061 -[NSView displayIfNeeded] (in AppKit) + 755 [0x7fff37eef213]
  • ! 2055 -[NSView _oldDisplayRectIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:] (in AppKit) + 2126 [0x7fff37ef305b]
  • ! : 1985 -[NSView _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] (in AppKit) + 3503 [0x7fff37ef5ebb]
  • ! : | 1735 -[NSView _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] (in AppKit) + 3503 [0x7fff37ef5ebb]
  • ! : | + 1723 -[NSView _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] (in AppKit) + 2143 [0x7fff37ef596b]
  • ! : | + ! 1723 -[NSView _drawRect:clip:] (in AppKit) + 1103 [0x7fff37ef7444]
  • ! : | + ! 1723 _NSViewDrawRect (in AppKit) + 139 [0x7fff37f0807b]
  • ! : | + ! 1723 wxOSX_drawRect(NSView*, objc_selector*, CGRect) (in libwx_osx_cocoau-3.0.0.3.0.dylib) + 358 [0x1058d0376]
  • ! : | + ! 1722 wxWidgetCocoaImpl::drawRect(void*, NSView*, void*) (in libwx_osx_cocoau-3.0.0.3.0.dylib) + 690 [0x1058d06f2]
  • ! : | + ! : 1720 wxWindow::MacDoRedraw(long) (in libwx_osx_cocoau-3.0.0.3.0.dylib) + 857 [0x1057ef819]

I discussed with Apple and they don’t think it’s hardware or OS related and suggest to find help in library’s forum. Does anybody have the same issue? Can anybody give me some hint on how to improve my app on bad Macs?

Do you have the same issue with wxPython 4.0 or 4.1 builds? Can you create a small, runnable test application that demonstrates the issue?

Hi Robin,
I wrote a simple program to run on 3 Macs. My own app keep reading video from usb and draw on screen. To simplify the case, the test program simply switching pictures with eraseBackground=True.

On two Macs(mid 2015 and mid 2019 ), it did 2.7 EVT_PAINT /second, and many other heavy duty programs running meanwhile.
One the other Mac (late 2018), it did 1.7 EVT_PAINT/second, with no other programs running at all.

The last one is much slower than the first two. That kind of explains why my app runs slow on the last Mac.
(Threads are similar on two Mac, so I believe number of thread is not the cause of bad performance for my app.)

I wonder how is dc.drawBitmap() implemented, how it’s interacting with Apple APIs, etc. I really don’t understand why the newer Mac with better CPU and GPU runs worse than an old Mac.

#!/usr/bin/env python
import wx
class VideoCanvas(wx.Panel):
         buffer=None
        def __init__(self,
                     parent,
                     ID=-1,
                     pos=wx.DefaultPosition,
                     size=(1980,1200),
                     style=wx.NO_FULL_REPAINT_ON_RESIZE | wx.NO_BORDER):
                wx.Panel.__init__(self,parent,ID,pos,size,style)
                self.Bind(wx.EVT_PAINT,self.onPaint)
                self.timerObject = wx.Timer(self)
                self.Bind(wx.EVT_TIMER, self.update,self.timerObject)
                self.timerObject.Start(50)
               #pixel of my jpgs.
                self.x=3000
                self.y=4000
                self.onSize(None)
                self.i=0
        def draw(self,dc):
                 if self.i:
                      self.i=0  
                      bm = wx.Image("test0.jpg", wx.BITMAP_TYPE_ANY).ConvertToBitmap()
               else:
                        self.i=1
                        bm = wx.Image("test1.jpg", wx.BITMAP_TYPE_ANY).ConvertToBitmap()
                dc.DrawBitmap(bm,5,5)

        def update(self,event):
                dc=wx.MemoryDC()
                dc.SelectObject(self.buffer)
                self.draw(dc)
                self.Refresh(eraseBackground=True)

        def onPaint(self,event):
                dc=wx.BufferedPaintDC(self,self.buffer)

        def onSize(self,event):
                self.buffer = wx.EmptyBitmap(self.x, self.y)


class MainFrame(wx.Frame):
        def __init__(self,
                     parent=None,
                     ID=-1,
                     title='',
                     pos=wx.DefaultPosition,
                     size=(1980,1200),
                     style=wx.DEFAULT_FRAME_STYLE):
                wx.Frame.__init__(self,parent,ID,title,pos,size,style)
                self.canvas=VideoCanvas(self)

if __name__ == '__main__':
        app = wx.App(False)  
        app.SetAppName("dover")
        frame = MainFrame()
        frame.Show(True)     # Show the frame.
        app.MainLoop()

You are spending too much effort and overhead on double buffering that you do not need to do. Since the only things that need to be drawn in your sample is to draw the background and draw current image, there really isn’t anything that needs buffering. And since the bitmap is expected to fill almost all of the widget, you can pretty much think of the current image as the buffer. Plus, on Macs the UI is already buffering everything, so you’re basically doing double-double buffering.

I’ve tweaked your sample a little, pasted in below. The main things to notice are the use of SetBackgroundStyle, and using IsDoubleBuffered to determine what kind of DC to create. On my Mac I see the thread count mostly at 5 or 6, with occasional bumps to 8. A simple Hello World application will have 5 threads on macOS, so subtracting the base 5 I’m seeing just 0-3 extra threads. It says the same even if I reduce the timer to 20 ms. (All testing was done on wxPython 4.0.7.post2)

Another thing to be aware of is that just because you call Refresh doesn’t mean that it will immediately call theEVT_PAINT handler. It just tells the system something like “When it’s convenient for you, please let me repaint myself.” Consequently, the system may or may not have called your paint handler before the next time you call Refresh. In that case there may be only one paint for one or more calls to Refresh. There are lots of platform-specific complexities under the covers here, but the main thing to know is that the paint events happen when the system wants them or needs them to happen. Calling Refresh is just a request.

Here’s my updated version of your sample:

import wx

class VideoCanvas(wx.Panel):
    def __init__(self,
                 parent,
                 ID=-1,
                 pos=wx.DefaultPosition,
                 size=(1980,1200),
                 style=wx.NO_BORDER):
            wx.Panel.__init__(self,parent,ID,pos,size,style)
            self.Bind(wx.EVT_PAINT, self.onPaint)
            self.timerObject = wx.Timer(self)
            self.Bind(wx.EVT_TIMER, self.update, self.timerObject)
            self.timerObject.Start(50)
            self.i=0
            self.SetBackgroundStyle(wx.BG_STYLE_PAINT)
            self.update(None)

    def update(self, event):
        if self.i % 2 == 0:
            self.bmp = wx.Bitmap("test0.jpg", wx.BITMAP_TYPE_ANY)
        else:
            self.bmp = wx.Bitmap("test1.jpg", wx.BITMAP_TYPE_ANY)
        print(self.i)
        self.i += 1
        self.Refresh()

    def onPaint(self, event):
        if self.IsDoubleBuffered():
            dc = wx.PaintDC(self)
        else:
            dc = wx.BufferedPaintDC(self)

        # clear the full background
        dc.SetBackground(wx.Brush(self.GetBackgroundColour()))
        dc.Clear()

        # draw the current bitmap
        dc.DrawBitmap(self.bmp, 5, 5)

class MainFrame(wx.Frame):
    def __init__(self,
                 parent=None,
                 ID=-1,
                 title='',
                 pos=wx.DefaultPosition,
                 size=(1980,1200),
                 style=wx.DEFAULT_FRAME_STYLE):
        wx.Frame.__init__(self,parent,ID,title,pos,size,style)
        self.canvas=VideoCanvas(self)

if __name__ == '__main__':
    app = wx.App(False)
    app.SetAppName("dover")
    frame = MainFrame()
    frame.Show(True)     # Show the frame.
    app.MainLoop()

Hi Robin,
Your codes give me the idea of how to improv my app.
Different from my sample code, my real app is to update changed tiles , while keep unchanged part of the video on the screen. (An I-frame will cause a redraw of the whole screen, and for other frames, just update changed part. ) So I have to use double buffered.
I modified the sample to show my case more accurately.:

        def draw(self,dc):
               for line_by_line:
                     .......
                     # read  changed data from source and turn into bitmap
                     ........
                    dc.DrawBitmap(bm,5,5)

        def update(self,event):
                dc=wx.MemoryDC()
                dc.SelectObject(self.buffer)
                self.draw(dc)
                self.Refresh(eraseBackground=True)

        def onPaint(self,event):
                dc=wx.BufferedPaintDC(self,self.buffer)


According to the tutorial, BufferedPaintDC will draw to buffer first then draw buffer to screen which is supposed to be very fast. But my app runs very slow on new Mac (late 2018) and very good on an old Mac (mid 2015, which has fewer cores, older CPU and GPU). If I use wx.PaintDC to only draw the changed part of video on screen, it runs well on the late 2018 Mac. But if using MemoryDc+BufferedPaintDC to keep all data in buffer, it runs ok on old Mac, but very slow on new Mac. So this ensure me that it’s not my drawing procedure wrong, but somewhere in drawing buffer to screen that runs very slow on the new Mac. I suspected my new Mac has hardware issue, so did benchmark test on CPU,GPU and disk on late 2018 Mac, but the results look reasonable.
What else can cause a better Mac runs MemoryDc+BufferedPaintDC slower than an old Mac.

Thanks.

You are correct. When doing just partial updates to the window’s image then using a bitmap to collect the updates into a single image is the better approach. There are still a couple things that can help improve things however.

Since you are painting the whole window (using self.buffer) in the paint event, then there is no need for the eraseBackground=True. Also, you can combine the first two lines with dc = wx.MemoryDC(self.buffer). In the paint event handler you can also use a wx.PaintDC instead of the wx.BufferedPaintDC, which may save a step or two in the C++ code. Like this:

        def onPaint(self,event):
                dc = wx.PaintDC(self)
                dc.DrawBitmap(self.buffer, 0,0)

The next place to look for optimizations is probably in draw().

  • What kinds of things are you doing to transform the data into the bitmap tiles?

  • It’s not clear from the pseudo-code, but I assume that you are skipping the tiles that have no changes. (If not, then you should.)

  • You can save a lot of time by not drawing things to the buffer that are not visible, so you could try setting the size of the buffer image to the size of the widget. When the size or image offset changes you can change the size of the buffer and then fill in the newly exposed areas from the data.

  • Another possibility is to keep your raw image RGB data in a numpy array. That will probably be faster to update when new data arrives, and pulling out the subset needed for the visible portion of the widget and converting it to a bitmap will be faster that doing lots of tiles.

There are probably other possibilities, but those are the ones that come to mind now.