How to have a window transparent to mouse/keyboard events

Newbie Help
#1

Essentially I would like to draw a transparent overlay overtop of what’s on screen, but have mouse/keyboard events go to the window behind the wx window and not capture anything.

Edit: on Windows, as its likely OS-specific

It’s quite possible wxPython is just entirely the wrong thing to do this in, but if that’s so does anyone have recommendations for what to use instead?

It doesn’t need any of the various event or window handling features, all I need is the ability to semi-transparently draw things over the screen, while not preventing the program running from seeing my mouse/keyboard inputs. Googling how to draw an overlay with python has seemed to largely point towards wxPython which is why I thought I’d start here, but my initial blind hacking around hasn’t got me anywhere so far

Edit: thinking about it, the mouse events I want to send through are all going to be generated by the program and not provided by a user so I could in theory minimise the overlay, emulate a click and then bring it back up, hopefully without too much flickering. It’s not ideal but if all else fails it could work.

#2

Hi @purple_pixie. Could you provide the “big picture” of what you’re trying to accomplish here to avoid a XY problem? That may help someone to help you better. :slight_smile:

#3

Sure :slight_smile:

The program is an AI to play a computer game, using a lot of computer vision to determine what’s in the scene and what information it needs to get out of it.

While it’s doing feature detection etc I would like to be able to display bounding boxes etc overtop of features it detects in the game window, but do so in a way that doesn’t prevent the game from having focus (so it shows me mouseover highlights etc) or receiving the simulated mouse events I need to send to it to play it.

Currently I’m settling for a smaller window re-drawing a scaled down capture of the game screen with features drawn onto it, but outside of the area where I might need to click, but it’s not my ideal solution if this idea could be made to work.

#4

I am not really sure what the details are, but this might work:

For the screen overlay, this is probably what you were talking about in your OP, and it seems to work on Windows:
https://web.archive.org/web/20091215052811/http://www.michaelfogleman.com/2009/12/drawing-on-the-windows-desktop-using-python-and-wxpython/

You could try using that.

Another idea would be to go with a variant of your idea about minimizing the overlay window, etc, etc.

I might try creating a transparent window that fills the screen (How to create a transparent frame (Phoenix) - wxPyWiki) and draws your stuff on it (to be an overlay), then when a click happens, minimize the window and focus the other window.

Of course, the only way I think this could work is if the event is different than a click down (e.g: on click down, you could focus the other window and it would recieve the click up event and other events after that.)

Maybe someone else can help here as well with a better idea. :slight_smile:

#5

That’s a handy link and I’ll definitely fiddle around with it, cheers :slight_smile:

Though I suspect given how it draws directly onto the other running program that would mess with future screen captures which it has to do regularly.

I think the game wants to be catching mouse downs not just ups, and also needs to register when I move the mouse over something so I can’t easily just hide it when I down. Though if I’m doing hiding I can just do that by posting a “please hide now” event from the AI thread before it tries to do a click, it’s just a bit flickery and non-ideal.

The alternative I have looked at briefly but not put a lot of time into making it would would be using a shaped overlay window which is exactly and only the shape of the bitmap I’m trying to draw, which should significantly reduce the chance of that window getting in the way of my mouse events.

#6

In that case, your idea there may be best. :grin: