wx.FileSystemWatcher Only Showing Access Events

wxPython Users
#1

First of all, wxPython is just awesome. Many thanks to the developers and the community. Loving it so much.

I’m writing an IDE using a custom terminal emulator for the editor panels (this way people can use Emacs / Vim / whatever with all their features), and writing a custom 3D OpenGL graph / tree viewer to replace the standard file / directory tree display. (https://github.com/avose/GLShell)

So, I want to be able to detect when files are added / removed so that I can update the 3D graph of all the files / dirs. However, it seems like I’m only getting events for file accesses. I’m sure I’m just using it wrong, but not seeing the issue yet. Any help would be appreciated.

When I do an “ls” in the watched directory, I see the access print. When I open a file, I see the access print. However, when I “touch foo” or “rm foo”, I don’t see any prints for that.

https://docs.wxpython.org/wx.FileSystemWatcher.html
https://docs.wxpython.org/wx.FileSystemWatcherEvent.html
https://docs.wxpython.org/wx.FSWFlags.enumeration.html

The code that’s doing this follows:

class glsDirTree(glsFSObj, wx.FileSystemWatcher):
    def __init__(self, path, settings):
        path = os.path.abspath(path)
        glsFSObj.__init__(self, path)
        wx.FileSystemWatcher.__init__(self)
        self.settings = settings
        self.files = []
        self.dirs  = []
        self.graph = fdpGraph(self.settings)
        for root, dirs, files in os.walk(self.path):
            root_node = glsDir(root)
            for name in files:
                node = glsFile(os.path.join(root, name))
                self.files.append(node)
                self.graph.add_node(node)
                self.graph.add_edge( (root_node,node) )
            for name in dirs:
                node = glsDir(os.path.join(root, name))
                self.dirs.append(node)
                self.graph.add_node(node)
                self.graph.add_edge( (root_node,node) )
        self.thread = glsFDPThread(self.settings, self.graph, speed=0.01)
        self.Bind(wx.EVT_FSWATCHER, self.OnFSChange)
        self.thread.start()
        self.AddTree(self.path)
        return
    def OnFSChange(self, event):
        change_type = event.GetChangeType()
        if change_type == wx.FSW_EVENT_CREATE:
            print('create')
        elif change_type == wx.FSW_EVENT_DELETE:
            print('delete')
        elif change_type == wx.FSW_EVENT_WARNING:
            print('warn')
        elif change_type == wx.FSW_EVENT_ERROR:
            print('error')
        elif change_type == wx.FSW_EVENT_ACCESS:
            print('access')
        else:
            print(change_type)
        return

wxPython 4.2.1
Python 3.10.12

#2

Hi and welcome to Discuss wxPython.

I’ve not used wx.FileSystemWatcher before, so I tried a few experiments.

My test code doesn’t use any OpenGL for simplicity.

At first I got the same results as you - only ‘access’ events were triggered. However, when I created a subdirectory and started making changes in that subdirectory, I started getting more events being triggered.

Here is some output:

access /home/richardt/Development/python/exp/hci/wxpython/file_system_watcher/target_dir/subdir/
access /home/richardt/Development/python/exp/hci/wxpython/file_system_watcher/
rename /home/richardt/Development/python/exp/hci/wxpython/file_system_watcher/target_dir/subdir/b.txt
create /home/richardt/Development/python/exp/hci/wxpython/file_system_watcher/target_dir/subdir/a.txt
attrib /home/richardt/Development/python/exp/hci/wxpython/file_system_watcher/target_dir/subdir/a.txt
access /home/richardt/Development/python/exp/hci/wxpython/file_system_watcher/target_dir/subdir/
access /home/richardt/Development/python/exp/hci/wxpython/file_system_watcher/target_dir/subdir/
access /home/richardt/Development/python/exp/hci/wxpython/file_system_watcher/target_dir/subdir/
delete /home/richardt/Development/python/exp/hci/wxpython/file_system_watcher/target_dir/subdir/c.txt

It seems strange that it’s not triggering these events in the top-level directory?

Tested using: Python 3.10.12 + wxPython 4.2.1 gtk3 (phoenix) wxWidgets 3.2.2.1 on Linux Mint 21.2

Example code:

import os
import wx

class MyFrame(wx.Frame):

    def __init__(self, parent):
        wx.Frame.__init__(self, parent)
        self.SetTitle("File System Watcher")
        self.SetSize((100, 100))
        self.Bind(wx.EVT_FSWATCHER, self.OnFSChange)
        wx.CallAfter(self.addWatcher)

    def addWatcher(self):
        path = os.path.abspath("target_dir")
        self.watcher = wx.FileSystemWatcher()
        self.watcher.AddTree(path)
        self.watcher.SetOwner(self)

    def OnFSChange(self, event):
        change_type = event.GetChangeType()
        path = event.GetPath()

        if change_type == wx.FSW_EVENT_CREATE:
            change = 'create'
        elif change_type == wx.FSW_EVENT_DELETE:
            change = 'delete'
        elif change_type == wx.FSW_EVENT_WARNING:
            change = 'warn'
        elif change_type == wx.FSW_EVENT_ERROR:
            change = 'error'
        elif change_type == wx.FSW_EVENT_ACCESS:
            change = 'access'
        elif change_type == wx.FSW_EVENT_ATTRIB:
            change = 'attrib'
        elif change_type == wx.FSW_EVENT_RENAME:
            change = 'rename'
        elif change_type == wx.FSW_EVENT_MODIFY:
            change = 'modify'
        else:
            change = str(change_type)
        print(change, path)


if __name__ == "__main__":
    app = wx.App()
    frame = MyFrame(None)
    frame.Show()
    app.MainLoop()
#3

Ah, thanks much, RichardT. I was wondering if you would be the one to reply. I’ve been lurking a while before signing up, and you’re usually very helpful. :grinning:

I’m glad that I’m not the only one seeing the surprising behavior. I guess a solution here would be to use both “.Add(path)” and “.AddTree(path)”. That seems to work for existing directories. Then, whenever a new directory is created, do “.Add(new_path)”. Then it seems to give all the events I’m interested in. – Seems like a workable solution.

Thanks so much for your reply. :+1: I would have figured out what it was doing eventually, but your post was very useful.

#4

More surprises… Your output seems to indicate that it’s giving you the path of created files:

create /home/richardt/Development/python/exp/hci/wxpython/file_system_watcher/target_dir/subdir/a.txt

My system is returning the path of the directory in which the new file is created, not the path of the new file itself from “.GetPath()”.

prompt> touch foo
create /home/aaron/projects/GLShell/GLShell

prompt> touch too.txt
create /home/aaron/projects/GLShell/GLShell

Ah, I tested your example code, and your code is giving the full path to the new files, while my code is not. Maybe it’s something I’m doing with inheriting from “wx.FileSystemWatcher”…

#5

Ahhh, ok. I only get the file path including the file name if the file is created in one of the sub-directories… I’m using “.Add()” and “.AddTree()” as well, and when I get an event from creating a file in the parent directory, it doesn’t show the file name…

I also switched so I’m no longer inheriting from wx.FileSystemWatcher, so my code is much closer to yours now.

This is… strange.

I mean, at this point, the only way I see to get around this would be to use “os.path.dirname(path)”, and then rescan the directory for new / removed files myself, as I cannot rely on the event to give the full filename.

#6

Yes, that’s the same as what I get.

What’s even stranger is that when I change something in target_dir’s parent directory, then it triggers events also! (and they show the full path).

access /home/richardt/Development/python/exp/hci/wxpython/file_system_watcher/target_dir
create /home/richardt/Development/python/exp/hci/wxpython/file_system_watcher/b.txt
attrib /home/richardt/Development/python/exp/hci/wxpython/file_system_watcher/b.txt

I did find an open issue related to FileSystemWatcher in the wxWidgets GitHub repository which mentioned that directory pathnames should end with a trailing slash:

However, that didn’t seem to make any difference, so perhaps it’s only relevant in C++.

#7

Wow, you’re the best!! :flushed:

I’ve only done a little testing, but adding the “/” to the end of the path is working well for me now.

Yeah, all seems good. Now, to see if I can use something like os.path.join(path, “”) to get the OS independent “/” / “\” added on there. – and, yeah, os.path.join(path,"") does seem to do what I want.

And that’s fixing the issue with the parent directory entirely, so using “.AddTree()” is all I need here. I also don’t seem to need to add the newly created directories, as the “.AddTree()” seems to know the new directory was created and starts watching that as well. So… you fixed it!

#8

Annnnd, still has an issue.

For example:

mkdir -p foo/bar/baz/how/now/brown/cow

Only shows a create event for “foo”. If I make them one at a time, it all shows up.

Maybe I can’t use this, as it’s not reliable… Perhaps the only solution is to look for any creation or deletion event anywhere in the directory tree, and then manually rescan the entire thing looking for changes.

This is a real shame, however. Maybe I can go over to the wxWidgets GitHub page and file a bug report.

Maybe this makes some sense… The “foo” directory is created, and isn’t yet registered somehow as something who’s contents need to be watched. That takes some time to register… by the time it’s registered, the child directories are already created, so the changes there haven’t been seen. I guess it’s not too surprising this happens… Though, the wxWidgets code should be able to detect this case and make it all work.

I created an issue for this on the wxWidgets GitHub. Maybe someone gets to it, but quite possibly not. The solution is not a one line fix or anything, they probably need to scan the newly-created directory and send events for the children, probably not getting fixed anytime soon if ever.

I’ll use the workaround of scanning the directory myself.

#9

Hi avose,

Have you come across the watchdog python package?

pip install watchdog

Below is a very quick hack trying to use it with wxPython. It appears to work, including detecting the creation of a directory hierarchy (mkdir -p).

One problem with the example is that the Handler class is separate from the UI and I haven’t thought of a way to integrate it - any ideas?

import os
import wx
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class MyFrame(wx.Frame):

    def __init__(self, parent):
        wx.Frame.__init__(self, parent)
        self.SetTitle("File System Watcher")
        self.SetSize((100, 100))
        self.Bind(wx.EVT_CLOSE, self.OnClose)
        wx.CallAfter(self.addWatchdog)

    def addWatchdog(self):
        path = os.path.abspath("target_dir/")
        self.observer = Observer()
        event_handler = Handler()
        self.observer.schedule(event_handler, path, recursive=True)
        self.observer.start()

    def OnClose(self, _event):
        self.observer.stop()
        self.observer.join()
        self.Destroy()


class Handler(FileSystemEventHandler):

    @staticmethod
    def on_any_event(event):
        if event.is_directory:
            print("Watchdog received directory event - %s" % event.src_path)

        elif event.event_type == 'created':
            print("Watchdog received created event - % s." % event.src_path)

        elif event.event_type == 'deleted':
            print("Watchdog received deleted event - % s." % event.src_path)

        elif event.event_type == 'modified':
            print("Watchdog received modified event - % s." % event.src_path)

        elif event.event_type == 'moved':
            print("Watchdog received moved event - % s\n -> %s." % (event.src_path, event.dest_path))

        else:
            print("Unhandled event - '%s'" % event.event_type)


if __name__ == "__main__":
    app = wx.App()
    frame = MyFrame(None)
    frame.Show()
    app.MainLoop()
2 Likes
#10

Below is a version where the Handler class is able to pass the watchdog event to the frame.

Because the watchdog runs in child threads, the Handler uses wx.CallAfter() to ensure the UI code is executed in the main thread.

import os
import wx
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler


class MyFrame(wx.Frame):

    def __init__(self, parent):
        wx.Frame.__init__(self, parent)
        self.SetTitle("File System Watcher")
        self.SetSize((100, 100))
        self.Bind(wx.EVT_CLOSE, self.OnClose)
        wx.CallAfter(self.addWatchdog)


    def addWatchdog(self):
        path = os.path.abspath("target_dir/")
        self.observer = Observer()
        event_handler = Handler(self)
        self.observer.schedule(event_handler, path, recursive=True)
        self.observer.start()


    def OnClose(self, _event):
        self.observer.stop()
        self.observer.join()
        self.Destroy()


    @staticmethod
    def OnWatchdog(event):
        print("'%s' - %s" % (event.event_type, event.src_path))
        if hasattr(event, 'dest_path'):
            print(" -> %s" % event.dest_path)


class Handler(FileSystemEventHandler):
    def __init__(self, parent):
        self.parent = parent


    def on_any_event(self, event):
        wx.CallAfter(self.parent.OnWatchdog, event)


if __name__ == "__main__":
    app = wx.App()
    frame = MyFrame(None)
    frame.Show()
    app.MainLoop()
#11

Very interesting solution there, many thanks! :thinking: :slightly_smiling_face:

For now, I have written code which rescans everything any time a filesystem event is received. This does turn out to be a bit slow, however. – Your solution using the watchdog library is going to perform much better. I’ll give it a try and see how that works out.

So far I’m using only numpy and wxPython in my IDE project. Maybe this case will warrant pulling in another library. Poor wxPython must be sad to see this task being done by a different library. :cry:

#12

well, this sort of task is by far more suited for Python than wx, not only because of the manpower available (there are other features like this) :wink:

#13

Oh, don’t get me wrong; it wasn’t a subtle dig at wxPython or anything. I just feel like I’m cheating on poor wx. Don’t want it to get lonely… :frowning: It’s OK, buddy, I’ll always love you, wx, you sweet py, you! :sob:

#14

I found some time to try this out, and I have to say that it works just perfectly! Thanks much for the example code, very nice.