Open file without locking it

Tom12 · June 29, 2009, 3:32pm

Does anyone know how to open a file without "locking" it (python/
wxPython).

I currently have a small "tailview" application which allows you to
tail a logfile (and do some fancy stuff with it). The logfile is
created by an external application and is rotated after a while.

Unfortunately the file-rotation (i.e. the external application is
renaming the file from logfile.txt to logfile.1, after that a new file
logfile.txt is created) is blocked because my application still has it
open (in read-only mode).

Suggestions?

Best regards,
Tom.

Mike_Driscoll0 · June 29, 2009, 6:15pm

Hi,

Does anyone know how to open a file without "locking" it (python/
wxPython).

I currently have a small "tailview" application which allows you to
tail a logfile (and do some fancy stuff with it). The logfile is
created by an external application and is rotated after a while.

Unfortunately the file-rotation (i.e. the external application is
renaming the file from logfile.txt to logfile.1, after that a new file
logfile.txt is created) is blocked because my application still has it
open (in read-only mode).

Suggestions?

Best regards,
Tom.

If one process is using a file, the file gets locked. What you seem to
want to do is allow two processes to access the same file at the same
time. This is dangerous as it can corrupt files or just cause general
program instability.

See this fun thread for more info (although its topic is a little
different than yours): http://mail.python.org/pipermail/python-list/2009-March/702979.html

Anyway, what you'll probably want to do is do a quick open/read/close
process every so often or make a copy of the file and follow it,
refreshing it as needed.

One of the uber-Pythonistas may have other ideas...

- Mike

···

On Jun 29, 10:32 am, Tom <tcler...@gmail.com> wrote:

Stef_Mientki1 · June 29, 2009, 7:12pm

Mike Driscoll wrote:

Hi,

Does anyone know how to open a file without "locking" it (python/
wxPython).

I currently have a small "tailview" application which allows you to
tail a logfile (and do some fancy stuff with it). The logfile is
created by an external application and is rotated after a while.

Unfortunately the file-rotation (i.e. the external application is
renaming the file from logfile.txt to logfile.1, after that a new file
logfile.txt is created) is blocked because my application still has it
open (in read-only mode).

Suggestions?

Best regards,
Tom.

If one process is using a file, the file gets locked. What you seem to
want to do is allow two processes to access the same file at the same
time. This is dangerous as it can corrupt files or just cause general
program instability.

See this fun thread for more info (although its topic is a little
different than yours): http://mail.python.org/pipermail/python-list/2009-March/702979.html

Anyway, what you'll probably want to do is do a quick open/read/close
process every so often or make a copy of the file and follow it,
refreshing it as needed.

One of the uber-Pythonistas may have other ideas...

Well I'm certainly not a uber-...
but we use the following trick for fast data aquistion ( > 200 kbytes /
sec),
between a program-1 and a program-2 (under windows, because pipes works
bad there
Data has to be transported from program-1 to program-2:

Program-1 writes it's data to the first file,
until a certain length is reached ( e.g. 500 kB).
Then program-1 switches to the second file and so on.

Program-2 starts with deleting both files and then launches the Program-1.
Program-2 opens the first file for readonly,
checks if it's length has changed, and if so reads the data.
And you can imagine the rest of the story.

Works like a charm.

cheers,
Stef

···

On Jun 29, 10:32 am, Tom <tcler...@gmail.com> wrote:

- Mike
>

Dave_Angel1 · June 30, 2009, 12:34am

Tom wrote:

Does anyone know how to open a file without "locking" it (python/
wxPython).

I currently have a small "tailview" application which allows you to
tail a logfile (and do some fancy stuff with it). The logfile is
created by an external application and is rotated after a while.

Unfortunately the file-rotation (i.e. the external application is
renaming the file from logfile.txt to logfile.1, after that a new file
logfile.txt is created) is blocked because my application still has it
open (in read-only mode).

Suggestions?

Best regards,
Tom.

If you're on Windows, I suspect that the win32com addon package will do it for you. Try looking at

win32file.CreateFile(), which lets you specify file sharing modes. Default sharing mode for builtin open() is exclusive.

DaveA

Tom12 · June 30, 2009, 7:49am

Unfortunately, the program which creates the logfile is not under my
control, so I cannot change the behavior on that side.

One option as mentioned by Mike would be to:
* Check filesize
* If filesize changed: Open file;read new data;Close file
* else: do nothing

I'm not sure what the penalty would be in doing this open/close all
the time.

Tom.

···

On Jun 30, 2:34 am, Dave Angel <da...@ieee.org> wrote:

Tom wrote:
> Does anyone know how to open a file without "locking" it (python/
> wxPython).

> I currently have a small "tailview" application which allows you to
> tail a logfile (and do some fancy stuff with it). The logfile is
> created by an external application and is rotated after a while.

> Unfortunately the file-rotation (i.e. the external application is
> renaming the file from logfile.txt to logfile.1, after that a new file
> logfile.txt is created) is blocked because my application still has it
> open (in read-only mode).

> Suggestions?

> Best regards,
> Tom.

If you're on Windows, I suspect that the win32com addon package will do
it for you. Try looking at

win32file.CreateFile(), which lets you specify file sharing modes. Default sharing mode for builtin open() is exclusive.

DaveA

Stefanie_Luck · June 30, 2009, 8:51am

Hi!

I added a help to my application according to the Html help Demo of the wxPython book. Unfortunately after compiling with py2exe it's can't find the .hhp file.

Here's my code:

...
# Help
def OnHelp(self, event):
self.InitHelp()

    def InitHelp(self):
        def _addBook(filename):
            if not self.help.AddBook(filename):
                wx.MessageBox("Unable to open: " + filename,
                              "Error", wx.OK|wx.ICON_EXCLAMATION)

self.help = wx.html.HtmlHelpController()

_addBook("help.hhp")
self.OnShowHelpContents()

def OnShowHelpContents(self):
self.help.DisplayContents()
...

And here my setup.py:

from distutils.core import setup
import py2exe

#python setup.py py2exe

setup(
    windows = [
        {
            "script": "RNAiscan_GUI.py", ### Main Python script
            "icon_resources": [(0, "siFi.ico")] ### Icon to embed into the PE file.
        }
    ],
)

Does someone know what could be the problem? If I run the script from the IDLE everything is ok.
All files related to the help are correctly installed in the installation folder and should be found...

Regards
Stefanie

Andrea_Gavana · June 30, 2009, 9:05am

Hi,

2009/6/30 Stefanie Lück:

Hi!

I added a help to my application according to the Html help Demo of the
wxPython book. Unfortunately after compiling with py2exe it's can't find the
.hhp file.

Here's my code:

...
# Help
def OnHelp(self, event):
self.InitHelp()

def InitHelp(self):
def _addBook(filename):
if not self.help.AddBook(filename):
wx.MessageBox("Unable to open: " + filename,
"Error", wx.OK|wx.ICON_EXCLAMATION)
   self\.help = wx\.html\.HtmlHelpController\(\)

   \_addBook\(&quot;help\.hhp&quot;\)
   self\.OnShowHelpContents\(\)
def OnShowHelpContents(self):
self.help.DisplayContents()
...

And here my setup.py:

from distutils.core import setup
import py2exe

#python setup.py py2exe

setup(
windows = [
{
"script": "RNAiscan_GUI.py", ### Main Python
script
"icon_resources": [(0, "siFi.ico")] ### Icon to embed into
the PE file.
}
],
)

Does someone know what could be the problem? If I run the script from the
IDLE everything is ok.
All files related to the help are correctly installed in the installation
folder and should be found...

I don't think this is strictly related to py2exe, as the hhp files
have nothing to do with Python. In which folder does your executable
live? Is it the same folder as the hhp file? How are you starting the
executable?

Andrea.

"Imagination Is The Only Weapon In The War Against Reality."
http://xoomer.alice.it/infinity77/

Steven_Sproat · June 30, 2009, 9:14am

Andrea Gavana wrote:

Hi,

2009/6/30 Stefanie Lück:

Hi!

I added a help to my application according to the Html help Demo of the
wxPython book. Unfortunately after compiling with py2exe it's can't find the
.hhp file.

Here's my code:

...

I don't think this is strictly related to py2exe, as the hhp files
have nothing to do with Python. In which folder does your executable
live? Is it the same folder as the hhp file? How are you starting the
executable?

Andrea.

You could try something like

path = os.path.split(os.path.abspath(sys.argv[0]))[0]
_file = os.path.join(path, 'help-folder', 'help.hhp')

        if os.path.exists(_file):
            self.help = wx.html.HtmlHelpController()
            self.help.AddBook(_file)

Stefanie_Luck · June 30, 2009, 9:14am

Everything is in one folder. The helpfiles (including hhp) and my
executable. The myprogram.exe don't work directly from the dist folder
created by py2exe and not either after building an installer via InnoSetup.
But my myprogram.py in the dist folder can find the hhp file, if I run it.

It's very suspicious...

···

----- Original Message -----
From: "Andrea Gavana" <andrea.gavana@gmail.com>
To: <wxPython-users@googlegroups.com>
Sent: Tuesday, June 30, 2009 11:05 AM
Subject: [wxPython-users] Re: wxHtml Help and py2exe

Hi,

2009/6/30 Stefanie Lück:

Hi!

I added a help to my application according to the Html help Demo of the
wxPython book. Unfortunately after compiling with py2exe it's can't find
the
.hhp file.

Here's my code:

...
# Help
def OnHelp(self, event):
self.InitHelp()

def InitHelp(self):
def _addBook(filename):
if not self.help.AddBook(filename):
wx.MessageBox("Unable to open: " + filename,
"Error", wx.OK|wx.ICON_EXCLAMATION)

self.help = wx.html.HtmlHelpController()

_addBook("help.hhp")
self.OnShowHelpContents()

def OnShowHelpContents(self):
self.help.DisplayContents()
...

And here my setup.py:

from distutils.core import setup
import py2exe

#python setup.py py2exe

setup(
windows = [
{
"script": "RNAiscan_GUI.py", ### Main Python
script
"icon_resources": [(0, "siFi.ico")] ### Icon to embed into
the PE file.
}
],
)

Does someone know what could be the problem? If I run the script from the
IDLE everything is ok.
All files related to the help are correctly installed in the installation
folder and should be found...

I don't think this is strictly related to py2exe, as the hhp files
have nothing to do with Python. In which folder does your executable
live? Is it the same folder as the hhp file? How are you starting the
executable?

Andrea.

"Imagination Is The Only Weapon In The War Against Reality."
http://xoomer.alice.it/infinity77/

Enrico · June 30, 2009, 9:26am

Probably it's a path problem. Are you sure the help file is in the right place. I think in the setup.py you should add this:

data_files=[('',['myhelp.hhp'])]

and when you try to load it be sure to be pointing to the right path.

_addBook("help.hhp") doesn't say where the file should be, add a full path.

best regards, Enrico

···

At 10.51 30/06/2009 +0200, you wrote:

And here my setup.py:

from distutils.core import setup
import py2exe

#python setup.py py2exe

setup(
    windows = [
        {
            "script": "RNAiscan_GUI.py", ### Main Python
script
            "icon_resources": [(0, "siFi.ico")] ### Icon to embed into
the PE file.
        }
    ],
)

Does someone know what could be the problem? If I run the script from the
IDLE everything is ok.
All files related to the help are correctly installed in the installation
folder and should be found...

Regards
Stefanie

Stefanie_Luck · June 30, 2009, 9:31am

I forgot to say that I tried this. It's the same problem.

···

----- Original Message -----
From: "Steven Sproat" <sproaty@gmail.com>
To: <wxPython-users@googlegroups.com>
Sent: Tuesday, June 30, 2009 11:14 AM
Subject: [wxPython-users] Re: wxHtml Help and py2exe

Andrea Gavana wrote:

Hi,

2009/6/30 Stefanie Lück:

Hi!

I added a help to my application according to the Html help Demo of the
wxPython book. Unfortunately after compiling with py2exe it's can't find
the
.hhp file.

Here's my code:

...

I don't think this is strictly related to py2exe, as the hhp files
have nothing to do with Python. In which folder does your executable
live? Is it the same folder as the hhp file? How are you starting the
executable?

Andrea.

You could try something like

path = os.path.split(os.path.abspath(sys.argv[0]))[0]
_file = os.path.join(path, 'help-folder', 'help.hhp')

        if os.path.exists(_file):
            self.help = wx.html.HtmlHelpController()
            self.help.AddBook(_file)

Enrico · June 30, 2009, 9:47am

I forgot to say that I tried this. It's the same problem.

In this case a logging function could help to
discover the problem. I'd write down the file path.
Regards, Enrico

···

At 11.31 30/06/2009 +0200, you wrote:

----- Original Message -----
From: "Steven Sproat" <sproaty@gmail.com>
To: <wxPython-users@googlegroups.com>
Sent: Tuesday, June 30, 2009 11:14 AM
Subject: [wxPython-users] Re: wxHtml Help and py2exe

Andrea Gavana wrote:
> Hi,
>
> 2009/6/30 Stefanie Lück:
>
>> Hi!
>>
>> I added a help to my application according to the Html help Demo of the
>> wxPython book. Unfortunately after compiling with py2exe it's can't find
>> the
>> .hhp file.
>>
>> Here's my code:
>>
>> ...
>>
>
> I don't think this is strictly related to py2exe, as the hhp files
> have nothing to do with Python. In which folder does your executable
> live? Is it the same folder as the hhp file? How are you starting the
> executable?
>
> Andrea.
>
>
You could try something like

        path = os.path.split(os.path.abspath(sys.argv[0]))[0]
        _file = os.path.join(path, 'help-folder', 'help.hhp')

        if os.path.exists(_file):
            self.help = wx.html.HtmlHelpController()
            self.help.AddBook(_file)

Stefanie_Luck · June 30, 2009, 10:13am

Just to make it clear:
I'm sure for the files and paths. It doesn't matter whether I give the full
path or only the file. The path of the file is where it should be (after
printing the path out). And in addition, I have many other files used in my
app in the same folder and there're all found.
So it's definitive not a path problem.
Including the files to the py2exe also don't help...

Any ideas?

···

----- Original Message -----
From: <ricercar@infinito.it>
To: <wxPython-users@googlegroups.com>
Sent: Tuesday, June 30, 2009 11:26 AM
Subject: [wxPython-users] Re: wxHtml Help and py2exe

At 10.51 30/06/2009 +0200, you wrote:

And here my setup.py:

from distutils.core import setup
import py2exe

#python setup.py py2exe

setup(
    windows = [
        {
            "script": "RNAiscan_GUI.py", ### Main
Python
script
            "icon_resources": [(0, "siFi.ico")] ### Icon to embed
into
the PE file.
        }
    ],
)

Does someone know what could be the problem? If I run the script from the
IDLE everything is ok.
All files related to the help are correctly installed in the installation
folder and should be found...

Regards
Stefanie

Probably it's a path problem. Are you sure the help file is in the
right place. I think in the setup.py you should add this:

data_files=[('',['myhelp.hhp'])]

and when you try to load it be sure to be pointing to the right path.

_addBook("help.hhp") doesn't say where the file should be, add a full
path.

best regards, Enrico

>

Andrea_Gavana · June 30, 2009, 10:33am

Hi,

2009/6/30 Stefanie Lück:

Just to make it clear:
I'm sure for the files and paths. It doesn't matter whether I give the full
path or only the file. The path of the file is where it should be (after
printing the path out). And in addition, I have many other files used in my
app in the same folder and there're all found.
So it's definitive not a path problem.
Including the files to the py2exe also don't help...

Any ideas?

Yes, please post a small sample app that reproduces the problem so
that Windows users with py2exe can test it:

http://wiki.wxpython.org/MakingSampleApps

Andrea.

"Imagination Is The Only Weapon In The War Against Reality."
http://xoomer.alice.it/infinity77/

Stefanie_Luck · June 30, 2009, 12:14pm

Hi!

By generating a test example I found out that it works! The only change was
that I compilied into a new folder. All the rest is the same (folder content
etc.). I just copied the files to a new folder and made a new exe file,
instead to overwrite the existing dist folder...Strange but at least
solved...

Thanks for your help!
Stefanie

···

----- Original Message -----
From: "Andrea Gavana" <andrea.gavana@gmail.com>
To: <wxPython-users@googlegroups.com>
Sent: Tuesday, June 30, 2009 12:33 PM
Subject: [wxPython-users] Re: wxHtml Help and py2exe

Hi,

2009/6/30 Stefanie Lück:

Just to make it clear:
I'm sure for the files and paths. It doesn't matter whether I give the
full
path or only the file. The path of the file is where it should be (after
printing the path out). And in addition, I have many other files used in
my
app in the same folder and there're all found.
So it's definitive not a path problem.
Including the files to the py2exe also don't help...

Any ideas?

Yes, please post a small sample app that reproduces the problem so
that Windows users with py2exe can test it:

http://wiki.wxpython.org/MakingSampleApps

Andrea.

"Imagination Is The Only Weapon In The War Against Reality."
http://xoomer.alice.it/infinity77/