Cross-platform distribution

wxPython Users
1

Hi,

I'm trying to figure out a better way to ship extra so called
"resource" files with my wxPython-based application such that they can
be accessed in a cross-platform manner. I'm aware this is a common
problem, but google hasn't suggested a standard solution.

I currently use distutils rather clumsily and place these files in the
data_files array:
  data_files=[
    ('share/pallavi', [
        'config/pallavi_config.py',
        'config/pallavi128.xpm'
        ]),
    ('share/pallavi/lexing_languages', map(lambda x:
"config/lexing_languages/%s" % x ,filter(lambda x: x.endswith(".py"),
os.listdir('config/lexing_languages')))),
  ],

This installs the relevant files to /usr/share/pallavi or
/usr/local/share/pallavi under Linux systems (assuming the installer
doesn't pass strange prefixes to setup.py), and to
C:\Python2x\share\pallavi\ under Windows (with the same assumption). I
don't know what happens on a Mac and I'm not in a position to worry
about it until a Mac user becomes interested in my software or I buy
one (its on my wish list!)

I don't know how to reliably access these paths under wxPython.
wx.StandardPaths returns /usr/share/pallavi under Linux, which is
exactly what I need. However, under Windows, it currently returns
C:\Python25\lib\site-packages\wx-2.8-msw-unicode\wx and the
documentation tells me it should actually be returning C:\Program
Files\Pallavi\.

So I need to know how to either:
a) get wxPython to return C:\Python2x\share\pallavi or whichever
similar path is related to the Python installation OR
b) get wxPython to return C:\Program Files\Pallavi instead of that
site-packages url AND get my setup.py to install the data_files to
C:\Program Files\Pallavi instead of C:\Python2x\share\pallavi under
Windows.

Like I say, I assume this is a common problem, but I'm a bit clumsy
when it comes to Windows installations... I didn't expect as many
Windows users interested in my project as has turned out to be the
case. I apologize if this has been asked before; I couldn't find any
references.

Thanks for any insight,
Dusty

2

Dusty Phillips wrote:

I'm trying to figure out a better way to ship extra so called
"resource" files with my wxPython-based application such that they can
be accessed in a cross-platform manner.

I currently use distutils rather clumsily and place these files in the
data_files array:

distutils is really designed for distributing packages, rather than applications. That is, essentially libraries that folks will use to build their own apps.

For distributing applications, you're better off with Py2exe on Windows, Py2App on OS-X (those two are pretty compatible), and probably a regular old tarball for Linux (or maybe PyInstaller). If you want to make things really nice for Linux users, you'd build an rpm, and deb, and .....

Anyway, at least py2app and Py2exe have, I think, a standard place to put data files.

Another option is to make your "resources" python files instead. img2py is a script distributed with wxPython that turns various image types into python source code, and virtually any other type of data could be serialized in a similar way, I'm sure. I like this approach, it makes things pretty darn clean.
> I

don't know what happens on a Mac and I'm not in a position to worry
about it until a Mac user becomes interested in my software

I'm an interested Mac user -- at least if you're still working on meeting Robin's "give me a way for me to leave Emacs" challenge.

I don't know how to reliably access these paths under wxPython.
wx.StandardPaths returns /usr/share/pallavi under Linux ...

Right. The problem is that wx.StandardPaths and distutils are totally unrelated. I suppose you could use wx.StandardPaths in your setup.py file, as there wouldn't be much point in anyone running it without wx installed anyway.

if you do want to use distutils, this may be helpful:

http://wiki.python.org/moin/DistutilsInstallDataScattered

(the Wiki seems to be down for the moment, so I'm not sure how helpful that may be, I found the link in the matplotlib setup.py -- it uses a lot of data files)

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

3

Dusty Phillips wrote:

Hi,

I'm trying to figure out a better way to ship extra so called
"resource" files with my wxPython-based application such that they can
be accessed in a cross-platform manner. I'm aware this is a common
problem, but google hasn't suggested a standard solution.

I currently use distutils rather clumsily and place these files in the
data_files array:
  data_files=[
    ('share/pallavi', [
        'config/pallavi_config.py',
        'config/pallavi128.xpm'
        ]),
    ('share/pallavi/lexing_languages', map(lambda x:
"config/lexing_languages/%s" % x ,filter(lambda x: x.endswith(".py"),
os.listdir('config/lexing_languages')))),
  ],

This installs the relevant files to /usr/share/pallavi or
/usr/local/share/pallavi under Linux systems (assuming the installer
doesn't pass strange prefixes to setup.py), and to
C:\Python2x\share\pallavi\ under Windows (with the same assumption). I
don't know what happens on a Mac and I'm not in a position to worry
about it until a Mac user becomes interested in my software or I buy
one (its on my wish list!)

I don't know how to reliably access these paths under wxPython.
wx.StandardPaths returns /usr/share/pallavi under Linux, which is
exactly what I need. However, under Windows, it currently returns
C:\Python25\lib\site-packages\wx-2.8-msw-unicode\wx and the
documentation tells me it should actually be returning C:\Program
Files\Pallavi\.

It's actually based on the location of the main wx resource instance, which on Windows is linked to the wx DLL. The assumption is that either wx will be staticly linked to a C++ app, or the C++ app will have it's own resource instance that includes wx's. Neither of those is true for wxPython so it falls back to the DLL.

So I need to know how to either:
a) get wxPython to return C:\Python2x\share\pallavi or whichever
similar path is related to the Python installation OR

On Windows you can use something like os.path.join(sys.prefix, 'share/pallavi), although I'm not sure what that will do with a py2exe's app... On the other hand, a py2exe'd app would probably do the right thing with wx.StandardPaths because the location of the wx DLL file would be in the Program Files subdir then (or wherever the app was installed.)

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

4

> I'm trying to figure out a better way to ship extra so called
> "resource" files with my wxPython-based application such that they can
> be accessed in a cross-platform manner.

> I currently use distutils rather clumsily and place these files in the
> data_files array:

distutils is really designed for distributing packages, rather than
applications. That is, essentially libraries that folks will use to
build their own apps.

For distributing applications, you're better off with Py2exe on Windows,
Py2App on OS-X (those two are pretty compatible), and probably a regular
old tarball for Linux (or maybe PyInstaller). If you want to make things
really nice for Linux users, you'd build an rpm, and deb, and .....

Anyway, at least py2app and Py2exe have, I think, a standard place to
put data files.

Hmm. I've been reluctant to use Py2exe, as I understand that it ships
wxPython with the app. Maybe that's normal for Windows, but it seems
more sensible to me to download and install it once separately than to
have a large download with each app. I will experiment with it a bit
though. It looks like Py2exe is actually a distutils module, rather
than distinct from it, so I should theoretically be able to use it
without rearranging my linux installation stuff.

Another option is to make your "resources" python files instead. img2py
is a script distributed with wxPython that turns various image types
into python source code, and virtually any other type of data could be
serialized in a similar way, I'm sure. I like this approach, it makes
things pretty darn clean.
> I
> don't know what happens on a Mac and I'm not in a position to worry
> about it until a Mac user becomes interested in my software

I'm an interested Mac user -- at least if you're still working on
meeting Robin's "give me a way for me to leave Emacs" challenge.

Yup. I have your "multiple top-level windows" in svn... that's why I'm
trying to figure out how to make a more user-friendly release now. I
haven't got "multiple views of the same buffer" yet; that's for 0.7.
I've had it before though, but took it out when I switched to wx.aui;
it shouldn't be too hard.

So if I created a py2exe version of distutils you would probably be
able to make it work using py2app?

> I don't know how to reliably access these paths under wxPython.
> wx.StandardPaths returns /usr/share/pallavi under Linux ...

Right. The problem is that wx.StandardPaths and distutils are totally
unrelated. I suppose you could use wx.StandardPaths in your setup.py
file, as there wouldn't be much point in anyone running it without wx
installed anyway.

if you do want to use distutils, this may be helpful:

Distutils/Cookbook/InstallDataScattered - Python Wiki

(the Wiki seems to be down for the moment, so I'm not sure how helpful
that may be, I found the link in the matplotlib setup.py -- it uses a
lot of data files)

still down, I'll take a look later.

Thanks!

Dusty

5

> So I need to know how to either:
> a) get wxPython to return C:\Python2x\share\pallavi or whichever
> similar path is related to the Python installation OR

On Windows you can use something like os.path.join(sys.prefix,
'share/pallavi),

It looks like that would also work under Linux, since sys.prefix
returns /usr on my system. I think its possible for the situation to
arise where sys.prefix returns /usr and the files are installed to
/usr/local though. Would wx.StandardPaths know to do the right thing
in this case?

Probably my best bet is to look in wx.StandardPaths and use sys.prefix
as a fallback. That should cover 90% of the cases, and the other 10%
would probably be created by distributors who know how to patch it.

Thanks,
Dusty

6

You should consider this a feature. This way each app can have its own version of wxPython. You also don't end in dll hell or need to keep the wx version separate. After all wxPython has only a payload of maybe 10 MB or so. Even if you install 50 apps which use wxPython this takes up 0.5gb which is neglible imo.
The windows model handles this just a lot better. The worst thing ever about linux is package hell.

-Matthias

···

Am 01.10.2007, 22:21 Uhr, schrieb Dusty Phillips <buchuki@gmail.com>:

Hmm. I've been reluctant to use Py2exe, as I understand that it ships
wxPython with the app. Maybe that's normal for Windows, but it seems
more sensible to me to download and install it once separately than to
have a large download with each app. I will experiment with it a bit

7

Dusty Phillips wrote:

I think its possible for the situation to
arise where sys.prefix returns /usr and the files are installed to
/usr/local though.

Indeed, desirable! /usr is for stuff installed by the system vendor, and /usr/local is for stuff installed by the user. until you get official Ubuntu (or whatever) packages of your app, then python will probably be installed in /usr/, and your app should be installed in /usr/local/.

Probably my best bet is to look in wx.StandardPaths and use sys.prefix
as a fallback.

I think wxStandardPaths is the way to go -- but I'm not sure how to get distutils to match.

Hmm. I've been reluctant to use Py2exe, as I understand that it ships
wxPython with the app.

Yes, that's part of the point. I think you could build a py2exe executable that relies on the system python+wxPython (I know you can with py2app on OS-X), but I don't think that's the right thing to do for an app like Pallavi: IIUC, you want it to be a general purpose editor -- a user may or may not even have Python installed. Even if it is being used for wxPython development, the version of wxPython that the user is building their apps with may not be the same one that Pallavi is using. indeed, the same applies to Python itself, and any number of other modules.

In short, a Pallavi user should not need to know or care that Pallavi is written in Python (or which version, etc) That's what Py2exe is for, make a Python app a "normal" Windows app -- not an extension for python.

It looks like Py2exe is actually a distutils module, rather
than distinct from it, so I should theoretically be able to use it
without rearranging my linux installation stuff.

Yup, that's True.

Linux is a different story than Windows or OS-X. In the Linux world, people are used to software depending on an assortment of libraries and what not. There are also nifty package managers that handle all that for the user -- so it's less likely that you'll need to us PyInstaller on Linux (even if you do, there are still a bunch of lower-level dependencies!)

If you do depend on a system-installed Python and wxPython, make sure you specify versions so that the right one gets used if the user has multiple versions installed. i.e.:

/usr/bin/env python2.4

import wxversion
wxversion.select("2.8")

Yup. I have your "multiple top-level windows" in svn...

cool! Are you in touch with Rob of the Peppy Project? -- that one looked very promising, but he got a bit sidetracked when it was ALMOST ready for daily use....

I
haven't got "multiple views of the same buffer" yet; that's for 0.7.

That's a bit less critical, but I'm glad you have it on your radar.

So if I created a py2exe version of distutils you would probably be
able to make it work using py2app?

Sure, I'd give it a try. I can also help some with making it a good Mac citizen -- accepting Drag and Dropped files -- that sort of thing.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

8

> Yup. I have your "multiple top-level windows" in svn...

cool! Are you in touch with Rob of the Peppy Project? -- that one looked
very promising, but he got a bit sidetracked when it was ALMOST ready
for daily use....

Oof! Body blow! :slight_smile:

I wouldn't say sidetracked exactly; I discovered I had to do a whole
lot of refactoring, which is not the same as a bunch of sexy features.
The problem was that I didn't want to invest a lot of time writing a
variety of major modes with the architecture that I had in place. It
would have involved a lot of rewriting, because I knew there were some
fundamental architectural issues, and discovered even more while I was
"sidetracked" writing my music player daemon major mode.

But, anyway, the architecture is cleaner now. I'm releasing 0.6.5
today, actually, and 0.6.6 will be concentrating on a python editing
mode! I've also given up on trac plugins and have gone with Yapsy
plugins so I can have versioning.

I haven't forgotten about editing code, but my goals are more than
just an editor, and the refactoring was important. I want it to be
easy for other people to contribute major modes without having to poke
around in the guts of the code.

So, while 0.6.5 won't be that useful in itself, it will set the
groundwork for a useful 0.6.6.

Rob

9

Rob McMullen wrote:

Yup. I have your "multiple top-level windows" in svn...

cool! Are you in touch with Rob of the Peppy Project? -- that one looked
very promising, but he got a bit sidetracked when it was ALMOST ready
for daily use....

Oof! Body blow! :slight_smile:

Sorry, I didn't mean to be insulting -- just calling it as I saw it.

> while I was

"sidetracked" writing my music player daemon major mode.

Well, for someone that is looking for a code editor -- a music player mode really does look like a sidetrack!

But if it gets you a cleaner, more flexible architecture then great!

I want it to be
easy for other people to contribute major modes without having to poke
around in the guts of the code.

I think that is a key goal -- the easier it to contribute major modes the more you'll get them, and no one person is ever going to be able to write them all.

So, while 0.6.5 won't be that useful in itself, it will set the
groundwork for a useful 0.6.6.

I'm looking forward to it.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

10

Dusty Phillips wrote:

So if I created a py2exe version of distutils you would probably be
able to make it work using py2app?

Here is a simple example of supporting both from the same setup.py:

http://svn.wxwidgets.org/viewvc/wx/wxPython/branches/WX_2_8_BRANCH/samples/doodle/setup.py?revision=48418&view=markup

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

11

Christopher Barker wrote:

Rob McMullen wrote:

Yup. I have your "multiple top-level windows" in svn...

cool! Are you in touch with Rob of the Peppy Project? -- that one looked
very promising, but he got a bit sidetracked when it was ALMOST ready
for daily use....

Oof! Body blow! :slight_smile:

Sorry, I didn't mean to be insulting -- just calling it as I saw it.

> while I was

"sidetracked" writing my music player daemon major mode.

Well, for someone that is looking for a code editor -- a music player mode really does look like a sidetrack!

Or maybe a "soundtrack" ? :wink:

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!