Ubuntu and wxpython creating a package

wxPython Users
1

I have several wxpython apps that run on both Windows and MAc, and not they want them packaged
for Ubuntu 11.04. I have set up wxpython on an Ubuntu system and have the apps running great from
the command line in a terminal window. What I don't understand is how to make a setup.py to
build the wxpython app, and then what I have to do to make a Ubuntu package, they like the
way that wxpyton is installed where you point to a private archive and then install. I also have to
be able to create a folder and copy data files to the folder during install.

Could someone please point me to an example that does this in a simple straight forward manner,
as we may have over a dozen different apps to make packages for.

Thanks
Phil

Phil Schloss

2

There's lots of good info here: http://tinyurl.com/3th78h8

···

On 7/25/11 5:52 PM, Phil Schloss wrote:

I have several wxpython apps that run on both Windows and MAc, and not
they want them packaged
for Ubuntu 11.04. I have set up wxpython on an Ubuntu system and have
the apps running great from
the command line in a terminal window. What I don't understand is how to
make a setup.py to
build the wxpython app, and then what I have to do to make a Ubuntu
package, they like the
way that wxpyton is installed where you point to a private archive and
then install. I also have to
be able to create a folder and copy data files to the folder during
install.

Could someone please point me to an example that does this in a simple
straight forward manner,
as we may have over a dozen different apps to make packages for.

--
Robin Dunn
Software Craftsman

3

Hi Phil,

Packaging Python applications is a bit of a mess in some ways and there are lots of options with different pros and cons. One of the issues is the end users. Are they developers? Experienced Python users? Ordinary software users? People installing their own software? Administrators installing for others (who might have very strong opinions on The One Correct Way)?

There are two main approaches - language specific packages (cross platform e.g. Ruby gems) or platform-specific/distro-specific packages (dmgs for Macs, msis/zipped exes for Windows, debs for Ubuntu, rpms for Fedora etc). For ordinary end users I think the best approach is to give them the sorts of packages they are used to, which are the platform-specific ones.

For my own wxPython application, SOFA Statistics (http://www.sofastatistics.com) I made a deb package for Ubuntu/debian, and a tar.gz file for other Linux distros. I just recently managed to completely automate the process so it is pretty convenient now. I have heard of various tools to help make deb packages but there seemed to be problems (anyone else having a better experience?). I finally made a break through with a ShowMeDo video on making debs (http://showmedo.com/videotutorials/video?name=linuxJensMakingDeb). The process described in that involved lots of manual steps (copying files, adding text, running commands etc) but it worked. I now use a Python script to do that all for me. It is about 200 lines long and would require modification but the packages seem to work in the wild and I’ve been pumping out releases for a couple of years. I can help with specifics if you follow that path.

In my view, this is not a Solved Problem in general, even though I have finally figured out something satisfactory for myself. BTW how do you package your Mac apps? I use Platypus to make the bundle, Iceberg to make the mkpg, and a simple manual process to make the dmg. And I use NSIS for making Windows exes.

All the best, Grant

4

I use py2app to make the bundle -- it can be a bit of a pain, but it shares a lot of config with py2exe (which I use on Windows), and is under active development at the moment.

It also can make a "full-fledged" app - supporting drag and drop, etc. file type associations, etc.

I'd be interested in more about Platypus, though. It looks like it does not bundle the interpreter, so you'd have to use one of the apple-supplied ones, and I don't see how it would bundle up extra packages, etc. Are you doing all that by hand?

For delivery, I haven't had the need for a mkpg, so I just put the app bundle in a dmg -- a simple script does that for me.

You also might want to look at the Task Coach project -- I think he's got a nice complete set of installer scripts.

-Chris

···

On 7/26/11 1:07 PM, pyGrant wrote:

BTW how do you
package your Mac apps? I use Platypus to make the bundle, Iceberg to
make the mkpg, and a simple manual process to make the dmg.

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

5

Hi Chris,

BTW how do you
package your Mac apps? I use Platypus to make the bundle, Iceberg to
make the mkpg, and a simple manual process to make the dmg.

I use py2app to make the bundle -- it can be a bit of a pain, but it shares a lot of config with py2exe (which I use on Windows),

I have used pyinstaller to make a single executable for other projects but that is not what I use NSIS for. The installer exe it makes installs python, wxPython, all the other packages such as Numpy, and puts my application python modules into the correct folders etc. The user is given various options too e.g. which folder to install to. See ShowMeDo Video Tutorials | showmedo.com

and is under active development at the moment.

It also can make a "full-fledged" app - supporting drag and drop, etc. file type associations, etc.

I'd be interested in more about Platypus, though. It looks like it does not bundle the interpreter,

What do you mean by the interpreter? Do you mean Python?

so you'd have to use one of the apple-supplied ones, and I don't see how it would bundle up extra packages, etc. Are you doing all that by hand?

I set things up once only in Iceberg and then just update the version number and do a build. I make several subpackages, and then a master package which installs everything. Occasionally I have to add or delete a module from the setup but that is a once-off configuration.

For delivery, I haven't had the need for a mkpg, so I just put the app bundle in a dmg -- a simple script does that for me.

That sounds like a good approach where it is possible.

You also might want to look at the Task Coach project -- I think he's got a nice complete set of installer scripts.

I see they use hdiutil to make the dmgs - I think I will use that to remove a few manual steps. Thanks.

All the best, Grant

···

On 27/07/11 08:35, Christopher Barker wrote:

On 7/26/11 1:07 PM, pyGrant wrote:

-Chris

6

I use py2app to make the bundle -- it can be a bit of a pain, but it
shares a lot of config with py2exe (which I use on Windows),

I have used pyinstaller to make a single executable for other projects
but that is not what I use NSIS for. The installer exe it makes installs
python, wxPython, all the other packages such as Numpy, and puts my
application python modules into the correct folders etc.

Does this do a totally stand-alone install? or does it put in in the "usual" place -- i.e. C:\PYTHON27 ?

I'd hate to do that later.

I'd be interested in more about Platypus, though. It looks like it
does not bundle the interpreter,

What do you mean by the interpreter? Do you mean Python?

yes, exactly.

I set things up once only in Iceberg and then just update the version
number and do a build.

what is iceberg?

I make several subpackages, and then a master
package which installs everything. Occasionally I have to add or delete
a module from the setup but that is a once-off configuration.

but anyway -- what gets installed, where? With py2app, everything you need (the interpreter, all modules an packages you need, as *.so files, any other data files you've specified, all go into the .app bundle, so it looks exactly like a regular old mac app (though a big one, to be sure)

-Chris

···

On 7/26/11 3:13 PM, Grant Paton-Simpson wrote:

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

7

Hi Chris

I use py2app to make the bundle -- it can be a bit of a pain, but it
shares a lot of config with py2exe (which I use on Windows),

I have used pyinstaller to make a single executable for other projects
but that is not what I use NSIS for. The installer exe it makes installs
python, wxPython, all the other packages such as Numpy, and puts my
application python modules into the correct folders etc.

Does this do a totally stand-alone install? or does it put in in the "usual" place -- i.e. C:\PYTHON27 ?

I'd hate to do that later.

It is just the standard Windows installer for Python 2.6. It can be skipped, though if already installed.

I'd be interested in more about Platypus, though. It looks like it
does not bundle the interpreter,

What do you mean by the interpreter? Do you mean Python?

yes, exactly.

See above

I set things up once only in Iceberg and then just update the version
number and do a build.

what is iceberg?

http://s.sudre.free.fr/Software/Iceberg.html

And it saved my bacon.

I make several subpackages, and then a master
package which installs everything. Occasionally I have to add or delete
a module from the setup but that is a once-off configuration.

but anyway -- what gets installed, where?

Numpy, matplotlib, Python 2.6 (unlike the already-installed version) etc plus all my own scripts, images, an SQLite database, css files etc. Under the sofastats folder, I have _internal, css, etc folders - just like on the Windows and Linux installations. Users can read the Python scripts and edit them too.

All the best, Grant

···

On 27/07/11 10:22, Christopher Barker wrote:

On 7/26/11 3:13 PM, Grant Paton-Simpson wrote:

With py2app, everything you need (the interpreter, all modules an packages you need, as *.so files, any other data files you've specified, all go into the .app bundle, so it looks exactly like a regular old mac app (though a big one, to be sure)

-Chris

8

Does this do a totally stand-alone install? or does it put in in the
"usual" place -- i.e. C:\PYTHON27 ?

I'd hate to do that later.

It is just the standard Windows installer for Python 2.6. It can be
skipped, though if already installed.

I suggest that that is a "bad idea" (tm)

It assumes that your users aren't using Python for anything else, or at least, that they aren't using any of the same packages as your code, but in different versions.

This used to be a big pain in the *$^%*& with Redhat Linux ()and maybe still is), 'cause if you upgraded the system python or some key packages, you'd break system utilities. This is also true for OS-X

For Windows, MS doesn't install Python or use it for anything, but maybe the user does, or some other software does (I know Arc GIS does, for instance)

So I think it's really better to provide your users with a completely independent python install -- I find py2exe works well, but there are a number of other alternatives.

I'd be interested in more about Platypus, though. It looks like it
does not bundle the interpreter,

what is iceberg?

WhiteBox - Iceberg 1.3.1

And it saved my bacon.

It looks handy, yes.

I make several subpackages, and then a master
package which installs everything. Occasionally I have to add or delete
a module from the setup but that is a once-off configuration.

but anyway -- what gets installed, where?

Numpy, matplotlib, Python 2.6 (unlike the already-installed version) etc
plus all my own scripts, images, an SQLite database, css files etc.
Under the sofastats folder, I have _internal, css, etc folders - just
like on the Windows and Linux installations. Users can read the Python
scripts and edit them too.

Again, where does all this get installed? if you're putting it in:

/Library/Frameworks/Python (e.g. where the python.org python goes)

I'd suggest you don't do that -- same conflicts as above.

py2app would not be ideal for you, though, if you want your users to have access to the scripts. Though you could probably hack something, where you had your app, with all that, and a separate scripts directory that you put the stuff you want your users to access in.

I wonder if one could create a virtualenv for your stuff, and install that, so you could use the standard python, but all your stuff would be independent...

hmmm.

-Chris

···

On 7/26/11 5:05 PM, Grant Paton-Simpson wrote:

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

9

Hi Chris,

  On

7/26/11 5:05 PM, Grant Paton-Simpson wrote:

      Does this do a totally stand-alone

install? or does it put in in the

      "usual" place -- i.e. C:\PYTHON27 ?




      I'd hate to do that later.

    It is just the standard Windows installer for Python 2.6. It can

be

    skipped, though if already installed.

  I suggest that that is a "bad idea" (tm)




  It assumes that your users aren't using Python for anything else,

or at least, that they aren’t using any of the same packages as
your code, but in different versions.

You can run SOFA on a system with multiple versions of Python - as

long as the launcher is explicit about the version being used. And
as long as the version being used has the relevant libraries e.g.
Numpy running. I agree this is not ideal. Here is where I have
explained it for users striking that problem:
Am I right in thinking it would need to be a completely different
Python, wxPython, numpy, matplotlib etc etc install? Might be worth
it to keep everything self-contained.
Made me think of this from A good chapter to read in the context of the original point of this
thread too.
All the best, Grant

···

http://www.sofastatistics.com/wiki/doku.php?id=help:will_not_start#wrong_version_of_python

  This used to be a big pain in the *$^%*& with Redhat Linux

()and maybe still is), 'cause if you upgraded the system python or
some key packages, you’d break system utilities. This is also true
for OS-X

  For Windows, MS doesn't install Python or use it for anything, but

maybe the user does, or some other software does (I know Arc GIS
does, for instance)

  So I think it's really better to provide your users with a

completely independent python install – I find py2exe works well,
but there are a number of other alternatives.

http://www.aosabook.org/en/packaging.html

There are two schools of thought when it comes to
installing
applications. The first, common to Windows and Mac OS X, is that
applications should be self-contained, and their installation
should
not depend on anything else. This philosophy simplifies the
management of applications: each application is its own
standalone
“appliance”, and installing and removing them should not disturb
the
rest of the OS. If the application needs an uncommon library,
that
library is included in the application’s distribution.

    The second school, which is the norm for Linux-based systems,

treats
software as a collection of small self-contained units called
packages . Libraries are bundled into packages, any
given
library package might depend on other packages.
Installing an application might involve finding and installing
particular versions of dozens of other libraries. These
dependencies
are usually fetched from a central repository that contains
thousands
of packages. This philosophy is why Linux distributions use
complex
package management systems like dpkg and RPM
to track
dependencies and prevent installation of two applications that
use
incompatible versions of the same library.

There are pros and cons to each approach.

  [emphasis added]

          I'd be interested in more about

Platypus, though. It looks like it

          does not bundle the interpreter,

      what is iceberg?

And it saved my bacon.

  It looks handy, yes.

        I make several subpackages, and then a

master

        package which installs everything. Occasionally I have to

add or delete

        a module from the setup but that is a once-off

configuration.

      but anyway -- what gets installed, where?

    Numpy, matplotlib, Python 2.6 (unlike the already-installed

version) etc

    plus all my own scripts, images, an SQLite database, css files

etc.

    Under the sofastats folder, I have _internal, css, etc folders -

just

    like on the Windows and Linux installations. Users can read the

Python

    scripts and edit them too.

  Again, where does all this get installed? if you're putting it in:




  /Library/Frameworks/Python (e.g. where the python.org python goes)




  I'd suggest you don't do that -- same conflicts as above.




  py2app would not be ideal for you, though, if you want your users

to have access to the scripts. Though you could probably hack
something, where you had your app, with all that, and a separate
scripts directory that you put the stuff you want your users to
access in.

  I wonder if one could create a virtualenv for your stuff, and

install that, so you could use the standard python, but all your
stuff would be independent…

  hmmm.




  -Chris

http://s.sudre.free.fr/Software/Iceberg.html

10

I use py2exe for windows, and py2app for mac. Wish there was something like that to make debs for ubuntu.
Any thing you can email we would sure help. I have looked at that video but am still confused.

Thanks
Phil

···

At 03:07 PM 7/26/2011, pyGrant wrote:

Hi Phil,

Packaging Python applications is a bit of a mess in some ways and there are lots of options with different pros and cons. One of the issues is the end users. Are they developers? Experienced Python users? Ordinary software users? People installing their own software? Administrators installing for others (who might have very strong opinions on The One Correct Way)?

There are two main approaches - language specific packages (cross platform e.g. Ruby gems) or platform-specific/distro-specific packages (dmgs for Macs, msis/zipped exes for Windows, debs for Ubuntu, rpms for Fedora etc). For ordinary end users I think the best approach is to give them the sorts of packages they are used to, which are the platform-specific ones.

For my own wxPython application, SOFA Statistics (http://www.sofastatistics.com) I made a deb package for Ubuntu/debian, and a tar.gz file for other Linux distros. I just recently managed to completely automate the process so it is pretty convenient now. I have heard of various tools to help make deb packages but there seemed to be problems (anyone else having a better experience?). I finally made a break through with a ShowMeDo video on making debs (ShowMeDo Video Tutorials | showmedo.com). The process described in that involved lots of manual steps (copying files, adding text, running commands etc) but it worked. I now use a Python script to do that all for me. It is about 200 lines long and would require modification but the packages seem to work in the wild and I've been pumping out releases for a couple of years. I can help with specifics if you follow that path.

In my view, this is not a Solved Problem in general, even though I have finally figured out something satisfactory for myself. BTW how do you package your Mac apps? I use Platypus to make the bundle, Iceberg to make the mkpg, and a simple manual process to make the dmg. And I use NSIS for making Windows exes.

All the best, Grant

11

Yes, py2app, py2exe and similar tools will gather everything that the application needs at runtime into a single bundle, with options you can somewhat control how those things are organized. Sometimes you need to help it a bit to include things that it misses, or to not include some things that it shouldn't, but the tool will do most of the heavy lifting for you.

For Macs py2app organizes it into a .app bundle that the user normally sees as if it was a single file, and can simply drag it out of your disk image and into their applications folder or wherever they would like to put it. There is usually no need to make an Installer package for it unless you need to do something like install or change system files or run a script that modifies things outside of the application bundle. For Windows py2xe makes a folder full of files and you can use InnoSetup or similar to bundle together the files into the typical self-installer executable.

···

On 7/26/11 7:21 PM, Grant Paton-Simpson wrote:

So I think it's really better to provide your users with a completely
independent python install -- I find py2exe works well, but there are
a number of other alternatives.

Am I right in thinking it would need to be a completely different
Python, wxPython, numpy, matplotlib etc etc install? Might be worth it
to keep everything self-contained.

--
Robin Dunn
Software Craftsman

12

You can run SOFA on a system with multiple versions of Python - as long
as the launcher is explicit about the version being used.

multiple versions of python (i.e. 2.6 and 2.7) are not usually a problem -- on all systems I know of, they get installed independently. while typing "python" at a prompt will get you only the default, there are ways to get the one you want -- full path or "python2.6" or something.

However, the problem comes in when your users may need different versions of packages, i.e. numpy1.5 vs. numpy1.6 -- then things get ugly. wxPython (and a few others) have a way top manage versions (wxversion), but there is no standard way to do it in python, and most packages have no way to handle multiple versions installed. And when you have things like numpy, MPL, scipy -- they all have to be compatible.

help:will_not_start [SOFA Statistics]

this only helps with the python version -- not package versions. This whole mess is why virtual_env was invented, as well as py2exe and friends.

So I think it's really better to provide your users with a completely
independent python install -- I find py2exe works well, but there are
a number of other alternatives.

Am I right in thinking it would need to be a completely different
Python, wxPython, numpy, matplotlib etc etc install? Might be worth it
to keep everything self-contained.

yup.

Made me think of this from Redirecting…

right -- that link makes it clear -- on OS-X ans Windows there is no package manager, so you really want to keep your application independent.

Even on Linux, since python has no package versioning system, you have the same problems. For some reason the core Python devs have never seen this as an issue, but really, having no package versioning is like having shared libraries with no version information -- i.e. "dll hell". The result is that you need to use the python equivalent of static linking.

Note: if you want your users to have a regular python interpreter available, you might want to check out bb_freeze -- it provides some neat features like that.

-Chris

···

On 7/26/11 7:21 PM, Grant Paton-Simpson wrote:

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

13

[...]

this only helps with the python version -- not package versions. This
whole mess is why virtual_env was invented, as well as py2exe and friends.

[...]

Well. We should face the reality. Python is a scripting
language, a Python engine built to interpret a source
code. No more, no less. Due to that nature, the natural
way to distribute code is to distribute the Python
scripts [*]. I can even say, "to exchange code" instead
of "to distribute code".

Building exe, standalone applications may be nice. Python
is basically not the appropriate tool to do this kind
of job. [**]

So speaking of a "mess" seems to me a little bit strong.

The Python core developers are doing a more than
excellent job. If I can emit a serious concern:
thera are not controlling their product.

When I see, that a single man can decide to rename
site-package to dist-package or some people may
decide to change the sys.getdefaultencoding(), things
can not work properly.

This is what I'm calling a mess.

[*] Python 3.2 with the cached .pyc files emphasizes this
point. It is no more possible to run a .pyc file.

[**] I'm sometimes working in that way. I'm aware this
is not the correct way, if I'm doing this it is only for
the "fun", not for "serious" work.

jmf

···

On 27 juil, 18:44, Christopher Barker <Chris.Bar...@noaa.gov> wrote:

On 7/26/11 7:21 PM, Grant Paton-Simpson wrote:

14

Well. We should face the reality. Python is a scripting
language, a Python engine built to interpret a source
code. No more, no less.

I don't know if it is more or less, but Python is a dynamic programming language, and CPython is an implementation nof it that compiles the source to a byte code, and provides an interpreter to run that that byte code (and some people call it a virtual machine).

I don't know that "scripting language" has a clear definition, nor that it confines the language to any particular use-case.

Due to that nature, the natural
way to distribute code is to distribute the Python
scripts [*]. I can even say, "to exchange code" instead
of "to distribute code".

Absolutely -- if you want to distribute (or exchange) code. However, if you want to distribute an application, it's a different story.

No different that any other language -- if you write an app or library in C, and you want to distribute/exchange code, you distribute source code.

If you want to distribute an application, you distribute one or both of:
* the source and build scripts
* compiled binaries

It's really no different with Python.

Building exe, standalone applications may be nice. Python
is basically not the appropriate tool to do this kind
of job. [**]

sure it is. Many or us do, it works fine, what's the problem?

I like what a member of this list once wrote (paraphrasing):

My clients can't tell any difference between an app written in C++ and Python -- except that the Python one has more features and fewer bugs.

So speaking of a "mess" seems to me a little bit strong.

Python has gotten wider an wider usage of the years. And many many python programs (or scripts, for that matter) depend on a variety of third-party packages. Those packages may come in different versions, and those version may not be fully compatible (backwards or forwards). Therefor, if folks are going to use the "scripting language" model, and distribute their apps as source code (scripts), then things WIL break if users want a variety of unrelated scripts to work, if they rely on different versions of the same packages.

Linux distros provide package managers to handle these issues for C/C++., etc code -- keeping track of required version sof shared libraries, etc. But Python provides no way to install two versions of a package side-by-side, so you can't use that method to manage dependencies for python scripts.

That is the "mess" I was referring to.

Fortunately, there is virtualenv and friends for the folks that distribute (and run) applications from source code, and py2exe and friends for folks that want a "binary".

The Python core developers are doing a more than
excellent job.

Of course they are -- I'm just confused as to why they have never thought this was an issue worth addressing.

When I see, that a single man can decide to rename
site-package to dist-package or some people may
decide to change the sys.getdefaultencoding(), things
can not work properly.

This is what I'm calling a mess.

part of the same issue really -- and another good reason to use a py2exe type solution to distribute applications.

  > [**] I'm sometimes working in that way. I'm aware this

is not the correct way, if I'm doing this it is only for
the "fun", not for "serious" work.

what's incorrect about it-- and why not use it for serious work?

-Chris

···

On 7/29/11 2:51 AM, jmfauth wrote:

On 27 juil, 18:44, Christopher Barker<Chris.Bar...@noaa.gov> wrote:

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov