rpath in wxPython shared objects on UNIX

wxPython Users
1

Is there any particular reason why the shared object files
(in particular wxPython.wxc.so) are distributed linked with the
-rpath option pointing at /usr/lib/wxPython ?

For those that aren't familiar with this stuff, when a shared
library is linked with a -R/-rpath option, it has hardcoded into
it the location of other libraries it depends on. You cannot
override or augment the location (eg by using $LD_LIBRARY_PATH
or fixing /etc/ld.so.conf)

This is causing me huge grief as I can't package a wxPython
application.

I can't put the files in /usr/lib/wxPython as the user may
already have a different version of wxPython in that location.
If rpath hard linking is going to happen, it should at least
include the version number in the directory name to allow
multiple concurrent versions of the libraries to be installed.

And even if it did that, there is still the problem of multiple
standalone installations wanting to own the same fixed location.

I would settle for a program that can modify the .so file that
I package to point rpath at a different location. (objdump -p
shows where it is pointing). Unfortunately I can't seem to
find any.

wxPython really should be compiled with -rpath so that it
can be relocated. This will require /etc/ld.so.conf and/or
$LD_LIBRARY_PATH but gains far greater flexibility.

For those who are curious, I am using Anthony Tuininga's
cx_Freeze tool which does a really good job. Now if only
wxPython would play ball ...

Roger

2

Is there any particular reason why the shared object files
(in particular wxPython.wxc.so) are distributed linked with the
-rpath option pointing at /usr/lib/wxPython ?

Hoo, boy. You're in religious war territory. :slight_smile:

You cannot
override or augment the location (eg by using $LD_LIBRARY_PATH
or fixing /etc/ld.so.conf)

On some Unices you can (Solaris); on some you can't (the rest).

I can't put the files in /usr/lib/wxPython as the user may
already have a different version of wxPython in that location.
If rpath hard linking is going to happen, it should at least
include the version number in the directory name to allow
multiple concurrent versions of the libraries to be installed.

No argument there... not sure why wx doesn't do that now. That would
be my vote for a remedy.

And even if it did that, there is still the problem of multiple
standalone installations wanting to own the same fixed location.

If you absolutely need a standalone location, you'll probably want to
rebuild and point rpath at a relative location anyway. Maybe that
should be a configuration option, but I can't see changing the default
behavior for the 20% case.

I would settle for a program that can modify the .so file that
I package to point rpath at a different location. (objdump -p
shows where it is pointing). Unfortunately I can't seem to
find any.

Technically, you can always replace the path in the binary (so long as
it's shorter than the original), but... blechh!

wxPython really should be compiled with -rpath so that it
can be relocated. This will require /etc/ld.so.conf and/or
$LD_LIBRARY_PATH but gains far greater flexibility.

...for Unices that have an /etc/ld.so.conf or a $LD_LIBRARY_PATH that
works the way you want it to.

*sigh*

<soapbox>

-R/-rpath/LD_LIBRARY_PATH/LD_CONFIG is, without a doubt, the nastiest,
ugliest wart still remaining from the bad old days of Unix. It really
is a problem for which there is no really good solution (within the
parameters of binary installation, cross-platform compatibility, and
co-existance with "special" builds of the same library). It's a relic
from a bygone era when everyone either built from source, or else
could agree where things should go and never batted an eye telling a
user that s/he had to install dependencies.

ldconfig/crle/whathaveyou are all incompatible bandaids to the sucking
chest wound that is UNIX dynamically linked object runtime
paths. LD_LIBRARY_PATH has its own problems (there's a paper on it
called something like "Why LD_LIBRARY_PATH is bad"). This is one of
the few, and I do mean few, system-level areas where Windows seems to
have learned from the mistakes of its predecessor: no path information
is baked into the binaries, and the load precedence order is
well-defined.

</soapbox>

···

On Wed, Jul 16, 2003 at 05:53:01PM -0700, Roger Binns wrote:
--
Tim Lesher <tim@lesher.ws>
http://www.lesher.ws

3

Hoo, boy. You're in religious war territory. :slight_smile:

I spotted that in my google searches :slight_smile:

If you absolutely need a standalone location, you'll probably want to
rebuild and point rpath at a relative location anyway. Maybe that
should be a configuration option, but I can't see changing the default
behavior for the 20% case.

The problem I have is that there is no "runtime" package of wxPython. And
requiring users to install a recent Python, a recent wxPython plus the
few other packages I depend on is a nightmare.

A runtime wxPython that just put the shared libs in /usr/lib/wxPython-version
would work fine for me.

And I really don't want to recompile wxPython since what I produce
won't be the same as what everyone is using.

Technically, you can always replace the path in the binary (so long as
it's shorter than the original), but... blechh!

The path in the binary is /usr/lib/wxPython. There are no strings
that short that also include version information. (My install is
into /usr/lib/bitpim-0.6-test2)

Windows seems to
have learned from the mistakes

The one thing that Windows does well in this regard is that it looks
in the same directory as the binary first. If Linux did that, I
wouldn't have a problem.

Roger

4

> Hoo, boy. You're in religious war territory. :slight_smile:

I spotted that in my google searches :slight_smile:

We went several rounds over this at work last year. The scars are
healing nicely, thank you.

The problem I have is that there is no "runtime" package of wxPython. And
requiring users to install a recent Python, a recent wxPython plus the
few other packages I depend on is a nightmare.

What do you meant by a runtime package? You didn't mention what OS
you were using, but from your mention of ldconfig I assume it's a
flavor of Linux... there are RPMs for both Python and wxPython.

> Technically, you can always replace the path in the binary (so long as
> it's shorter than the original), but... blechh!

The path in the binary is /usr/lib/wxPython. There are no strings
that short that also include version information. (My install is
into /usr/lib/bitpim-0.6-test2)

Hence the 'blech'. The default paths are almost always shorter than
what you want, IME.

···

On Wed, Jul 16, 2003 at 10:09:18PM -0700, Roger Binns wrote:

--
Tim Lesher <tim@lesher.ws>
http://www.lesher.ws

5

Hi,

Roger Binns wrote:

wxPython really should be compiled with -rpath so that it
can be relocated. This will require /etc/ld.so.conf and/or
$LD_LIBRARY_PATH but gains far greater flexibility.

No, you're missing the point of wxPython's use of -rpath. It is used
so that wxPython does *not* use wxGTK library installed in library
paths from /etc/ld.so.conf or LD_LIBRARY_PATH. The reason it does it
is that it must coexist with existing installation of standalone
wxGTK (possibly in different version or built with different
options).

I agree the lib name should be /usr/lib/wxPython-%{exact_version},
though.

You can always download wxPython .src.rpm, remove the line with
--enable-rpath=%{wxpref}/lib and rebuild.

VS

···

--
PGP key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x465264C9

6

What do you meant by a runtime package?

I mean an RPM that only contains the shared objects only. ie
you could run a frozen application, but you couldn't do any
development.

I assume it's a flavor of Linux...

I am trying to distribute a wxPython based application that will run
on most recent Linuxes.

there are RPMs for both Python and wxPython.

And I am doing the distribution in binary form. On Windows, I use py2exe
and everything is easy and works well. In addition to those RPMS, I also
need pySerial, and the users would need to have the right versions of
everything. (And last I checked RPM isn't the native format for
Slackware or Debian).

So in theory I could do everything as seperate RPMS, require users to
download them, enjoy RPM dependency hell, and eventually they will be
able to run my software.

Hence the 'blech'. The default paths are almost always shorter than
what you want, IME.

Anthony (the author of cx_Freeze) has found that if you remove the
RPATH (using chrpath) then $LD_LIBRARY_PATH is honoured. That will
solve my problem.

So the good news is it is now finally possible to distribute wxPython
based apps as a binary distribution on Linux with no dependencies on Python
being installed (or any extensions).

Roger

7

Vaclav Slavik wrote:

Hi,

Roger Binns wrote:

wxPython really should be compiled with -rpath so that it
can be relocated. This will require /etc/ld.so.conf and/or
$LD_LIBRARY_PATH but gains far greater flexibility.

No, you're missing the point of wxPython's use of -rpath. It is used so that wxPython does *not* use wxGTK library installed in library paths from /etc/ld.so.conf or LD_LIBRARY_PATH. The reason it does it is that it must coexist with existing installation of standalone wxGTK (possibly in different version or built with different options).

Yep. wxPython essentially has a private version of wxGTK.

I agree the lib name should be /usr/lib/wxPython-%{exact_version}, though.

Yes, I can see the value of that. I'll make that change.

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

8

Roger Binns wrote:

Anthony (the author of cx_Freeze) has found that if you remove the
RPATH (using chrpath) then $LD_LIBRARY_PATH is honoured. That will
solve my problem.

Ah, I didn't know about chrpath. I'll have to remember that one...

So the good news is it is now finally possible to distribute wxPython
based apps as a binary distribution on Linux with no dependencies on Python
being installed (or any extensions).

Cool! Could you please write a HOWTO for the wiki about it?

Thanks!

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

9

Robin Dunn wrote:

> I agree the lib name should be
> /usr/lib/wxPython-%{exact_version}, though.

Yes, I can see the value of that. I'll make that change.

Could there be a symlink (/usr/lib/wxPython) to it in -devel package?
it will be impossible to find wxPython's wx-config otherwise and you
need it sometimes (e.g. wxMozilla now).

VS

···

--
PGP key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x465264C9

10

Vaclav Slavik wrote:

Robin Dunn wrote:

I agree the lib name should be
/usr/lib/wxPython-%{exact_version}, though.

Yes, I can see the value of that. I'll make that change.

Could there be a symlink (/usr/lib/wxPython) to it in -devel package? it will be impossible to find wxPython's wx-config otherwise and you need it sometimes (e.g. wxMozilla now).

Yep.

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

11

Unless I understand your requirements incorrectly, Gordon McMillan's
Installer has been able to do this for quite some time. I use it to
distribute quite a large VTK + wxPython (and soon ITK) application on Linux
and Windows. The Linux and Windows machines in question have no wxPython or
Python installed, or at the very least a far too old version of Python.

···

On Thu, Jul 17, 2003 at 11:00:59AM -0700, Roger Binns wrote:

So the good news is it is now finally possible to distribute wxPython
based apps as a binary distribution on Linux with no dependencies on Python
being installed (or any extensions).

--
charl p. botha http://cpbotha.net/ http://visualisation.tudelft.nl/

12

Unless I understand your requirements incorrectly, Gordon McMillan's
Installer has been able to do this for quite some time.

The reason why I don't use the McMillan installer is because as far
as I could tell, it is actually an executable that self extracts
everytime you run it. I REALLY don't like that.

The Linux and Windows machines in question have no wxPython or
Python installed, or at the very least a far too old version of Python.

Windows is a non-issue since you can't hard code the location of DLLS
anyway, and by default it looks in the same directory as the binary
first.

On Linux I can reassure you that there is a problem, unless the Installer
removes the rpath information. If the rpath is removed, or points to
a non-existent directory, then LD_LIBRARY_PATH is obeyed.

Roger

13

If I understand it correctly, that might have a lot to do with the fact
that Installer (on Linux anyway) extracts all of the files to a
(temporary) directory and then executes the binary from there. That is
(in my opinion) unacceptable but that is how Gordon McMillan chose to
solve the problem. I have now written a binary that will simply set
LD_LIBRARY_PATH and then re-execute itself as needed and that seems to
work quite well.

That said, I don't know he managed to get around the RPATH issue (if he
did) since on Linux if the value is present it is searched first. So if
the target machine has an incompatible wxPython installed, the program
blows up. Removing the rpath value from the wxc.so library solves the
problem (or as Robin suggested, using a very specific directory so that
incompatibilities do not get in the road).

···

On Thu, 2003-07-17 at 16:45, Charl P. Botha wrote:

On Thu, Jul 17, 2003 at 11:00:59AM -0700, Roger Binns wrote:
> So the good news is it is now finally possible to distribute wxPython
> based apps as a binary distribution on Linux with no dependencies on Python
> being installed (or any extensions).

Unless I understand your requirements incorrectly, Gordon McMillan's
Installer has been able to do this for quite some time. I use it to
distribute quite a large VTK + wxPython (and soon ITK) application on Linux
and Windows. The Linux and Windows machines in question have no wxPython or
Python installed, or at the very least a far too old version of Python.

--
Anthony Tuininga <anthony@computronix.com>

14

Cool! Could you please write a HOWTO for the wiki about it?

Here is a temporary HOWTO. Anthony is enhancing cx_Freeze
so the wrapper script won't be necessary. When he has
finished that, I'll update these and someone else can
dump them in the wiki.

   http://www.rogerbinns.com/howtowxpython.html

Roger