Distributing 'frozen' wx apps

wxPython Users
1

I need to be able to distribute wxPython apps to users who may not have all the prerequisites installed. Thus, I'm building a 'frozen' version of the app using cx_Freeze. The results have been mixed, though.

  My development machine is FC2, and those who try the frozen app who are also on FC2 can run the app without problem. Others, though, on RH8 machines, report that they receive errors, and these errors indicate that they are not able to load wxPython classes. Someone suggested to me that wxPython dynamically links to some system libraries, which differ on FC2 and RH8.

  Looking through the docs, I saw a reference to a problem with multilib builds on OS X, and the option to create a monolithic build. Is this related to the problem I'm having? IOW, would it be worth the effort to rebuild wxWidgets in monolithic mode? Or is this not related at all?

      ___/
     /
    __/
   /
  ____/
  Ed Leafe
  http://leafe.com/
  http://dabodev.com/

2

  My development machine is FC2, and those who try the frozen app who are also on FC2 can run the app without problem. Others, though, on RH8 machines, report that they receive errors, and these errors indicate that they are not able to load wxPython classes. Someone suggested to me that wxPython dynamically links to some system libraries, which differ on FC2 and RH8.

This is a problem with distributing any binary linux app. In general, the commercial folks support a given distribution, and that's it. Uses can get it working on other distros, but may have to install different versions of various libraries. A lot of distros also distribute older versions of various libraries (sometimes as a "compat" package or something like that. This allows users to run binary apps built for older systems.

First of all, I'd recommend building on a older version of the OS. It's going to be much easier for people to run an app build for Redhat 8 on FC2 than the other way around. With a lot of work, you could probably put older versions of al the libs linked to on your machine, and make sure your build uses those, but it'd be much easier to just build on an older distro.

The other option is to include as many libs as possible with our build. You could probably figure out how to statically link wxWidgets to most libs, but it's going to get pretty big! What I've seen instead is that the app comes with copies of various *.so files, and a start-up script sets a path for linking, so they get used. You'll still probably want to use the base system files, so I would still build on the oldest distro you want to support.

If a number of people need to do this, perhaps you could get together and build a more-or-less official statically linked binary distro wxPython for Linux.

This is all a big pain in the a??, so I really hope the LSB project or something like it gets established!

Of course, the other solution is to just distribute the source, with a list of required libs.

  Looking through the docs, I saw a reference to a problem with multilib builds on OS X, and the option to create a monolithic build. Is this related to the problem I'm having? IOW, would it be worth the effort to rebuild wxWidgets in monolithic mode? Or is this not related at all?

I doubt that's your problem. I think Monolithic means that you get one so for wxWidgets, rather than a bunch, but you're including them all anyway.

-Chris

Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

···

On Aug 19, 2004, at 8:21 AM, Ed Leafe wrote:

3

Ed Leafe wrote:

My development machine is FC2, and those who try the frozen app who
are also on FC2 can run the app without problem. Others, though, on
RH8 machines, report that they receive errors, and these errors
indicate that they are not able to load wxPython classes. Someone
suggested to me that wxPython dynamically links to some system
libraries, which differ on FC2 and RH8.

Your problem is a simple artifact of the way that UNIX and Linux
systems work. Generally they are backwards compatible (you can
run older programs on newer versions of the OS). It is *extremely*
rare for them to be forwards compatible (running newer programs
on the older OS versions).

Sometimes you can do things like compile the program and link
in all libraries statically. This is increasingly likely to
not work since many system services such as name resolution,
user and group listing as well as authentication are provided
dynamically. They use files (pam, nsswitch.conf) and dynamic
libraries as specified in those files. Even the presence of
/tls causes various important side effects with threading
on Linux.

The simplest solution is to build on the oldest version of
UNIX/Linux you want to support. Note however that people using
newer ones may have to install backwards compatibility libraries,
but these are often installed by default.

I use VMWare/VirtualPC to make a pristine build environment
that is only used for building and has the least amount
of software installed.

If you are feeling masochistic, you can have multiple guest
systems for each major version of the distros you want to
support and have a selections of rpm/debs/tgz for people
to grab depending on what they use.

Roger

4

The simplest solution is to build on the oldest version of
UNIX/Linux you want to support. Note however that people using
newer ones may have to install backwards compatibility libraries,
but these are often installed by default.

  I guess it was too much to expect that everything would work seamlessly. I understand about the dynamic links; I thought those were "frozen" to create static distributions, but apparently that's not the case.

I use VMWare/VirtualPC to make a pristine build environment
that is only used for building and has the least amount
of software installed.

  That's not a bad idea, as I don't have extra systems lying around.

  Let's say I create a RedHat 7.1 system for building (that's the oldest distro for which I have disks). I assume based on what you wrote that there is a good chance that later releases of RH and FC should then be able to run apps "frozen" in RH7.1. Now what about people running Mandrake, SuSE, Gentoo, etc.? Will they have their own quirks that will make things incompatible? Has it been your experience that different distros from the same time period are pretty compatible in this regard?

      ___/
     /
    __/
   /
  ____/
  Ed Leafe
  http://leafe.com/
  http://dabodev.com/

···

On Aug 19, 2004, at 5:25 PM, Roger Binns wrote:

5

Ed Leafe wrote:

    I guess it was too much to expect that everything would work seamlessly. I understand about the dynamic links; I thought those were "frozen" to create static distributions, but apparently that's not the case.

the various Python packaging tools "freeze" the libs directly required by your app, but not the ones that those libs call.

The *nix tradition is to dynamically link everything. This works particularly well if apps are compiled on the system they will run on, but not so well if you are distributing a binary. It is possible to compile a completely statically linked app, but that would probably result in a very large app, and you'd have to re-compile the libs your app depends on (i.e. wxGTK). In theory, you could re-compile Python itself with wxPython and wxGTK statically linked in. In fact, I've thought that might be a good idea as a way to create a wxPython runtime environment.

    Let's say I create a RedHat 7.1 system for building (that's the oldest distro for which I have disks). I assume based on what you wrote that there is a good chance that later releases of RH and FC should then be able to run apps "frozen" in RH7.1.

Yes, though they may need some backwards compatible libraries installed, but those should be available as rpms.

Now what about people running Mandrake, SuSE, Gentoo, etc.? Will they have their own quirks that will make things incompatible?

Well, yes, but the odds are pretty good.

Has it been your experience that different distros from the same time period are pretty compatible in this regard?

The basic system libraries are pretty similar, but more obscure stuff less so. You'll never know 'till you test.

Linux does have it's difficulties.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

6

Ed Leafe wrote:

    I need to be able to distribute wxPython apps to users who may not have all the prerequisites installed. Thus, I'm building a 'frozen' version of the app using cx_Freeze. The results have been mixed, though.

    My development machine is FC2, and those who try the frozen app who are also on FC2 can run the app without problem. Others, though, on RH8 machines, report that they receive errors, and these errors indicate that they are not able to load wxPython classes. Someone suggested to me that wxPython dynamically links to some system libraries, which differ on FC2 and RH8.

Probably because of different gtk+ versions (2.2 vs. 2.4 although this incompatibility should be fixed in wxPython 2.5.2.7) as well as because of the switch to X.org from XFree86.

    Looking through the docs, I saw a reference to a problem with multilib builds on OS X, and the option to create a monolithic build. Is this related to the problem I'm having? IOW, would it be worth the effort to rebuild wxWidgets in monolithic mode? Or is this not related at all?

Not related. The issue is with the external libs that wxWidgets was built/linked with.

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

7

OK, I'm now trying to build wxWidgets on a RH8 system so that all the libraries are compiled staticly. I have the system variable 'LDFLAGS=-static' set, and I've compile other source (such as Python 2.3.4) so that ldd reports that the file is not a dynamic executable. However, following the instructions for building wx doesn't produce the same results. Here's a sample output after running the .make script:

[ed@moe bld]$ set | grep LD
LDFLAGS=-static
[ed@moe bld]$ ldd lib/libwx_gtkd_core-2.5.so.2.0.0
         libz.so.1 => /usr/lib/libz.so.1 (0x4035b000)
         libdl.so.2 => /lib/libdl.so.2 (0x4036a000)
         libgtk-1.2.so.0 => /usr/lib/libgtk-1.2.so.0 (0x4036d000)
         libgdk-1.2.so.0 => /usr/lib/libgdk-1.2.so.0 (0x404c7000)
         libgmodule-1.2.so.0 => /usr/lib/libgmodule-1.2.so.0 (0x40500000)
         libgthread-1.2.so.0 => /usr/lib/libgthread-1.2.so.0 (0x40503000)
         libglib-1.2.so.0 => /usr/lib/libglib-1.2.so.0 (0x40506000)
         libpthread.so.0 => /lib/i686/libpthread.so.0 (0x4052d000)
         libXi.so.6 => /usr/X11R6/lib/libXi.so.6 (0x4057d000)
         libXext.so.6 => /usr/X11R6/lib/libXext.so.6 (0x40585000)
         libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0x40593000)
         libm.so.6 => /lib/i686/libm.so.6 (0x40671000)
         libpng12.so.0 => /usr/lib/libpng12.so.0 (0x40693000)
         libjpeg.so.62 => /usr/lib/libjpeg.so.62 (0x406b7000)
         libtiff.so.3 => /usr/lib/libtiff.so.3 (0x406d5000)
         libwx_based-2.5.so.2 => not found
         libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x40717000)
         libc.so.6 => /lib/i686/libc.so.6 (0x42000000)
         /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)
         libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x407ca000)

  As you can see, there are quite a few dynamic dependencies here. How can I get wxWidgets to compile statically?

      ___/
     /
    __/
   /
  ____/
  Ed Leafe
  http://leafe.com/
  http://dabodev.com/

···

On Aug 23, 2004, at 6:51 PM, Robin Dunn wrote:

    I need to be able to distribute wxPython apps to users who may not have all the prerequisites installed. Thus, I'm building a 'frozen' version of the app using cx_Freeze. The results have been mixed, though.
    My development machine is FC2, and those who try the frozen app who are also on FC2 can run the app without problem. Others, though, on RH8 machines, report that they receive errors, and these errors indicate that they are not able to load wxPython classes. Someone suggested to me that wxPython dynamically links to some system libraries, which differ on FC2 and RH8.

Probably because of different gtk+ versions (2.2 vs. 2.4 although this incompatibility should be fixed in wxPython 2.5.2.7) as well as because of the switch to X.org from XFree86.

8

Ed Leafe wrote:

···

On Aug 23, 2004, at 6:51 PM, Robin Dunn wrote:

    I need to be able to distribute wxPython apps to users who may not have all the prerequisites installed. Thus, I'm building a 'frozen' version of the app using cx_Freeze. The results have been mixed, though.
    My development machine is FC2, and those who try the frozen app who are also on FC2 can run the app without problem. Others, though, on RH8 machines, report that they receive errors, and these errors indicate that they are not able to load wxPython classes. Someone suggested to me that wxPython dynamically links to some system libraries, which differ on FC2 and RH8.

Probably because of different gtk+ versions (2.2 vs. 2.4 although this incompatibility should be fixed in wxPython 2.5.2.7) as well as because of the switch to X.org from XFree86.

    OK, I'm now trying to build wxWidgets on a RH8 system so that all the libraries are compiled staticly. I have the system variable 'LDFLAGS=-static' set, and I've compile other source (such as Python 2.3.4) so that ldd reports that the file is not a dynamic executable. However, following the instructions for building wx doesn't produce the same results. Here's a sample output after running the .make script:

[ed@moe bld]$ set | grep LD
LDFLAGS=-static
[ed@moe bld]$ ldd lib/libwx_gtkd_core-2.5.so.2.0.0
        libz.so.1 => /usr/lib/libz.so.1 (0x4035b000)
        libdl.so.2 => /lib/libdl.so.2 (0x4036a000)
        libgtk-1.2.so.0 => /usr/lib/libgtk-1.2.so.0 (0x4036d000)
        libgdk-1.2.so.0 => /usr/lib/libgdk-1.2.so.0 (0x404c7000)
        libgmodule-1.2.so.0 => /usr/lib/libgmodule-1.2.so.0 (0x40500000)
        libgthread-1.2.so.0 => /usr/lib/libgthread-1.2.so.0 (0x40503000)
        libglib-1.2.so.0 => /usr/lib/libglib-1.2.so.0 (0x40506000)
        libpthread.so.0 => /lib/i686/libpthread.so.0 (0x4052d000)
        libXi.so.6 => /usr/X11R6/lib/libXi.so.6 (0x4057d000)
        libXext.so.6 => /usr/X11R6/lib/libXext.so.6 (0x40585000)
        libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0x40593000)
        libm.so.6 => /lib/i686/libm.so.6 (0x40671000)
        libpng12.so.0 => /usr/lib/libpng12.so.0 (0x40693000)
        libjpeg.so.62 => /usr/lib/libjpeg.so.62 (0x406b7000)
        libtiff.so.3 => /usr/lib/libtiff.so.3 (0x406d5000)
        libwx_based-2.5.so.2 => not found
        libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x40717000)
        libc.so.6 => /lib/i686/libc.so.6 (0x42000000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x407ca000)

    As you can see, there are quite a few dynamic dependencies here. How can I get wxWidgets to compile statically?

Do you have static versions of all these libs (*.a files) installed? BTW, from what I've heard you don't want to do a completely static link with the gtk libs as they will then have problems dynamically loading some themes, etc.

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

9

Right now I'd be willing to trade some of the trimmings, such as theme support, for portability.

  The problem is the dynamic loader: /lib/ld-linux.so.2. Since that path is hardcoded into the generated libs, I can't create a frozen version that will work on different platforms, since they will always look to their local copy of that library rather than any bundled into the frozen version.

  I was able to build Python 2.3.4 into a static executable; why is it not possible (or at least obvious) to build a static version of wxWidgets?

      ___/
     /
    __/
   /
  ____/
  Ed Leafe
  http://leafe.com/
  http://dabodev.com/

···

On Aug 30, 2004, at 5:56 PM, Robin Dunn wrote:

Do you have static versions of all these libs (*.a files) installed? BTW, from what I've heard you don't want to do a completely static link with the gtk libs as they will then have problems dynamically loading some themes, etc.

10

Ed Leafe wrote:

···

On Aug 30, 2004, at 5:56 PM, Robin Dunn wrote:

Do you have static versions of all these libs (*.a files) installed? BTW, from what I've heard you don't want to do a completely static link with the gtk libs as they will then have problems dynamically loading some themes, etc.

    Right now I'd be willing to trade some of the trimmings, such as theme support, for portability.

    The problem is the dynamic loader: /lib/ld-linux.so.2. Since that path is hardcoded into the generated libs, I can't create a frozen version that will work on different platforms, since they will always look to their local copy of that library rather than any bundled into the frozen version.

    I was able to build Python 2.3.4 into a static executable; why is it not possible (or at least obvious) to build a static version of wxWidgets?

It should be possible. It's been discussed on wx-dev or wx-users before, but I don't know any details.

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

11

Ed Leafe wrote:

    The problem is the dynamic loader: /lib/ld-linux.so.2. Since that path is hardcoded into the generated libs, I can't create a frozen version that will work on different platforms, since they will always look to their local copy of that library rather than any bundled into the frozen version.

/lib/ld-linux.so.2 is one thing you should expect to find on any moderately recent linux distro (Redhat had it as of 7.0 at least)

When ld-linux.so.3 comes out, distros should have backward-compatible packages that would include it.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

12

But it doesn't seem to be compatible across various versions. Test apps I've frozen always choked on something that threw an error saying that something was "not defined in file ld-linux.so.2 with link time reference". Apparently there are many different versions out there; the ld-linux.so.2 is just a symlink to the one that's installed. That's why I'm trying to build on a RH8 machine; I'm hoping that later systems will be backwards-compatible.

      ___/
     /
    __/
   /
  ____/
  Ed Leafe
  http://leafe.com/
  http://dabodev.com/

···

On Aug 30, 2004, at 7:11 PM, Chris Barker wrote:

/lib/ld-linux.so.2 is one thing you should expect to find on any moderately recent linux distro (Redhat had it as of 7.0 at least)

13

Chris Barker wrote:

/lib/ld-linux.so.2 is one thing you should expect to find on any
moderately recent linux distro (Redhat had it as of 7.0 at least)

You should probably be aware of what Ed is trying to do. He is
attempting to freeze a wxPython application (using cx-Freeze) on some random Linux distribution and then have it work on any other version of any other Linux distribution out there including
old ones such as Mandrake 7 and new ones such as FC2.

Obviously just doing a freeze on say Redhat 8 won't work since
newer distros will require various compat libraries to be installed
and older ones won't have needed pieces.

Consequently he is trying to build up everything needed including
*all* libraries (including ld-linux) into a single directory with
absolutely no external dependencies and have $LD_LIBRARY_PATH
and other linker type stuff done to ensure only files out of
that directory are used.

Personally I think it is an admirable goal, and exactly what
I want for the Linux/wxPython stuff I distribute as well.
However I don't think it is possible. That hasn't stopped
Ed though :slight_smile:

Currently the way this problem is "solved" is two fold.
The majority of apps just pick some random Linux distro
version (typically Redhat 9) and build on that. Newer
distros are backwards compatible usually requiring the
installation of compat-libs. Only that single "binary"
is shipped. That is what I do for my software, and
what most corporate software does as well. Even Robin does
it for the binary versions of wxPython, although FC2
packages are also made available.

A few brave souls go out and package their software up on many different versions of Linux. A good example is Gaim (in the top 10 projects on SourceForge):

http://gaim.sourceforge.net/downloads.php

Note how there is exactly one Windows download, and 8 Linux downloads.

This whole issue isn't the fault of wxPython, but is an
indirect consequence of wxPython because it sucks in GTK
which then sucks in a whole bunch of complementary libraries
and then they all drag in various X libraries, which then
drag in all sorts of operating system libraries and functionality
such as networking, threading, dynamic linking etc.

If I was in Ed's shoes, and desperately wanted to solve this
problem enough, I would use something like VMWare and automate freezing on all the different Linux distro versions.

Alternately I did suggest using a Knoppix style bootable
CD to demo the software which is where this issue is
most important.

Roger

14

You should probably be aware of what Ed is trying to do. He is
attempting to freeze a wxPython application (using cx-Freeze) on some random Linux distribution and then have it work on any other version of any other Linux distribution out there including
old ones such as Mandrake 7 and new ones such as FC2.

  Slight correction: I'm attempting to freeze on RH8, in the hope of having it work on any later version of RH/FC, and possibly other distros.

Obviously just doing a freeze on say Redhat 8 won't work since
newer distros will require various compat libraries to be installed
and older ones won't have needed pieces.

  Why would newer versions require these other things? I understand that older versions will not be forward-compatible. If the distributed code doesn't contain references to stuff outside of itself, why shouldn't it run?

Personally I think it is an admirable goal, and exactly what
I want for the Linux/wxPython stuff I distribute as well.
However I don't think it is possible. That hasn't stopped
Ed though :slight_smile:

  IMO, this is probably the single biggest problem standing in the way of the growth of Linux. Not being able to build an app that can run on slightly different distros is absurd, and businesses will not invest in developing apps for a platform that is actually hundreds of mini-platforms, each requiring their own particular way of doing things.

Note how there is exactly one Windows download, and 8 Linux downloads.

  Exactly my point. Sure, I could do that, but that sucks.

This whole issue isn't the fault of wxPython, but is an
indirect consequence of wxPython because it sucks in GTK
which then sucks in a whole bunch of complementary libraries
and then they all drag in various X libraries, which then
drag in all sorts of operating system libraries and functionality
such as networking, threading, dynamic linking etc.

  But why is it that I am not able to build wxWidgets so that when it sucks all of those things in, it actually sucks them in instead of linking to them dynamically? Is that really more involved than something like Python itself, which is no problem to build statically?

If I was in Ed's shoes, and desperately wanted to solve this
problem enough, I would use something like VMWare and automate freezing on all the different Linux distro versions.

  Ah, but that's not "solving" the problem. That's coping with it.

Alternately I did suggest using a Knoppix style bootable
CD to demo the software which is where this issue is
most important.

  I'm not interested in a demo disk. I am developing a framework for building applications, much like Visual FoxPro or Visual Basic. My target is developers who are moving from the Windows world to the cross-platform world. Making them boot from a CD every time they want to work on their development project is not feasible.

      ___/
     /
    __/
   /
  ____/
  Ed Leafe
  http://leafe.com/
  http://dabodev.com/

···

On Aug 30, 2004, at 8:21 PM, Roger Binns wrote:

15

Ed Leafe wrote:

Slight correction: I'm attempting to freeze on RH8, in the hope of
having it work on any later version of RH/FC, and possibly other
distros.

Ah, you reduced your goals :slight_smile: Incidentally I do so on RH9 and
all seems to be fine.

Why would newer versions require these other things?

For example your file may link against libfoo-3. The newer distro
has libfoo-4. When you install compat-libs on the newer distro,
it will include libfoo-3. In the case of wxPython 'foo' is usually
the C++ libraries.

older versions will not be forward-compatible. If the distributed code
doesn't contain references to stuff outside of itself, why shouldn't
it run?

It will run if you catch *every* reference. The default install
of wxPython ends up linking dynamically to libstdc++ which is
the usual culprit. (It is notoriously hard to do ABI version
compatibility with C++)

IMO, this is probably the single biggest problem standing in the way
of the growth of Linux. Not being able to build an app that can run on
slightly different distros is absurd, and businesses will not invest
in developing apps for a platform that is actually hundreds of
mini-platforms, each requiring their own particular way of doing
things.

UNIX has always been like this, but with something like Solaris
there weren't multiple incompatible vendors so it wasn't a big
issue.

For the Linux vendors, the big ones (ok, one - Redhat) didn't
care since everyone made sure their stuff ran on Redhat.
This was a self reinforcing circle. For the little Linux
vendors, they had no choice but to make sure anything that
ran on Redhat could run on their distro.

The second attempt to fix it resulted in United Linux. That
was vendor driven and basically consisted of the tier of
Redhat major competitors. See http://www.unitedlinux.com/
You will also note that they became defunct shortly after
one of their members renamed themselves to SCO and started
a bunch of stupid lawsuits.

The first attempt at a fix was the Linux Standards Base
(LSB). http://www.linuxbase.org/ Some vendors tried to
comply but you ended up with bizarre things like Mandrake
having extra packages if you wanted LSB compliance.

LSB also doesn't include C++. The new version 2 of LSB
will probably include C++, but there are many issues.
You can read about it LWN.net Weekly Edition for August 5, 2004 [LWN.net]
The other problem is that even after LSB 2 comes out,
it will take several years for the majority of the
user base to be running compliant systems. That
won't help you next month, but may fix this problem
for any releases you do in 3 years time.

But why is it that I am not able to build wxWidgets so that when it
sucks all of those things in, it actually sucks them in instead of
linking to them dynamically? Is that really more involved than
something like Python itself, which is no problem to build statically?

The short answer is that developers have been focusing their
attention on shared libraries for quite a while now. While
it is technically possibly to link things statically, noone
puts in the effort any more.

Teh long answer:

You should understand that Unix in general and Linux in particular
has been moving very strongly to shared libraries over the
years. Shared libraries are significantly more efficient in
terms of disk space and memory consumption, and also have the
advantage that if there is a security hole or bug, you just replace
the library and that fixes all programs that use it. Nowadays
about the only statically linked programs on a Linux box are
sash and the package manager.

Additionally components of the system are using shared libraries
for flexibility. For example authentication is done using
PAM which allows the behaviour to be controlled by loading
libraries. Name service is done by libraries that do various
protocols or conventions. Themes in gui environments are
done using dynamic loading. Programs load dynamically load
their plugins. Languages like Python load their modules
dynamically.

If I was in Ed's shoes, and desperately wanted to solve this
problem enough, I would use something like VMWare and automate
freezing on all the different Linux distro versions.

Ah, but that's not "solving" the problem. That's coping with it.

I did say that I don't think it can be solved :slight_smile: I do think
it is possible to hide it from users by doing the above
and then having a setup program that does download the
right version of the package. Ximian's Red Carpet
does that for example.

I'm not interested in a demo disk. I am developing a framework for
building applications, much like Visual FoxPro or Visual Basic. My
target is developers who are moving from the Windows world to the
cross-platform world. Making them boot from a CD every time they want
to work on their development project is not feasible.

Trying to make Linux work like Windows won't solve the problem for
you :slight_smile: (A lot of people try though). Linux is the way it is.
Additionally your packaging will not be accepted by many of the
distros. For example both Debian and Gentoo refuse my stuff
that I use cx-Freeze on. (Of course the funny thing is that although they claim to have superior packaging mechanisms and
processes, neither can cope with a two week release cycle or
have an easy robust install that doesn't impose version
constraints on other applications).

Lastly Linux users will expect that your stuff will integrate
with their packaging system, in exactly the same way that
Windows users expect stuff to end up in the Add/Remove programs
in the Control Panel. The only way you can realistically
do this is by building the package on each distro version.
(You didn't think they gave the same name and version number
to X libraries, system libraries etc did you :slight_smile:

I think you can see again why most companies (and most non-companies
for that matter) just pick one random version of Redhat and be done
with it.

Roger

16

Roger Binns wrote:

Chris Barker wrote:

/lib/ld-linux.so.2 is one thing you should expect to find on any
moderately recent linux distro (Redhat had it as of 7.0 at least)

You should probably be aware of what Ed is trying to do.

I understood what he is trying to do. My point was that it may not be necessary to statically link absolutely everything, just most things ;-). I don't know all that much about this, but It think ld-linux.so.2 is one thing that's pretty consistent across distributions. I partly say that because I have a comercial app (matlab) that was distributed for RedHat 5, and I've run it on Redhat 7.2 and now Gentoo 2004.2. It requires /lib/ld-linux.so.1 ! (yes it was a bit of a pain to get working on Gentoo. Gentoo has a bug in their compat package, which is suppose hasn't been fixed because not a lot of Gentoo users are running old pre-compiled binary apps)

> Obviously just doing a freeze on say Redhat 8 won't work since
> newer distros will require various compat libraries to be installed

yes, that was already mentioned before in this thread. I think he was willing to accept that limitation. Most distros do provide the compat libs, and they will probably be installed by a "install everything" installer. Other wise the user can install it if required. After all, that's how windows does it, Newer systems include older libs by default.

The difference is that with Linux, there is often an expectation that everything will get re-compiled for the new system. This of course only works with open source, but even then, only works if the developer has provided an idiot-proof build system, and the users don't mind typing:

./configure && make && make install

This is a LOT messier with a wxPython app, as you could potentially need to build: python, GTK, wxGTK, and wxPython, and there are various versions of each of these.

There has been some discussion of developing a wxPython runtime, that would include python and wxPython. If we did that, that might be a place to focus on making it as independent of other libraries as possible.

old ones such as Mandrake 7 and new ones such as FC2.

If he only has to go that old, he's probably OK, at least with ld-linux!

Consequently he is trying to build up everything needed including
*all* libraries (including ld-linux) into a single directory with
absolutely no external dependencies and have $LD_LIBRARY_PATH
and other linker type stuff done to ensure only files out of
that directory are used.

That appears to be the approach commercial software takes. Here's a sample of what matlab has with it:

/usr/local/matlab5/sys/os/lnx86/libm.so.5
/usr/local/matlab5/sys/os/lnx86/libstdc++.so.27
/usr/local/matlab5/sys/os/lnx86/libstdc++.so.27.2.8
/usr/local/matlab5/sys/os/lnx86/libc.so.5.4.38
/usr/local/matlab5/sys/os/lnx86/libg++.so.27
/usr/local/matlab5/sys/os/lnx86/libm.so.5.0.9
/usr/local/matlab5/sys/os/lnx86/libXpm.so.4
/usr/local/matlab5/sys/os/lnx86/libg++.so.27.2.8
/usr/local/matlab5/sys/os/lnx86/libc.so.5
/usr/local/matlab5/sys/os/lnx86/libXpm.so.4.9

Personally I think it is an admirable goal, and exactly what
I want for the Linux/wxPython stuff I distribute as well.
However I don't think it is possible.

ALL linux distros, probably not, but MOST FAIRLY recent probably is possible.

> The first attempt at a fix was the Linux Standards Base
(LSB).

I'm very disappointed in that, but I suppose it's to be expected. On the one hand, the distros are competing on what they provide, and on the other, a _binary_ interface is kind of anathema to hard core open-sourcers.

Making them boot from a CD every time they want
to work on their development project is not feasible.

I think the suggestion was that you could provide a demo that worked that way, and once they saw how nice your product is, they'd be willing to go through a little pain to get it installed, which I think is a good idea.

However, it sounds like you want to create a tool for developers to write apps in, and the issue is probably not so much to make it easy for those developers, but to make it easy for the users of the apps created with your tools, and thus you need to provide a pretty complete runtime that they can then re-distribute. I think the wxPython Run time idea would be good here!

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

17

I don't know all that much about this, but It think ld-linux.so.2 is one thing that's pretty consistent across distributions.

  Odd, because that is the lib cited in all the error messages reported by my testers on non-FC systems when I 'froze' the app on FC2. According to someone who knows way more about this than I do, it seems that while the lib may be roughly the same, the actual routine in the library cannot be located by the different system, perhaps because an address is coded?

  I think I could make it work if I could figure out how to specify that the libraries are to use a copy of ld-linux.so.2 in the current directory, instead of having them hard-code an absolute path of /lib/ld-linux.so.2 into them. So far my Googling hasn't turned up anything.

This of course only works with open source, but even then, only works if the developer has provided an idiot-proof build system, and the users don't mind typing:

./configure && make && make install

This is a LOT messier with a wxPython app, as you could potentially need to build: python, GTK, wxGTK, and wxPython, and there are various versions of each of these.

  Further complicating things is the fact that since this is a database development system, they would also need to install the desired database and then also compile the database adapter.

However, it sounds like you want to create a tool for developers to write apps in, and the issue is probably not so much to make it easy for those developers, but to make it easy for the users of the apps created with your tools, and thus you need to provide a pretty complete runtime that they can then re-distribute. I think the wxPython Run time idea would be good here!

  The issue is to make it easy for everyone. Those with experience in Linux will not have a problem building and installing various components, even though they may grumble a bit; however, I'm trying to attract those who have never used Linux before to switch over from the "dark side". So far, the few people who have attempted to get things up and running in Linux after using nothing but Windows for the last decade have run away screaming. It seems odd, but they are willing to put up with installers, registry keys, DLL hell, and all the rest that comes with WIndows, but not going through all the steps required to get a Linux system running.

  OK, back to my original question: does anyone know enough about the whole compilation process for wxWidgets to help me figure out how to either a) build the libraries completely static, or b) change the link to ld-linux.so.2 to a relative one?

      ___/
     /
    __/
   /
  ____/
  Ed Leafe
  http://leafe.com/
  http://dabodev.com/

···

On Aug 31, 2004, at 2:20 PM, Chris Barker wrote:

18

Chris Barker writes:

I think the wxPython Run time idea
would be good here!

A wxPython runtime would go a long way to making wxPython
applications much easier to distribute, hell, even to test
different versions.

I would do it if I knew how. I'd start it if I knew how. Did
anything fruitful come out of the various discussions of
approximately 6 months ago on the topic?

As I recall, we were looking for ways to both specify a specific
wxPython version to use, or to fall back to the default
version, as well as provide a runtime library distribution for
wxPython. I don't know if the wxPython runtime would be
separate from the Python library, GTK, etc. or if those would
be statically linked, or even if such details are orthagonal to
this discussion.

I also recall some discussion of how to mangle module loading to
'do the right thing', and the hope that this would be easier
once Python supports relative imports.

In short, I agree that the idea of a runtime library for
wxPython should be pursued. I nominate the R'Bot to head up
this effort. <g>

···

--
Paul McNett
Independent Software Consultant
http://www.paulmcnett.com

19

Ed Leafe wrote:

    OK, back to my original question: does anyone know enough about the whole compilation process for wxWidgets to help me figure out how to either a) build the libraries completely static,

Ask on wx-users.

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

20

Paul McNett wrote:

Chris Barker writes:

I think the wxPython Run time idea
would be good here!

A wxPython runtime would go a long way to making wxPython applications much easier to distribute, hell, even to test different versions.

I would do it if I knew how. I'd start it if I knew how. Did anything fruitful come out of the various discussions of approximately 6 months ago on the topic?

There was a couple strawmans and a wiki page: http://wiki.wxpython.org/index.cgi/wxPRE

In short, I agree that the idea of a runtime library for wxPython should be pursued. I nominate the R'Bot to head up this effort. <g>

It's definitly a dream of mine. Maybe one of these days...

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!