Phoenix build testing needed

Hi all,

Sorry for the huge email here. I intended for it to be a lot shorter, but I think it is worth it. If you make it all the way through it you have my thanks. :slight_smile:

If you've been watching the Phoenix commits or are getting the notification emails you probably noticed that I checked in a truck-load of stuff last night, mostly all of them related to the build. Here are a few highlights:

* On non-Windows platforms build.py can now use an already installed wxWidgets instead of assuming that it will be building and using the one in the same source tree. It will still need to be a version fairly close to the version of the Phoenix source code though, at least for a while. Build.py can also perform the build and install of wxWidgets for you of course, and then you can have it build Phoenix to use that installed wxWidgets too, instead of bundling them together.

* The build.py + waf scripts and setup.py are no longer duplicating the same build actions. Setup.py will now invoke build.py if it thinks it needs to do a build, and build.py will invoke setup.py to do things like installs of the wxPython package. This avoids duplication of effort, testing, and allows each to do what they do best. Using waf for building a huge library like this is light-years better than relying on distutils to do it. On the other hand, distutils/setuptools is very good at knowing how to properly install python packages, either into a real Python site-packages or into a virtualenv, how to make eggs, and do other things that people typically want to do with python packages.

* In support of the above build.py has added a few commands that basically just invoke setup.py to do them: install_py, bdist_egg, bdist_wininst. More can be added. I also tried bdist_msi but apparently the msi code requires the the version number is only 3 components long, and as you know wxPython uses 4. If anybody knows of a workaround for that please let me know.

* You can now tell build.py exactly which python executable to build for by using the --python parameter, instead of only being able to tell it the Python version and relying on it to find the right one. If neither a --python flag nor a N.N version number are used then the Python that invoked build.py will be used, just like how setup.py builds typically do it.

* Sip has been updated to 4.14.4, that's a lot of 4s.

* Magic. ??? Yes, Phoenix has become even more magical than before. You may have noticed my use of the term "magic" in the commit messages and in the comments and help text of build.py. This is just my whimsical way of saying that the wxWidgets shared libraries can now be bundled with wxPython in a relocatable way, on all platforms not just Windows, and without needing to set any environment variables. Yes, really. That's why I call it magic. Of course this has always worked on Windows and in Classic we always installed the wx DLLs in the same folder as wxPython's .pyd's and all was peachy without needing to do anything extra. Now we can do the same thing for Mac and any unix that uses the ELF style binaries and dynamic loader (I think). On Mac build.py rewrites the .so/.dylib headers that specify what dylibs are to be loaded so they use "@loader_path" and on Linux it will cause that the extension modules use an rpath of "$ORIGIN", which basically does the same thing. I've mentioned eggs a couple times now, and it's the magic that allows that to happen. In other words you can have a binary Phoenix egg that is totally self-contained, and install it in to any virtualenv no matter where it is located, and it will be able to find its own wx shared libs all by itself. Or you could have different versions of Phoenix eggs installed and choose between them at runtime using package manager (no more wxversion required.) The magic can be turned off :frowning: if you use --no_magic or --use_syswx build.py flags. That probably will make linux package maintainers and other purists happier since they tend to abhor duplicating anything on the system, but personally I feel that the relocatability and the independence from interference by whatever wxWidgets may happen to be installed on the system beats out non-duplication of libraries any day.

How you can help:

* Obviously since so much has changed there is a greater chance that something has gone wrong in some corner cases somewhere, so it would be nice to try and discover those cases and to fix or work around them. So if you're able to do your own builds (IOW, have a compiler and know how to use it) please pick a scenario or two and try it out, either with the current trunk of wxWidgets and Phoenix, or with the source tarball that will be created tonight or later. Also, look at the output of "build.py --help" and see if it makes sense. If you run into problems please try to fix them and submit a patch, otherwise let me know and I'll try to look into it.

* I've already mentioned that build_msi won't work with more than 3 components in the version number. It would be great if somebody could look in to that and see if there is a monkey patch or some other workaround we could do to allow it.

* There is one more big thing that I want to do and that is to be compatible with builds performed by easy_install and pip. That will require that there is a setup.py in the root folder of the source tarball, but currently we have just the wxWidgets and Phoenix folders there, and the builds need to be performed in the Phoenix folder. Obviously we could move the wxWidgets folder into the Phoenix folder and then make the current Phoenix folder be the root, but I would rather avoid that if possible for various reasons. I expect that putting a new setup.py in the root that simply changes the CWD to Phoenix and then delegates to the real setup.py there would do the job, but it would be nice to avoid reinventing the wheel or rediscovering problems if somebody has already worked this out in another project. So if you've seen another project that does something like that then please let me know about it. Or perhaps setuptools/distribute already has a mechanism for redirecting the auto-build into a subfolder and I don't know about it yet. That would be nice too.

Thanks!

···

--
Robin Dunn
Software Craftsman
http://wxPython.org