OK, I'm working on getting wxMozilla to use wx.build.config. For the most part, things seem to work fine (it is building the source without problems), but I am having difficulty getting SWIG to work with it. There are a couple of issues:
1) the swig_deps are of course not included with the bundle and they assume the wxPython src path to be "./" which won't be the case for wxMozilla. And of course there will be issues when the .i file includes other .i files from the source distro. Lastly, the XML docs creation and xml_renamers expect the current directory to be in the wxPython src dir. I think it makes sense to assume that someone running SWIG will have a wxPythonSrc tree, but I think we need a way to specify the path to that. What do you think?
I tried adding WX_SRC to build options and did a "global WX_SRC" before accessing it (as you do with WX_CONFIG), but I still get undefined variable errors if the parameter isn't passed in on the command line. :-/ In any case, once I specified WX_SRC and pointed run_swig to the wxPythonSrc dir, things ran fine.
2) It looks like wx/wxPython/wxPython.h and pyclasses.h don't get copied over during a make install operation, meaning SWIG can't find them. However, it looks like putting them in include/wx shouldn't cause any problems.
3) Install copies over the extension itself, but not its associated .py file. Is there an easy way to do this? When I added it as a script or data_file, distutils kept wanting to put it in the Python binary dir (?).
Anyways, these are the major issues. If I have a solution to #1, I think I can submit a working patch to you for that part, at least!
OK, I'm working on getting wxMozilla to use wx.build.config. For the most part, things seem to work fine (it is building the source without problems), but I am having difficulty getting SWIG to work with it. There are a couple of issues:
1) the swig_deps are of course not included with the bundle and they assume the wxPython src path to be "./" which won't be the case for wxMozilla. And of course there will be issues when the .i file includes other .i files from the source distro. Lastly, the XML docs creation and xml_renamers expect the current directory to be in the wxPython src dir. I think it makes sense to assume that someone running SWIG will have a wxPythonSrc tree, but I think we need a way to specify the path to that. What do you think?
I knew there would be some rough and broken spots...
I tried adding WX_SRC to build options and did a "global WX_SRC" before accessing it (as you do with WX_CONFIG), but I still get undefined variable errors if the parameter isn't passed in on the command line. :-/ In any case, once I specified WX_SRC and pointed run_swig to the wxPythonSrc dir, things ran fine.
Sounds like a good idea.
2) It looks like wx/wxPython/wxPython.h and pyclasses.h don't get copied over during a make install operation, meaning SWIG can't find them. However, it looks like putting them in include/wx shouldn't cause any problems.
They should be copied to {prefix}/include/wx/wxPython as part of the wxPython install step. Also the core *.i files are copied as well, they go to {prefix}/include/wx/wxPython/i_files.
3) Install copies over the extension itself, but not its associated .py file. Is there an easy way to do this? When I added it as a script or data_file, distutils kept wanting to put it in the Python binary dir (?).
The extension is handled by distutils. The swig-generated .py file is copied by run_swig into the local copy of the package dir. It is then installed by distutils by virtue of being located in the local source package dir.
BTW, are you building things such that the wxMozilla modules will go directly into the wx package, or using a sub-package such as wx.mozilla?
···
--
Robin Dunn
Software Craftsman http://wxPython.org Java give you jitters? Relax with wxPython!
3) Install copies over the extension itself, but not its associated .py file. Is there an easy way to do this? When I added it as a script or data_file, distutils kept wanting to put it in the Python binary dir (?).
The extension is handled by distutils. The swig-generated .py file is copied by run_swig into the local copy of the package dir. It is then installed by distutils by virtue of being located in the local source package dir.
OK, I think I'm missing something pretty fundamental here. When I build wxPython or (now) wxMozilla, the SWIG-generated .py and .cpp file go into the directory specified by GENDIR thanks to run_swig. The object files (and resulting library files) all go into the distutils-created build folder that is created when the build starts.
The question is, how does distutils know to copy the .py files from GENDIR to site-packages/PKGDIR? Obviously, everything in the build/lib folder goes into PKGDIR, but I don't see how distutils knows about GENDIR. I can't seem to find this magic anywhere in the source, and my .py file still isn't copying, so I must be missing something. ^_^;
BTW, are you building things such that the wxMozilla modules will go directly into the wx package, or using a sub-package such as wx.mozilla?
At the moment, it goes into the root directory of the wx package. Do you have any preference?
3) Install copies over the extension itself, but not its associated .py file. Is there an easy way to do this? When I added it as a script or data_file, distutils kept wanting to put it in the Python binary dir (?).
The extension is handled by distutils. The swig-generated .py file is copied by run_swig into the local copy of the package dir. It is then installed by distutils by virtue of being located in the local source package dir.
OK, I think I'm missing something pretty fundamental here. When I build wxPython or (now) wxMozilla, the SWIG-generated .py and .cpp file go into the directory specified by GENDIR thanks to run_swig.
Does it also copy the .py file to PKGDIR?
The object files (and resulting library files) all go into the distutils-created build folder that is created when the build starts.
The question is, how does distutils know to copy the .py files from GENDIR to site-packages/PKGDIR? Obviously, everything in the build/lib
folder goes into PKGDIR, but I don't see how distutils knows about GENDIR. I can't seem to find this magic anywhere in the source, and my .py file still isn't copying, so I must be missing something. ^_^;
Distutils doesn't take the .py files from GENDIR. One of the parameters you pass to setup() is packages, which is the list of packages/subpackages (relative to the current directory IIRC) that you want distutils to deal with. It should automatically scan those directories to find the .py files that it will then install.
BTW, are you building things such that the wxMozilla modules will go directly into the wx package, or using a sub-package such as wx.mozilla?
At the moment, it goes into the root directory of the wx package. Do you have any preference?
I think that some things may be easier to deal with if you use a sub-package. For example there won't be any confusion down the road about which files in wx/* belong to particular installations of 3rd party modules. Try resetting PKGDIR after you import config to "wx/mozilla" and then set packages=["wx.mozilla"] in your call to setup(). If your wx/mozilla/__init__.py does a "from mozilla import *" then importing the pacakge will appear from the importer's perspective to be exactly the same as importing the module.
Hmmm... I just thought of one potential problem with this. The SWIG-generated mozilla.py is probably doing a "import _core" expecting to be able to find the wx._core module but since they will now be at different levels of the package heirarchy that won't work. One way to work around this is to temporarily adjust sys.path in your __init__.py. Another way is to do yet another tweak to swig to allow you to specify the pacakge that the %import'ed modules should be found in...
···
On May 24, 2004, at 11:44 PM, Robin Dunn wrote:
--
Robin Dunn
Software Craftsman http://wxPython.org Java give you jitters? Relax with wxPython!
Hmmm... I just thought of one potential problem with this. The
SWIG-generated mozilla.py is probably doing a "import _core"
expecting to be able to find the wx._core module but since they
will now be at different levels of the package heirarchy that won't
work. One way to work around this is to temporarily adjust
sys.path in your __init__.py. Another way is to do yet another
tweak to swig to allow you to specify the pacakge that the
%import'ed modules should be found in...
It would be nice to have a solution that would also enable developers
to install wxPython extensions outside of (installed) wxPython tree.
There would be two useful applications to it:
* People that don't have root access could install wxPython extensions
locally and still be able to share system-wide wxPython installation.
* It would make it possible to, say, ship private copy of wxMozilla or
your custom control implemented in C++ together with your app and
install it into /usr/lib/yourapp instead
of /usr/lib/python2.3/site-packages.
Distutils doesn't take the .py files from GENDIR. One of the parameters you pass to setup() is packages, which is the list of packages/subpackages (relative to the current directory IIRC) that you want distutils to deal with. It should automatically scan those directories to find the .py files that it will then install.
Ahhhh... I think this is what I was missing. I had thought of packages and ext_packages as separate things. Adding a 'wx' package resolves the problem.
I think that some things may be easier to deal with if you use a sub-package. For example there won't be any confusion down the road about which files in wx/* belong to particular installations of 3rd party modules. Try resetting PKGDIR after you import config to "wx/mozilla" and then set packages=["wx.mozilla"] in your call to setup(). If your wx/mozilla/__init__.py does a "from mozilla import *" then importing the pacakge will appear from the importer's perspective to be exactly the same as importing the module.
Hmmm... I just thought of one potential problem with this. The SWIG-generated mozilla.py is probably doing a "import _core" expecting to be able to find the wx._core module but since they will now be at different levels of the package heirarchy that won't work. One way to work around this is to temporarily adjust sys.path in your __init__.py. Another way is to do yet another tweak to swig to allow you to specify the pacakge that the %import'ed modules should be found in...
So the rule that sub-packages can refer to each other doesn't apply with SWIGged modules? Bummer. ;-/
As a status update, everything is building and installing now, except for build_renamers. =) Speaking of which, should build_renamers become part of config.py? What do you think?
So the rule that sub-packages can refer to each other doesn't apply with SWIGged modules? Bummer. ;-/
The issue is that if you put this in your .i file
%import othermodule.i
Then SWIG will generate this in you .py output file:
import othermodule
There is no way (that I know of) to tell it to generate this instead:
import pacakgename.othermodule
As a status update, everything is building and installing now, except for build_renamers. =) Speaking of which, should build_renamers become part of config.py? What do you think?
I think that for this build model it certainly makes sense. If I'm able to switch to bakefile then it will probably have to become a standalone script again but we can cross that bridge later.
BTW, if you've looked at the code for build_renamers you've probably see that it can use either libxml2 or xml.sax for parsing. I'm pretty sure that the xml.sax version is no longer correctly generating what it is supposed to (because the libxml2 version migrated as I was developing and I don't think I ever went back and made the xml.sax version do the same things...) So to be on the safe side make sure you have the libxml2 module installed (it's faster anyway.)
···
--
Robin Dunn
Software Craftsman http://wxPython.org Java give you jitters? Relax with wxPython!
So the rule that sub-packages can refer to each other doesn't apply with SWIGged modules? Bummer. ;-/
The issue is that if you put this in your .i file
%import othermodule.i
Then SWIG will generate this in you .py output file:
import othermodule
There is no way (that I know of) to tell it to generate this instead:
import pacakgename.othermodule
OK, I'll play with this once I move build_renamers over.
As a status update, everything is building and installing now, except for build_renamers. =) Speaking of which, should build_renamers become part of config.py? What do you think?
I think that for this build model it certainly makes sense. If I'm able to switch to bakefile then it will probably have to become a standalone script again but we can cross that bridge later.
Do you mean use bakefile in place of distutils? I thought distutils was supposed to make things easier for people building Python modules/extensions. =)
BTW, if you've looked at the code for build_renamers you've probably see that it can use either libxml2 or xml.sax for parsing. I'm pretty sure that the xml.sax version is no longer correctly generating what it is supposed to (because the libxml2 version migrated as I was developing and I don't think I ever went back and made the xml.sax version do the same things...) So to be on the safe side make sure you have the libxml2 module installed (it's faster anyway.)
OK, why don't I just remove the xml.sax version then? A small number of people will be running the script anyways, and setting up libxml is a one time thing, so I think it's better just to remove the xml.sax version so that no one tries to use it. =)
I think that for this build model it certainly makes sense. If I'm able to switch to bakefile then it will probably have to become a standalone script again but we can cross that bridge later.
Do you mean use bakefile in place of distutils? I thought distutils was supposed to make things easier for people building Python modules/extensions. =)
Yes. As can be seen from the complexity and fragility of the current setup.py/config.py I don't think that distutils is a very good solution. It's fine for python-only pacakges or for simple extensions, but it has always been a struggle getting everything working right for wxPython's build. The proposed distutils2 might get there, but who knows how long it will take and I don't really have time to make sure that it does get there.
Now that we have the bakefile tool that can generate makefiles and project files from a single set of build definitions I think that it is time to try it out and see what happens.
BTW, if you've looked at the code for build_renamers you've probably see that it can use either libxml2 or xml.sax for parsing. I'm pretty sure that the xml.sax version is no longer correctly generating what it is supposed to (because the libxml2 version migrated as I was developing and I don't think I ever went back and made the xml.sax version do the same things...) So to be on the safe side make sure you have the libxml2 module installed (it's faster anyway.)
OK, why don't I just remove the xml.sax version then?
That's fine.
···
--
Robin Dunn
Software Craftsman http://wxPython.org Java give you jitters? Relax with wxPython!
So the rule that sub-packages can refer to each other doesn't apply with SWIGged modules? Bummer. ;-/
The issue is that if you put this in your .i file
%import othermodule.i
Then SWIG will generate this in you .py output file:
import othermodule
There is no way (that I know of) to tell it to generate this instead:
import pacakgename.othermodule
Ok, try it with the new swig patch I just checked in. (It's in the docstring patch file although it's not really related to docstrings. But since the source files are the same the patches overlap...)
I added support for a new option for the %module directive that let's you specify the package that the generated module will live in. Then when other .i files %import that one they can compare the package name with their own and if they are different packages then it will generate the import and the class statements in the .py file appropriately. For example, all the wxPython .i files now have this:
%module(pacakge="wx") whatever
So if wxMozilla uses this:
%module(pacakge="wx.mozilla") mozilla
%import _core.i
Then it will generate code like this:
import wx._core
class MozillaWindow(wx._core.Window):
...
instead of this
import _core
class MozillaWindow(_core.Window):
...
If you don't specify a package name in your .i file and the one you %import does then it will still prepend the package name like the above. It is only in the case where the importer and the importee have matching package names (or both have none) that the pacakge names are not used.
···
--
Robin Dunn
Software Craftsman http://wxPython.org Java give you jitters? Relax with wxPython!
