Interesting Swig Enhancements

wxPython Dev
1

Hello,

I was surprised that to wrap C++ classes, Swig must first "flatten"
all methods in (say) "core_wrap.cpp", then rebuild the classes in
"core.py".

Since python 2.2, it is possible to build classes from the C API.

So I've made some changes to Swig, which I think could be interesting to
share.

I heavily changed the code in "Python.cxx", and now:
- classes and methods are all built from the "wrap.cpp" code
- "core.py" shrunk a lot: it contains a "from _core import *",
  plus all the %pythoncode...
- All the "thisown" stuff is done at the C level
- No more "xxxPtr" classes
- all objects derive from the "swig_base" type
- "pythonappend" and "pythonprepend" are included in "wrap.cpp"
- I removed "-modern" and "-classic" options
  (all is "supermodern" now :wink:

There are many advantages:
- my "windows.py" module is only 66 lines instead of 3260!
  (removing blank lines and comments)
- less memory: after "import wx",
  python.exe uses 11Mb instead of 14Mb before,
  82,251 python refs instead of 205,276 (python_d says)
- seems twice faster to load (Though I did not measure it precisely)
- function calls should be faster too.

Still, it it not an easy task, and the work far from complete.
Also, It's possible that I did not understand everyting correctly.
That's why I'm also asking for help...

Enough of the demo is working to be sure that we can work around
the remaining problems.

The main problem is that changes are not backward compatible.
But I guess that it's because wxPython codes is too much aware
of the underlying Swig structure.
So I consider this not so important.

Here are the current issues:
- Lot of Python objects are not correctly freed
  (problems in the garbage collector are not easy to correct...)
- Tested only on Windows 2000
- Removed the "_UnbornObject" and "_DeadObject" classes.
  I think this should be handled with a special attribute of the
  object instead of special classes.
- static methods are no more available as "Class_StaticMethod(args)".
  Use "Class.StaticMethod(args)" instead.
- "%pythoncode { asTuple=Get }" don't work any more.
  "%pythoncode { asTuple=Size.Get }" should work though.
- special methods (__nonzero__, __getitem__) don't work.
  We need some Swig magic to detect this and fill the appropriate
  slots of the PyTypeObject.
  I've done this for Size.__getitem__, but it is far from perfect.
- the Swig code (as well as libpy.c,helpers.cpp) needs tidying seriously...

I would attach the files I changed, but 600Kb seems big for a mailing list.
I'll send it to anybody who asks!
I'm not subscribed to the mailing list but I often check the archives.
(my name is amaury dot forgeotdarc and my domain name is ubitrade dot com)

···

--
Amaury Forgeot d'Arc

2

Hello,

I was surprised that to wrap C++ classes, Swig must first "flatten"
all methods in (say) "core_wrap.cpp", then rebuild the classes in
"core.py".

Since python 2.2, it is possible to build classes from the C API.

So I've made some changes to Swig, which I think could be interesting to
share.

I've considered doing this several times in the past, but always came to the conclusion that having the flexibility to be able to graft things in to the Python proxy classes was very valuable (for example, almost everything in your "current issues" list) and I didn't want to give it up. I'd still be interested though in seeing if there is a way to combine the two approaches...

I heavily changed the code in "Python.cxx", and now:
- classes and methods are all built from the "wrap.cpp" code
- "core.py" shrunk a lot: it contains a "from _core import *",
  plus all the %pythoncode...
- All the "thisown" stuff is done at the C level
- No more "xxxPtr" classes
- all objects derive from the "swig_base" type
- "pythonappend" and "pythonprepend" are included in "wrap.cpp"

Interesting. Do you embed and call out to the Python code from the C/C++ wrappers?

- I removed "-modern" and "-classic" options
  (all is "supermodern" now :wink:

There are many advantages:
- my "windows.py" module is only 66 lines instead of 3260!
  (removing blank lines and comments)
- less memory: after "import wx",
  python.exe uses 11Mb instead of 14Mb before,

For real apps this isn't really as big as it sounds.

  82,251 python refs instead of 205,276 (python_d says)

But this is certainly nice.

- seems twice faster to load (Though I did not measure it precisely)
- function calls should be faster too.

As well as this. In fact the function call speed improvement was one of the main reasons I looked into doing this before.

Are you also getting the new docstrings for the classes/methods/functions into the extension code?

Here are the current issues:
- Lot of Python objects are not correctly freed
  (problems in the garbage collector are not easy to correct...)
- Tested only on Windows 2000

- Removed the "_UnbornObject" and "_DeadObject" classes.
  I think this should be handled with a special attribute of the
  object instead of special classes.

So every single method and atrtribute access would have to check this special attribute?

- static methods are no more available as "Class_StaticMethod(args)".
  Use "Class.StaticMethod(args)" instead.
- "%pythoncode { asTuple=Get }" don't work any more.
  "%pythoncode { asTuple=Size.Get }" should work though.
- special methods (__nonzero__, __getitem__) don't work.
  We need some Swig magic to detect this and fill the appropriate
  slots of the PyTypeObject.
  I've done this for Size.__getitem__, but it is far from perfect.
- the Swig code (as well as libpy.c,helpers.cpp) needs tidying seriously...

I would attach the files I changed, but 600Kb seems big for a mailing list.
I'll send it to anybody who asks!

Please send me a copy.

···

amaury.forgeotdarc@ubitrade.com wrote:

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

3

In general I agree, however this is a problem for some people. I ported
python (not wxPython :wink: )onto PS2,Xbox,and GC and used SWIG to generate a
scripting interface to our games.
The overhead of the shadow classes was huge. We really don't have memory to
waste here. I also cannot stress just how slow python is on a processor such
as the PS2 - especially in the function call mechanism from python to c++
via swig.
Andrew

···

>
> There are many advantages:
> - my "windows.py" module is only 66 lines instead of 3260!
> (removing blank lines and comments)
> - less memory: after "import wx",
> python.exe uses 11Mb instead of 14Mb before,

For real apps this isn't really as big as it sounds.

> 82,251 python refs instead of 205,276 (python_d says)

_____________________________________________________________________
This e-mail is confidential and may be privileged. It may be read, copied and used only by the intended recipient. No communication sent by e-mail to or from Eutechnyx is intended to give rise to contractual or other legal liability, apart from liability which cannot be excluded under English law.

This message has been checked for all known viruses by Star Internet delivered through the MessageLabs Virus Control Centre.

www.eutechnyx.com Eutechnyx Limited. Registered in England No: 2172322