Linux wheels with manylinux2010

But it being based on CentOS 6 makes me think that it may not be better. CentOS 7 is probably the minimum needed.

OK, well I’ll be interested to hear if you make any progress there. I was working on a project that needed XInput2 and the step to Centos6 was enough to get that one over the line, so I was hoping it would help wx too

There is now a manylinux2014 built on Centos7 and it looks like they’re adding newer ones based on debian versions:

It would sure be great to get linux wheels fixed for wxPython. It’s still always a pain point for our (PsychoPy’s) ubuntu users

Revisiting this and have made some progress. Since it was last discussed:

  • there is now a manylinux2014 (based on Centos 7)
  • there is now a cibuildwheel available pretty easily on GitHub Actions that makes it pretty easy to use manylinux without handling docker environments directly

With the new tools I’ve got quite far (I think). So far I’m just trying this for Py3.8 but the system should extend up to any versions that are supported, as well as handling win/mac builds all still using cibuildwheel. The c-compiling steps all now seem to run, but the building the wheel isn’t working due to python packaging.

You can see logs of the builds here
Actions · peircej/wxpython_buildwheels · GitHub
I’m still tweaking the dependency configs - it’s currently failing because sip/siplib license isn’t where waf expects it to be :cry: but I hope this can be made to work and anyone with time to help debug the build config would be welcome to join in!

Ah, the current issue, with failing to find sip/siplib/license.txt is that that file was deleted in this commit:
Update to use sip 6.6.x · wxWidgets/Phoenix@90171ba · GitHub

We either need to add sip/siplib/license.txt back in or we need to remove the line in updateLicenseFiles that requires it

Yep, looks like sip-module isn’t including the license files in the sdist it generates. I reported that to upstream sip.

Cool, in the meantime I’m testing from a separate copy of Phoenix with tweaked updateLicenseFiles to see what the next barrier will be! :laughing:

It seems quite a bit of the build architecture is expecting sip and siplib to be within this repo so when that got removed the build system broke. When I fix the license issue above, we get hit with errors about missing sip/siplib.sip.h but I can see in wscript (which is called during build) there are lots of references to the siplib files, all hard-coded to this location

The conversation around the commit suggests that builds have successfully completed on other systems though so I’m puzzled how that was possible! :thinking:

@peircej, yeah the sip/siplib files are removed from the repository, but they are now being automatically generated by this step:

BTW, the issue with the license files being missing seems to be an issue with older versions of setuptools.

There was also an issue recently with some of the sip files not being included in the snapshots, see: Fix inclusion of siplib and sip.h in sdist with sip 6.6.2 · wxWidgets/Phoenix@3a20dd4 · GitHub

I took a quick look at your log files. Please note this warning:

    running bdist_wheel
    running build
    WARNING: Building this way assumes that all generated files have been
    generated already.  If that is not the case then use build.py directly
    to generate the source and perform the build stage.  You can use
    --skip-build with the bdist_* or install commands to avoid this
    message and the wxWidgets and Phoenix build steps in the future.

Basically, you can’t use bdist_wheel without generating all of the generated files first. You might want to take a look at how the project’s CI builds its wheels.

So (taking this from buildbot/master.cfg) there need to be the following commands preceding pip build bdist_wheel?

python build.py build_wx --gtk3
python build.py dox etg sip --gtk3
python build.py build_py --gtk3

Is that what creates the files you mean? Or is this more/less than needed?

Apologies for the slightly dumb questions - I’ve never worked with a 2000-line multi-step build script before! :laughing:

Yes. That should get you what you need. Note that you don’t need --gtk3
anymore as that’s the default.

You may wish to refer to the Azure CI configuration also:

You should be able to skip the build_py command, as bdist_wheel will do that step too. (Although it’s been a while since I tested it that way…) The build.py script can set up and launch the bdist_weel command too, so if desired you can do it all on one command-line:

python build.py etg sip build_wx bdist_wheel

You should probably also toss in a --no_doc to save some time by skipping the generation of the Sphinx docs. And while develooping/testing, once you’ve generated the code one time, you don’t need to run the etg or sip steps again unless you clean the build folders.

Thanks both. I’ve made some progress.

Using:

python build.py build_wx
python build.py dox etg --nodoc sip
python build.py build_py

the build now completes but using the wheel in a separate test environment fails to import wx with a segfault. You can see the full logs for that fail here:
Move all build scripts into Phoenix project · peircej/wxpython_buildwheels@9e9c785 · GitHub

@Robin this is on a GitHub VM so it’s a clean environment every time. I’m sure there would be ways to cache intermediate files but I haven’t had time to look into that. FYI the build process is taking around 40 mins (to build the Py3.8 wheel for manylinux flavours)

Note that cibuildwheel, which I used here to provide the support for manylinux, does also run on Azure pipelines, so you wouldn’t need to change much of your current workflow to get the many linux flavours in one go. I just used Github Actions to test the principle because I know it better from my own projects

Give this a try:

It’s not based on cibuildwheel but I thought I would give this approach a try when I was playing around with it last night. It came together much easier than the last time I attempted manylinux.

1 Like

Sounds great @Robin

I was in no way wedded to using cibuildwheel - i just saw that as the easiest way to get it moving. If you’ve got a build working using an even newer variant of manylinux that’s great. If you want me to tyre-kick the resulting wheel then let me know! :slight_smile:

I’ve put a set of tires to be kicked here. Have fun.

1 Like

That’s progress! Installing the wheel worked fine and we can now load wx:

$ python3.8
Python 3.8.13 (default, Apr 19 2022, 02:32:06) 
[GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import wx
>>> print(wx.version())
4.2.0 gtk3 (phoenix) wxWidgets 3.2.0

That’s progress!

Unfortunately, @Robin was right that there are still Glib version incompatibilities that might be showstoppers:

>>> app = wx.App()

(python3.8:4795): GLib-GIO-ERROR **: 10:19:48.441: Settings schema 'org.gnome.settings-daemon.plugins.xsettings' does not contain a key named 'antialiasing'
Trace/breakpoint trap (core dumped)

This is on an x64 VM running a clean, updated Ubuntu 22.04, and Python3.8 from sudo apt install

For what is worth, I was able to build a manylinux_2_31 wheel in ubuntu 20.04 and a manylinux_2_35 wheel in ubuntu 22.04 from the regular master branch simply running:

auditwheel repair --plat manylinux_2_35_x86_64 --only-plat -w dist dist/wxPython-*.whl (replace with 2_31 on 20.04)

after building a platform specific wheel the usual way with python build.py bdist_wheel

But unfortonately neither of them --in particular the 2_35, which at the time of writing is the most new manylinux spec-- runs on a current debian stable (bookworm). These are the some of the critical errors I get on start up:

g_once_init_leave: assertion 'result != 0' failed
g_type_register_dynamic: assertion 'parent_type > 0' failed
g_type_interface_add_prerequisite: assertion 'G_TYPE_IS_INTERFACE (interface_type)' failed

You can download the wheel file with pip download --extra-index-url https://pkgs.dev.azure.com/jorgemoraleda/wxPythonMeticy/_packaging/release_artifacts/pypi/simple wxPythonMeticy (For why it is called wxPythonMeticy and not wxPython see wxPythonMeticy: A release to PyPI of the latest master branch code