I’ve got an insanely long build and installation process that times out in Travis CI. wxPython 4 takes forever to build, but it’s built, things are fine. Travis, however, seems to time out somewhere during tox integration.
I think I can solve this by building a Docker container first, pulling it in the test, and running everything there, but I’m not sure where to start. Does anyone have any suggestions or know of similar projects I could look at? Has anyone else encountered this problem and found a good solution? I used this method to test Ansible roles on specific Fedora versions, but I haven’t been able to figure out how to apply it here.
I’ve experimented with a few different Travis-builtins, like travis wait, all without any luck. tox seems to hang for an inordinate period of time. When I ran it verbosely, via tox -vv, it looks like it was rebuilding things and then Travis quit because the log exceeded acceptable limits. This might suggest that I have a bad tox configuration, but I’m not quite sure how to debug that. I’m still fairly new to Python development, and rarely (if ever) run a CI build that has to compile so much before beginning tests.
I couldn’t figure out how to make tox or tox-travis work with the build matrix. No matter what, it kept creating its own virtualenv and ignoring sitepackages = True. This means I was both installing the wheel in the Travis virtualenv and then attempting to install (and build) the raw package in the toxvirtualenv.
pytest makes it easy to use the test subcommand: docs
I was using tox to test in multiple environments, which is basically the same thing as specifying multiple python versions in the .travis.yml
Current setup.cfg:
However, there are some things about this solution that bug me:
The whl is limited to either GTK 2 or GTK 3
The whl locks in the OS, so the test isn’t portable
I still have to have tox.ini to run tests locally across Python versions (which are fine because I built it already)
I have to include pytest-runner as an install dependency even though no one not running the tests needs it
So my original question about a Docker build still applies, but I do have a temporary solution. Getting tox help, which could also solve this, probably isn’t pertinent to this group.
Unless you really need to build wxPython yourself for some reason you can just have pip install the pre-built wheel from Index of /wxPython4/extras/linux in your travis install section.
While on the subject, wxPython’s own Travis-CI builds are now timing out most of the time. If anybody has ideas for improving the build times, or extending the time allotted, please let me know.
···
On Sunday, February 18, 2018 at 11:47:08 AM UTC-8, CJ Harries wrote:
I’ve got an insanely long build and installation process that times out in Travis CI. wxPython 4 takes forever to build, but it’s built, things are fine.
Might be a bit involved, and potentially result in inconsistent
builds, but considering that building wx itself accounts for 10
minutes of the build, maybe there's a chance you could setup automated
Docker Hub builds of wx (tagged for each branch), and then have the
wxPython Travis builds pull docker images with those pre-built
binaries to use instead.
Of course, some downsides to that idea:
- Some wxPython/Phoenix builds could pull an image that's a commit or
two behind what the build might be expecting.
- If your wxPython build matrix uses different settings for the wx
build itself, this will be even tougher to setup.
I think the general idea is that you're testing wxPython stuff in
these builds, not wx compilation itself (which should be well covered
in the wxWidgets Travis builds), so that's mostly why I suggest this.
Seems like the longest task is "python build.py build_py", coming in
around 24 or 25 minutes. I can't really tell if the "--jobs=4" option
is actually being passed all the way down to SIP's "-j4" option, but
it seems like nothing is being processed in parallel there, and if so,
there's likely a bit win you can make here, possibly shaving off
another 10 minutes from the build.
···
On Mon, Feb 19, 2018 at 3:01 PM, Robin Dunn <robin@alldunn.com> wrote:
While on the subject, wxPython's own Travis-CI builds are now timing out
most of the time. If anybody has ideas for improving the build times, or
extending the time allotted, please let me know.
While on the subject, wxPython’s own Travis-CI builds are now timing out most of the time. If anybody has ideas for improving the build times, or extending the time allotted, please let me know.
Not sure if it is worth the hassle for you, but I have some experience with long running builds of the FEniCS project (see, e.g. Bitbucket ) and we have switched to CircleCI. You get 120 minute timeouts per command instead of 50 minutes on TravisCI and open source projects gets 4x parallelism so you can enable parallel building if the tool chain supports it (?). I believe we cut our CI time more than in half (now under one hour total). In general I have been very happy with their service and use it for building, testing and also building of docs and automatically pushing static pages to web hosting for personal projects as well. Hope they make enough money to stay around, but I have never paid anything. It seems like a good service to try if you need more than TravisCI can offer (note that I have only limited experience with other CI services, there may be better ones out there).
Thanks! I’m switching to CircleCI right now. Once I was able to filter out the instructions and other docs for their old 1.0 version it wasn’t too hard to put together a 2.0 version config for wxPython builds. It’s faster too. The wxPython builds are complete and successful in about 25-30 minutes, where the TravisCI builds had about a 85% chance of failing due to timing out at 50 minutes.
···
On Tuesday, February 20, 2018 at 8:15:58 AM UTC-8, Tormod Landet wrote:
While on the subject, wxPython’s own Travis-CI builds are now timing out most of the time. If anybody has ideas for improving the build times, or extending the time allotted, please let me know.
Not sure if it is worth the hassle for you, but I have some experience with long running builds of the FEniCS project (see, e.g. https://bitbucket.org/fenics-project/dolfin/commits/all ) and we have switched to CircleCI. You get 120 minute timeouts per command instead of 50 minutes on TravisCI and open source projects gets 4x parallelism so you can enable parallel building if the tool chain supports it (?). I believe we cut our CI time more than in half (now under one hour total). In general I have been very happy with their service and use it for building, testing and also building of docs and automatically pushing static pages to web hosting for personal projects as well. Hope they make enough money to stay around, but I have never paid anything. It seems like a good service to try if you need more than TravisCI can offer (note that I have only limited experience with other CI services, there may be better ones out there).