Unable to Compile Python C Extensions


#1

A user reports:

When explicitly setting the Python version to use, as described in https://circleci.com/docs/configuration/#python-version, the python env we get doesn’t contain libpython.so.
libpython.so is needed to “pip install” some Python C extensions.

For example, try installing the Shiboken package when setting the python version to either 2.7.11, 2.7.10, 2.7.9, 2.7.5 or 2.7.6, and you’ll get an error, because the build tries to use libpython.a when libpython.so is not found.

Even when setting the Python version to 2.7.6, the one that’s used if no version is explicitly selected, building some packages fails.


#2

While still a bug, here is a workaround.

For 2.7.11, adjust destination path as needed for version:

- cp /usr/lib/x86_64-linux-gnu/libpython2.7.so /opt/circleci/python/2.7.11/lib/libpython2.7.so

This will allow libpython2.7.so to be found.

example build of Shiboken: https://circleci.com/gh/drazisil/circleci-testing/267


#3

Hi, OP here.

The system-wide Python version on the Ubuntu 12.04 image is 2.7.6.
Is it safe to build Python C extensions against a libpython2.7.so of a different version than the one that will import them later?

Also, on the Ubuntu 12.04 image the appropriate place to copy the libpython2.7.so file seems to be /home/ubuntu/.pyenv/versions/2.7.11/lib/libpython2.7.so. There’s no /opt/circleci/python/2.7.11/lib dir on that image.


#4

My work around was for Trusty, give me a few to take a look around the Precise image.

It is this then, for that version:

- cp /usr/lib/x86_64-linux-gnu/libpython2.7.so /home/ubuntu/.pyenv/versions/2.7.11/lib/libpython2.7.so

I copied it to where I saw it was looking for it, you can change location if you perfer.

However, a build under 12.04 fails with qt errors following the instructions from https://pypi.python.org/pypi/Shiboken/1.2.2

ref: https://circleci.com/gh/drazisil/circleci-testing/270


#5

I found a fix that doesn’t depend on the image:

machine:
  pre:
    - test $(pyenv install --list | grep 2.7.12) || (cd /home/ubuntu/.pyenv/plugins/python-build/../.. && git pull && cd -)
    - pyenv local 2.7.12 || (PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 2.7.12 -v)
    - pyenv local 2.7.12
    - pip install virtualenv
  python:
    version:
      2.7.12

The first ‘pre’ line updates pyenv (if needed) so that 2.7.12 is available.
The second line installs Python 2.7.12 (if not installed already) in the system, with the ‘–enable_shared’ flag that’s missing in all the pre-installed available pyenv versions.
The third chooses Python 2.7.12 as the currently active python.
The fourth installs virtualenv for our Python 2.7.12, so that the automatic ‘machine: virtualenv’ step can run.

Shiboken builds successfully on Precise this way, but it does involve re-building Python each time, which takes ~2:13 minutes.

Then I tried to bypass building Python over and over by using the share-enabled executable that is cached in the project’s virtualenv at the end of the build, but I couldn’t get it to work.


Don't try to manage virtualenvs
#6