Circleci/python:3 doesn't search system path for headers, libraries

For my Python projects, I’ve switched from the ubuntu-18:04 images to circleci/python:3 recently, and it dramatically sped up my builds. Yeay!
Unfortunately, since this week, I see all kinds of weird error popping up in my builds. Some of the errors are due to the fact that apt-installed python libraries don’t seem to be recognized. That’s fair enough, probably the build is not using system Python, so none of the apt-packages matter anyway. One issue that I cannot fix, however, is the fact that the pip-installable packages don’t search system paths for dependent libraries and headers, e.g., libhdf5-dev. This results in build errors [1].

Any hints here?

Nico

[1] https://circleci.com/gh/nschloe/meshio/1276

I’m not really a fan of convenience images. I understand they’re handy to try things out quickly, but they are susceptible to upstream breakages precisely because they are trying to be all things to all people.

If you can, I would suggest building your own Python image and setting up a pipeline for this that runs weekly, and pushes the image to a Docker registry (free ones are available). My preference is to use Alpine as a base, so that resulting images are 100-200M in size, rather than 1G+.

In the short term, you could see if the upstream CircleCI image has changed, and then pin to the last known working hash. That will probably get you back up and running.

You’re using a very generic version tag for the Python image which I wouldn’t suggest. circleci/python:3 will always refer to the latest minor release of Python, which can have changes.

About a week ago Python v3.8 was release meaning shortly after that, the tag you’re using should have switched from pointing to Python v3.7 to v3.8. I wonder if that has something to do with this.

Can you please try changing the image you’re using from circleci/python:3 to circleci/python:3.7 and let us know if that helps?

Thanks,
Ricardo

1 Like

Good call, switching to circleci/python:3.7 fixes it.

1 Like