Issue with python install on machine executor

Build Environment
1

We are having issues trying to use the python pandas lib in the machine executor with image ubuntu-1604:201903-01. It seems that a the lzma module from the stdlib is missing from the python installs on that machine. I’m getting the following error when trying to import pandas:


 File "/home/circleci/project/env/lib/python3.6/site-packages/pandas/io/common.py", line 9, in <module>
    import lzma
  File "/opt/circleci/.pyenv/versions/3.6.5/lib/python3.6/lzma.py", line 27, in <module>
    from _lzma import *
ModuleNotFoundError: No module named '_lzma'

The test command is as follows:

            pyenv global 3.6.5
            python -m venv env
            source env/bin/activate
            python --version
            pip install pandas==0.25.0
            python -c "import pandas"

This works just fine if instead of using machine we use the docker image for the corresponding python version. To me, it seems that the issue is with the way the python versions for pyenv were compiled for this machine image. I tried installing everything that the pyenv docs list as prerequisites and noticed that some of those libs are in fact not installed on the machine images. This, however, did not fix my problem as I’m guessing the python version would have to be recompiled after those libraries are installed.

2

FYI my earlier suspicions seem to be correct as preceding the build with the following command fixes the issue:

   
       command: |
            pyenv uninstall  -f 3.6.5
            sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \
            libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
            xz-utils tk-dev libffi-dev liblzma-dev python-openssl git
            pyenv install 3.6.5

if anyone experiences a similar issue you can use that as work around and cache the image though is is sub-optimal

3

If anyone still ends up here, this issue has been fixed in machine image tags 202101-01 tag, i.e. ubuntu-2004:202101-01

You can verify with this config:

version: 2.1
jobs:
  sample:
    machine:
      image: ubuntu-2004:202010-01
    steps:
      - run:
          name: load latest
          command: |
            python3 -c 'import lzma; print(dir(lzma))'
  sample-latest:
    machine:
      image: ubuntu-2004:202101-01
    steps:
      - run:
          name: load latest
          command: |
            python3 -c 'import lzma; print(dir(lzma))'

workflows:
  main:
    jobs:
      - sample  # this one fails
      - sample-latest  # this one passes