Cache issues with Python virtualenvs

python

#1

I’m trying to use Circle CI to test Buildbot project.

From time to time following issue occurs during installation of python packages:

$ pip install -e pkg -e master -e slave -e worker

Traceback (most recent call last):
  File "/home/ubuntu/virtualenvs/venv-2.7.11/bin/pip", line 7, in <module>
    from pip import main
  File "/home/ubuntu/virtualenvs/venv-2.7.11/lib/python2.7/site-packages/pip/__init__.py", line 4, in <module>
    import logging
  File "/home/ubuntu/.pyenv/versions/2.7.11/lib/python2.7/logging/__init__.py", line 26, in <module>
    import sys, os, time, cStringIO, traceback, warnings, weakref, collections
  File "/home/ubuntu/.pyenv/versions/2.7.11/lib/python2.7/collections.py", line 9, in <module>
    from operator import itemgetter as _itemgetter, eq as _eq

pip install -e pkg -e master -e slave -e worker returned exit code 1

ImportError: /home/ubuntu/virtualenvs/venv-2.7.11/lib/python2.7/lib-dynload/operator.so: undefined symbol: _PyUnicodeUCS4_AsDefaultEncodedString Action failed: pip install -e pkg -e master -e slave -e worker

For example here: https://circleci.com/gh/buildbot/buildbot/112

This issue is not correlating with author of PR, is this PR or project branch, or with changed code (this error occurs before any actual project testing is started).

I debugged this issue here and my conclusion was following: sometimes Circle CI uses cached virtualenv directory that is incompatible with Python environment on used Circle CI VM.

Restarting build without cache usually “fixes” specific build (failed: circleci.com/gh/buildbot/buildbot/77), fixed: circleci.com/gh/buildbot/buildbot/78).

Here is circle.yml that I use:

machine:
  python:
    version: 2.7.11

dependencies:
  override:
    - pyenv global 2.7.11
    - pip install -e pkg -e master -e slave

test:
  override:
    - make docs

general:
  artifacts:
    - "master/docs/_build/html/"

Notice, that it’s not important what is being installed, since pip startup fails.

Is this is a Circle CI bug? (I think it is.)

Or is this my misconfiguration?


#2