Don't try to manage virtualenvs

Build Environment Programming Languages
1

The CircleCI documentation mentions that Python environments automatically set up a virtualenv:

When Circle detects Python, we automatically use virtualenv to create an isolated Python environment.

I would highly prefer that it did not attempt to do any management of the virtual environment. One reason why is that users do not run shells inside a virtualenv locally, so our test setup scripts need to create an environment. It would be good to keep this behavior consistent between our CircleCI runner and our local environment.

Another reason why I donā€™t want this behavior is because the virtualenv version is 1.10; the latest virtualenv version is 15.1, and contains numerous stability and security fixes.

Furthermore it appears to complicate getting things working; for example when I try to run the following in dependencies/override:

pip install --upgrade pip virtualenv
virtualenv venv

fails with the following baffling error:

virtualenv venv
Please use the *system* python to run this script
Traceback (most recent call last):
  File "/home/ubuntu/webhooks/venv/bin/virtualenv", line 9, in <module>
    load_entry_point('virtualenv==15.0.1', 'console_scripts', 'virtualenv')()
  File "/home/ubuntu/webhooks/venv/lib/python2.7/site-packages/virtualenv.py", line 708, in main
    symlink=options.symlink)
  File "/home/ubuntu/webhooks/venv/lib/python2.7/site-packages/virtualenv.py", line 921, in create_environment
    site_packages=site_packages, clear=clear, symlink=symlink))
  File "/home/ubuntu/webhooks/venv/lib/python2.7/posixpath.py", line 360, in abspath
    if not isabs(path):
  File "/home/ubuntu/webhooks/venv/lib/python2.7/posixpath.py", line 54, in isabs
    return s.startswith('/')
AttributeError: 'NoneType' object has no attribute 'startswith'
make: *** [venv] Error 1

Iā€™m not sure how to work around this; I guess I could call deactivate before literally every command I run ?

It would be nice if there was an option to opt out of this.

2 Likes
2

To get a newer version of virtualenv (15.0.0) by default, you can switch to the Ubuntu 14.04 image via you projectā€™s ā€˜Build Environmentā€™ settings.

I donā€™t have an answer to the first part of you question. Perhaps someone else will have a workaround.

3

OK. I guess I am confused why the newer image with the upgraded software version is not the default image.

4

In our project virtualenv is also picked up as an inference. I agree that it would be nice if we could disable this.

5

I would also prefer to disable virtualenv entirely. For our use case it doesnā€™t make sense and only adds unneeded complexity (we use conda and would prefer it manage environments for us)

6

Iā€™d be happy with at least some documentation of how the virtualenv is setupā€¦ i.e. what is the path, at which stage does it get activated etc

the docs are not very systematic, itā€™s a bunch of stories designed to make everything look super easy but lacking detail

1 Like
7

This wonā€™t happen overnight but I opened a new issue to improve virtualenv information.

If you have any additional feedback to help improve docs, please feel free to open some issues and/or comment on existing ones. Weā€™re working hard to improve docs.

8

Using both virtualenv and pyenv is quite confusing because they conflict with each other. The Python configuration only mentions pyenv and virtualenv is hidden away in Python-specific pages but then the examples only talk about pyenv!

As a concrete example, if I add a machine post step for pyenv global 3.5.3, then I expect Python 3 to be the default for both the python and pip commands. But the virtualenv is created with Python 2 and placed at the beginning of the path, so now those both refer to Python 2.

The reason for using a post step is to pick a Python version based on container, giving some easy parallelism. But this is totally broken if I canā€™t just write python and get the version I picked at the beginning.

Some documentation recommendations:

  • Both pyenv and virtualenv should be mentioned in both of the above linked spots
  • The machine ā€œpostā€ step should not be called ā€œpostā€ if it runs before inferred virtualenv.

Implementation recommendations:

  • Why is a virtualenv even needed when running in a container anyway?
  • Make ā€œpostā€ actually be post!

Iā€™ve resorted to deleting the whole virtualenv directory in the dependencies step just to get the right versions.

9

I think Iā€™ve found a slightly better workaround for my case here. The important step being to install virtualenv after changing Python versions. This makes the virtual env used by CircleCI use the Python version I want without some hacky deletion. It doesnā€™t matter whether itā€™s pre: or post: since the inferred virtualenv command always runs after.