Cached venv in tox

python

#1

Venv created by tox is not cached between builds, which slows down the build significantly. Do you have any plans to fix this?

Cheers P :smile:


#2

Where is the created venv stored? You should be able to cache it manually with the dependencies: cache_directories section in your circle.yml.


#3

tox creates virtual env in test section when calling command tox
As far as I know circle can cache dependencies only created in dependencies section.

This is actual problem, how to (even explicitly) cache something between builds outside dependency section.

Example:

machine:
  python:
    version: 2.7.3

test:
  override:
    - tox -- --junit-xml=$CIRCLE_ARTIFACTS/tests_results.xml --cov asdf --cov-report term-missing --cov-report html --cov-config=.coveragerccircle tests/ # venv will be created here

#4

Oh, I get your point. Unfortunately right now the cache is collected at the end of the dependencies step, and there is no way to change that. You could potentially compress the venv at the end of the build and store it on S3, but you’ll have to fully maintain that yourself.


#5

I think I may have a fix for this. I have two separate tox.ini files, The first is for “setup”:

# tox.setup.ini
[tox]
envlist = py27,py340

[testenv]
commands = python setup.py test --dry-run

And the second for testing:

[tox]
envlist = py27,py340

[testenv]
commands = python setup.py test

Then in my circle.yml:

machine:
  python:
    version: 2.7.9

dependencies:
    cache_directories:
      - "~/repo/.tox"
      - "~/repo/.eggs"
    override:
      - echo "Not pip-ing"
    post:
      - if [[ ! -d ~/repo/.eggs ]]; then cd ~/repo/ && tox -c tox.setup.ini; fi
      - if [[ ! -d ~/repo/.tox ]]; then cd ~/repo/ && tox -c tox.setup.ini; fi
test:
    override:
        - make test

Seems to be working for me!


#6

I am actually using this, which looks like a shorter version of your approach :slightly_smiling:

dependencies:
  override:
    - "tox --notest"
    - ".tox/py273/bin/pip install -r requirements.txt"
    - ".tox/py273/bin/pip install -r test_requirements.txt"


#7

That worked for me as well – the combination of the correct cache_directories setting and tox --notest. Thanks!


#8