Inconsistent caching behaviour

Build Environment Caching Dependencies
,
1

Hi All!

I’m experiencing inconsistent behavour when doing pip install over restored venv. Pip install has some version conflict warning (another issue), which passes and gets things installed first time. But when rebuild happens and venv is restored from the cache, pip fails hard with version inconsistencies. The weird part is that if I rebuild with ssh enabled and then execute the same pip install commands in the container, everything is fine.

Any ideas?
A.

  test:
    docker:
      - image: circleci/python:3.6.5
    steps:
      - checkout
      - restore_cache:
          keys:
          - v3-dependencies-{{ checksum "requirements.txt" }}-{{ checksum "setup.py"}}
      - run:
          name: Install dependencies
          command: |
            virtualenv venv --python=python3.6
            . venv/bin/activate
            pip install -r requirements.txt --extra-index-url https://xxxx --trusted-host xxxx
      - save_cache:
          paths:
            - ./venv
          key: v3-dependencies-{{ checksum "requirements.txt" }}-{{ checksum "setup.py"}}
      - run:
          name: run tests
          command: |
            . venv/bin/activate
            pytest
2

When you rebuild, are you doing so without cache? If so, it’s quite possible that the cache is causing this, and incrementing the cache key would be a fix (v3 to v4).

3

I rebuild with cache. Bumping cache version helps, because install succeedes when there is no cache found. But when a cache is found, it fails.

4

Could you post a link to a build where you’re building without cache, and one where you’re building with cache? Only CircleCI employees will be able to see the builds, unless they’re open source.

5

He is the build where cache is not found (success):
https://circleci.com/gh/integrateai/webservice/294
Here is the rebuild of the same one and it found cache from the previous run:
https://circleci.com/gh/integrateai/webservice/295

6

It looks like the venv is succeeding, and the pip command is failing. I’m not exactly sure why that is though. I’ll look at this further.

7

Exactly. Because the same pip command is OK the first time. Moreover, if you rerun with ssh, login there and try executing pip from shell, it runs with no problem.
We fixed this build by fixing pip warnings, but I’m just curious about this behaviour. It might suggest some CircleCI bug.

8

Glad to hear you fixed it at least – I’ll look into this further after the weekend.