How to use python version or image name in cache key

Build Environment Caching Dependencies
1

I was using cimg/python:3.9 but it updated the hotfix version, and now my cache is all messed up. I’d like to include python version as the cache key but I’m not finding an easy way to do this. Only option I have is to export an env var with the python version in a separate step and then then use that in the key. Is there a better way?

2

Hey. Not really. I think this is a good idea though and I created a GitHub Issue to track it: https://github.com/CircleCI-Public/cimg-shared/issues/38

For now, my suggestion would be to use python --version to output the version to a text file. Then use the checksum feature of CircleCI cache and checkson the version file. This way, whenever the version changes, the checksum changes thus the cache busts.

Lastly, the image tag you are using a designed to change as you left out the patch version. So instead, I would suggest using a full SemVer tag such as cimg/python:3.9.0 or maybe cimg/python:3.9.3. These won’t change Python versions on you.

3

Seeing the same issue (and the discussion board help actually found this topic to offer me!)

I can’t use a composite key that includes, say, {{ checksum ".venv/bin/python" }}, because the restore job doesn’t yet have that file to compute the checksum on.

This basic idea works for me:

commands:
  checkout-and-restore-deps:
    steps:
      - checkout
      - run: python --version > python_version
      - restore_cache:
          keys:
            - pipenv-v3-{{ .Branch }}-{{ checksum "Pipfile.lock" }}-{{ checksum "python_version" }}
            - pipenv-v3-{{ .Branch }}
            - pipenv-v3
jobs:
  restore-deps:
    executor: python
    environment: &common-env
      - PIPENV_VENV_IN_PROJECT: 1
      - PIPENV_NOSPIN: "TRUE"
    steps:
      - checkout-and-restore-deps
      - run:
          name: Wipe venv in case of stale symlink to old pyenv version
          command: |
            if [[ -d .venv/bin ]] && [[ ! -e .venv/bin/python ]]
              then pipenv --rm
            fi
      - run: pipenv sync --dev
      - save_cache:
          key: pipenv-v3-{{ .Branch }}-{{ checksum "Pipfile.lock" }}-{{ checksum "python_version" }}
          paths:
            - ".venv"