Cannot use circle.yml environment variables in cache keys


#1

Per https://circleci.com/docs/2.0/configuration-reference/, the {{ .Environment.variableName }} template can be used to get the content of environment variables to use as part of a cache key.

However, it appears that only the built-in CircleCI 2.0 environment variables (https://circleci.com/docs/2.0/env-vars/) are accessible in this manner – when I try to reference an environment variable defined in the environment: block, it renders as <no value>.

Not sure if this is a bug or new feature request, but being able to do this would be helpful in our builds.


#3

I went ahead and opened a ticket with our team. I’ll move this post over to feature requests.


#4

Thanks! For the time being, I’m working around this by writing the contents of the environment variable to a file and computing the checksum of that file to use as part of the cache key:

      - type: shell
        name: "Write GODIST to file"
        command: echo "$GODIST" > ~/GODIST.txt
      - type: cache-restore
        key: GODIST-cache-{{ checksum "~/GODIST.txt" }}

#5

I’m still seeing this issue … is there any progress on resolving this?

I’m trying to use it in following context:

version: 2
jobs:
  build:
    # ...
    environment:
      PHANTOMJS_VERSION: 2.1.1
    steps:
      # ...
      - restore_cache:
          key: v1-phantomjs-{{ .Environment.PHANTOMJS_VERSION }}
      - run:
          name: Install PhantomJS
          command: |
            [ -f /usr/local/bin/phantomjs ] || curl -sSo /usr/local/bin/phantomjs https://s3.amazonaws.com/circle-downloads/phantomjs-"${PHANTOMJS_VERSION}"
            chmod +x /usr/local/bin/phantomjs
      - save_cache:
          key: v1-phantomjs-{{ .Environment.PHANTOMJS_VERSION }}
          paths:
            - /usr/local/bin/phantomjs
      # ...

But the restore_cache step produces the following output:

Found a cache from build 1314 at v1-phantomjs-<no value>

I’was a little bit confused about the “<no value>” there.


#6

At this time, that is expected functionality. We do not yet fully support environment variables throughout the configuration file. This thread is a feature request not a bug report :slight_smile:


#7

Then I’d recommend updating the docs in any of two ways:

  • remove mention of {{ .Environment.variableName }} until it’s actually implemented
  • add a note that it will not work yet and is planned for a future

If it’s in the docs I expect it to work. If it doesn’t: it’s a bug … either in the implementation or in the docs. :wink:


#8

The docs should definitely be updated- the phrasing is unclear. It supports any env var we supply, not any arbitrary env var.


#9

This works for me as a workaround:

run: echo "$MY_ENV_VAR" > _tmp_file
- restore_cache:
    key: cache-{{ checksum "_tmp_file" }}

#10

This is clearly documented yet doesn’t work so is definitely a bug, the bug is either in the docs or a bug in the implementation.

I’ve just wasted 15 mins trying to get this to work because the docs say it should, how many more people have to waste their time before you deal with this?


#11

I have a PR up for this doc change.


#12

Agreed. I’ve just spend 30 minutes getting very confused why this doesn’t work as documented.

Doesn’t work with generated env vars either.

      - run:
          name: Configure env
          command: |
            echo 'export CACHE_DATE=$(date +%F)' >> $BASH_ENV

#13

Thanks for the workarounds, everyone.

I just ran into this as well. After carefully re-reading the docs to make sure I wasn’t crazy, I spent a good while debugging the issue to figure out what I’d done wrong, but got nowhere. Finally I gave up, started searching, and found this thread.

It sounds like a doc update was ready to go almost two months ago, but as far as I can tell that hasn’t happened and people are still losing time on this. CircleCI, could you please at least expedite the doc update?


#14

It looks like this is properly documented in some places, but not others.

For example, it’s not properly documented in the Caching Dependencies guide, but it is properly documented in the Configuration Reference.


Nodejs version
#15

Is there any chance this feature gets implemented?

I was trying to factor out the cache key repetition for multiple workflow jobs (which ideally would look like {{ checksum "$LOCATION/dev-requirements.txt" }}) into a yaml reference.

This way each job would roughly look like

environment:
  LOCATION: x
steps:
  - checkout
  - <<: *restore-cache
  - <<: *install-dependencies
  - <<: *save-cache

#16

Is that the description below is not accurate?

https://circleci.com/docs/2.0/configuration-reference/#save_cache

The environment variable variableName (supports any environment variable exported by CircleCI or added to a specific Context—not any arbitrary environment variable).

#17

circleci sure doesn’t like to make things easy ! I just found this message after starting my issue. Maybe you could fix the bug instead of the doc and provide better errors than yaml: line 10: did not find expected key like the name of the key ?


#18

Do you have checksum inside all of those three blocks? I am wondering if you could use individual key replacement instead, so you wouldn’t need to use an env var in checksum. See the entry for YAML on Wikipedia, it is surprisingly detailed. It shows that if you have a predefined block, you can still replace individual keys as well.

So in your case you’ve have the above, and then three checksums with hardwired paths.