Cache gcloud updates


#1

I’d like to keep gcloud updated by caching the latest version after I run gcloud components update -q. What’s the best strategy for saving and restoring this cache key ?


#2

You might need something creative like dumping the directory structure into a file and getting the checksum of it.


#3

There is a VERSION file I can checksum, but the problem is restoring the cache. For example, version 148 is what’s on my image. If I run gcloud components update, it updates to version 149, I can save the cache with the new version, but I wont’ know the version to restore on the next run.


#4

Just include the branch name in the key and drop the checksum on the cache restore so it restores the latest one.


#5

Maybe I’m missing something - won’t that skip the save_cache step, because the cache already exists ? Here is a snippet of what I have.

  # update gcloud
  - restore_cache:
      key: gcloud-latest-{{ .Branch }}
  - run:
      name: Update gcloud SDK
      command: $GOOGLE_DIR/google-cloud-sdk/bin/gcloud components update -q
  - save_cache:
      key: gcloud-latest-{{ .Branch }}
      paths:
        - "$GOOGLE_DIR/google-cloud-sdk"

#6

Only drop the checksum for restore_cache. Keep save_cache unique enough that it’ll generate a new cache for version changes.

The key for restore_cache matches as a prefix, not an exact string. Given multiple matches, it selects the more recent one.

So for your desired behavior:

  # update gcloud
  - restore_cache:
      key: gcloud-latest-{{ .Branch }}
  - run:
      name: Update gcloud SDK
      command: $GOOGLE_DIR/google-cloud-sdk/bin/gcloud components update -q
  - save_cache:
      key: gcloud-latest-{{ .Branch }}-{{ checksum "VERSION_FILE" }}
      paths:
        - "$GOOGLE_DIR/google-cloud-sdk"

#7

So I made this change, but I’m not seeing the expected prefix behavior for restore_cache. Must I Include {{ .Branch }} ? I actually don’t want the cache to be branch specific…


#8

No. That’s totally up to you.


My 2.0 experience
#9

There may be an existing exact match? Perhaps it’s preferring that older exact match, even though there exists newer prefix matches?


#10

The save and restore steps show the key being used.


#11

So, it does seem to be restoring the exactly named key over a prefix-matched one. Is there any way I can delete this key?

  • restoring myproject-gcloud-latest
  • saving myproject-gcloud-latest-{{ checksum "VERSION" }}

#12

No but you can change the name of the key


#13

OK I think I’ve solved the key issue, now my issue is the path. It doesn’t seem to like environment variables in save_cache paths… is there a workaround for that?


#14

Just specify the directory as a string.


#15

I already am specifying it as a string. If you look at the config I posted:

- save_cache:
  key: gcloud-latest
  paths:
    - "$GOOGLE_DIR/google-cloud-sdk"

where $GOOGLE_DIR=/google
However it doesn’t seem like restore_cache is restoring the correct path:

Cached paths:
  * /myworkdir/$GOOGLE_DIR/google-cloud-sdk

#16

No, I mean the env vars aren’t yet supported there, so you need to use a string. Support for env vars there should arrive somewhat soon as far as I understand.


#17