Cache steps takes longer than test and build

Build Environment Caching Dependencies
1

Hi!

We have a workflow which restores cache:

    steps:
  - checkout
  - restore_cache:
      keys:
        - ubiquity-bazel-cache-v2-{{ .Branch }}
        - ubiquity-bazel-cache-v2-
  - restore_cache:
      keys:
        - ubiquity-web-cache-{{ checksum "ubq/atlas/service/web-server/package-lock.json" }}
        - ubiquity-web-cache-

And then builds, tests using bazel.

And then saves cache:

      - save_cache:
      key: ubiquity-web-cache-{{ checksum "ubq/atlas/service/web-server/package-lock.json" }}
      paths:
        - /root/ubiquity/ubq/atlas/service/web-server/node_modules
  - save_cache:
      key: ubiquity-lint-cache-{{ checksum "package-lock.json" }}
      paths:
        - /root/ubiquity/node_modules
  - save_cache:
      key: ubiquity-python-cache-{{ .Branch }}-{{ .Revision }}
      paths:
        - /root/venv
  - save_cache:
      key: ubiquity-bazel-cache-v2-{{ .Branch }}-{{ .Revision }}
      paths:
        - /root/.cache/bazel
      when: always

The cache save step takes forever. It is a large repo with a lot of code (I guess).

Its pretty stupid because build and test is usually very fast (3m) and we end up waiting 25m more just for cache.

We are wondering if you have any recommendations for how to fix this. Any idea why cache is so slow? Suggestions for ways to debug? Is there some way for us to view our cache?

If we can’t make caching faster, can you tell us if there’s an easy way to send the github check notification to github BEFORE we try to save cache? The main problem is checks block our checkins for way longer than they should. We like the checks, but we don’t like needlessly waiting for 25 mins.

Thanks!

2

What is the size of your cache, in megabytes/gigabytes, in each of these?

  • /root/ubiquity/node_modules
  • /root/venv
  • /root/.cache/bazel
3

Not sure how i can check those things in circle easily.
In any case, we figure out we can use a custom check and then just hit an http endpoint to satisfy the check after tests pass.

4

Use this in a run step:

ls -lh /root/venv
5

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.