Unable to reliably cache files in home directory

cache

#1

I have a project where I was attempting to cache yarn’s global cache that is located at ~/.cache/yarn. In doing so, I was seeing subsequent builds fail on cache restores, in particular when they occurred in any docker container that did not have circleci configured as the home directory.

When running it in a container that had another username as $HOME, the cache restore step would attempt to place things in the non-existent /home/circleci/.yarn/cache and ran into permissions issues.

My cache and restore steps are as below:

  restore_cache:
    keys:
      - yarn-deps-v1-{{ .Branch }}-{{ checksum "yarn.lock" }}
      - yarn-deps-v1-{{ .Branch }}
      - yarn-deps-v1-master
  save_cache:
    key: yarn-deps-v1-{{ .Branch }}-{{ checksum "yarn.lock" }}
    paths:
      - ~/.cache/yarn

This seems like a bug in the way that ~ is handled by the caching?


#2

If the tilde is a problem, can you replace it with the actual fully-qualified home directory path?


#3

Yes, but unfortunately that isn’t a complete solution, because the cache is used in different jobs within the workflow, some of which have different user names. e.g. I use a circleci/node-browsers image for E2E testing, which has a “circleci” user, and a more vanilla ubuntu image with an “ubuntu” user for other work. When you have different usernames, then the hardcoded paths don’t work unfortunately.


#4

I resolved it (annoyingly) by setting all images we’re using to use the circleci user, mostly by basing our customizations on Circle’s images.

My stab at a better solution:
save_cache should archive files as relative paths underneath a base (can keep the base around) and allowed restore_cache via a separate base (default to the path the files were picked up from).


#5

This topic was automatically closed 41 days after the last reply. New replies are no longer allowed.