Workaround Docker cache issue with docker-cache-shim

Helpful Resources Tips, Tricks, and Hacks
1

Currently, CircleCI doesn’t support Docker cache very well. There is some workaround but sometimes it’s hard to use correctly.

Slightly better and easier approach is pushing your image to Docker Hub and pull the image before running docker build. In this way, you are using Docker Hub as cache store for your image.

I made docker-cache-shim to make this workflow easier for me. You can see it in action https://circleci.com/gh/kimh/docker-cache-shim/66#config

Disclaimer
I work in CircleCI but I wrote this post and docker-cache-shim personally, so please make Github issues when you have questions/problems, but not in CircleCI support ticket. Thanks!

4 Likes
2

Thanks for creating this. I haven’t looked too in depth at the code, but would it be straightforward and make sense to add support for other registries (AWS ECR or quay.io for example)?

3

You should be able to use other repositories for caching. You just need to change the image path to pull/push.

docker-cache-shim pull quay.io/username/reponame

docker-cache-shim push quay.io/username/reponame:tag
4

Just to confirm, will this work for Docker 1.10 as well?

5

Unfortunately not :frowning: Both load/save and pulling cache workarounds are broken in Docker 1.10.

1 Like
6

Do you know when does CircleCI’s caching docker image work 100% ?

7

Wanted to share what I’m doing to cache docker images. I noticed that when I implemented CircleCI’s docker caching suggestion, the build was still pulling from the docker hub when running docker pull, even after the image was loaded from docker load.

This works very well for loading docker images from cache:

dependencies:
  cache_directories:
    - "~/docker"
  override:
    - >-
      if [[ -e ~/docker/go.1.7.3.tar ]]; then
        echo "Loading golang docker image from cache"
        docker load -i ~/docker/go.1.7.3.tar
      else
        echo "Pulling golang docker image from Docker Hub"
        docker pull golang:1.7.3
        mkdir -p ~/docker; docker save -o ~/docker/go.1.7.3.tar golang:1.7.3
      fi
1 Like