# Does docker\_layer\_caching support buildx?

**URL:** <https://discuss.circleci.com/t/does-docker-layer-caching-support-buildx/41401>\
**Category:** Build Environment\
**Created:** [October 2, 2021, 12:07am UTC](https://discuss.circleci.com/t/does-docker-layer-caching-support-buildx/41401 "2021-10-02T00:07:46Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![stackptr](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/stackptr/32/28232_2.png) [@stackptr](https://discuss.circleci.com/u/stackptr)\
**Post date:** [October 2, 2021, 12:07am UTC](https://discuss.circleci.com/t/does-docker-layer-caching-support-buildx/41401/1 "2021-10-02T00:07:47Z")

</div>

My CircleCI configuration has a job defined with a machine executor using docker\_layer\_caching. This currently works as expected using `docker build`. However, when moving to a custom builder to target multiple platforms, I do not see subsequent runs using a cache. The [documentation for this option](https://circleci.com/docs/2.0/docker-layer-caching/) does not say anything regarding using buildx.

The main change in the job is:

docker buildx create --use --name myname  
docker buildx build …

It’s plausible to me that a custom builder may need to be configured in some way to leverage the cache that CircleCI is meant to add. Is there any advice on what that could look like?

Thanks

---

<div class="post-metadata">

**Author:** ![kelvintaywl](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/kelvintaywl/32/28141_2.png) [@kelvintaywl](https://discuss.circleci.com/u/kelvintaywl)\
**Post date:** [August 26, 2022, 1:59am UTC](https://discuss.circleci.com/t/does-docker-layer-caching-support-buildx/41401/2 "2022-08-26T01:59:21Z")

</div>

[Updated]

## **Docker Buildx works with DLC on CircleCI if Docker contexts are not used.** Please see [Brian’s sharing below](https://discuss.circleci.com/t/does-docker-layer-caching-support-buildx/41401/4)!

hi @stackptr

Apologies for the late follow-up here!

I would like to share that indeed, I can confirm that `docker buildx` is not playing nice with Docker Layer Caching (DLC) on CircleCI builds;  
I also could not get my own builds utilizing DLC when using `docker buildx build` specifically.

We also see this mentioned in the CircleCI aws-ecr Orb here:

> <https://github.com/CircleCI-Public/aws-ecr-orb/issues/202>
>
> Hi can someone please help me with docker layer caching?
> I've tried to use like… specified on the Orb site, but i still didn't figured out how to have docker layer caching.
> I'm currently using a free plan that has the feature for the DLC (is no longer required to have Performace or better).
> Also I've already contacted the support to let me activate DLC.
> 
> Every time i set another run, the docker start downloading again and again. The whole workflow take about 25m of which the build process take about 15-18 min, so I'm trying desperately to use DLC.
> 
> this is my current file:
> \`\`\`version: 2.1
> 
> orbs:
> aws-ecr: circleci/aws-ecr@8.1.2
> aws-ecs: circleci/aws-ecs@2.2.1
> 
> workflows:
> build-and-deploy:
> jobs:
> - build-and-push-image
> - aws-ecs/deploy-service-update: # create a new task definition
> requires:
> - build-and-push-image
> container-image-name-updates: # name of the container defined in the task definition
> cluster-name: # ECS cluster name
> family: # name of the task definition's family.
> service-name: # name of the service
> force-new-deployment: true
> 
> jobs:
> build-and-push-image:
> docker:
> - image: cimg/base:2021.04 # CircleCI base Ubuntu-based image
> steps:
> - aws-ecr/build-and-push-image:
> remote-docker-layer-caching: true
> setup-remote-docker: true
> repo: a\_repo
> tag: a\_tag
> no-output-timeout: 15m # 10 minutes could let to workflow stop
> \`\`\`
> 
> \### Orb version
> 8.1.2
> 
> \### What happened
> 
> Didn't cache the docker layers
> 
> \### Expected behavior
> 
> Cache the docker layers

In particular, the aws-ecr Orb has moved to using `docker buildx` for building images.  
As such, developers using the current aws-ecr Orb (at 8.1.2) are also seeing DLC not utilized in their builds.

Unfortunately, I do not have an answer / solution for this yet!  
Please allow us to reach out and confirm with our Engineering team on this, and follow up here 🙇‍♂️

If you have any findings, please let us know here too as well.

Thank you for your understanding in this matter.

---

<div class="post-metadata">

**Author:** ![kelvintaywl](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/kelvintaywl/32/28141_2.png) [@kelvintaywl](https://discuss.circleci.com/u/kelvintaywl)\
**Post date:** [September 2, 2022, 6:01am UTC](https://discuss.circleci.com/t/does-docker-layer-caching-support-buildx/41401/3 "2022-09-02T06:01:05Z")

</div>

As a follow-up, I would like to share solutions and some explanation around [Docker Buildx](https://docs.docker.com/build/buildx/) and the Docker Layer Cache (DLC) feature on CircleCI.

**Re: DLC on CircleCI**

When you build a Docker image, its layers are saved within the Docker root directory.  
The Docker root directory is typically `/var/lib/docker` for most setups.

However, builds on CircleCI (or any CI) are ephemeral.  
The next build is likely on a different virtual machine (VM) and so we need a way to “pass” around the stored layers between builds.

Hence, layer caching on CircleCI works by storing and retrieving these layers in an external volume instead.  
These volumes are attached to the remote machine running the Docker daemon.  
The idea is that these volumes will likely include all, if not some of the layers as they get passed around.

> More information: [https://circleci.com/docs/docker-layer-caching#how-dlc-works](https://circleci.com/docs/docker-layer-caching#how-dlc-works)

**Re: How does this impact Docker Buildx**

With Docker Buildx, developers can choose where the cached layers should be stored and retrieved.

There are many options, but this includes the local cache (folder), or a registry cache (i.e., saving the layers in a Docker repo on a registry like Docker Hub).

See the following options for the `docker buildx build` command:

- [docker buildx build | Docker Docs](https://docs.docker.com/engine/reference/commandline/buildx_build/#cache-from)
- [docker buildx build | Docker Docs](https://docs.docker.com/engine/reference/commandline/buildx_build/#cache-to)

Note that you **do not need the DLC feature** in order to pass around the cache storage between builds.

**Example: Cache from local folder**

```yaml
jobs:
  build-push-local-docker:
    docker:
    - image: cimg/base:stable
    resource_class: medium
    environment:
    - DOCKER_REGISTRY: docker.io
    - DOCKER_USER: <user_or_org>
    - DOCKER_LOGIN: <login_user>
    steps:
    - checkout
    - setup_remote_docker:
        version: 20.10.14
    - run:
        name: Check Docker settings (for buildx)
        command: |
          docker version
          docker buildx version
          docker context inspect
    - run:
        name: Setup docker buildx
        command: |
          docker context create circleci
          docker buildx create --use circleci

          docker buildx ls
          docker context inspect circleci
    - run:
        name: Login to registry
        command: |
          # login credentials should be provided via context or project environment variables.
          echo $DOCKER_PASSWORD | docker login $DOCKER_REGISTRY --username $DOCKER_LOGIN --password-stdin
    - restore_cache:
        keys:
        - hello-buildx-docker-{{ arch }}-{{ .Branch }}-
        - hello-buildx-docker-{{ arch }}-
    - run:
        name: Docker buildx with local cache
        command: |
          docker buildx build --progress=plain \
            --tag="${DOCKER_REGISTRY}/${DOCKER_USER}/hello-buildx:${CIRCLE_SHA1}" \
            --cache-to=type=local,mode=max,dest=/tmp/dockercache \
            --cache-from=type=local,src=/tmp/dockercache \
            --output=type=docker \
            .

          docker image ls
    - save_cache:
        key: hello-buildx-docker-{{ arch }}-{{ .Branch }}-{{ checksum "/tmp/dockercache/index.json" }}
        paths:
        - /tmp/dockercache
    - run:
        name: Publish image
        command: |
          docker image tag "${DOCKER_REGISTRY}/${DOCKER_USER}/hello-buildx:${CIRCLE_SHA1}" "${DOCKER_REGISTRY}/${DOCKER_USER}/hello-buildx:local-docker"
          docker image push "${DOCKER_REGISTRY}/${DOCKER_USER}/hello-buildx:local-docker"
    - run:
        name: Prune cache
        command: |
          docker buildx prune

```

This solution uses the `local` cache option for `--cache-from` and `--cache-to`.  
With the `local` cache strategy, we can take advantage of CircleCI’s `restore_cache` and `save_cache` steps to load and save the cached layers between builds!

Hope this helps! 🤓

I have a sample repo showcasing the registry and local cache strategies:

> **[GitHub - kelvintaywl-cci/docker-buildx: I can docker buildx on CircleCI](https://github.com/kelvintaywl-cci/docker-buildx)**
>
> I can docker buildx on CircleCI. Contribute to kelvintaywl-cci/docker-buildx development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![Brian](https://avatars.discourse-cdn.com/v4/letter/b/97f17d/32.png) [@Brian](https://discuss.circleci.com/u/Brian)\
**Post date:** [December 15, 2022, 7:30pm UTC](https://discuss.circleci.com/t/does-docker-layer-caching-support-buildx/41401/4 "2022-12-15T19:30:11Z")

</div>

Hey Everyone,

We just released the version 8.2.1 of the `aws-ecr` orb which contains a fix for using `docker layer caching` with `docker buildx`.

`DLC` only works with `docker buildx` when there is no `docker context` being used. The `aws-ecr` orb only uses a context to support multi-architecture builds. Therefore, the orb now removes the `docker context` if there is only 1 platform provided (`linux/arm64`,`linux/amd64`, etc), enabling the use of `DLC`

If more than one platform is provided, a context will be used but `DLC` is not supported at this time.

We’re currently working on adding `DLC` support for multi-architecture builds and it should be available in the upcoming months.

Thank you and hope that helps!

Brian

---

<div class="post-metadata">

**Author:** ![sina](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/sina/32/11255_2.png) [@sina](https://discuss.circleci.com/u/sina)\
**Post date:** [February 8, 2023, 6:51pm UTC](https://discuss.circleci.com/t/does-docker-layer-caching-support-buildx/41401/5 "2023-02-08T18:51:00Z")

</div>

Hi @Brian . If we want to build using buildx, and want to hook into buildkit (e.g., for zstd building like `docker buildx use zstd-builder`), we do seem to need to use a context, otherwise we get an error `could not create a builder instance with TLS data loaded from environment`.

And if we create a context, DLC is not supported.

Is there a way around this? This is frustrating cause we seem to now have to pick between DLC or zstd builds.

---

<div class="post-metadata">

**Author:** ![Brian](https://avatars.discourse-cdn.com/v4/letter/b/97f17d/32.png) [@Brian](https://discuss.circleci.com/u/Brian)\
**Post date:** [March 1, 2023, 8:03pm UTC](https://discuss.circleci.com/t/does-docker-layer-caching-support-buildx/41401/6 "2023-03-01T20:03:51Z")

</div>

Hey @sina,

From my understanding, DLC won’t work with the builder context that enables parallel multiarch builds.

Having said that, I am working on adding the docker buildx build command to the docker orb with the ability to specify a context.

I’m gong to have to test zstd to see if it works with docker layer caching but if it does, it using the docker orb should be a viable solution.

Let me check with the team to see if we can test ztsd with DLC and get back to you.

Thank you!

Brian

---

<div class="post-metadata">

**Author:** ![sina](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/sina/32/11255_2.png) [@sina](https://discuss.circleci.com/u/sina)\
**Post date:** [March 9, 2023, 12:27am UTC](https://discuss.circleci.com/t/does-docker-layer-caching-support-buildx/41401/7 "2023-03-09T00:27:54Z")

</div>

Thanks a lot Brian. I think the main issue is really to use zstd here, whether through buildx or not.

Zstd can have a big impact for many customers (e.g., [AWS fargate startup times](https://aws.amazon.com/blogs/containers/reducing-aws-fargate-startup-times-with-zstd-compressed-container-images/)).

However, we don’t want this to come at a substantial increase in build times without DLC, and when using local caching (using docker’s caching mechanisms and then save-cache on circle), the job takes even longer, because more time is spent on saving and restoring caches, compared to the time it saves from the build time.

Would look forward to what you find out re. using zstd with DLC. many thanks.