Caching with DLC and fan-out building

Build Environment
1

I have a workflow that is building in parallel all the services of my application:

workflows:
  build-test-publish:
    jobs:
      - prepare_bakefile
      - build_svc1:
          requires: 
          - prepare_bakefile
      - build_svc2:
          requires: 
          - prepare_bakefile
      - build_svc3:
          requires:
          - prepare_bakefile
      - build_svc4:
          requires: 
          - prepare_bakefile
      - build_svc5:
          requires: 
          - prepare_bakefile

And each service is built with something like this:

jobs:
  build_svc1:
    machine:
      image: ubuntu-2204:current
      docker_layer_caching: true
      - run: 
          command: |
            docker buildx bake --progress=plain --file=docker-bake.hcl --file=baked-metadata.json svc1

I’m trying to make this setup work properly with DLC, but I cannot figure out a way to make it works: the cache does not seems to be used properly between reruns of the job, is there something obvious I’m missing ?

Also I was wondering if the DLC was scoped to jobs and not to workflow ?

2

@Ely DLC caching is per job, with some special handling for if you are using parallelism. Did you see somewhere in the docs that said otherwise, if so please let me know, so we can get it corrected! (Or make it clear for others)

In your case, if you are using buildx you might need to name your builder instance, see Docker layer caching overview - CircleCI for more details.

Lastly, I would encourage you to use the Docker orb, CircleCI Developer Hub - circleci/docker, which would handle this for you.

1 Like