we are using a machine executor and using docker.
Let’s say that job1 is a dependency for job2, so it’s:
job1 → job2
And in job1 we have docker-compose build --pull and this is pulling an building all the images that we need for job1 and job2.
Both job1 and job2 have docker_layer_caching enabled.
If an image img1 is built into job1, then the layers are cached and it’s not built again on job2, but if an image img2 is just pulled from the repository, then job2 has to pull again all the layers.
Is there a way to cache also img2 so that job2 does not have to contact again the repository etc.?
Thanks
docker_layer_caching is documented as only providing any advantage in a normal machine environment when the image is built in a job and then reused while the cached layers are still held within the CircleCI-provided cache. So what you are seeing is the expected result where it does not operate as a local cache for images that are requested directly from the repository. I Do not know what design decisions caused this configuration, but providing a full general cache would be a major undertaking for CircleCI due to storage and bandwidth requirements.
If you are looking for a full cache that you have full control over you have to look at using a self-hosted runner. This results in you running your own instance of the CircleCI build agent on your own hardware or rented system instance. By having your own system deployed you can then create an environment where there is persistence between jobs and workflows. This allows you to have docker operate a full image cache with the only real limitation being the amount of storage space you can allocate to the task.