Speed up `spin up environment` with custom easticsearch docker image

Hi all,

I am trying to run my integration tests in CircleCI with elasticsearch image.
I need to install some extra plugins for better analytic result.

Currently I am using my own custom docker image as a dependency.
However, I found that using my own custom image will make the spin up environment process to be much slower than what we had previously.

Would there be any better practices that we can speed up the spin up environment process?

Hello @adamaiken89

Could you share what image you were using previously?

There are two ways to get new tools into an environment on CircleCI:

  • Installation at run time
  • Via the Docker image.

Each has its own pros and cons.

Via the Docker image as you have indicated is often the best option when available. However, if it greatly increases the size of the Docker image it is possible that the extra download time can outweigh the benefits, though this is somewhat rare.

To speed up the Docker spin-up time, the best thing to do is maximize the number of potentially cached layers in the host.

The host which runs your project contains the cache of the many images used by other users (assuming they are publically accessible images). Because CircleCI provides a suite of Docker images that are used commonly by our users, they are often highly cached. If you base your Docker image off one of our existing images, the majority of the image should be cached, resulting in only the difference in layers needing to be downloaded.

We have just announced our next generation of Docker images of which each is based off the same “base” image for maximum cachability. These images are however new so at this very moment, they are potentially not as cached as our older convenience images. I would, however, recommend starting with the base image provided here as we expect this to rapidly change soon as more and more images are built from these.

This is the base image which we will be building these new images off of: GitHub - CircleCI-Public/cimg-base: The CircleCI Base (Ubuntu) Docker Convenience Image.

If you are looking for maximum speed in the short term, you could use the elasticsearch image you were using previously as the base image for your custom image. It may be that this older image is still more likely to be cached for the time being.

1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.