Build process in an already dockerized project, speed up strategy

Hello There!

I have not used circleci since 2014, been a while and things have changed a bit.
I am looking for feedback on what I am trying to do.

My project foobar relies on Docker on Docker compose to run and build prod ready images.
I have a few make commands for this purpose.

To give you an example the command to run tests is:

version: 2
    working_directory: ~/foobar
      - image: chefclub/circleci
      - checkout
      - setup_remote_docker

      - run:
          name: Activate gcloud with service account auth
          command: |
            echo $GCLOUD_KEY | base64 --decode > gcloud.json
            gcloud auth activate-service-account $GCLOUD_EMAIL --key-file gcloud.json
      - run:
          name: Build image
          command: make build
      - run:
          name: Run docker-compose
          command: docker-compose up -d
      - run:
          name: Run tests
          command: make test

This works, but:

Those steps takes time. At least half a minute. Allocating a remote Docker Engine is not instantaneous.
If somehow the setup_remote_docker was always there with a docker client ready and up to date with the previous builds context the Docker layer caching could kick in and the build be done in a few seconds.

I used docker build --cache-from to pull the cache and that goes a bit faster.
Anything I can do to speed up things

Have you looked into a direct machine build? This might help to build things faster.

Docker Build CircleCI 2.0 Config Example

By loading the container directly into the machine you could save build time.

Not really, same speed with machine the ~30 boot is nulled by the 30s gain.

What we would need is a smart sticky docker executor, reusing the same docker host would speed up build thanks to docker cache.

Circleci is not the only one with this issue.

1 Like