Build process in an already dockerized project, speed up strategy

Build Environment
1

Hello There!

I have not used circleci since 2014, been a while and things have changed a bit.
I am looking for feedback on what I am trying to do.

My project foobar relies on Docker on Docker compose to run and build prod ready images.
I have a few make commands for this purpose.

To give you an example the command to run tests is:

version: 2
jobs:
  build:
    working_directory: ~/foobar
    docker:
      - image: chefclub/circleci
    steps:
      - checkout
      - setup_remote_docker

      - run:
          name: Activate gcloud with service account auth
          command: |
            echo $GCLOUD_KEY | base64 --decode > gcloud.json
            gcloud auth activate-service-account $GCLOUD_EMAIL --key-file gcloud.json
      - run:
          name: Build image
          command: make build
      - run:
          name: Run docker-compose
          command: docker-compose up -d
      - run:
          name: Run tests
          command: make test

This works, but:

Those steps takes time. At least half a minute. Allocating a remote Docker Engine is not instantaneous.
If somehow the setup_remote_docker was always there with a docker client ready and up to date with the previous builds context the Docker layer caching could kick in and the build be done in a few seconds.

I used docker build --cache-from to pull the cache and that goes a bit faster.
Anything I can do to speed up things https://circleci.com/docs/2.0/caching/?

2

Have you looked into a direct machine build? This might help to build things faster.

Docker Build CircleCI 2.0 Config Example

By loading the container directly into the machine you could save build time.

https://circleci.com/docs/2.0/executor-types/

3

Not really, same speed with machine the ~30 boot is nulled by the 30s gain.

What we would need is a smart sticky docker executor, reusing the same docker host would speed up build thanks to docker cache.

Circleci is not the only one with this issue.

1 Like