How to run a single build at a time

[jggc]

We are a small team (2-3 devs) and we want only a single build running at a time for our github repo.

The reason is that we are doing continuous delivery + acceptance testing in our staging and production environments, but when multiple builds run at the same time the deployment of a build can break acceptance tests of another, especially on staging where we deploy automatically feature branches.

Our account is part of a larger organization which I don’t control.

Ideally I would be able to disable parallelization only for a few steps of the build (deployment and acceptance testing) but that does not seem to be possible to configure for an entire repository.

[grahamw]

Hi @jggc, just to clarify your requirement. It sounds like your saying that for this one specific repo, you’d like to ensure that if there are 2 pushes to you repo, let says say 15 seconds apart that the 2nd build from the second push should queue entirely behind the first. Is that correct?

Also, do you have workflows setup on that repo? The current default behaviour that I’ve observed is that if you have a workflow with 2 steps and the first job from the first build is still running when a subsequent push occurs, then the 2nd job from the first push will get queued behind the 1st job of the 2nd queue. I hope that makes sense, it seems to. So, the results is that the jobs from both workflows end up interleaved.

Probably need clarification on what your setup is as well. Are you able to supply your config.yml either here or in a gist?

[jggc]

Yep you got it right I want builds to get queued instead of ran at the same time.

I am trying with workflow though and it runs in parallel. I pushed two dummy commits about 30 seconds apart and both workflows started immediately and all jobs of the workflow were run without any queuing.

So your solution does not work… Plus using workflow makes the build much longer as circle rebuilds the entire context between each step.

[halfer]

Plus using workflow makes the build much longer as circle rebuilds the entire context between each step.

This is a separate issue, but you might be able to use workspaces to fix this. As @grahamw suggests, readers could do with seeing your config to be able to offer more specific help.

[jggc]

but you might be able to use workspaces to fix this.

Thanks I’ll try that.

Aaand here is my (anonymized) circle config.xml.

version: 2
jobs:
  unit_tests:
    docker:
      - image: someimage:latest
        auth:
          username: somedockeruser
          password: $DOCKERHUB_PASSWORD
    working_directory: ~/repo
    steps:
      - checkout
      # Download and cache dependencies
      - restore_cache:
          keys:
          - main-dependencies-v3-{{ checksum "package-lock.json" }}
          - main-dependencies-v3-
      - restore_cache:
          keys:
          - cloud-functions-dependencies-v3-{{ checksum "functions/package-lock.json" }}
          - cloud-functions-dependencies-v3-
      - run: ./.circleci/dependencies.sh
      - save_cache:
          paths:
            - node_modules
          key: main-dependencies-v3-{{ checksum "package-lock.json" }}
      - save_cache:
          paths:
            - functions/node_modules
          key: cloud-functions-dependencies-v3-{{ checksum "functions/package-lock.json" }}
      - run: ./.circleci/server-tests-unit.sh
      - run: ./.circleci/tests-unit.sh
  accept_test_staging:
    docker:
      - image: someimage:latest
        auth:
          username: someuser
          password: $DOCKERHUB_PASSWORD
    working_directory: ~/repo
    steps:
      - checkout
      - restore_cache:
          keys:
          - main-dependencies-v3-{{ checksum "package-lock.json" }}
          - main-dependencies-v3-
      - restore_cache:
          keys:
          - cloud-functions-dependencies-v3-{{ checksum "functions/package-lock.json" }}
          - cloud-functions-dependencies-v3-
      - run: ./.circleci/build-staging.sh
      - run: ./.circleci/deploy-staging.sh
      - run: ./.circleci/wait-staging.sh
      - run: ./.circleci/tests-accept.sh
      - store_artifacts:
          path: screenshots
  deploy_prod:
    docker:
      - image: someimage:latest
        auth:
          username: someuser
          password: $DOCKERHUB_PASSWORD
    working_directory: ~/repo
    steps:
      - checkout
      - restore_cache:
          keys:
          - main-dependencies-v3-{{ checksum "package-lock.json" }}
          - main-dependencies-v3-
      - restore_cache:
          keys:
          - cloud-functions-dependencies-v3-{{ checksum "functions/package-lock.json" }}
          - cloud-functions-dependencies-v3-
      - run: ./.circleci/build-prod.sh
      - run: ./.circleci/deploy-prod.sh
      - run: ./.circleci/wait-prod.sh
      - run: ./.circleci/deploy-mobile-code-push.sh
workflows:
  version: 2
  test_and_deploy:
    jobs:
      - unit_tests
      - accept_test_staging:
          requires:
            - unit_tests
      - deploy_prod:
          requires:
            - accept_test_staging
          filters:
            branches:
              only: 'master'

Thanks for the help so far, I hope we’ll figure it out!