Workflow: run single job multiple times with different Docker images

Build Environment
,
1

I have a very simple config.yml with a single job of multiple steps:

version: 2
jobs:
  build:
    docker:
      - image: ...
    steps:
      - run:
          name: Install
          command: ...
      - checkout
      - run:
          name: CMake
          command: ...
      - run:
          name: Build
          command: ...

I run it in Docker container with C++ GCC 8.0 compiler and it works fine. Now I want to build a next workflow:

  • perform my job with GCC 8.0
  • if it fails - fail faster and don’t do anything else
  • if it passes - repeat in in several more Docker images with GCC 7.0, GCC 6.0 etc.

so the first job would check if my project compiles at all, if it does - check for backward compatibility, if it doesn’t - don’t spam me with failing builds on many other compilers. I see the setup like this:

workflows:
  version: 2
  check_compile:
    jobs:
      - build:
          name: latest
          docker: docker/gcc8
      - build:
          name: gcc7
          docker: docker/gcc7
          requires: build-latest
      - build:
          name: gcc7
          docker: docker/gcc7
          requires: build-latest
      - build:
          name: gcc6
          docker: docker/gcc6
          requires: build-latest

Two problems:

  1. I need to repeat a single job multiple times with different names
  2. I need to change Docker image for different jobs

Is there a way to achieve this?

2

I would be inclined to use Docker-in-Docker for this. Use one of the Docker images in your docker key, and then use a shell script to start/stop Docker containers based on the logic in your list.

I have an example of D-in-D here.

1 Like
3

Sounds interesting, TY. Do I understand it right that if I run a rather long shell script, I cannot mirror its steps to Circle CI steps – so instead of several foldable points in build log I will see a single long log of the whole script execution?

4

Correct. There is a suggestion floating around (either on this board or in the Ideas page) to be able to force separate steps from a single step (e.g. using a specific stdout character sequence) but for now it would be one block.

Of course, you have total stdout control in a step, so perhaps you can create a nice summary output, and make the detailed logs into artefacts.

1 Like
5

I found a solution that goes pretty much along my initial thought o.o It works around the YAML features I believe are called aliasing and merging. The result is slightly more verbose then I hoped, but really fits nicely with my wish to have different steps in config.yml instead of putting them into scripts.

"-": &dockerbuild
  steps:
    - run:
        name: Install
        command: ...
    - checkout
    - run:
        name: CMake
        command: ...
    - run:
        name: Build
        command: ...
jobs:
  build-latest:
    <<: *dockerbuild
    docker:
      - image: lasote/conangcc8
  build-gcc7:
    <<: *dockerbuild
    docker:
      - image: lasote/conangcc7
  build-clang5:
    <<: *dockerbuild
    docker:
      - image: lasote/conanclang50
workflows:
  version: 2
  check_compile:
    jobs:
      - build-latest
      - build-gcc7:
          requires:
            - build-latest
      - build-clang5:
          requires:
            - build-latest

I kinda wish you could use terser syntax with strings instead of arrays, like docker: lasote/conangcc8 and requires: build-latest, to omit excessive line feeding, but this is nice too.

Edit: There is! docker: [ image: lasote/conangcc8 ], build-gcc7: { requires: [ build-latest] }

6

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.