Do I have to duplicate jobs in config if the same job will be used in multiple places in a workflow?

Say I have two kinds of jobs in my standard workflow, a job that builds something, and a job that tests the thing that was just built. The build job has a step that takes a parameter, which affects what it builds. But the test job takes no parameters, it just reads an ID from a file in the workspace (which was persisted at the end of the build job), and tests that build which is identified by that ID.

In my config.yml I might have something like this for the build:

  build_someting_a: &buildsomething_a
    run:
      shell: /bin/bash +ex -uo pipefail
      command: |-
        bin/build_something.sh param-a

  build_someting_b: &buildsomething_c
    run:
      shell: /bin/bash +ex -uo pipefail
      command: |-
        bin/build_something.sh param-c

 tests:
      - run:
          shell: /bin/bash +ex -uo pipefail
          command: |-
            source BUILD_ID_FILE
            bin/run-tests.sh

cleanup:
  [...]

...
jobs:
 build_a:
    <<: *defaults
    steps:
      - checkout
      - *build_something_a
      - *persist

 build_b:
    <<: *defaults
    steps:
      - checkout
      - *build_something_b
      - *persist

 tests:
    <<: *defaults
    steps:
      - checkout
      - *tests

...

workflows:
  version: 2

  build_things:
    jobs:
      - build_a:
          <<: *filter

      - build_b:
          <<: *filter

What I’d like is for the workflow to run tests after each build, and then run a single cleanup after all of the tests jobs have completed. Something kind of like this, though this exactly as-is won’t work:

  - tests:
      <<: *filter
      requires:
        - build_a

  - tests:
      <<: *filter
      requires:
        - build_b

  - cleanup:
      <<: *filter
      requires:
        - [all tests]

Is there a way to do this?

  • Include the same job in different places in a workflow, with different dependencies, and have it read the persisted file from the job it depends on?
  • Have one job require all of the above, even if they have the same name?

Yes, I realize that in this toy example, you might be tempted to suggest that the test not be a separate job, just another step in the build job. In this contrived example, yes, that’s the reasonable thing to do, but that’s not the question I’m asking; I just made this minimal example to illustrate my more general question.

What I have been doing is duplicating all of the job configs, so that instead of tests I have tests_a, tests_b, etc, all of them with identical bodies, just different names. But imagine this with a) more builds than just _a and _b, and b) a lot more steps in each job, not just a single script. It adds up to a lot of duplication, makes it harder to make small updates since every change has to be made in all the copies, and creates a possibility of mistakenly failing to keep them consistent with each other and not noticing. So I’d like to collapse all that duplication, and define a job just once, even if it’s going to be used in multiple places in a workflow.

Can I do something like that?

Hi @cos,

Have you considered reusable commands?

It allows you to define a sequence of steps as a map to be executed in a job; enabling you to reuse a single command definition across multiple jobs.