# Do I have to duplicate jobs in config if the same job will be used in multiple places in a workflow?

**URL:** <https://discuss.circleci.com/t/do-i-have-to-duplicate-jobs-in-config-if-the-same-job-will-be-used-in-multiple-places-in-a-workflow/36559>\
**Category:** Build Environment\
**Created:** [July 6, 2020, 4:54pm UTC](https://discuss.circleci.com/t/do-i-have-to-duplicate-jobs-in-config-if-the-same-job-will-be-used-in-multiple-places-in-a-workflow/36559 "2020-07-06T16:54:02Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![cos](https://avatars.discourse-cdn.com/v4/letter/c/34f0e0/32.png) [@cos](https://discuss.circleci.com/u/cos)\
**Post date:** [July 6, 2020, 4:54pm UTC](https://discuss.circleci.com/t/do-i-have-to-duplicate-jobs-in-config-if-the-same-job-will-be-used-in-multiple-places-in-a-workflow/36559/1 "2020-07-06T16:54:03Z")

</div>

Say I have two kinds of jobs in my standard workflow, a job that builds something, and a job that tests the thing that was just built. The build job has a step that takes a parameter, which affects what it builds. But the test job takes no parameters, it just reads an ID from a file in the workspace (which was persisted at the end of the build job), and tests that build which is identified by that ID.

In my config.yml I might have something like this for the build:

```
  build_someting_a: &buildsomething_a
    run:
      shell: /bin/bash +ex -uo pipefail
      command: |-
        bin/build_something.sh param-a

  build_someting_b: &buildsomething_c
    run:
      shell: /bin/bash +ex -uo pipefail
      command: |-
        bin/build_something.sh param-c

 tests:
      - run:
          shell: /bin/bash +ex -uo pipefail
          command: |-
            source BUILD_ID_FILE
            bin/run-tests.sh

cleanup:
  [...]

...
jobs:
 build_a:
    <<: *defaults
    steps:
      - checkout
      - *build_something_a
      - *persist

 build_b:
    <<: *defaults
    steps:
      - checkout
      - *build_something_b
      - *persist

 tests:
    <<: *defaults
    steps:
      - checkout
      - *tests

...

workflows:
  version: 2

  build_things:
    jobs:
      - build_a:
          <<: *filter

      - build_b:
          <<: *filter

```

What I’d like is for the workflow to run tests after each build, and then run a single cleanup after all of the tests jobs have completed. Something kind of like this, though this exactly as-is won’t work:

> ```
> - tests:
> <<: *filter
> requires:
> - build_a
> 
> - tests:
> <<: *filter
> requires:
> - build_b
> 
> - cleanup:
> <<: *filter
> requires:
> - [all tests]
> 
> ```

Is there a way to do this?

- Include the same job in different places in a workflow, with different dependencies, and have it read the persisted file from the job it depends on?
- Have one job require all of the above, even if they have the same name?

Yes, I realize that in this toy example, you might be tempted to suggest that the test not be a separate job, just another step in the build job. In this contrived example, yes, that’s the reasonable thing to do, but that’s not the question I’m asking; I just made this minimal example to illustrate my more general question.

What I have been doing is duplicating all of the job configs, so that instead of tests I have tests\_a, tests\_b, etc, all of them with identical bodies, just different names. But imagine this with a) more builds than just \_a and \_b, and b) a lot more steps in each job, not just a single script. It adds up to a lot of duplication, makes it harder to make small updates since every change has to be made in all the copies, and creates a possibility of mistakenly failing to keep them consistent with each other and not noticing. So I’d like to collapse all that duplication, and define a job just once, even if it’s going to be used in multiple places in a workflow.

Can I do something like that?

---

<div class="post-metadata">

**Author:** ![yannCI](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/yannci/32/24866_2.png) [@yannCI](https://discuss.circleci.com/u/yannCI)\
**Post date:** [September 16, 2020, 11:02pm UTC](https://discuss.circleci.com/t/do-i-have-to-duplicate-jobs-in-config-if-the-same-job-will-be-used-in-multiple-places-in-a-workflow/36559/2 "2020-09-16T23:02:28Z")

</div>

Hi @cos,

Have you considered [reusable commands](https://circleci.com/docs/2.0/reusing-config/#authoring-reusable-commands)?

It allows you to define a sequence of steps as a map to be executed in a job; enabling you to reuse a single command definition across multiple jobs.

---

<div class="post-metadata">

**Author:** ![cos](https://avatars.discourse-cdn.com/v4/letter/c/34f0e0/32.png) [@cos](https://discuss.circleci.com/u/cos)\
**Post date:** [February 9, 2021, 10:56pm UTC](https://discuss.circleci.com/t/do-i-have-to-duplicate-jobs-in-config-if-the-same-job-will-be-used-in-multiple-places-in-a-workflow/36559/3 "2021-02-09T22:56:36Z")

</div>

I had been using yaml anchors, but I will try reusable commands, it might be a bit nicer for some uses.

However, it still seems to be intended for situations where you have slightly different jobs that have a lot in common, and you need to use parameters to express the small differences between two mostly-similar jobs.

What I’m asking about is multiple instances of **exactly identical jobs** , so that I can stop littering my config with jobs named jobname-1, jobname-2, jobname-3, etc. which all have EXACTLY the same couple of lines - basically invoking the same yaml anchor or the same reusable command, without any parameters. I’d like to instead define the job just once, and then give it different instance-names in each workflow if it is used in that workflow multiple times.

---

<div class="post-metadata">

**Author:** ![yannCI](https://sea2.discourse-cdn.com/circleci/user_avatar/discuss.circleci.com/yannci/32/24866_2.png) [@yannCI](https://discuss.circleci.com/u/yannCI)\
**Post date:** [April 28, 2021, 10:10pm UTC](https://discuss.circleci.com/t/do-i-have-to-duplicate-jobs-in-config-if-the-same-job-will-be-used-in-multiple-places-in-a-workflow/36559/4 "2021-04-28T22:10:19Z")

</div>

Hi @cos,

I’m reviving this thread as I’m realizing [matrix jobs](https://circleci.com/docs/2.0/configuration-reference/#matrix-requires-version-21) might be the solution here.

**Note** : [CircleCI Server v3.x](https://circleci.com/docs/2.0/server-3-whats-new/#new-features) supports matrix jobs

Here’s a config suggestion:

```auto
jobs:
  build:
    parameters:
      argument:
        type: string
        
    <<: *defaults
    steps:
      - checkout
      - run:
          shell: /bin/bash +ex -uo pipefail
          command: |-
              bin/build_something.sh << parameter.argument >>
      - *persist
  
  tests:
    parameters:
      argument:
        type: string

    <<: *defaults
    steps:
      - checkout
      - run:
          shell: /bin/bash +ex -uo pipefail
          command: |-
                source BUILD_ID_FILE
                bin/run-tests.sh

  cleanup:
  [...]

workflows:
  workflow:
    jobs:
      - build:
          name: Build << matrix.argument >>
          matrix:
            parameters:
              argument: ["arg_a", "arg_b"]
              
              
      - test:
          name: Test << matrix.argument >>
          matrix:
            parameters:
              argument: ["arg_a", "arg_b"]
          requires:
            - Build << matrix.argument >>
            
      - cleanup:
          requires:
            - test

```

&nbsp;  
Let me know if this works for you.

---

<div class="post-metadata">

**Author:** ![cos](https://avatars.discourse-cdn.com/v4/letter/c/34f0e0/32.png) [@cos](https://discuss.circleci.com/u/cos)\
**Post date:** [February 14, 2022, 5:06pm UTC](https://discuss.circleci.com/t/do-i-have-to-duplicate-jobs-in-config-if-the-same-job-will-be-used-in-multiple-places-in-a-workflow/36559/5 "2022-02-14T17:06:06Z")

</div>

Thanks. CircleCI Server was still stuck on 2.x when I wrote this. However, CircleCI Server 3 finally arrived a couple of months ago (after I think at least two years of waiting!) and we just upgraded to it a couple of weeks ago. So now I can look at CircleCI 3.x features if issues like this come up in the future.