Run a job after another job even on failure

Build Environment
1

I’d like to run a collect job after a fan-out finish even if one of the jobs of the fan-out was a failure.
Any ideas?

2

Hello

This is building upon the previous post here.

You will need to set a personal API token called CCI_TOKEN using the guide here.

Below is an example config that works from my testing the way it works is that there is a job polling an API to check for running jobs and once it finds there are no longer any running jobs it then starts the final job. This does look odd when looing at the workflow but it will work and the job will always run even if every job fails.

version: 2.1

jobs:
  upstream-job:
    docker: 
      - image: cimg/node:current
    steps:
      - run: echo "Hello I am '$CIRCLE_JOB', the upstream job"
      - run: sleep 10

  required-job:
    docker: 
      - image: cimg/node:current
    steps:
      - run: echo "Hello I am '$CIRCLE_JOB', a required job"
      - run: sleep 10

  other-required-job:
    docker: 
      - image: cimg/node:current
    steps:
      - run: echo "Hello I am '$CIRCLE_JOB', another required job"
      - run: sleep 10

  dependent-job:
    docker: 
      - image: circleci/node:current
    steps:
      - run: echo "Hello I am '$CIRCLE_JOB', the dependent job"
      - run: sleep 10
      
  waiter:
    docker: 
      - image: cimg/node:current
    steps:
      - run: |
          while [[ $(curl --location --request GET "https://circleci.com/api/v2/workflow/$CIRCLE_WORKFLOW_ID/job" --header "Circle-Token: $CCI_TOKEN"| jq -r '.items[]|select(.name != "waiter")|.status' | grep -c "running") -gt 0 ]]
            do
              sleep 5
            done
      - run: echo "All required jobs have now completed"

workflows:
  my-workflow:
    jobs:
      - upstream-job
      - required-job:
          requires:
            - upstream-job
      - other-required-job:
          requires:
            - upstream-job
      - waiter
      - dependent-job:
          requires:
            - waiter

Kind Regards
Owen Oliver

1 Like
3

What does this command do?

This was working perfectly for me until yesterday. Nothing in my workflow changed but now the waiter never stops. I’m trying to figure out where the issue is.

4

It is testing the return value from the curl/jq/grep statements placed inside $(curl…) and will return true if the return value is greater than (gt) 0).

Basically what it is doing is using a while loop to monitor the status of the workflow by reading the info provided by the API. If any info is returned the job is still active as ‘-gt 0’ returns true and so the while loop sleeps for 5 seconds and checks again. If no job status info is returned from the API the check ends up returning a zero value and so ‘-gt 0’ returns false and the while loop stops.

It is a complicated line as shell’s don’t really have any direct understanding of Rest APIs and their json output. So

curl makes the API call
jq processes the returned json 
grep looks for a certain value in the output from jq