Run another workflow after a workflow succeeds?

[cerino-ligutom]

Hello, I just recently dived to CircleCI and I’m trying to achieve something what seems to be common but can’t find resources about it nor did I find it in the docs. Is it possible to run another workflow after the main workflow finishes successfully with a git branch condition that is not via schedule since I want to run it right away? Basically I don’t want all workflows specified in the config to run concurrently. Take the example config below, I want to run deploy_test workflow after ci workflow finishes only on the test git branch. Same for deploy_prod workflow after ci workflow finishes but against the master branch.

workflows:
  version: 2.1
  ci:
    # ...jobs

  deploy_test:
    # ...jobs

  deploy_prod:
    # ...jobs

Ultimately, the goal is I want to reuse jobs (same job name) and pass parameters to it while having a branch/tag filter on the workflow level rather than creating individual jobs (job name with probably a suffix) for each environment on a single workflow and having each job keep repeating the branch/tag filters. Maybe using some sort of trigger could do the trick here but I’m not sure how to pull that off. There’s also the when but it doesn’t seem to track the status of another workflow. Or probably a support for requires option on a workflow level similar to what we have on jobs under a workflow currently would be succinct.

I could move things down to commands but then the “namespacing” (suffix) of the environments now moves to the jobs level and the condition for branches can now exist on two places, jobs and workflows. Also the former doesn’t have tags filter while the latter does. I want to keep the branch/tag filters in a single level and I think the workflow is the best place but I don’t want them to run concurrently so yep. Is there a way to achieve running workflows sequentially? If not, then I’d like to verify, is it common practice or expected to have the jobs know the environment (e.g. test, staging, prod) they’re operating on?

Thanks in advance!

[stig]

Hi @cerino-ligutom,

We don’t have a way to declare dependencies between workflows in a pipeline. However, here are some alternatives.

One is to use a human in the loop (HITL) and make the first job in the deploy workflows an approval job. The last job in the ci workflow could be to notify somone via Slack that a job is ready for approval. I’ve used the Slack Orb’s approval-notification for this in the past.

If you don’t want to use a HITL approach you could use conditional workflows and use the API to trigger deploy_test from the ci job, and deploy_prod from the deploy_test workflow.

It’s also possible to pass parameters to jobs, and use the same job more than once in a single workflow. This is probably the approach I would recommend, unless you have a real need for separate workflows. (If so, I would be interested in hearing about it!) Here’s an example:

version: 2.1

jobs:
  hello:
    parameters:
      env:
        type: string
    docker:
      - image: circleci/clojure:lein-2.9.1
    steps:
      - run: echo hi from << parameters.env >>

workflows:
  version: 2
  ci-and-deploy:
    jobs:
      - hello:
          name: hello-ci
          env: ci

      - hello:
          name: hello-test
          env: test
          requires: [hello-ci]

      - hello:
          name: hello-prod
          env: prod
          requires: [hello-test]

This would result in a workflow like this:

You can also add branch filters to jobs at any point in a workflow, so you could make the hello-ci run for every branch, but hello-test and hello-prod only run for master by tweaking the hello step to be something like:

  - hello:
      name: hello-test
      env: test
      requires: [hello-ci]
      filters:
        branches:
          only: master

Hope this helps!

[cerino-ligutom]

Hello @stig,

Thanks for getting back! I ended up with a config similar to your example of passing parameters to jobs and using the same job more than once in a single workflow.

This is probably the approach I would recommend, unless you have a real need for separate workflows. (If so, I would be interested in hearing about it!)

Nothing serious, just a matter of convenience in readability since repeating long job names doesn’t really look nice imo and I ended up adding comments to separate each environments since the name doesn’t really stick out too much. Also just the convenience of being able to filter only once on the workflow level would be nice rather than repeating the filters on each job when you know it applies to a number of jobs.

Regarding the last point above, one could probably say make reusable commands then group them all up on a single job for an environment then filter that job on a workflow. That’s what I initially thought and tried but had no luck making that work with the aws-ecr orb. I tried running the aws-ecr/build-and-push-image command as a step on another job but I get an error about not being able to connect to the docker Daemon and it seems to be looking for a different executor. Particularly a machine executor w/ an ubuntu image upon looking at the source of the orb. I use the Docker executor with the circleci/node:12 image for the other jobs and commands. Since a job can only have one executor, they’d have to be separated so now we have multiple jobs (and more to filter). There’s an aws-ecr/build-and-push-image job from the orb so that’s good but kind of “bad” at the same time, at least in my scenario. “Bad” because now the filters have to keep repeating for each job per environment and I think it could’ve been declared just once on a workflow level and let the jobs follow that workflow filter rule, possibly merging/overriding rules as well if the job has a filter rule. I’ve read about the YAML anchors and aliases to reduce the code but not having to resort to that would be ideal.

There might be other things to consider but that’s my opinion from the limited knowledge I have currently with the platform. That concern aside, I achieved what I wanted to do and learned other alternatives that we might need in the future so all is good for now