How to add an automated rollback to a CircleCI workflow?

[cos]

I would like to configure a deployment workflow that runs integration tests after a code deploy job. If the integration test job fails, then I want it to automatically run a code rollback job. I’m not asking here how to write the actual code deploy and rollback jobs, that’s a separate issue; what I’m asking is how I can configure a workflow to do that.

I searched and didn’t find a lot of posts about this. I did find this 2018 post from @krhubert :

Configure workflows to run jobs when an other failed

How to achive rollback startegy with cirleci 2.0? Sample config: build: # build project publish: # put on storage deploy: # deploy to cloud rollback: # remove from storage & cloud e2e: # tests workflows: build_and_deploy: jobs: - e2e: requires: - deploy - .... on_failed: # (1) - # run rollback What is configuration to rollback deploy on e2e test failed?

They asked pretty much the same question I’m asking, but nobody really answered it. @ThiagoBarradas suggested a workflow that has an approval step after the integration test, and a rollback job that depends on the approval: Auto rollback with circleci 2.0 · GitHub

However, there are two issues with this suggestion:

1. I think it allows you to run the rollback only if the integration test succeeds, not if it fails.
2. It’s not automatic, it requires a person to notice the failure and click the approval. I want an automatic rollback to known good code.

Anyone have any solutions for either (or, ideally, both) of the problems above?

[ven2575]

I’ m also searching for the same solution. @cos please let me know if you find any, i’ll let you know if i found any solution.

[cos]

As far as I can tell, there is no way to configure a job to run if and only if another job fails.

However, if I put both the code deploy and the tests in the same job, there is a way to have a step within a job run if any previous step in the same job failed, using the “when” attribute, documented here: Configuring CircleCI - CircleCI

In other words, I have a job like this:

steps:
  - checkout
  - *deploy-code
  - *test-environment
  - *rollback-code

The rollback-code step has a “when: on_fail” for its run command. That means that if the deploy and test steps succeed, the rollback step will not run. However, if either the deploy fails, or the deploy succeeds but then the tests fail, then the rollback step will run. If the deploy fails, the test step is skipped; only the rollback runs.

This is not ideal, because I want to use the same rollback step as a separate job, and put it behind an approve step. That would give people the ability to manually roll back when they choose to, even after a successful deploy and test. So I basically need to configure both a rollback step and a rollback job and use them in different contexts, which is a confusing and error-prone configuration. It also makes the flow less clear to users.

CircleCI really needs to add better flow control logic for workflows, I think. But this workaround can work in the meantime.