I need a job in my workflow to be optional, a la Gitlab’s when statement (when: manual).
Note that this isn’t the same as a conditional job, since that presumes the job should be run based on some knowable criteria. Nor is it an approval type, since that’s not an optional job, but a special job that puts mandatory job after it on hold for approval.
It might make it clearer if I spell out the use-case. We’re using Terraform Cloud for deployments, so the flow is:
Compile our application
Run terraform init, terraform plan, terraform apply, all in different jobs, in a Circle CI workflow
It’s possible that after terraform plan, we may not wish to continue. In this case, we need to send a call to the Terraform Cloud API to cancel the run. If that call is not made, the run will sit there forever, blocking any further runs, until it is either cancelled or approved by an operator.
As you have noted conditional workflows are only possible when using known information at the time the workflow is started - this comes from the fact that the config file is processed at the start and so conditional checks can not be made on any information that changes or comes into existence during the run.
The result is that you should look at pushing the init, plan and apply steps with the check to see if you wish to continue down into a single job rather than having them defined as independent jobs.
There have been other solutions/workarounds put forward using the API interface in the past and I have used a third-party service to store state information between jobs, but those complicate the overall design and so should only be considered if you can not refactor your process.
so conditional checks can not be made on any information that changes or comes into existence during the run.
The result is that you should look at pushing the init, plan and apply steps with the check to see if you wish to continue down into a single job rather than having them defined as independent jobs.
That’s the thing, there’s nothing to check The decision to run the cancel job which kills the TFC run can come from a variety of reasons, most of which are not knowable.
Gitlab’s when: manual solves this neatly, and requires no knowledge of the state of anything. It’s simply a type of job that you can choose to run if you want to. Having dug into this a bit further since I posted this question, I’m now fairly certain that Circle CI doesn’t have this functionality.
As a workaround, I’ve set our projects to kill old pipelines when a newer one wants to run (“Auto-cancel redundant workflows”), and have the cancel job always run at the start of a new pipeline. I’ve also ensured that TFC runs can only be cancelled if they’re in a pre-apply stage, so it’s safe.
The decision to run the cancel job which kills the TFC run can come from a variety of reasons, most of which are not knowable.