I have a pipeline that executes some Terraform code, I only ever want one of these pipelines to run at any given time.
By default it seems that Circle will try to run a pipeline for each commit to a repo even if another one is already running, this causes any pipelines past the first one to fail since Terraform will not be able to obtain a state lock.
Is it possible to “lock” a pipeline so only one can run at any given time for a project/repo?
Hi @lobsterdore
I strongly recommend that you manage state for your Terraform on terraform cloud. It’s def the cleanest way to enable Terraform within pipelines. You can reference this blog post on IaC + pipelines to learn a bit more about implementation.
I think using Terraform Cloud (TFC) further highlights the issue.
When using TFC workspaces, running terraform apply (or sending the request via their API, which is the same thing) starts a “run”. In non-interactive environments such as CI, you will need to supply -auto-approve. You will probably want to see the plan first though, so you might add a job that executes terraform plan before that. Bear in mind however, that you can not use -out to save execution plans when using TFC, since TFC handles the run queues and needs to guarantee consistency.
Once you start a new run after this, it will go into the queue and execute as soon as the runs in front of it are finished (with -auto-approve!). This means that you can not rely on your speculative plan output in the previous job, since the state of the system could have changed since that plan was generated by other pipeline runs that execute the next (apply) job before your pipeline (which could be sitting around waiting).
Does Circle CI have anything like Gitlab’s “Resource Groups” to get around this race condition?