Update 6-Dec-2022: We previously announced that we will be sunsetting scheduled workflows at the end of this year. Since that announcement went live, we’ve been closely tracking your feedback and feature requests, and have decided to postpone the deprecation - no new timelines are set yet. We know that there is more work we need to do in order to improve the existing Scheduled Pipelines experience and also make migration easier for all. We will share updates as we have them. Thanks for your patience and understanding.
Scheduled Pipelines are a new functionality we are introducing to help customers run scheduled work. They are intended to replace scheduled workflows, which will eventually be deprecated.
What are scheduled pipelines?
A scheduled pipeline is a way to trigger a pipeline based upon a schedule. The schedule is defined by the following:
- One or more days of the week
- Hours in which the schedule runs
- A number of times per hour that it runs (i.e. 1 time per hour, 2 times per hour/every 30 minutes, etc.)
Outside of the scheduled aspect, these pipelines are triggered similarly to the API, so they will run all workflows in a pipeline and you can set pipeline parameters as part of your schedule as well.
How do I use a scheduled pipeline?
Right now schedules can be created via an API. You can get more information about the process in our preview docs.
FAQs
Don’t you already have a way to schedule work?
Yes, we have a feature called scheduled workflows. However, this solution has a number of limitations including incompatibility with dynamic config and restricted contexts. We are planning to deprecate scheduled workflows in the future and scheduled pipelines will be its replacement.
What if I already use scheduled workflows?
You will need to migrate to scheduled pipelines before the deprecation date. This will involve creating a scheduled pipeline and removing your scheduled workflow. The deprecation date will occur when scheduled pipelines is GA and will be about 6 months from that date. Any scheduled workflows will stop running after the deprecation date.
How do I run a particular workflow in a schedule?
By default, a scheduled pipeline will run every workflow in the same way a webhook or API triggered pipeline would. Since scheduled pipelines let you set pipeline parameters, if you define a parameter in your config you can use it to filter specific workflows with a “when” or “unless” clause.
Do I have to use an API to manage this?
For now, this is API only. We are working on a UI for managing this, it will be available by GA. Both the UI and the API are using the same backend, so you will be able to use either or both. We are removing the option to manage these changes via code since storing in code has caused a number of reliability issues. However, this new API-focused approach will enable management via Infra as Code and/or Config Management tools (Terraform, Ansible, Chef, Puppet, etc.)
What is the “actor” thing?
By default, a schedule will be run as a “scheduled user”. However, by selecting an alternate actor, these schedules can be run as that user and have access to the same restricted contexts that user has access to. That actor will need to be the one creating/modifying the schedule.
Where did the cron syntax go?
We’ve found that the overwhelming majority of schedules that are being run now generally fit a set of simple patterns. Keeping schedules to this simpler syntax makes scheduled pipelines more stable while also covering the most common use cases.
