[Workaround] Using scheduled workflows with Dynamic config

[Workaround] Using scheduled workflows with Dynamic config

We recently released a feature called Dynamic config that unlocks a lot of options in using CircleCI. However, at the moment this feature is unable to be used in conjunction with scheduled workflows. The team is actively working on addressing this to allow for them to be used together. Since there isn’t a specific date of that release this guide is to help ensure both features can be used at the same time while that is being worked on.

This guide will walk through setting up a secondary repository that will not have dynamic config enabled. It will simply contain a config.yml file that has scheduled workflows. Those workflows will execute a job that sends an API call to trigger a build at the repository that has dynamic config enabled, allowing for builds to be triggered on a specific cron schedule.

Requirements

  1. This guide utilizes Pipeline parameters, so you will need to be on at least version 2.1
  2. This guide requires the creation of a new repository, you will need the proper permissions at your organization to do that

Create new ‘schedule’ repository

First step will create a new repository, ideally under the same organization where you have your main repository you want to have scheduled workflows on.

Within that new repository you will create a directory called .circleci at the root of the project. Within that directory create a file called config.yml.

The contents of config.yml will be similar to the following:

version: 2.1

jobs:
  run-scheduled-workflow:
    docker:
      - image: cimg/base:stable
    steps:
      - run:
          name: Run scheduled workflow
          command: |
            curl --location --request POST 'https://circleci.com/api/v2/project/(vcs)/(org)/(project)/pipeline' \
            --header 'Content-Type: application/json' \
            -u "${CIRCLE_TOKEN}:" \
            --data-raw '{
              "branch": "main",
                "parameters": {
                  "scheduled-workflow": true
              }
            }' 
      
workflows:
  scheduled-workflow:
    triggers:
       - schedule:
           cron: "30 9 * * *"
           filters:
             branches:
               only:
                - main
    jobs:
      - run-scheduled-workflow

You will want to modify the following items:

  1. The URL of the curl you need to change (vcs) to either github or bitbucket, (org) to the organization where your dynamic config repo lives, (project) to the repository of your dynamic config project.
  2. In the --data-raw section adjust "branch": "main" to the branch where you want to run the scheduled workflow.
  3. In the --data-raw section add any additional parameters you may need/want to pass along.
  4. Adjust the cron: "30 9 * * *" to be the schedule in which you want to run on.

Once the above changes are made and committed to the repository, set this project up in CircleCI.

Configure Environment Variables

The curl in the job requires a personal API key to successfully trigger the workflow. This needs to be stored as a variable and can be done at the project level, or, if you plan to use it across projects – set these up in a context.

Variable Name Description
CIRCLE_TOKEN This will be a personal API token.

Note: The personal API token used for the curl needs to have write access to the dynamic config repository, which means as long as the token is under your user and you have write access, it will work.

Add ‘scheduled-workflow’ parameter to dynamic config

Now that we have the 2nd repository setup and have it running on a schedule, we need to ensure our dynamic config knows how to act when it receives the API call.

You will need to add scheduled-workflow as a pipeline parameter, like:

version: 2.1

setup: true

parameters:
  scheduled-workflow:
    type: boolean
    default: false

Next specify a new workflow that will only execute when scheduled-workflow is true:

workflows:
  version: 2
  scheduled-workflow:
    when: << pipeline.parameters.scheduled-workflow >>
    jobs:
      - schedule-job

Set the jobs to be the jobs you want to run on a schedule. If you don’t want your other workflows to trigger when this API call happens you will also need to add the following condition to the rest of your workflows:

when:
  equal: [false, << pipeline.parameters.scheduled-workflow >>]

Resources

  1. Here is a config.yml example for the schedule repository.
  2. Here is a config.yml example for the dynamic config repository.