I am experiencing an issue with CircleCI dynamic configuration where the continuation step fails, despite “Enable dynamic config using setup workflows” properly enabled.
I have followed instructions from past issues in this forum but they all lead to examples with old continuation versions that are not quite doing what I want to do.
My goal is to generate a dynamic config from scratch using a golang script in the setup config as follows:
# Dynamic configuration file https://circleci.com/docs/using-dynamic-configuration/
version: 2.1
# This enables dynamic configuration generation
setup: true
# invoke the continuation orb to make the continuation/continue command available
orbs:
continuation: circleci/continuation@1
executors:
go-executor:
docker:
- image: cimg/go:1.20
# Define pipeline parameters that can be set when triggering the pipeline
parameters:
force_execute:
type: boolean
default: false
description: "If true, forces execution of all projects regardless of file changes"
project_path:
type: string
default: ""
description: "Specific project path to run, if empty all projects are considered"
# Define jobs for the setup workflow
jobs:
# Job to generate the dynamic CircleCI configuration
setup-job:
executor: go-executor
steps:
- checkout # Check out the repository code
- run:
name: Generate CircleCI Config
command: |
# Change to the .circleci directory where the Go script is located
cd .circleci
# Compile the Go script
go build -o generate_pipeline generate_pipeline.go
# Initialize empty args string
ARGS=""
# If force_execute is true, add the -force flag with all projects
if [[ << pipeline.parameters.force_execute >> == true ]]; then
ARGS="$ARGS -force '*'"
fi
# If a specific project_path is provided, add it to the arguments
if [[ -n "<< pipeline.parameters.project_path >>" ]]; then
ARGS="$ARGS -path << pipeline.parameters.project_path >>"
fi
# Generate the configuration using the compiled Go script
./generate_pipeline $ARGS > generated_config.yml
# Store the generated configuration as an artifact for debugging purposes
- store_artifacts:
path: .circleci/generated_config.yml
destination: generated_config.yml
# Persist the generated configuration to the workspace
# This allows it to be used in subsequent jobs if needed
- persist_to_workspace:
root: .
paths:
- .circleci/generated_config.yml
# Job to run the generated configuration
continue-pipeline:
executor: continuation/default
steps:
- attach_workspace:
at: .
- continuation/continue:
configuration_path: .circleci/generated_config.yml # use newly generated config to continue
# Define the setup workflow
workflows:
# This workflow is responsible for generating the dynamic configuration
setup-workflow:
jobs:
- setup-job # Run the generate-config job
- continue-pipeline: # Wait for the setup-job to complete
requires:
- setup-job
Then this generates the following dynamic config, which I verified to be present in the artifacts of the setup-job job. The content of the generated config is the following:
version: 2.1
# Use the path-filtering orb to determine which files have changed
orbs:
path-filtering: circleci/path-filtering@0.1.3
# Define parameters for each project to determine if its job should run
parameters:
run-hello_world-job:
type: boolean
default: false
run-sample_tofu-job:
type: boolean
default: false
# Define jobs for each project
jobs:
hello_world_job:
docker:
- image: cimg/base:stable
steps:
- checkout
- run:
name: Run hello_world job
command: |
echo "Running hello_world job"
echo "CONTEXT_NAME: ${CONTEXT_NAME}"
sample_tofu_job:
docker:
- image: cimg/base:stable
steps:
- checkout
- run:
name: Run sample_tofu job
command: |
echo "Running sample_tofu job"
echo "CONTEXT_NAME: ${CONTEXT_NAME}"
# Define workflows
workflows:
version: 2
# Setup workflow to determine which files have changed
setup:
jobs:
- path-filtering/filter:
name: check-updated-files
mapping: |
hello_world/.* run-hello_world-job true
sample_tofu/.* run-sample_tofu-job true
base-revision: main
# Define a workflow for each project
hello_world-workflow:
when: << pipeline.parameters.run-hello_world-job >>
jobs:
- hello_world_job:
context:
- hello-world
sample_tofu-workflow:
when: << pipeline.parameters.run-sample_tofu-job >>
jobs:
- sample_tofu_job:
context:
- infrastructure
Would upload some images of the UI but I’m a new user. It fails at the setup workflow of the generated config file with error CIRCLE_CONTINUATION_KEY is required. Make sure setup workflows are enabled., I have searched about solutions to this error but none solve my issue.
Here are the relevant details:
Issue Overview:
The pipeline fails with the error “CIRCLE_CONTINUATION_KEY is required. Make sure setup workflows are enabled.” Upon inspection of the environment variables log, I notice that CIRCLE_CONTINUATION_KEY is not present among the CircleCI environment variables, which appears to be the root cause of the failure.
Configuration Status:
I have confirmed in Project Settings > Advanced that “Enable dynamic config using setup workflows” is enabled.
Current Setup:
- Using continuation orb: circleci/continuation@1
- Workspace persistence and attachment configured correctly
- Dynamic configuration
setup: true
Environment Variables Present:
The logs show standard CircleCI variables (CIRCLE_BRANCH, CIRCLE_BUILD_NUM, etc.) and project-specific variables, but notably absent is the CIRCLE_CONTINUATION_KEY that should be automatically generated for dynamic configurations.
Pipeline Flow:
- Environment setup completes successfully
- Environment variables are initialized (CIRCLE_CONTINUATION_KEY missing)
- Code checkout completes successfully
- Parameters are set correctly
- Pipeline fails at continuation step due to missing key
Given that the setup workflows are enabled but the CIRCLE_CONTINUATION_KEY is not being generated, I would appreciate guidance on what might be preventing the generation of this key or how to properly configure the pipeline to ensure this key is available.