Locally run jobs with dynamic config setup in monorepo

Build Environment
,
1

I have a monorepo setup with Dynamic Config. Here is the directory structure:

  • monorepo
    • .circleci
      • config.yml
      • workflows.yml

I cannot pack/process this into one config.yml to be then able to run individual jobs locally. Any ideas?

2

Hello,

CircleCI’s Dynamic Config isn’t designed for running individual jobs locally. However, there are workarounds for your monorepo setup:

Temporary Config Split: Create a separate .circleci/config.yml file for local development. This file can reference specific workflows from workflows.yml using the & syntax. This allows local execution but requires maintaining two configs.

Local Development Tools: Consider tools like Skaffold or Docker Compose. These tools can spin up environments that mimic your CI pipeline locally, allowing you to test individual jobs without modifying CircleCI configs.

I hope the solution may help you.

3

Do you have a working example of the temporary config split? As I understand, you cannot alias between YAML files.

I don’t think that Dockers is a viable solution since I cannot reproduce the circleci agent actions which is where I see most issues.

4

Do you have a working example of the temporary config split?

I haven’t done this personally, and haven’t had your use case, but maybe the circleci config pack command would be useful here? I think it’s similar to the way circleci orb pack works, and this link about a similar use case mentions it. At the least, this might make the various components a bit easier to break apart and more self-contained?

Could also look at other higher level abstractions for building / generating yaml like cuelang.

[edit: from your comment in another thread, looks like you also started going down this rabbit hole)

As I understand, you cannot alias between YAML files.

From a quick search, I think you’re definitely right on that one.

5

I have tried circleci config pack, and unfortunately, it doesn’t work. It doesn’t pull the other configs into one file. Because they are not in the FYAML format. They are completely different YAML files.