How build monorepo with multiple sub-projects, where only trigger if applicable?

Build Environment
,
1

I have a monorepo with projects that build several apps (server, mobile, cli):

├── .circleci
├── Share
├── app1
│   ├── Share
│   ├── server
│   ├── sync
│   ├── mobile
├── app2
│   ├── Share
│   ├── server
│   ├── sync
│   ├── mobile

I wanna run app1, app2 only when files changed there or in Share.

Then, only run each subprojects if their ‘Share’ or the subproject is affected.

Now for what I see about dynamic config is not clear to me how do it in a way that avoid a very large file (ideally, is a config.yml per project)

2

Short term: Using dynamic configuration - CircleCI is indeed your best option here. To avoid a really long config.yml, you’d write a script to generate dynamically the “continuation config” that has logic specific to each project.

Medium term: @Benny and team are working on a solution where you’ll be able to create a custom expression where you define a regex for specific directories that had files that changed. and then you can have a YML config file that runs for that particular “trigger”. You can create multiple triggers / YML config file combinations in your project. Attaching an image that shows an example (you can see the “trigger rules” at the top that say "build only when files in ‘/src’ are changed’). You’d use something like that. Let us know if you think that would be an easier approach for your use case

image
image3456×2674 378 KB

1 Like
3

Ok,I have tried following GitHub - CircleCI-Public/dynamic-configuration-examples at generate-config-file-main.

This not work for me, the jobs not get triggered. I have dynamic support enabled for the project.

config:

version: 2.1

setup: true

orbs:
  path-filtering: circleci/path-filtering@1.0.0

workflows:
  setup-workflow:
    jobs:
      - path-filtering/filter:
          base-revision: master
          config-path: .circleci/no-updates.yml
          mapping: | # The mapping will be used to generate the dynamic configuration for all conditions that match.
            .* always-continue true .circleci/shared.yml
            RustStore/.* build-store true .circleci/rust-store.yml
            RustErp/.* build-erp true .circleci/rust-erp.yml

shared:

version: 2.1

# define the parameters from the setup config.
parameters:
  always-continue:
    type: boolean
    default: false
  build-store:
    type: boolean
    default: false
  build-erp:
    type: boolean
    default: false

jobs:
  install_pg:
    steps:
      - run:
          name: Install PG 16
          command: |
            nix-env -iA nixpkgs.postgresql_16

  any-change:
    docker:
      - image: cimg/base:stable
    steps:
      - run: echo '`always-continue` "<< pipeline.parameters.always-continue >>"'
      - run: echo '`build-store` "<< pipeline.parameters.build-store >>"'
      - run: echo '`build-erp` "<< pipeline.parameters.build-erp >>"'

workflows:
  run-on-any-change:
    jobs:
      - any-change

It only show setup in the ui

4

Interesting…can you confirm that “Enable dynamic config using setup workflows” is toggled on for the project in Project Settings > Advanced?

5

I find need to add the tag filters:

workflows:
setup-workflow:
jobs:
- path-filtering/filter:
base-revision: master
config-path: .circleci/no-updates.yml
mapping: | # The mapping will be used to generate the dynamic configuration for all conditions that match.
.* always-continue true .circleci/shared.yml
RustStore/.* build-store true .circleci/rust-store.yml
RustErp/.* build-erp true .circleci/rust-erp.yml
filters:
tags:
only: /.*/

Now is working

1 Like
6

aha, okay cool. glad you got it working