Dynamic config (path-filtering + continuation wf) vs Script detecting file changes

Build Environment
1

Hi!

I am working on a single application repo which pipeline should publish only specific NPM packages based on file changes of the repo.
Most of the workflow would stay the same, but let’s say two jobs (build and deploy) should change behavior based on which files have changed current PR vs the main branch.

Reading on the internet I could see there have been a few different approaches.
In the past years, as far as I understood ppl used custom scripts to get which files were changing, then read from a file what was changed and let the command or circleci config behave accordingly.

@lerna/changed - npm is an example.
Or simply using git diff

Then now I see an amazing implementation come out,
Dynamic configs
It allows me, potentially, to

  • Execute conditional workflows/commands
  • Pass pipeline parameter values and/or generate additional configuration
  • Trigger separate config.yml configurations, which exist outside the default parent ./circleci/ directory

Now my question is… is dynamic the best practice I should follow? I believe would work, used in combination with path-finder + continuation.
This approach, just feels a bit overkilling having to introduce a setup workflows and those orbs, when all I need is to know which files have changed. A simple script such as lerna would work great for me. I am not using Lerna so I would eventually need to find one that works for me but sounds simple and specific for my needs.

Am I not seeing the benefits dynamic config would bring me in close or long run ?
Or a script is still an efficient approach for me?

Would like to collect some opinions!

Thank you.

2 Likes
2

Hi @Vanals,
Like you mentioned, a custom script should be possible.

But CircleCI’s dynamic config with their path-filtering orb will probably be simpler and more declarative.

Here’s what your .circleci/config.yml could be like. The npm packages would be in packages/

version: 2.1

setup: true

orbs:
  path-filtering: circleci/path-filtering@0.1

workflows:
  setup:
    jobs:
      - path-filtering/filter:
          name: Create dynamic jobs
          mapping: |
            packages/foo/.* run-foo true
            packages/bar/.* run-bar true
            packages/baz/.* run-baz true

The mapping value has regexes for the changed files.

So if packages/foo/package.json changed, run-foo would be true.

Then, .circleci/continue_config.yml would accept the mapping values as parameters:

version: 2.1

parameters:
  run-foo:
    type: boolean
    default: false
  run-bar:
    type: boolean
    default: false
  run-baz:
    type: boolean
    default: false

orbs:
  node: circleci/node@5.0

jobs:
  foo:
    executor: node/default
    steps:
      - when:
          condition: << pipeline.parameters.run-foo >>
          steps:
            - checkout
            - run:
                command: npm ci && npm test
                working_directory: packages/foo
  bar:
    executor: node/default
    steps:
      - when:
          condition: << pipeline.parameters.run-baz >>
          steps:
            - checkout
            - run:
                command: npm ci && npm test
                working_directory: packages/baz
  baz:
    executor: node/default
    steps:
      - when:
          condition: << pipeline.parameters.run-bar >>
          steps:
            - checkout
            - run:
                command: npm ci && npm test
                working_directory: packages/bar

workflows:
  build:
    jobs:
      - foo
      - bar
      - baz

Here’s that code in a repo: circle-advanced-setup-workflow/config.yml at example/node-modules · getlocalci/circle-advanced-setup-workflow · GitHub

Props to Makoto Mizukami’s great repo, which that example is forked from.

1 Like