Dynamic configuration vs Github Required checks

Build Environment
,
1

Hi,

Some weeks ago, we started to switch our circleci configuration to dynamic configuration, as you can see here open-build-service/config.yml at master · openSUSE/open-build-service · GitHub

The issue that we are facing right now is that, we have a set of “required checks” to merge PR into master, and one of the “required” checks, will just run if some set of files were changed. Therefore we need as well a conditional way to require a specific check just if a specific workflow runs or not. Otherwise either I have to remove the “required check” or never have a PR with all checks “green” again :frowning:

Thanks

Victor

1 Like
2

currently working on a monorepo with many services and having the exact same problem, since the Github merge protection requires the admin to specify the exact job or workflow name, the path-filtering orb doesn’t work really well with it

If I have parallel workflows A and B and specify both on the Github required status check dashboard, the Github PR merge check will be stuck if I only trigger either workflow A or B (not both) based on path-filtering orb mapping.

I wonder if there’s a way on circleci (or from Github side) to dynamically use all existing workflow as a check rather than specifying it manually from the Github dashboard

2 Likes
3

I have the same issue. The way I did it for now is creating a all-ok job that I trigger at the end of my workflow:

    frontend:
        when:
            and:
                - << pipeline.parameters.test-frontend >>
                - not: << pipeline.parameters.test-backend >>
        jobs:
            - test-e2e
            - test-frontend
            - test-pdf
            - test-js-utils
            - all-ok:
                  requires: [test-frontend, test-pdf, test-e2e, test-js-utils]

but it forces me to specify each case individually (“when (and (test-frontend, not test-backend))” instead of just “when test-frontend”) otherwise I would have the all-ok job run in another workflow and allow merge on github.

Now I’m adding more paths and dependencies (run test-frontend and test-pdf when js-utils/* is modified) and the configuration gets really tedious to write by hand…

4

I have this same problem and I’m trying to think of a solution to it.

I’ve been thinking about implementing a similar solution that works with workflows.
I don’t want a single workflow for all our code - it would be unmanageable.

I’m surprised that CircleCI hasn’t come up with a good solution here.

5

Hmm … interestingly, the Circletron orb might solve this issue we have with CircleCI and Github branch protection.

It will replace jobs that are not required with echo statements, so you can require any number of final outputs.

The negative side of that is that you’ll see a huge spam in GitHub in big monorepos

6

It would be nice to have an official response from CircleCI as this affects everyone working with monorepos and github checks…

2 Likes
7

It would be nice to have an official response from CircleCI

yes definitely. But getting no response is a powerful response in itself .

8

Anyone come up with any new / creative solutions to this. I had the same thought (in terms of dummy checks), but doesn’t seem like a good idea for various reasons.

9

I’m dealing with the same issue.
Any solution that doesn’t force me to implement a new job that requires the other jobs like @cosmith implemented?

Maybe after a year we’ll be able to get CircleCI’s take on it? :sweat_smile:

10

Hi folks,

Thanks for raising this up, and sharing some of the ideas here re: Dynamic Config + GitHub required checks (i.e., via GitHub Branch Protection rules).

I wanted to share a possible solution that may work for you, especially if your project:

  • uses Dynamic Config, and path-filtering
  • generates workflows depending on folders changed
  • would like to have GitHub Branch protection rules not blocking your PRs
  • may be a monorepo

kelvintaywl-cci/dynamic-config-showcase: Showcasing features and limitations of Dynamic Config (github.com)

In this project, the working solution is similar to what the circletron Orb is trying to do (as shared by @sigurdur).

Essentially, we:

  • run all workflows, but
  • depending on the pipeline parameters, make jobs exit early via circleci-agent step halt
  • ensure all jobs’ statuses are reported back to GitHub then

You can see a working example via this PR.

I understand that everyone’s setup may differ, but hopefully this gives you some idea / next steps!

1 Like
11

Hola,
Is there any newer alternative?
My main concern is that our conditional jobs are running on a docker machine, and having the condition within the job’s steps require spinning up the environment before we can break from additional steps…

12

Hi folks, we have a feature request here that I believe is in line with your concerns:

To help our Product team understand the need for this and how this will improve your workflows, I would like to also ask if you can:

  1. vote on this feature request
  2. (optional) share any additional comments on the feature request (e.g., how you envision the developer experience for this new feature can be, etc)

Thanks so much folks!
I’m hoping the increase in votes can also formally signal to our team on this request.

1 Like
13

I just wanted to say that even though this solution works, it has the undesirable side-effect of cluttering the CircleCI UI with a bunch of jobs that are essentially no-ops. It makes it harder to nagivate to the job(s) that you might care about.

14

Here is an alternative workaround that will hopefully help with that. It was created with the goal of being more visually clear about what workflows are being “skipped”.

The main differences:

  • The branch protection rule uses the final job(s) in each path-filtered workflow
  • A stand-in workflow is triggered when a path-filtered workflow isn’t run
  • The stand-in workflow contains a stand-in job that has the same name as the final job in the path-filtered workflow