Hello. I am looking for a way to implement following pattern:
a first job runs a script that analyze git history to identify which components are impacted by changes
according to result, one or more jobs have to be triggered with different parameters
aggregate jobs’ results to mark PR as valid or rejected
To be more concrete, here is the repository I would like to process: https://github.com/SynoCommunity/spksrc/
As a set of cross-compilation Makefiles, splitting everything means hundred of small (3-5 files each) repositories.
Do you have a monorepo? If so, search for that word in this forum - it has been asked about a few times, but I don’t think there is official support for it.
I’ve not had to do this, but I wonder if I would set up Git mirrors for each sub-project in the monorepo, and then on each Git push, pull each mirror, copy the monorepo folder into each mirror, commit, and push. You could then attach a CircleCI project to each mirror.
The nice thing with this arrangement is that if there is no change in a mirror then the commit will fail, and pushing will have no effect. Thus, mirrors that are not changed during a push will not get a CI trigger.
Thanks for proposal but that is not so easy - even with Git submodules…
Content in repository is shared between components - there are buildable “terminal components” (which produces a package) and common “dependencies” (libraries) which are buildable on their own but not relevant alone.
Complexity is that a change in one dependency/library has to trigger builds on impacted “terminal components” (one or many) - but not necessarily all components in repository.
I will go on reading advices here in varia posts to see if I can achieve my goals with as less refactoring in build chain as possible.
You know your projects, so if you are certain it would not work, that’s fine. But there is nothing to say that you could not copy some shared folders in the monorepo to all downstream mirrors. Equally, the mirrors are not obligated to behave similarly, either - they would each have their own build config.
I wouldn’t touch submodules though, I still have nightmares about them
I’ve usually found using workflows to run all the jobs to be appropriate, fanning in as per the workflows guide, then adding a step in each that guards whether it should run or not a la:
- run:
name: skip the rest of this job if no changes since dev in this branch on ./path/of/interest
command: |
if git diff origin/dev.. --quiet -- ./path/of-interest; then
circleci step halt
fi
- run: # further steps happen if there were changes
Thanks. In my case, a PR may trigger multiple workflows (when a common library changes impact multiple packages)… so I wonder how to vote PR (accept/reject) in such a context.
Is there a way to “join” workflows result?
May not be the answer you want, but perhaps you could do one big workflow and others as (logical) subworkflows comprised of dependencies between each of the builds. Each build fans into a single “yay - finished” build, then a final fan-in brings the logical subworkflows together for a final success/failure.