Does CircleCI 2.0 work with monorepos?

Sounds like you’d need to rebase or change what the commit range is.

This is how I solved this: we have a step in every job that checks if the relevant files have changed. If so, we continue that job

# Example usage:
# ./exit-if-path-not-changed.sh path/to/dir1 path/to/dir2

# 1. Get all the arguments of the script
# https://unix.stackexchange.com/a/197794
PATHS_TO_SEARCH="$*"

# 2. Get the latest commit
LATEST_COMMIT=$(git rev-parse HEAD)

# 3. Get the latest commit in the searched paths
LATEST_COMMIT_IN_PATH=$(git log -1 --format=format:%H --full-diff $PATHS_TO_SEARCH)

if [ $LATEST_COMMIT != $LATEST_COMMIT_IN_PATH ]; then
    echo "Exiting this CircleCI job because code in the following paths have not changed:"
    echo $PATHS_TO_SEARCH
    circleci step halt
fi

For reference:

We use CircleCI with golang monorepo.
Here’s how it’s done: https://github.com/Tufin/circleci-monorepo

It looks great but the biggest disadvantage is that the dependent build will be triggered by a user who owns the token. In my opinion it could confuse developers in the team (I pushed the commit why the build is running/failed as someone else?).

Did you find a solution for that problem?

With the CircleCI API v2 release which is currently in Preview it’ll be possible to trigger entire workflows.
See Monorepo with CircleCI Conditional Workflows.
@lucassus have a look at this idea about user tokens.

3 Likes

Based on what @dimadeveatii and @reuvenharrison did. I’ve made an example using CircleCI with a JavaScript monorepo

Hope it help someone :blush:

1 Like

Based on ideas in this thread I put together showcase setup for monorepo using Gradle as build tool and CircleCI as ci tool.

First implementation is around one year old and meanwhile I used it in two projects.

Gradle specific stuff should be replaceable quite easily by something else. Just change tools/ci/core/list-dependencies.sh.

Thanks to @reuvenharrison for inspiration.

I have a built an orb to help with monorepos. And one command specifically tackles this problem. Any PRs are welcome.

Orb: https://circleci.com/orbs/registry/orb/roopakv/swissknife

specific command: https://circleci.com/orbs/registry/orb/roopakv/swissknife#commands-trigger-workflows-for-all-modified

example repo: https://github.com/roopakv/circle-monorepo-via-swissknife

Thanks for all the information here.
Have any of you figured out a way to work with required checks in Github. Since not the same list of workflows will run on each commit, how would one prevent merge on checks?

I have a solution that I discovered after a bit of work. it may or may not work fro you. This i use in addition / tandom with the swissknife orb mentioned above

workflow A

jobA--------|
            |------ commitSuccessfulJob
jobB--------|

jobC

All jobs that are necessary for a commit to pass are required for commitSuccessfulJob
The commit successful job does nothing. just. echo “I’m done”

Github requires only commitSuccessfulJob in pull requests. This also makes a very thing such that if you ever change the jobs required people don’t have to merge master etc since success is defined by circle and the branch :smiley:

For more discussion around this, @punkdata will be discussing monorepos and how to implement them in CircleCI pipelines TODAY (Tue Sept 15) at 2p ET with @roopakv on Twitch!