Centralize circleci configurations for multiple github repositories

Community Community Projects
1

We are trying to come up with an approach to centralize circleci configurations at one place for multiple github based projects.

What would be the right approach for individual project repos to pull the circleci configurations dynamically from central repo whenever there is a change in central repo?
Any known issues that we should keep in mind while we come up with this approach?

Any help would be greatly appreciated!

2

Hi @sudeepvarma133. Welcome to the CircleCI community!

The approach I’m thinking of is to have a dedicated CircleCI build that pushes the configuration to a list of repositories by leveraging the GitHub API.

The repository this CircleCI project is built on would contain:

  • The separate configuration file to be pushed to the other repositories --> global-ci.yml in my example
  • A config.yml that defines the job to push the above configuration file
  • A file containing the list of repositories (in the form “user/repo” or “org/repo”) --> satellites.txt in my example

You would also need to add an environment variable containing a GitHub API token with appropriate permissions/scopes.

The config.yml would be:

version: 2.1

jobs:
  deploy-config:
    docker:
      - image: circleci/python
    steps:
      - checkout
      - run:
          name: Test
          command: |
            echo GLOBAL_64=$(base64 -w 0 global-ci.yml) >> $BASH_ENV
            while IFS= read -r repo
              do
                echo BLOB_SHA=$(curl --location --request GET "https://api.github.com/repos/$repo/contents/.circleci/config.yml" --header "Authorization: token ${GH_TOKEN}" --header "Accept: application/vnd.github.v3+json"|jq -r .sha) >> $BASH_ENV
                source $BASH_ENV
                curl --location --request PUT "https://api.github.com/repos/$repo/contents/.circleci/config.yml" --header "Authorization: token ${GH_TOKEN}" --header "Accept: application/vnd.github.v3+json" --data-raw "{\"message\":\"Global config update\", \"content\":\"$GLOBAL_64\", \"sha\":\"${BLOB_SHA}\"}"
              done < "satellites.txt"
          
workflows:
  main:
    jobs:
      - deploy-config

 

GitHub API references:

Let me know if this helps.

3

My method is that we use public orb. Then in each repo that wants to use that orb, we use version “@volatile”. That way the repo always use the latest version of that orb.