Thanks. Is there a script that could be used to identify all secrets within a circleci org? We have hundreds of repos and a variety of platform teams working in CircleCI, as a security team it’s difficult to ensure we have rotated everything.
2 Likes
I used the github CLI to get a list of all github repos
gh api -H "Accept: application/vnd.github+json" '/orgs/ORG/repos?per_page=100&page=1' | jq 'map(.full_name) | .[]' -r > ./repos.txt
gh api -H "Accept: application/vnd.github+json" '/orgs/ORG/repos?per_page=100&page=2' | jq 'map(.full_name) | .[]' -r >> ./repos.txt
gh api -H "Accept: application/vnd.github+json" '/orgs/ORG/repos?per_page=100&page=3' | jq 'map(.full_name) | .[]' -r >> ./repos.txt
gh api -H "Accept: application/vnd.github+json" '/orgs/ORG/repos?per_page=100&page=4' | jq 'map(.full_name) | .[]' -r >> ./repos.txt
gh api -H "Accept: application/vnd.github+json" '/orgs/ORG/repos?per_page=100&page=5' | jq 'map(.full_name) | .[]' -r >> ./repos.txt
And then this script to list variables attached to a repo
curl -su $(sed -nr 's/token: (.*)/\1/p' ~/.circleci/cli.yml): \
https://circleci.com/api/v2/project/gh/${1}/envvar | \
jq '.items | map(.name) | .[]' -r | xargs -n1 echo ${1}
And then xargs to combine the two
cat ./repos.txt | xargs -n1 sh ./get-repo-vars.sh
For contexts I just used the CircleCI CLI.
To be able to see everything it works best when you’re a github admin.
2 Likes
I wrote a script. If it’s helpful then I’m happy.
1 Like
Is CircleCI able to disable the once in 30 day limit for audit logs? Our org did one 3 weeks ago and are now unable to verify anything. I imagine I’m not the only one in this situation. Thanks
1 Like
Were repositories also exposed. We deploy using an ansible playbook repository. Secretes are stored encrypted in the playbook but we un-encrypt at deploy time. Do we need to assume that all secrets that are stored in the playbook have also been exposed?
Thank you, this looks promising!
1 Like
So, are we safe rotating all the environment variables? How do we know that this is not still open/vulnerable? To be honest, the information provided about the issue is really poor.
We need to rotate the personal tokens too?
Thank you, this was very helpful.
I used it to extend to Legacy AWS Configuration too, since those keys are still dangerous but not exposed in the UI or new API (which is a very silly thing, in my opinion):
const getProjectSettings = async (
projectName: string,
pageToken?: string
): Promise<any[]> => {
const headers = {
authorization: `Basic ${Buffer.from(API_TOKEN).toString("base64")}`,
};
const response = await fetch(
`https://circleci.com/api/v1.1/project/${CVS}/${OWNER_NAME}/${projectName}/settings${pageToken ? `?page-token=${pageToken}` : ""
}`,
{ headers }
);
const data = await response.json();
return data.aws;
};
Update: submitted this as a PR against xhiroga’s repo, in case anyone wants it working. Thank you again.
1 Like
With Jira integration, each project has a token generated by a CircleCI plugin in Jira and then added to each project’s settings page via the Jira section.
- Do we need to rotate this secret?
- How do we rotate this secret in the CircleCI plugin in Jira? The secret appears to be hardcoded and I can’t see how to regenerate it.
@z00b ,
Would it be possible to confirm if “Deploy Keys” are on the list of things that need to be rotated
I know a few others have asked but it would be great to get a clear list of elements that need to be rotated as part of this advisory (With where to find them on projects and if possible scripts to enumerate where they all exist on an org wide basis. Also if so update to the original advisory to make it completely clear as to what needs rotating. Given I don’t think it was made clear that “secrets” included SSH keys as part of the original advisory. I feel adding that adding this to the messaging would be a great help to people just picking up on this / over the next few days 
)
Thanks for any help on this front!
The lack of engagement by CircleCI staff on this thread is concerning, to say the least
5 Likes
For those working with circleCI and GitHub, here is a script we did today
QuickStart
python main.py list-ctx
python main.py list-env
python main.py list-keys
python main.py replace-keys
with the following main.py file
import requests
import argparse
from config import API_TOKEN, GH_TOKEN, ORG # CircleCi api Token, Github api token, Github organisation name
ORG_SLUG = f"github/{ORG}"
auth_header = { 'Circle-Token': API_TOKEN }
github_headers = {
"X-GitHub-Api-Version": "2022-11-28",
"Accept": "application/vnd.github+json",
"Authorization": f"Bearer {GH_TOKEN}"
}
def list_contexts():
url = "https://circleci.com/api/v2/context"
req_params = {
"owner-slug": ORG_SLUG,
"owner-type": "organization",
}
page_token = ""
contexts = []
while True:
params = {**req_params, "page-token": page_token}
response = requests.get(url, params=params, headers=auth_header).json()
contexts += response.get("items", [])
page_token = response.get("next_page_token", None)
if page_token is None:
return contexts
def list_context_variables(context_id):
url = f"https://circleci.com/api/v2/context/{context_id}/environment-variable"
req_params = {}
page_token = ""
env_vars = []
while True:
params = {**req_params, "page-token": page_token}
response = requests.get(url, params=params, headers=auth_header).json()
env_vars += response.get("items", [])
page_token = response.get("next_page_token", None)
if page_token is None:
return env_vars
def list_github_repositories():
url = f"https://api.github.com/orgs/{ORG}/repos"
req_params = {
"per_page": 100
}
page_idx = 1
repos = []
while True:
params = {**req_params, "page": page_idx}
response = requests.get(url, params=params, headers=github_headers).json()
repos += response
if len(response) == 0:
return repos
page_idx += 1
def list_project_env_variables(project):
url = f"https://circleci.com/api/v2/project/gh/{ORG}/{project}/envvar"
req_params = {}
page_token = ""
env_vars = []
while True:
params = {**req_params, "page-token": page_token}
response = requests.get(url, params=params, headers=auth_header).json()
env_vars += response.get("items", [])
page_token = response.get("next_page_token", None)
if page_token is None:
return env_vars
def list_project_keys(project):
url = f"https://circleci.com/api/v2/project/gh/{ORG}/{project}/checkout-key"
req_params = {}
page_token = ""
keys = []
while True:
params = {**req_params, "page-token": page_token}
response = requests.get(url, params=params, headers=auth_header).json()
keys += response.get("items", [])
page_token = response.get("next_page_token", None)
if page_token is None:
return keys
def list_github_deploy_keys(project):
url = f"https://api.github.com/repos/{ORG}/{project}/keys"
req_params = {
"per_page": 100
}
page_idx = 1
keys = []
while True:
params = {**req_params, "page": page_idx}
response = requests.get(url, params=params, headers=github_headers).json()
keys += response
if len(response) == 0:
return keys
page_idx += 1
def delete_github_deploy_key(project, key):
url = f"https://api.github.com/repos/{ORG}/{project}/keys/{key['id']}"
response = requests.delete(url, headers=github_headers)
if response.status_code != 204:
print("Could not delete github key", key['title'], response.json())
def get_project_checkout_keys(project, fingerprint):
url = f"https://circleci.com/api/v2/project/gh/{ORG}/{project}/checkout-key/{fingerprint}"
response = requests.get(url, headers=auth_header).json()
return response
def delete_project_checkout_keys(project, fingerprint):
url = f"https://circleci.com/api/v2/project/gh/{ORG}/{project}/checkout-key/{fingerprint}"
response = requests.delete(url, headers=auth_header)
if response.status_code != 200:
print("Could not delete circleci key for project", project)
def create_project_checkout_keys(project, key_type):
url = f"https://circleci.com/api/v2/project/gh/{ORG}/{project}/checkout-key"
response = requests.post(url, headers=auth_header, json={"type": key_type})
if response.status_code != 201:
print("Could not create circleci key for project", project, response.json())
def main_list_project_env_var():
repos = list_github_repositories()
print("project;var")
for repo in repos:
env_vars = list_project_env_variables(repo["name"])
for var in env_vars:
print(f'{repo["name"]};{var["name"]}')
def main_list_project_checkout_keys():
repos = list_github_repositories()
print("project;type;fingerprint;created_at")
for repo in repos:
keys = list_project_keys(repo["name"])
for key in keys:
print(f'{repo["name"]};{key["type"]};{key["fingerprint"]};{key["created_at"]}')
def main_list_contexts():
contexts = list_contexts()
print("context;var;last_modified")
for ctx in contexts:
env_vars = list_context_variables(ctx["id"])
for var in env_vars:
print(f'{ctx["name"]};{var["variable"]};{var["created_at"]}')
def main_replace_project_checkout_keys():
repos = list_github_repositories()
for repo in repos:
project = repo["name"]
gh_keys = list_github_deploy_keys(project)
for key in gh_keys:
delete_github_deploy_key(project, key)
ci_keys = list_project_keys(project)
for key in ci_keys:
delete_project_checkout_keys(project, key["fingerprint"])
create_project_checkout_keys(project, key_type=key['type'])
command_dict = {
"list-ctx": main_list_contexts,
"list-env": main_list_project_env_var,
"list-keys": main_list_project_checkout_keys,
"replace-keys": main_replace_project_checkout_keys
}
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("cmd", choices=list(command_dict.keys()))
args = parser.parse_args()
command_dict[args.cmd]()
2 Likes
Hello CircleCI!
Do you have any script already done, or a docker image, that lists all the secrets that could have been compromised?
We have lots of repositories, so we need a tool to dump everything we have configured, rotate credentials and then run again to be sure everything was rotated.
I see some scripts but not sure if they are listing everything that could have been compromised.
Thanks!
For the various teams at CircleCI, working day & night on this - Thank you!
It was not an easy decision to go public without having all the details, even though this now means additional pressure to work the case faster # YouDidTheRightThing.
3 Likes
When does CircleCI plan to notify customers by email?
1 Like
Will CircleCI be updating the official blog post with additional guidance from questions/answers here? Or is this the place to look for the most accurate and updated information?
3 Likes
Is there a particular thing we should be looking for in the circle org audit log? I see a fair number of ‘unregistered’ values in the ACTOR_TYPE column but don’t know what that indicates.
They did at around 3am GMT last night!
Hey folks, thanks for sticking with us here. The team is understandably busy fielding a higher than usual level of questions across multiple channels.
We’ve just posted the following support article that aims to provide a comprehensive list of what we advise to rotate out of an abundance of caution.
2 Likes