[CircleCI Security Alert] Rotate any secrets stored in CircleCI

UPDATE: 13-Jan-2022 21:22 UTC

Today we published incident report for our security incident disclosed on January 4 on our blog here:


Details about the incident and action required:

At this point, we are confident that there are no unauthorized actors active in our systems; however, out of an abundance of caution, we want to ensure that all customers take certain preventative measures to protect their data.

We will provide updates about this incident as soon as they become available.

Support article for more information on impact and steps to remedy:

FAQ compilation of answered questions:

1 Like

The announcement only mentions secrets in contexts and environment variables, but what about SSH keys, Jira and Slack integration tokens, webhook secrets, etc?

3 Likes

Does this also affect the BitBucket integration key?

And what about the source code itself?

1 Like

Immediately rotate any and all secrets stored in CircleCI. These may be stored in project environment variables or in contexts.

Does this mean the secret stored in “Project Settings > Environment Variables” in CircleCI?

@t-asaeda Yes - as well as in “Organization Settings / Contexts” (if your company uses contexts).

1 Like

Is it safe to add new secrets in CircleCI?

Yes. At this time we recommend rotating SSH Keys and all tokens or secrets.

1 Like

How does CircleCI encrypt secrets in it’s backend? Was it a single encryption key the was “leaked”? What actions were taken to ensure environment integrity was restored?

2 Likes

Yes. At this point, we are confident that there are no unauthorized actors active in our systems. Thanks for your vigilance in keeping systems safe.

If you are looking for audit logs from CircleCI, you can request them from settings. Documentation on this feature is here: Audit logs - CircleCI

We also advise you check the logs of any systems which had secrets stored in CircleCI.

1 Like

Have you rotated the OIDC signing keys for orgs?

1 Like

Do I need to rotate “Deploy key” in SSH Keys?

2 Likes

Hi, thank you for your incident response.

I noticed that projects that don’t already use CircleCI (deleted .circleci/config.yml) still have environment variables. (This can be viewed by specifying the repository name directly in the URL https://app.circleci.com/settings/project/github///environment-variables).

Do I need to rotate credentials for such projects as well?

1 Like

Do self hosted runner have been compromised too ? And so all others secrets injection outside of circleci context/variables ?

2 Likes

Does this incident affect CircleCI Enterprise version?

1 Like

Other than leaked credentials & secrets, is there any chance for attackers to have injected code or tampered with our builds?

1 Like

will this discussion serve as a way to share any indicators of compromise that we can use to search through our logs or will some other form of communication be shared?

2 Likes

I am concerned if “Deploy keys” shall also be rotated
@z00b can you help us with that? Will you share more IOCs?

2 Likes

Two quick questions:

  • Do we know whether CircleCI’s platform was used for any access to customers’ systems or whether secrets were just grabbed and run off with?
  • How regularly do the IPs on all.knownips.circleci.com change? And if they’ve changed recently, please can we have a list of all the previous IPs?
2 Likes