We have some fairly sensitive information stored in environment variables for our build plan (ex. API keys). They are masked in the the settings which is great; however, if a user opts to “Rebuild with SSH” then they have full access to the container environment, including those sensitive environment variables.
Is there a way to disable SSH for some/all users to avoid this security issue. We don’t want these environment variables accessible to every Github user. Alternatively, is there a better way to store these keys?
Any user with write capabilities will also have SSH access. We are working on providing this ability but it is currently not of high priority because any user with write permissions could theoretically also expose secrets via commands issues through the config.yml file.
You should not be at any greater risk by allowing SSH access to users that have write access to the environment.
Any advice on how you then go about storing sensitive data in a build? For example, what if we want to deploy to a service like Docker, AWS, etc. as part of the build and need to authenticate. We don’t necessarily want everyone with write access in the organization knowing the tokens, keys, passwords to authenticate with those services. We could store those in a key store but I still see the Circle job having to connect to that.
You can put sensitive data in env vars in the CircleCI config - once they are in there they cannot be viewed or edited. However, of course, someone could commit a change to echo secrets to the build stdout, so you need to prevent that from happening. A good first step is to ensure your deploy keys only have read access to your repos.
Some other CI systems have a scanning system to detect when a secret has been printed to stdout, and will star them out. However, I think this is a misleading security feature - any attacker worth their salt will know this, and will echo out in Base64/Rot13/etc, which are trivially reversible.
Hey @KyleTryon, what don’t you add permissions around the environment variables? For example, if a certain user triggers a job, only expose the environment variables that particular user has permission to access in the builds.
One thing you could try, if you particularly want to lock things down, is to use the API to kick off builds. It allows the addition of build parameters, which you could use for secrets that you don’t want to make accessible to the build:
You mentioned that another way to lock things down is to run builds via API and pass secrets as pipeline parameters. API v2 documentation on the topic has a following note:
Note: Please note that the parameters key passed in this POST request is NOT secret.
Does that mean that literal values passed that way won’t be actively masked in the logs?
For anyone considering triggering builds via API as a way to extract sensitive information from build - that’s not going to work. Build that was triggered via API can be rerun with SSH and it will have access to the pipeline parameters that were passed via API on the first run.
@KyleTryon I would like to join Brian in asking if CircleCI has a recommended best practice for securing sensitive data (including from users with write access to the repository)?
I’d also like to know if perhaps something has changed since this thread was started and there now is a way to disable ability to “rerun with SSH” for certain builds?
We are currently able to disable to rerun with SSH for a particular project.
We also have an active idea available for voting on our ideas portal to create a feature to limit SSH access to Admins only. If this sounds like a viable solution please consider voting here: https://ideas.circleci.com/ideas/CCI-I-823
Or if you have your own idea, please feel free to comment on the existing post or create and post your own ideas here.
If you would like to totally disable SSH login for a particular project, please have an admin of the organization create a support ticket requesting access be turned off.
Disable option mentioned above, would that be on account level or individual project level? And if it is needed for debugging issues what kind of turn around time is expected from support tickets.
We can certainly help you with this. For free support, we do aim for best effort but generally respond within 2-3 days. If you create a support ticket, I will be happy to help you out there!
Hi @Brian-Triplett as Jaryt mentioned, the Support team aims to reply to our free-tier customers with the best effort, but try to respond within 2-3 days. This will apply to both disabling and enabling the rerun with SSH feature per project.
I definitely want to echo Kyle’s suggestion to vote or create a feature request such as limiting ssh access to admins that will fit your needs. This will be especially helpful as our PM’s review customer use-cases and plan their roadmap.
This doesn’t look right. Even though CircleCI doesn’t let anyone see the values on the “Environemnt Variables” dashboard, these values could still be seen by someone connecting with SSH.
However, of course, someone could commit a change to echo secrets to the build stdout, so you need to prevent that from happening.
That will be spotted during Pull Request review process. And looks like CircleCI is already hiding it from output: even if “echo $MY_SECRET” will be added, it will be hidden with “******” in the build output. But re-running the build with SSH access is available to all developers at any time (since developers commit the code - they need write permissions on the repository).
@KyleTryon That is not completely true. With forced PR reviews on, someone would catch that in a review (hopefully). Currently, every developer including our offshore developers will have access by doing an SSH AND there is no auditing of doing that SSH session (where a PR is auditable and we can see who caused the security offense).
This seems like it should be a pretty high priority though it is 3 years later. is it complete now?
