We are running pre-commit to check our linting and few other things.
Our configuration is as follows:
Blockquote
repos:
Everything was working fine last night, but in the morning today (with the exact same code, and same cimg/python:3.11 image) installation started failing with:
[INFO] Initializing environment for https ://github dot com/hhatto/autopep8.
[INFO] Initializing environment for https ://github dot com/pre-commit/pre-commit-hooks.
An unexpected error has occurred: CalledProcessError: command: (‘/usr/bin/git’, ‘fetch’, ‘origin’, ‘–tags’)
return code: 128
stdout: (none)
stderr:
ERROR: Permission to pre-commit/pre-commit-hooks.git denied to deploy key
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Check the log at /home/circleci/.cache/pre-commit/pre-commit.log
Exited with code exit status 3
Note that we are pulling from public repos over https, so no authentication should be needed.
If I ssh into the box, I can clone the https ://github dot com/hhatto/autopep8, but I can see the same failure with pre-commit-hooks repo.
Note that I have added spaces etc to the addresses because the forum does not allow posting links.
We are at our wit’s end. Does anyone have any idea what could be going on?
Thanks.
4 Likes
This is happening for a lot of folks across a lot of different repos, I would link the github issue if I was allowed.
Looks like either something on github or the circleci side got changed? Seems like this would be pretty wide spread, interesting there is no incident yet.
4 Likes
I re-ran a failed CI job and the pre-commit pulling started working again the the deploy key. Not sure of what caused the issue, but hopefully it’s fixed.
This seems to be something super mysterious. We run exactly same things (down to same docker image hash). 2/3 repos fail and one works.
Happy to see I’m not alone with this, and hope everything gets fixed soon.
As a temporary fix, I forked the pre-commit-hooks repo.
-Otto
Can you please let me know where this github issue is. Just write the link as in https domain(dot).org(slash)etc format.
-Otto
We’ve seen this go from 100% failing to much less as of around 16 hours ago. We opened a support ticket with high priority for this and circleci has missed their SLA, so far it’s been 22 hours out of the 12 hour SLA. We’ve opened another ticket with medium priority for failing builds with infrastructure failures and broken web identity tokens and AWS assume roles. (SLAs: https://support.circleci.com/hc/en-us/articles/4415357235995-Support-Plans-Priority-Response)
Another workaround is to add a SSH key with access to clone what you need, I’d create a restricted service account on github for this and avoid using your account as it likely has higher privileges. Add additional SSH keys to CircleCI - CircleCI
1 Like
Hey folks, are you still seeing this issue? We had reports from other users who opened up tickets with GitHub and were told by GitHub that it was an issue on the GitHub side. GitHub has said that this has been resolved. Let me know if you’re still seeing the issue though
Seems to be working normally now. Did you ever find out what the problem was?
-Otto
I don’t have specifics handy unfortunately. We didn’t make any underlying changes to the code that would have caused this kind of error so we expect it was some sort of issue with GitHub. I’ll let you know if I’m able to get specifics on what exactly it was.
It’s a new security improvement from CCI where your git clone https://... gets rewritten as ssh://...
To disable temporarily just do this
git config --global --unset url.ssh://git@github.com.insteadOf
git clone https://github.com/...
# restore the config. For security reasons
git config --global url."ssh://git@github.com".insteadOf https://github.com