After running tests, we deploy our app with Capistrano. Just today we ran into an issue where that task was failing with:
SSHKit::Runner::ExecuteError: Exception while executing as user@staging: disconnected: Too many authentication failures (2)
In auth logs on the server I can indeed see that it failed
Apr 21 06:48:07 staging sshd: error: maximum authentication attempts exceeded for user from 188.8.131.52 port 41100 ssh2 [preauth]
Ultimately after re-running the job everything finished successfully (as it used to in the past as well).
When trying to debug the issue, I noticed that the SSH key needed to deploy to
host is added to our " Additional SSH Keys" with the correct host. However, our CircleCI config is missing the required
But it worked all the time without this step, so I restarted a job with SSH access to see what keys are on the CircleCI container. I found that there’s only one key in
.ssh/id_rsa which doesn’t match the one added in “Additional SSH Keys”.
So then I ran
ssh user@staging -vvv and to my surprise, I was able to log in! I was curious which key CircleCI uses then. This is what I found in
ssh -vvv output:
debug1: Next authentication method: publickey debug1: Offering RSA public key: debug3: send_pubkey_test debug3: send packet: type 50 debug2: we sent a publickey packet, wait for reply debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,password debug1: Offering RSA public key: debug3: send_pubkey_test debug3: send packet: type 50 debug2: we sent a publickey packet, wait for reply debug3: receive packet: type 60 debug1: Server accepts key: pkalg rsa-sha2-512 blen 535 debug2: input_userauth_pk_ok: fp SHA256:(key fingerprint here) debug3: sign_and_send_pubkey: RSA SHA256:(key fingerprint here) debug3: send packet: type 50 debug3: receive packet: type 52 debug1: Authentication succeeded (publickey).
So it looks like CircleCI first offered some invalid key, and then another one which fingerprint matches the key that we have under “Additional SSH Keys” in project settings.
Offering RSA public key: would be followed by the path of the key, but here it’s… empty?
How does CircleCI know which key to use then? How does it work without
add_ssh_keys? Should I add
add_ssh_keys to my config to prevent issues with going over maximum authentication attempts?