We are facing a weird issue where when we try to ssh to our EC2 host with a jumpbox in between, the connection to EC2 happens but never responds. What I mean is that when the ssh command executes, I see a sshd: [accepted] and sshd: [net] processes spin up but the connection never responds back and it eventually timesout. I have added the ssh key to the hosts and jumpbox and also using add_ssh_keys in the job. Even when we try to execute the command after ssh (like ls -ltr), it does not execute.
I am running the ssh command from circleci job. When the ssh command executes, it makes the connection to the server - from log debug1: Connection established.
Then further down it i authenticated by the server debug1: Authentication succeeded (publickey).
But then after this the connection never happens, it just hangs with below in the log (-vvv option) debug3: Ignored env _ debug2: channel 0: request shell confirm 1 debug3: send packet: type 98 debug2: callback done debug2: channel 0: open confirm rwindow 0 rmax 32768 debug2: channel 0: rcvd adjust 2097152 debug3: receive packet: type 99 debug2: channel_input_status_confirm: type 99 id 0 debug2: shell request accepted on channel 0
When I run the ssh command using -n (along with -vvv), i get few more lines and then connection is exited
debug2: channel 0: open confirm rwindow 0 rmax 32768 debug2: channel 0: rcvd adjust 2097152 debug3: receive packet: type 99 debug2: channel_input_status_confirm: type 99 id 0 debug2: shell request accepted on channel 0 debug2: channel 0: read<=0 rfd 5 len 0 debug2: channel 0: read failed debug2: channel 0: close_read debug2: channel 0: input open -> drain debug2: channel 0: ibuf empty debug2: channel 0: send eof debug3: send packet: type 96 debug2: channel 0: input drain -> closed debug3: receive packet: type 98 debug1: client_input_channel_req: channel 0 rtype exit-status reply 0 debug3: receive packet: type 96 debug2: channel 0: rcvd eof debug2: channel 0: output open -> drain debug2: channel 0: obuf empty debug2: channel 0: close_write debug2: channel 0: output drain -> closed debug3: receive packet: type 97 debug2: channel 0: rcvd close debug3: channel 0: will not send data after close debug2: channel 0: almost dead debug2: channel 0: gc: notify user debug2: channel 0: gc: user detached debug2: channel 0: send close debug3: send packet: type 97 debug2: channel 0: is dead debug2: channel 0: garbage collecting debug1: channel 0: free: client-session, nchannels 1] debug3: channel 0: status: The following connections are open: #0 client-session (t4 r0 i3/0 o3/0 fd -1/-1 cc -1) debug3: send packet: type 1 Transferred: sent 3160, received 2732 bytes, in 0.4 seconds Bytes per second: sent 8719.4, received 7538.4 debug1: Exit status 0