Checkout step not injecting known_hosts for custom docker images

[jaygorrell]

Starting sometime today, my builds with custom docker images started failing – actually just hanging on a yes/no prompt for an unknown host. Switching to a circleci-maintained image fixes the issue.

Upon further investigation, I found that the checkout step had stopped injecting the known_hosts file; only for custom images. When using those images, the checkout step is very short without the Circle magic that normally goes on… it just checks out the code and nothing more. This results in the mentioned yes/no dialog if you interact with the git origin after that.

This worked since before Circle 2.0 was a thing, and only stopped today. Was there something changed internally that could explain this? I’m hoping it doesn’t mean I have to switch all my repos.

[drazisil]

That is very strange. Are you able to share your config?

[jaygorrell]

Sure thing. Here you go. I guess it’s worth noting I have 100+ repos that all worked until today so it’s not a one-off or anything.

version: 2
jobs:
  build:
    parallelism: 2
    working_directory: ~/src
    docker:
      - image: docker:18.06.0-ce
    steps:
      - setup_remote_docker:
          reusable: true
          exclusive: true
      - run:
          name: Dependencies
          command: apk add --no-cache curl git jq make openssh
      - checkout
      - run:
          name: Build and Test
          command: |
            case $CIRCLE_NODE_INDEX in
              0) make image runtests
              ;;
              1) make snyk
              ;;
            esac
      - deploy:
          command: |
            if [ -z "${CIRCLE_NO_DEPLOY}" ]; then
              if [[ `echo $CIRCLE_BRANCH | grep -E "^(master|.*hotfix.*)$"` ]]; then 
                make push
              fi
              if [[ `echo $CIRCLE_BRANCH | grep -E "^(master)$"` ]]; then 
                make deploy
              fi
            fi
workflows:
  version: 2
  commit-workflow:
    jobs:
      - build
  scheduled-workflow:
    triggers:
      - schedule:
          cron: "0 1 * * 1"
          filters:
            branches:
              only: master
    jobs:
      - build

Here’s the checkout text from Circle when using this config (after today):

Either git or ssh (required by git to clone through SSH) is not installed in the image. Falling back to CircleCI's native git client but the behavior may be different from official git. If this is an issue, please use an image that has official git and ssh installed.
Enumerating objects: 286, done.
Counting objects: 100% (286/286), done.
Compressing objects: 100% (38/38), done.
Total 1507 (delta 269), reused 253 (delta 248), pack-reused 1221

If I ssh into the instance, the code is checked out, but the extra known_hosts data and everything else Circle usually does during checkout isn’t there.

[drazisil]

Thanks Jay. Having folks take a look now, will give an update when I have one.

[jaygorrell]

Thank you! Apologies in advance but I need to confess that I sent a support ticket as well. I did get a response to it but they just said that I don’t have git/ssh available on that image, which is technically true but they do get installed – and the code does get checked out. Definitely not the root cause.

I really appreciate the prompt attention to this!

[drazisil]

Can you share the ticket number? That way we can look at the builds. (On phone, search is hard)

Edit: found it

[drazisil]

@jaygorrell we reverted a change that might have caused this, can you test with a fresh commit and let me know here and in the ticket?

It’s super late for me so I have to go to bed, but I’ve assigned the ticket and will check back tomorrow. Fingers crossed this does it

[jaygorrell]

That did it. Thank you so much! I’ll update the ticket now.