Accessing private repos inside docker-compose container

Build Environment
1

I’m trying to migrate to using docker-compose and multiple containers for builds (mirroring our local dev environment). In one of the containers I’m running ‘composer install’ to add dependencies. Some of the packages I’m attempting to install our in our own private Bitbucket repo. I have added an access key from Bitbucket and it works fine when I’m not using the new docker-compose setup.
I’m running the following command:
docker-compose exec --user=root app /usr/local/bin/composer install --prefer-dist --no-interaction

And it’s returning this when hitting the first private repo:

The authenticity of host 'bitbucket.org (18.205.93.0)' can't be established.
RSA key fingerprint is SHA256:zzXQOXSRBEiUtuE8AikJYKwbHaxvSc0ojez9YXaGp1A.
Are you sure you want to continue connecting (yes/no)?

Is there way to get around this? Am I approaching this the wrong way? Do I have to use my own docker image as the host and run this command there outside of the containers?

2

Hi there, welcome to our community!

This is a common issue, you have a couple of options. Either disable strict host key checking or run ssh-keyscan to add Bitbucket to the known hosts file.

More details can be found here in the Atlassian forums: https://community.atlassian.com/t5/Bitbucket-questions/What-and-how-to-add-a-Known-hosts/qaq-p/843444

Note, this has nothing to do with Bitbucket, it’s an ssh issue but the use case and solution above is the same.

3

I’ll add a third option to that. You can do the above manually in CircleCI after a failed build (using SSH), obtain the known hosts file, then add it to your build process. Copy it into place before doing your composer install. This is a bit more paranoid than the other options - I think it fixates on a single IP, so protects against DNS poisoning. However it might be a pain if Bitbucket change their IP address structure.

The other option with BitBucket is to use OAuth tokens. They are per Bitbucket user rather than repo, but you can specify that it’s read-only, which is nice. Tokens are effectively just a username and password that you can store in CircleCI env vars. You need to write them into a auth.json file for Composer, but that’s pretty easy.

4

Here’s a snippet of the config.yml file that I’ve was attempting to get working:

version: 2
jobs:
  build:
    machine: true 
    working_directory: ~/var/www
    steps:
      - checkout
      - run: docker-compose up -d nginx app db selenium
...
      - run: docker-compose exec app /usr/local/bin/composer install --prefer-dist --no-interaction
...

I’ve gotten past my original issue by running a build with ssh, and grabbing everything in the circleci user’s .ssh folder and replicating that in the app container’s www user’s .ssh folder. I just won’t work inside a container that’s inside the docker host unless all of that is there.

I have new permissions issues now when trying to save/restore the composer cache that I think I can overcome, and I would prefer to get something working to just copy the circleci users’s .ssh folder into the app container on each build without having to store them myself in case they change, etc.

5

Ah right, yes - if you are doing this in a container then you will need to solve the issue there (Docker in Docker) rather than in the (Docker) CircleCI build container.

I just solved this problem, in fact, last week. My tips:

  • Use BitBucket OAuth, it’s easier to work with than SSH, and is directly supported by Composer.
  • Put the generated token username and password in CircleCI env vars (in the UI) where they will remain secret
  • Pull your dependencies in a separate Docker build stage - remember that once secrets are written to an image layer they are stored in the final image, even if you delete them.
1 Like