/etc/hosts not shared between job and a remote docker

Hello,

In order to test an nginx docker in CircleCI before pushing it on my register, I use dgoss (which use docker commands) and fake some internal DNS by overwriting the /etc/hosts file like this:

version: 2
workflows:
  version: 2
  build_deploy:
    jobs:
      - deploy_on_registery
jobs:
  deploy_on_registery:
    docker:
     - image: node:lts
    steps:
      - checkout
      - setup_remote_docker
      - run:
          name: Dockerize project
          command: docker build -t someapp:test .
      - run:
          name: Install Goss (https://github.com/aelsabbahy/goss)
          command: curl -fsSL https://goss.rocks/install | sh
      - run: 
          name: Install test dependency
          command: npm install -g node-static
      - run:
          name: Fake server
          background: true
          command: |
            static -p 80
      - run:
          name: Fake server
          background: true
          command: |
            static -p 8000
      - run:
          name: Configure network
          command: |
            echo "127.0.0.1 someinternaldns" >> /etc/hosts
            echo "127.0.0.1 someotherinternaldns" >> /etc/hosts
      - run:
          name: Test Docker image
          command: |
            curl http://someinternaldns # this works
            GOSS_FILES_STRATEGY=cp dgoss run -v $PWD/secret/:/secret/ someapp:test # this doesn't work and returns `nginx: [emerg] host not found in upstream "someinternaldns" in /etc/nginx/conf.d/default.conf:171`
      

It worked perfectly on my computer but it doesn’t on CircleCI. I got an error which said that the host doesn’t exist like if the /etc/hosts wasn’t used by the dgoss. Isn’t the /etc/hosts shared between the runner and the docker commands inside it?
I see that the instruction to be able to use docker commands (setup_remote_docker) is quite explicit about the fact that it is a remote (unlike my local docker). So my question is: is it a bug? a missing feature? or if it is desired, how can I modify my /etc/hosts or simply add my internal DNS without altering my docker?

Thank you in advance for your help!

Best regards,
Fabrice

In general, Docker containers do not like their /etc/hosts being written to. The way to do this, at least with the docker run command, is to use --add-host. That might work here, though it depends on whether your dgoss allows it.

It also depends on whether the underlying Docker platform gives you enough permissions to modify networking features. For example, you cannot publish ports in CircleCI to the build host, I think because if you could do that, you would have enough control over the networking stack to affect other customers on the same bare-metal machine.

My way around that is to use Docker Compose, which works very well indeed. I am pretty sure the DC YAML config format allows extra hosts to be created, so if docker switches do not work, try moving to DC. I have not tried this in CircleCI, but it is what I would try.

The other thing you can try is to move to a Machine executor. This would be one layer of Docker, instead of Docker-in-Docker, and you would be able to modify your hosts file there. However you get less control over your build image - it is not Docker any more - and technically the charging model has not been finalised for it (the docs note that it could get more expensive).

1 Like

Awesome, that’s exactly what I need. I didn’t know this parameter exists on docker command.
It works perfectly like this:

version: 2
workflows:
  version: 2
  build_deploy:
    jobs:
      - deploy_on_registery
jobs:
  deploy_on_registery:
    docker:
     - image: node:lts
    steps:
      - checkout
      - setup_remote_docker
      - run:
          name: Dockerize project
          command: docker build -t someapp:test .
      - run:
          name: Install Goss (https://github.com/aelsabbahy/goss)
          command: curl -fsSL https://goss.rocks/install | sh
      - run: 
          name: Install test dependency
          command: npm install -g node-static
      - run:
          name: Fake server
          background: true
          command: |
            static -p 80
      - run:
          name: Fake server
          background: true
          command: |
            static -p 8000
      - run:
          name: Test Docker image
          command: GOSS_FILES_STRATEGY=cp dgoss run -v $PWD/secret/:/secret/ --add-host someinternaldns:127.0.0.1 --add-host someotherinternaldns:127.0.0.1 someapp:test

Thank you very much!

1 Like

Ah, nice work! :trophy:

Actually, mounting volume in CircleCI doesn’t work like I did (cf: https://circleci.com/docs/2.0/building-docker-images/#mounting-folders), so here is my working solution:

version: 2
workflows:
  version: 2
  build_deploy:
    jobs:
      - deploy_on_registery
jobs:
  deploy_on_registery:
    docker:
     - image: node:lts
    steps:
      - checkout
      - setup_remote_docker
      - run:
          name: Dockerize project
          command: docker build -t someapp:test .
      - run:
          name: Install Goss (https://github.com/aelsabbahy/goss)
          command: curl -fsSL https://goss.rocks/install | sh
      - run: 
          name: Install test dependency
          command: npm install -g node-static
      - run:
          name: Fake server
          background: true
          command: |
            static -p 80
      - run:
          name: Fake server
          background: true
          command: |
            static -p 8000
      - run:
          name: Test Docker image
          command: |
            # create a dummy container which will hold a volume with the secret folder
            docker create -v /secret --name secret alpine:3.4 /bin/true
            # copy the secret folder content into this volume
            docker cp "$(pwd)"/secret/. secrets:/secret/
            # run goss with the mounted volume
            GOSS_FILES_STRATEGY=cp dgoss run --volumes-from secrets --add-host someinternaldns:127.0.0.1 --add-host someotherinternaldns:127.0.0.1 someapp:test

Correct, I got bitten by that 18 months ago! I seem to recall that container volumes work, but not bind mounts (on-host volumes).