Cannot communicate with docker container

Build Environment
, ,
1

I have a simple workflow where a suite of tests run in circleci and they depend on a service running on a localhost port. I spin off this dependent service as a docker container in circleci, but I just cannot get the test suite from communicating with the dependent service.

This is the docker-compose.yml file containing the dependent service container that needs to be spawn:

version: "2"

services:
  duckling:
    container_name: duckling
    image: vijay120/duckling:latest
    networks:
      - internal
    expose:
      - '8000'
    ports:
      - "8000:8000"

networks:
  internal:
    driver: bridge

This is the part of the config.yml file:

version: 2

# Workflows

workflows:
  version: 2
  test-and-deploy:
    jobs:
      - test-py3.6

definitions:

  images:

    es-image: &es-image
        image: docker.elastic.co/elasticsearch/elasticsearch:5.6.10
        environment:
          - discovery.type=single-node
          - http.host=0.0.0.0
          - transport.host=127.0.0.1
          - xpack.security.enabled=false
          - ES_JAVA_OPTS=-Xms768m -Xmx768m
        command: /bin/bash -c "bin/elasticsearch-plugin install analysis-phonetic && bin/es-docker"

  steps:
    test-steps: &test-steps

      - checkout

      - setup_remote_docker

      # This should go into custom primary image, here's only for the sake of explanation
      - run:
          name: Install Docker Compose
          command: |
            sudo apt-get install docker-compose

      - run:
          name: Start container and verify it's working
          command: |
            set -x
            docker-compose up -d
            docker run --network container:duckling \
              appropriate/curl --retry 10 --retry-delay 1 --retry-connrefused http://localhost:8000/parse

      - run:
          name: Run python tests
          # run this even when linting fails
          when: always
          command: |
            # run tests
            mkdir ~/test-reports
            pytest --junitxml=~/test-reports/junit.xml


jobs:
  test-py3.6:
    docker:
      - image: circleci/python:3.6.6
        environment:
          PYTHON_VERSION: "3.6.6"
          _JAVA_OPTIONS: "-Xms512m -Xmx1024m"
      - image: docker:18.03.0-ce-git
      - *es-image
    steps: *test-steps

So the behavior is that the tests all fail due to “Connection refused” issues since they depend on the “Duckling” service to run.

When I ssh into the circleci machine, I see the following. On the parent container, I do not see any port bindings to the “Duckling” dependent service. Moreover, I get connection refused issues when I try to make a request to the service.

circleci@540f93e59632:~$ netstat -lntp
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:2626            0.0.0.0:*               LISTEN      4043/java           
tcp        0      0 0.0.0.0:9200            0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.11:45488        0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:9300          0.0.0.0:*               LISTEN      -                   
tcp6       0      0 :::54782                :::*                    LISTEN      78/circleci-agent  
circleci@540f93e59632:~$ wget http://0.0.0.0:8000/parse --post-data='locale=en_GB&text=7am&latent=True'
--2018-12-19 20:26:24--  http://0.0.0.0:8000/parse
Connecting to 0.0.0.0:8000... failed: Connection refused.

In the dependent “Duckling” service, the port bindings show that the service is listening and I can make successful requests to the service:

circleci@540f93e59632:~$ docker exec -it ccb6218e0a83 sh
# netstat -lntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:8000            0.0.0.0:*               LISTEN      8/duckling-example- 
tcp        0      0 127.0.0.11:37665        0.0.0.0:*               LISTEN      -                   
# wget http://0.0.0.0:8000/parse --post-data='locale=en_GB&text=7am&latent=True'
--2018-12-19 20:27:52--  http://0.0.0.0:8000/parse
Connecting to 0.0.0.0:8000... connected.
HTTP request sent, awaiting response... 200 OK

I understand from the CircleCI documentation that using the setup_remote_docker construct will make the child docker container not accessible to the outside world. However, I do not want to use docker run to make calls against the service since my tests just want to use requests calls. How do I solve this issue?

2

The Docker/CircleCI security model does not allow you to expose ports in your primary build container. If you were to have permissions to do that, you’d be able to do all sorts of monkey business with the networking stack on the build server, which affects all customers. That would be Not Good. (There are plenty more reports/descriptions about this on the forum).

There are two solutions. You could get rid of your docker-compose.yml file and launch your container as a secondary/remote container, like you have done for es-image. You are permitted to listen to ports there.

Or you could create another image and add it as a service inside your docker-compose.yml, where the services will be able to communicate with each other without needing to have any ports exposed.

3

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.