How to communicate with a Docker Container over TCP?

docker
circle.yml
workflow

#1

Can someone please tell me how to properly configure a config.yml in Circle 2.0 to establish communication with a Docker Container over TCP.

I basically have a set of tests that I have been running in a CircleCI Node Docker Container for awhile which works great. I need to setup a proxy within another Container, pass the IP/Port for that Container to the script that runs the tests. I will need to communicate with the Container hosting the proxy from outside Cloud Services like SauceLabs as well.

Below is the current state of my config.yml. Really appreciate some help setting this up correctly.

version: 2

general:
  branches:
    only:
      - never-build

jobs:
  build:
    docker:
      - image: circleci/node:8.11
    steps:
      - checkout
      - setup_remote_docker
      - run:
          name: Run proxy
          command: |
            docker pull owasp/zap2docker-stable
            docker run --name zap -u zap -p 8080:8080 -i owasp/zap2docker-stable zap.sh -daemon -host 0.0.0.0 -port 8080
          background: true
      - run:
          name: Wait for Proxy
          command: |
            echo Wait for container/port to become active
            echo Get/save container ip address (IP_ADDRESS)
      - run:
          name: Install project
          command: |
            cd project
            npm install
      - run:
          name: Run tests through proxy
          command: |
            cd project
            npm test -- --ip $IP_ADDRESS --port 8080

workflows:
  version: 2
  scheduled-workflow:
    triggers:
      - schedule:
          cron: "0 4 * * *"
          filters:
            branches:
              only: master
    jobs:
      - build

#2

You can do most of what you want. The command docker run ... -p 8080:8080 will publish a port 8080 on localhost, so the IP you need to supply to NPM is merely localhost or 127.0.0.1.

You’ve correctly identified that it’s worth waiting for the server to start up, but you don’t actually do so - there are some port testing utilities noted elsewhere in the forum if you’re stuck.

If you want to reach into the build server from outside, that’s not possible. Firstly CircleCI is thankfully heavily firewalled from the internet, and so their bare-metal servers are as well. Also, your port 8080 is instantiated by Docker-in-Docker, so although this is available in the first-level Docker, there is probably no connection between that Docker and the IP on the bare-metal box.

And, when I say “bare-metal”, it’ll probably be VMWare, so that’s around three levels of firewall nope! :fire:

In general if you want to communicate with external services, you need to push something to them, rather than ask them to push to you. What do you want to do here?


#3

You can do most of what you want. The command docker run … -p 8080:8080 will publish a port 8080 on localhost, so the IP you need to supply to NPM is merely localhost or 127.0.0.1.

I tried both localhost:8080 and 127.0.0.1:8080. Both fail for me. Any suggestions? At the bottom are updated sections to my config.yml that now waits for the Docker HealthCheck used by the Docker Image to flag it as healthy and tries to test one of it’s REST endpoints. I have the same proxy tool running locally on my laptop and the same REST endpoint works successfully. Just ruling that out.

Regarding your other question, basically we want to setup this proxy, point the browser’s proxy for our Selenium tests running on Saucelabs at it and when those tests finish end up with a report this proxy tool generates showing security vulneralities. Security regression tests piggy-backing onto our functional UI tests basically. I’m not sure if it would help the situation but Saucelabs provides a tool called SauceConnect that allows it to provide a tunnel into private networks. Maybe it could connect to DOCKER_HOST=tcp://35.196.8.108:2376 ?

  - run:
      name: Run proxy
      command: |
        docker pull owasp/zap2docker-stable
        docker run --name zap -u zap -p 8080:8080 -i owasp/zap2docker-stable zap.sh -daemon -host 0.0.0.0 -port 8080 -config api.disablekey=true -config api.addrs.addr.name=.* -config api.addrs.addr.regex=true
      background: true
  - run:
      name: Wait for proxy
      command: |
        status="unhealthy"
        until [ $status == "healthy" ]
        do
          sleep 10
          {
            status=$(docker inspect --format='{{.State.Health.Status}}' zap)
          } || {
            status="unhealthy"
          }
        done

        echo get container ip address
        curl http://127.0.0.1:8080/JSON/core/view/version/?zapapiformat=JSON

And in CircleCI I see the following for output

4898 [ZAP-daemon] INFO org.zaproxy.zap.DaemonBootstrap  - ZAP is now listening on 0.0.0.0:8080
status="unhealthy"
until [ $status == "healthy" ]
do
  sleep 10
  {
    status=$(docker inspect --format='{{.State.Health.Status}}' zap)
  } || {
    status="unhealthy"
  }
done

echo get container ip address
curl http://127.0.0.1:8080/JSON/core/view/version/?zapapiformat=JSON


Error: No such object: zap
Error: No such object: zap
Error: No such object: zap
Error: No such object: zap
get container ip address
curl: (7) Failed to connect to 127.0.0.1 port 8080: Connection refused
Exited with code 7

#5

I was able to get this to work with commands to the docker container prefixed with docker exec. The below example attempting to make a REST call to the docker container through localhost:8080 to get the version of ZAP running. This solves part of my problem. The next would be any possible way to connect to the docker container from Saucelabs via Sauce Connect. It can access private networks through firewalls. https://wiki.saucelabs.com/display/DOCS/Sauce+Connect+Proxy

Is it still impossible to access docker containers in CircleCI with a tool like this?

echo get container ip address
docker exec zap curl http://localhost:8080/JSON/core/view/version/?zapapiformat=JSON | grep ‘version’
Error: No such object: zap
Error: No such object: zap
Error: No such object: zap
get container ip address
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

100 19 100 19 0 0 589 0 --:–:-- --:–:-- --:–:-- 593
{“version”:“2.7.0”}


#6

I’d start off with using the SSH option on a failing build so you can play around with Docker (just remember to cancel the build afterwards in the Circle interface, otherwise it’ll eat your build minutes for breakfast).

I’d make sure that your docker run thing is still running (it could have failed). Have a look at ps aux to see what’s running.

If it is running, have a look at docker logs <container_name> to see if it failing in any fashion. If it has stopped, you can look at docker ps --all to get the name of the exited container, so you do can logs on it.

Also you could try dropping the background flag and using the --detach flag to background it in a Docker-friendly way.

Finally you can of course do your curl in SSH, see if that gives you any HTTP status codes.

Edit

Our posts cross! :grin:


#7

Righto. At a guess that’s a background process that you’d run in Circle, which creates a connection out to Sauce, so Sauce can push commands back the other way. Clever stuff; I should think that would work. Try it?


#8

Thanks for all the help. What would --detach do for me? I really just went with background: true because that run step just sat there blocking downstream run steps and I had to kill the build.

Definitely going to try Sauce Connect as it seems viable. First I need to “process” a beer and enjoy a partial accomplishment. :smiley:


#9

It’s pretty much the Docker version of what you’ve done already. I was just kicking out ideas - don’t change it if it is now working!

Heh, nice. I just got a bunch of Docker stuff working today, so I just processed a Lagavulin 8yo single malt :tumbler_glass:


#10

Mission accomplished. That’s a great beer.

I’ll leave background: true for now. As long as it won’t be deprecated when Circle 1.1 is sunsetted. One follow up question on this proxy tunnel tool from Sauce. Would I need to run that tunnel tool at the Node docker container or the ZAP docker container? I’m assuming if I tunnel to the Node container that should give access to the ZAP container?


#11

While I think I dimly understand what the tunnel does, I don’t understand enough about your use case to answer your question. Given that it is for general CI, I would be tempted to say “neither”, and that it should be run as a service at the same level as your Docker-in-Docker containers.

It may be worth my pointing out that the tunnel will only live as long as your build process, so it is worth considering how you will make sure your build remains alive (and keeping the tunnel open) until SauceLabs has done whatever it needs to do. Will it send some sort of finished message?


#12

The tunnel basically forwards all HTTP traffic from the browser running on Saucelab’s Cloud to which ever Docker container the tunnel(Sauce Connect) is launched from. The idea is to have the HTTP traffic flow from the Saucelab’s browser to the proxy which is on the Zap container. Once it hit’s that proxy it will then forward the request out to a public Internet site.


#13

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.