Can someone please tell me how to properly configure a config.yml in Circle 2.0 to establish communication with a Docker Container over TCP.
I basically have a set of tests that I have been running in a CircleCI Node Docker Container for awhile which works great. I need to setup a proxy within another Container, pass the IP/Port for that Container to the script that runs the tests. I will need to communicate with the Container hosting the proxy from outside Cloud Services like SauceLabs as well.
Below is the current state of my config.yml. Really appreciate some help setting this up correctly.
version: 2
general:
branches:
only:
- never-build
jobs:
build:
docker:
- image: circleci/node:8.11
steps:
- checkout
- setup_remote_docker
- run:
name: Run proxy
command: |
docker pull owasp/zap2docker-stable
docker run --name zap -u zap -p 8080:8080 -i owasp/zap2docker-stable zap.sh -daemon -host 0.0.0.0 -port 8080
background: true
- run:
name: Wait for Proxy
command: |
echo Wait for container/port to become active
echo Get/save container ip address (IP_ADDRESS)
- run:
name: Install project
command: |
cd project
npm install
- run:
name: Run tests through proxy
command: |
cd project
npm test -- --ip $IP_ADDRESS --port 8080
workflows:
version: 2
scheduled-workflow:
triggers:
- schedule:
cron: "0 4 * * *"
filters:
branches:
only: master
jobs:
- build
You can do most of what you want. The command docker run ... -p 8080:8080 will publish a port 8080 on localhost, so the IP you need to supply to NPM is merely localhost or 127.0.0.1.
Youâve correctly identified that itâs worth waiting for the server to start up, but you donât actually do so - there are some port testing utilities noted elsewhere in the forum if youâre stuck.
If you want to reach into the build server from outside, thatâs not possible. Firstly CircleCI is thankfully heavily firewalled from the internet, and so their bare-metal servers are as well. Also, your port 8080 is instantiated by Docker-in-Docker, so although this is available in the first-level Docker, there is probably no connection between that Docker and the IP on the bare-metal box.
And, when I say âbare-metalâ, itâll probably be VMWare, so thatâs around three levels of firewall nope!
In general if you want to communicate with external services, you need to push something to them, rather than ask them to push to you. What do you want to do here?
You can do most of what you want. The command docker run ⌠-p 8080:8080 will publish a port 8080 on localhost, so the IP you need to supply to NPM is merely localhost or 127.0.0.1.
I tried both localhost:8080 and 127.0.0.1:8080. Both fail for me. Any suggestions? At the bottom are updated sections to my config.yml that now waits for the Docker HealthCheck used by the Docker Image to flag it as healthy and tries to test one of itâs REST endpoints. I have the same proxy tool running locally on my laptop and the same REST endpoint works successfully. Just ruling that out.
Regarding your other question, basically we want to setup this proxy, point the browserâs proxy for our Selenium tests running on Saucelabs at it and when those tests finish end up with a report this proxy tool generates showing security vulneralities. Security regression tests piggy-backing onto our functional UI tests basically. Iâm not sure if it would help the situation but Saucelabs provides a tool called SauceConnect that allows it to provide a tunnel into private networks. Maybe it could connect to DOCKER_HOST=tcp://35.196.8.108:2376 ?
- run:
name: Run proxy
command: |
docker pull owasp/zap2docker-stable
docker run --name zap -u zap -p 8080:8080 -i owasp/zap2docker-stable zap.sh -daemon -host 0.0.0.0 -port 8080 -config api.disablekey=true -config api.addrs.addr.name=.* -config api.addrs.addr.regex=true
background: true
- run:
name: Wait for proxy
command: |
status="unhealthy"
until [ $status == "healthy" ]
do
sleep 10
{
status=$(docker inspect --format='{{.State.Health.Status}}' zap)
} || {
status="unhealthy"
}
done
echo get container ip address
curl http://127.0.0.1:8080/JSON/core/view/version/?zapapiformat=JSON
And in CircleCI I see the following for output
4898 [ZAP-daemon] INFO org.zaproxy.zap.DaemonBootstrap - ZAP is now listening on 0.0.0.0:8080
status="unhealthy"
until [ $status == "healthy" ]
do
sleep 10
{
status=$(docker inspect --format='{{.State.Health.Status}}' zap)
} || {
status="unhealthy"
}
done
echo get container ip address
curl http://127.0.0.1:8080/JSON/core/view/version/?zapapiformat=JSON
Error: No such object: zap
Error: No such object: zap
Error: No such object: zap
Error: No such object: zap
get container ip address
curl: (7) Failed to connect to 127.0.0.1 port 8080: Connection refused
Exited with code 7
I was able to get this to work with commands to the docker container prefixed with docker exec. The below example attempting to make a REST call to the docker container through localhost:8080 to get the version of ZAP running. This solves part of my problem. The next would be any possible way to connect to the docker container from Saucelabs via Sauce Connect. It can access private networks through firewalls. https://wiki.saucelabs.com/display/DOCS/Sauce+Connect+Proxy
Is it still impossible to access docker containers in CircleCI with a tool like this?
echo get container ip address
docker exec zap curl http://localhost:8080/JSON/core/view/version/?zapapiformat=JSON | grep âversionâ
Error: No such object: zap
Error: No such object: zap
Error: No such object: zap
get container ip address
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
Iâd start off with using the SSH option on a failing build so you can play around with Docker (just remember to cancel the build afterwards in the Circle interface, otherwise itâll eat your build minutes for breakfast).
Iâd make sure that your docker run thing is still running (it could have failed). Have a look at ps aux to see whatâs running.
If it is running, have a look at docker logs <container_name> to see if it failing in any fashion. If it has stopped, you can look at docker ps --all to get the name of the exited container, so you do can logs on it.
Also you could try dropping the background flag and using the --detach flag to background it in a Docker-friendly way.
Finally you can of course do your curl in SSH, see if that gives you any HTTP status codes.
Righto. At a guess thatâs a background process that youâd run in Circle, which creates a connection out to Sauce, so Sauce can push commands back the other way. Clever stuff; I should think that would work. Try it?
Thanks for all the help. What would --detach do for me? I really just went with background: true because that run step just sat there blocking downstream run steps and I had to kill the build.
Definitely going to try Sauce Connect as it seems viable. First I need to âprocessâ a beer and enjoy a partial accomplishment.
Iâll leave background: true for now. As long as it wonât be deprecated when Circle 1.1 is sunsetted. One follow up question on this proxy tunnel tool from Sauce. Would I need to run that tunnel tool at the Node docker container or the ZAP docker container? Iâm assuming if I tunnel to the Node container that should give access to the ZAP container?
While I think I dimly understand what the tunnel does, I donât understand enough about your use case to answer your question. Given that it is for general CI, I would be tempted to say âneitherâ, and that it should be run as a service at the same level as your Docker-in-Docker containers.
It may be worth my pointing out that the tunnel will only live as long as your build process, so it is worth considering how you will make sure your build remains alive (and keeping the tunnel open) until SauceLabs has done whatever it needs to do. Will it send some sort of finished message?
The tunnel basically forwards all HTTP traffic from the browser running on Saucelabâs Cloud to which ever Docker container the tunnel(Sauce Connect) is launched from. The idea is to have the HTTP traffic flow from the Saucelabâs browser to the proxy which is on the Zap container. Once it hitâs that proxy it will then forward the request out to a public Internet site.
