I am reading through the executors part (https://circleci.com/docs/2.0/configuration-reference/#executors) but I failed to find a way to allow my non-primary docker containers to link against each other. It isn’t specified in the reference so I am wondering if this is supported (in the docker type of build, not `machine’)
also I do not want to use remote executor nor docker-compose because in my Java project the primary docker image is maven which i do not want to build into final docker image - i am only using the primary docker environment to test, not building docker image.
I don’t fully understand your question, but basically, any containers you specify in the
docker section can connect to each other via TCP, just be wary of startup times for Postgres, MySQL, and such. If you
setup_docker_engine and bring up more containers from within
docker, you can’t connect to them because they are running elsewhere.
By link, I think that the author of the thread was referring to something like this.
I haven’t found a way to make it work.
hub: image: selenium/hub:3.4.0 ports: - 4444:4444 chrome: image: selenium/node-chrome:3.4.0 links: - hub
Those two containers can talk to each other there, but you can’t connect to them.
On what IP/Port can they talk to each other? Is there any way to see the logs? We need to run kafka for some of our tests and it depends on zookeeper.
I think it would be a lot more useful if you allowed specifying a docker compose file, rather than the subset of duplicate commands documented here: https://circleci.com/docs/2.0/configuration-reference/#docker--machine-executor. Which don’t seem to cover this fairly important case.
docker-compose up on the
Machine doesn’t offer all of the same features, specifically custom build images. In addition using docker compose within the docker image is non-trivial and also lacking in functionality. See my blog post here: https://medium.com/chaotic-snippets/circleci-2-0-and-docker-compose-a6183b9b6d05 for the details. It would be far more valuable to be able to run the containers in the same network as the build container, but also to be able to link them to each other.
There is also the issue of getting access to the logs of the containers that are started by the config.yaml, which seems like a pretty big gap.
You can certainly obtain any of the logs in question. The containers from circle.yml are accessible within a given job and you can export your custom container logs as artifacts.
Having looked through your article, this is not a valid config.yml in any capacity:
The machine only lacks layer caching. You can achieve everything else on the machine. It’s a full virtual machine with root privileges.
Ya, the circle config example was from earlier in the beta.
I don’t understand how I can achieve everything else on machine without having to orchestrate my own workflow within a job. Especially since it doesn’t have particularly current tools installed and as far as I understand there is in fact no way to provide a custom environment with newer tools. As an example the version of compose is 2.0, so running
docker-compose up does not in fact “just work”.
In addition, when I tested using machine for our integration test job, it broke pretty badly when restoring a cache from another job because the cache while defined in the save step with a relative path, actually seem to be stored with an absolute path. Since it happen that the path is /root the restore fails with permission denied.
Can you show me an example where it doesn’t work?
Well, our compose files are pretty much all 3.x at this point. I don’t think any compose file after 2.x will work. Certainly you could install a more recent version perhaps, but that just adds even more to the build time.
This is why I really like the custom build containers that were added as part of 2.0, but they don’t work well with machine (you would have to do a bunch of orchestration to run a build container, clone repos, pull out artifacts, etc) inside of the build job (maybe I am missing something here?). This seems like re-inventing the wheel to a certain extent, which why it would be nice of the docker option in circle jobs had a bit more flexibility.
any containers you specify in the docker section can connect to each other via TCP, just be wary of startup times for Postgres, MySQL, and such.
What IP/host and port can they connect to each other on?
Looks like it’s just localhost. I was expecting some kind of named host similar to how docker-compose works.
This topic was automatically closed 41 days after the last reply. New replies are no longer allowed.