Just a clarification needed,
In docker-compose in non host mode we can reference other containers using the container name.
Originally I thought docker inside of docker would be encapsulated however I now believe the daemon is passed down so the new docker containers are actually created elsewhere in the cloud, which means the reference will be wrong.
Hence when I ran this it says the container <long_id> can’t be found.
So the only solution is to use the Linux VM method, is that correct?
CircleCI’s docker environment is very much designed around the user case they present within their documentation. This does make it a very limited environment and one that you should not consider a general docker-compose environment to be managed in the same way that you would normally.
Out of the direct offerings from CircleCI the Linux VM environment would provide you with an environment in which you could correctly work with a docker-compose based configuration. The main limitation you will find is there is a time penalty incurred as every time the Linux VM is started it must configured the way you need and retrieve all the docker images needed by the docker environment.
CircleCI does have another offering, that of self-hosted runners. Depending on your requirements these can be a very good solution for complex Docker based environments. The CircleCI workflows are run on dedicated environments that you provide (I have a few VMs defined) this means you can provide persistence for things like docker images within your runner between workflow runs.
I resolved it and it’s now running in Linux VM. As you say there’s a bit of overhead in terms of time spinning things up, but that’s one I can kick down the road for a bit.