Setting up minio storage docker

I’m using minio (mock s3 storage) to help test our S3 uploading/configurations, like so:

jobs:
    test:
        docker:
           - image: circleci/python:3.6-jessie-browsers
           - image: minio/minio:RELEASE.2018-12-06T01-27-43Z
             name: minio
           - image: minio/mc:RELEASE.2019-02-06T20-26-56Z

This all works fine, but I cannot access this storage inside the test running section, is this true? I cannot access it via the network for some reason?

How are you trying to access it?

Normally through my application, which is through boto python libraries (S3). I tried cURLing to the server and that doesn’t work either – it seems to be a documented problem, you cannot access docker containers via network on circleCI. I solved this by not using docker at all and building things within the circleci container.

If you spin up your own Docker containers then yes, you cannot expose ports to the (Docker) host. I believe the reason for this is that the permissions required to do so would also give you permissions to the networking stack that would affect other customers, and that would not be acceptable for security reasons.

However, the two Minio images you have in your config can expose ports, and they will be connected to the same port on the build server localhost, using some CircleCI networking magic.

Finally, you can solve the first restriction by using Docker Compose - while you cannot publish ports in Docker, you can use internal Docker networking. So, if you want to connect to a Docker service, do it from another container on the same virtual network.

I could only talk to the dockers through other dockers right? My normal test py.test won’t be able to talk to the docker network? We could maybe use docker-compose… but have moved on from this now :frowning:

Yes, that’s right - if you run your tests in another container and connect them with a private virtual network, then it will work. This is because you don’t need to expose the port to get the connectivity.

Maybe when your CI is ripe for a refactoring! :grin:

1 Like