Circleci/mongo conn issue

Build Environment Databases
1

I have a go app that needs mongo and redis as services, everything was working fine. all in a sudden mongo service is not reachable anymore. redis container is still working fine though.

I found in couple of topics that it might be a race issue, go app already started while mongo container is not completely running. so I added dockerize to wait for mongo but it gave me connection refused all the time. looks like it is something with running mongo itself inside circleci environment.

  docker:
      - image: circleci/golang:1.11
      - image: circleci/mongo:4.1.9
        port: 27017:27017
        environment:
          - MONGODB_USERNAME: "x"
          - MONGODB_PASSWORD: "x"
          - MONGO_INITDB_ROOT_USERNAME: "x"
          - MONGO_INITDB_ROOT_PASSWORD: "x"
      - image: circleci/redis:alpine
        port: 6379:6379
      - image: circleci/dynamodb

    steps:
      - setup_remote_docker
      - run:
          name: Waiting for Redis
          command: dockerize -wait tcp://localhost:6379 -timeout 1m
      - run:
          name: Waiting for Mongo
          command: dockerize -wait tcp://localhost:27017 -timeout 1m
      - checkout
      - run:
          name: Test  
          command: |
            ./go.test.sh

Any advises

2

Hi,

It looks like the error is caused by the use of the MONGO_INITDB_ROOT_USERNAME and MONGO_INITDB_ROOT_PASSWORD environment variables, which is related to this open issue with the upstream mongo image: https://github.com/docker-library/mongo/issues/323

As a workaround for now, could you try removing the MONGO_INITDB_ROOT_USERNAME and MONGO_INITDB_ROOT_PASSWORD environment variables? The root user could be added in a separate step, for example via the following command:

# Requires installation of the mongo client
mongo --eval 'db.createUser({ user: "myrootuser", pwd: "mypassword", roles: [ { role: "userAdminAnyDatabase", db: "admin" } ] });'
3

I have the exact same issue as above, all of a sudden our workflow is broken and we can’t test and deploy anymore. The above solution doesn’t work for us…also it’s very generic, could you please be more specific and propose a complete solution?

2 Likes
4

I doubt this is the solution to the problem, I found the same ‘FileOpenFailed’ error in a job that previously successfully ran. This issue has started to happen 30hr ago, without any change at our end in the configuration.

5

I fixed the issue by removing both MONGO_INITDB_ROOT_USERNAME and MONGO_INITDB_ROOT_PASSWORD. Also, I had to change the code as well not to use any auth.

1 Like
6

@stella It was working before, it was broken in CircleCI all of a sudden and I don’t think it has any relation the the mongo issue you linked. from the logs I can see the mongo was up and the user was created. may be there’s something related to docker network I don’t know I couldn’t even trace the issue after I SSH to the machine because it doesn;t show current running containers even with enable_remote_docker

7

@stella I am having the same issue.

MongoDB init process complete; ready for start up.

should come after it creates the user and roles. but it exits just before this.

When i run the same docker image in my local it works.

error in the end for pymongo when it exits. -

pymongo.errors.ServerSelectionTimeoutError: localhost:27017: [Errno 111] Connection refused

the container does not maintain the state also. All the processes are in defunct state. So there is no point of ssh.
I cant even see the /proc/ logs for the mongo PID, says access denied.

8

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.