Connecting Service with DB using Docker-Compose

Build Environment
, ,
1

I’m trying to using docker-compose to build my service and database and then run my unit tests.
I’m able to get both images built and running with docker-compose and begin running the tests. All my tests that require a database connection fail with an error that my database does not exist.

error (DatabaseError) @Error {
    code: '3D000',
    column: undefined,
    constraint: undefined,
    dataType: undefined,
    detail: undefined,
    file: 'postinit.c',
    hint: undefined,
    internalPosition: undefined,
    internalQuery: undefined,
    length: 96,
    line: '885',
    position: undefined,
    routine: 'InitPostgres',
    schema: undefined,
    severity: 'FATAL',
    table: undefined,
    where: undefined,
    message: 'database "lnd-onboard" does not exist',
  }

I can confirm the database container at least exists, but haven’t figured out how to test specifically if the actual database exists. I have however run the setup process with the whole db container absent and I get the same error as above so I assume the issue is with connecting to the container, not the existence of the db.

#!/bin/bash -eo pipefail
set -e
docker ps -a

CONTAINER ID   IMAGE                 COMMAND                  CREATED         STATUS                              PORTS                                       NAMES
5fbc7b4dc2b3   project-svc           "docker-entrypoint.s…"   9 seconds ago   Exited (1) Less than a second ago                                               project-svc-1
b94a4695f56a   flyway/flyway:9.8.3   "flyway -locations=f…"   9 seconds ago   Exited (0) Less than a second ago                                               project-migrations-up-1
a3c0323f7028   flyway/flyway:9.8.3   "flyway clean"           9 seconds ago   Exited (0) 4 seconds ago                                                        project-migrations-clean-1
b4131715291e   postgres:15.1         "docker-entrypoint.s…"   9 seconds ago   Up 8 seconds                        0.0.0.0:5432->5432/tcp, :::5432->5432/tcp   project-db-1
CircleCI received exit code 0

Here’s my circle config and my docker-compose.yaml

version: 2.1

jobs:
  build_and_test:
    docker: 
      - image: cimg/node:16.16.0
      - image: cimg/postgres:14.4
    steps: 
      - checkout
      - restore_cache:
          keys:
              - mychache-{{ checksum "yarn.lock" }}
      - run:
          name: Install Dependencies
          command: yarn install
      - run:
          name: Install Flyway
          command: wget -qO- https://repo1.maven.org/maven2/org/flywaydb/flyway-commandline/9.8.3/flyway-commandline-9.8.3-linux-x64.tar.gz | tar xvz && sudo ln -s `pwd`/flyway-9.8.3/flyway /usr/local/bin
      - save_cache:
          paths:
            - flyway-9.8.3
            - node_modules
          key: mycache-{{ checksum "yarn.lock" }}
      - run:
          name: Install Docker Compose
          command: |
            set -x
            curl -L https://github.com/docker/compose/releases/download/v2.14.0/docker-compose-`uname -s`-`uname -m` > ~/docker-compose
            chmod +x ~/docker-compose
            sudo mv ~/docker-compose /usr/local/bin/docker-compose
      - setup_remote_docker
      - run:
          name: Build images and start services using 'docker-compose'
          command: docker-compose up -d
      - run: 
          name: testing
          command: |
            set -e
            docker ps -a

      - run:
          name: Run tests
          command: yarn ava --serial

workflows:
    run_tests:
      jobs:
        - build_and_test

services:
  db:
    image: postgres:15.1
    restart: always
    ports: 
      - 5432:5432
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=Iforgot
      - POSTGRES_DB=lnd-onboard
    volumes:
      - database:/var/lib/postgresql/data

  migrations-clean:
    image: flyway/flyway:9.8.3
    environment:
      - FLYWAY_USER=postgres
      - FLYWAY_PASSWORD=Iforgot
      - FLYWAY_URL=jdbc:postgresql://db:5432/lnd-onboard
      - FLYWAY_CLEAN_DISABLED=false
    command: clean
    volumes:
      - .:/migrations
    depends_on:
      - db

  migrations-up:
    image: flyway/flyway:9.8.3
    environment:
      - FLYWAY_USER=postgres
      - FLYWAY_PASSWORD=Iforgot
      - FLYWAY_URL=jdbc:postgresql://db:5432/lnd-onboard
    command: -locations=filesystem:/migrations -connectRetries=60 migrate
    volumes:
      - .:/migrations
    depends_on:
      migrations-clean:
        condition: service_completed_successfully

  svc:
    build:
      context: .
      dockerfile: rest.dockerfile
    environment:
      - ENVIRONMENT=local
      - PGHOST=db
      - PGUSER=postgres
      - PGPASSWORD=Iforgot
      - PGDATABASE=lnd-onboard
    ports:
      - 8080:8080
    links:
      - db
    depends_on:
      migrations-clean:
        condition: service_completed_successfully
      migrations-up:
        condition: service_completed_successfully
        
volumes:
  database:

Is there something I’m missing here to properly create this connection? Everything works locally so I have to assume the issue is something pertaining to circleci that I’m unaware of. Any advice would be greatly appreciated!

2

:wave: Hello @bsheridan,

Could it be that the DB is not yet ready? :thinking:

Try adding the following step just before your Run tests step:

  - run:
      name: Waiting for DB to be ready
      command: dockerize -wait tcp://localhost:5432 -timeout 1m

Let me know if this helps.

3

Hi @yannCI,

Thank you for the response!
Unfortunately that did not do the trick, got the same error message as before.

4

Thanks for getting back to me, @bsheridan.

How are you connecting to the DB in your tests? What hostname:port are you specifying?

5

I’m attempting to connect using pg PoolClient after fetching the info from the docker-compose file as it starts up:

export const EnvConfig = {
    environment: process.env.ENVIRONMENT as Environment || Environment.DEV,
    pool: new Pool({
        user: process.env.PGUSER || 'postgres',
        host: process.env.PGHOST || 'localhost',
        database: process.env.PGDATABASE || 'lnd-onboard',
        password: process.env.PGPASSWORD || 'Iforgot',
    }),
    port: process.env.PORT || 8080,
}

--------------------------------------------------------------------------------------------------------

export const connectPgClient = async (): Promise<PoolClient> => EnvConfig.pool.connect();