Postgres container strangeness - `could not connect to server` yet DB is created, migrations run, tests run

docker
postgresql
circle.yml
2.0

#1

I’m getting some strange output in one of our builds…

version: 2
executorType: docker
containerInfo:
  - image: ruby:2.3.1
  - image: postgres:9.4.1
    env:
      - POSTGRES_USER=ubuntu
stages:
  build:
    workDir: ~/appName
    steps:
      - type: checkout
      - type: shell
        name: Install System Dependencies
        command: apt-get update -qq && apt-get install -y build-essential postgresql libpq-dev nodejs rake
      - type: shell
        name: Install Ruby Dependencies
        command: bundle install
      - type: shell
        name: Run Tests
        environment:
          DATABASE_URL: "postgres://ubuntu@localhost:5432/db_name"
        command: |
          bundle exec rake db:create db:schema:load --trace

Then produces this output at that db:create step:

bundle exec rake db:create db:schema:load --trace
** Invoke db:create (first_time)
** Invoke db:load_config (first_time)
** Execute db:load_config
** Execute db:create
Created database 'db_name'
could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Couldn't create database for {"adapter"=>"postgresql", "pool"=>5, "timeout"=>5000, "database"=>"ctb_sda_test"}
rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

The weird thing is if I collect together the shell steps for the next parts like so:

      - type: shell
        name: Run Tests
        environment:
          DATABASE_URL: "postgres://ubuntu@localhost:5432/db_name"
        command: |
          bundle exec rake db:create db:schema:load --trace
          bundle exec rake db:migrate
          bundle exec rake

Then the migrations will run fine and the tests will run fine against the DB, even despite the supposed output that Postgres isn’t running.

Another bizarre thing is that I have another absolutely identical build (save for the app names and db names) with a similar side-along Postgres container which works just fine.

I’ve also tried the following type of syntax to set things up but the same output happens:

  - type: shell
    name: Set up DB
    command: |
      createuser -h localhost --superuser ubuntu &&
      createdb -h localhost db_name
  - type: shell
    name: Install Project Dependencies
    command: bundle install
  - type: shell
    name: Create DB
    command: bundle exec rake db:create db:schema:load --trace

I’m a bit stumped as to what to try next, any thoughts?


#2
version: 2
executorType: docker
containerInfo:
  - image: ruby:2.3.1
    env:
      - REDIS_URL="redis://localhost:6379/"
      - RAILS_ENV=test
      - RACK_ENV=test
      - PG_HOST=localhost
      - PG_USER=ubuntu
  - image: redis:2.8.19
  - image: memcached:1.4
  - image: elasticsearch:1.4.2
  - image: postgres:9.5
    env:
      - POSTGRES_USER=ubuntu
      - POSTGRES_DB=circle_ruby_test

When using multiple commands chained together, it’s good practice to combine them with && or prefix them all with set -e to ensure the step returns 1 and fails on any errors.


#3

Hi rohara,
thanks for responding. I’ve tried setting the explicit PG_HOST and PG_USER env variables as you suggest now, and the appropriate ones for the postgres container as well, but still getting the same output:

$ bundle exec rake db:create db:schema:load --trace &&  
bundle exec rake db:migrate
** Invoke db:create (first_time)
** Invoke db:load_config (first_time)
** Execute db:load_config
** Execute db:create
could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Couldn't create database for {"adapter"=>"postgresql", "pool"=>5, "timeout"=>5000, "database"=>"db_name"}
rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

The updated parts of circle.yml follows (I even changed the postgres version to match your example):

version: 2
executorType: docker
containerInfo:
  - image: ruby:2.3.1
    env:
      - PG_HOST=localhost
      - PG_USER=ubuntu
      - RAILS_ENV=test
      - RACK_ENV=test
  - image: postgres:9.5
    env:
      - POSTGRES_USER=ubuntu
      - POSTGRES_DB=db_name
stages:
  build:
    workDir: ~/app_name
    steps:
      - type: checkout
      - type: shell
        name: Install System Dependencies
        command: apt-get update -qq && apt-get install -y build-essential postgresql libpq-dev nodejs rake
      - type: shell
        name: Install Ruby Dependencies
        command: bundle install
      - type: shell
        name: Set up DB
        command: |
          bundle exec rake db:create db:schema:load --trace &&  
          bundle exec rake db:migrate

As I said previously, it is very strange as I have a practically identical build which functions fine.


Getting CircleCICucumberFormatter::CircleCIJson error when upgrading Cucumber
#4

So it seems that it’s now working, but only if I combine the env set against the image in the containerInfo section and also include the environment configuration in the shell command.

ie, this works, and doesn’t complain about the DB connection:

version: 2
executorType: docker
containerInfo:
  - image: ruby:2.3.1
    env:
      - PG_HOST=localhost
      - PG_USER=ubuntu
      - RAILS_ENV=test
      - RACK_ENV=test
  - image: postgres:9.5
    env:
      - POSTGRES_USER=ubuntu
      - POSTGRES_DB=db_name
stages:
  build:
    workDir: ~/appName
    steps:
      - type: checkout
      - type: shell
        name: Install System Dependencies
        command: apt-get update -qq && apt-get install -y build-essential postgresql libpq-dev nodejs rake
      - type: shell
        name: Install Ruby Dependencies
        command: bundle install
      - type: shell
        name: Set up DB
        environment:
          DATABASE_URL: "postgres://ubuntu@localhost:5432/db_name"
        command: |
          bundle exec rake db:create db:schema:load --trace &&
          bundle exec rake db:migrate

#5

Okay, I just wanted to point out why this worked.

Rails will try to use a database URL in the following order:

  1. The DATABASE_URL environment variable, if set
  2. The config for appropriate environment in config.yml (usually test for your test suite)

In CircleCI 1.0, you didn’t have to think about this, since our Rails inference would rewrite your config.yml test section to use the database config matching what was in our image.


#6

Gotcha, thanks for clarifying that Eric!


#7