How to securely set environment variables for Postgres Image?

Build Environment
1

I’ve been searching all over the documentation, this forum, and the rest of the Internet, and I can not seem to find any answer to this.

When you are using a container image, in my case, the postgres one, how do you securely set the environment variables (POSTGRES_USER, POSTGRES_DB, POSTGRES_PASSWORD) via other environment variables or any other mechanism? Is everyone seriously just hard-coding their postgres credentials right into their config.yml files for the entire world to see? Is everyone using the default root user or automatic trust settings that are specifically warned against?

This is the most relevant thread I could find and it did not help very much at all.

I understand that there is no environment variable interpolation at that spot, so I already know that this straightforward approach will not work. I do think however, that this is a serious problem. There’s no reason it shouldn’t work other than “it doesn’t”. I urge CircleCI to make whatever changes are necessary so that this does work.

 - image: circleci/postgres:latest
   environment:
     POSTGRES_USER: ${DB_USER}
     POSTGRES_DB: ${DB_NAME}
     POSTGRES_PASSWORD: ${DB_PASSWORD}

Ok, so I tried a parameterized approach. It also did not work. Unlike the previous example, I don’t understand why this doesn’t work. Is it the same reason that there is no interpolation here? What are environment variables for if we can’t use them for anything?

...orb stuff...
  executors:
    default:
      description: CircleCI python image
      docker:
        - image: circleci/python:<< parameters.tag >>
        - image: circleci/postgres:latest
          environment:
            POSTGRES_USER: << parameters.db_user >>
            POSTGRES_DB: << parameters.db_name >>
            POSTGRES_PASSWORD: << parameters.db_password >>
      parameters:
        tag:
          default: latest
          description: Tag of the python docker image to use. Must include poetry.
          type: string
        db_user:
          default: dbuser
          description: PostgreSQL database username
          type: string
        db_name:
          default: dbname
          description: PostgreSQL database name
          type: string
        db_password:
          default: dbpassword
          description: PostgreSQL database password
          type: string

jobs:
  build-and-test:
    executor:
      name: python-poetry/default
      db_user: ${DB_USER}
      db_name: ${DB_NAME}
      db_password: ${DB_PASSWORD}

So I tried removing all the environment stuff from config.yml and just going with

- image: circleci/postgres:/latest

And then I put the variables directly into my project settings. This didn’t work either.

image

I’m at the end of my rope. What is the answer here? How are other people handling this? Thanks.

2

Currently, there isn’t a great way to get secrets into Docker containers when using multi-docker builds.

As you have discovered, the environment variables are not expanded in the environment section like they are in the auth section

- image: example/example:1.2.3
  auth:
    username: $USERNAME
    password: $PASSWORD

The secondary container does not have the environment variables passed in like they are in the primary container. So, as you also discovered, you cannot set these in your Project Settings in the UI.

Many configurations are using Postgres as a test database which is seeded with test data. Because of that, and the fact that these databases are short-lived ephemeral containers, the username and password are not security concerns. They are often just things like root or the name of the project.

With that said, everyone’s use-case is unique so I’ll assume you have a good reason for needing production or other sensitive secrets in your test database for CI. If that is the case, you probably want to look at making use of the machine executors. Machine executors are full VMs and have things like Docker and Docker Compose preconfigured for you. That way you could just run your Postgres container directly and pass whatever values you want into it. Either via docker run or docker-compose. Think of it more like what you would be able to do locally on your development machine.

If you really did need to pass those secrets while use the Docker builds, you can create a custom image based on the Postgres image and pass those values in over the network. There is an article that talks about some techniques, while complex, that can accomplish that.

1 Like
3

Thanks! I’ll have to try this and see if it’s worth the hassle of doing the complex setup or if I should just give up and hard-code the credentials.

1 Like
4

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.