Prevent race conditions by waiting for services with Dockerize


#1

When you’re using multiple Docker containers for your jobs in CircleCI 2.0 you can encounter race conditions when a service in a container hasn’t started by the time the job tries to use it. For example your PostgreSQL container might be ‘up’ but Postgres itself might not be ready to accept connections.

We plan to add a built-in solution for this. For now, you can work around it by using Dockerize to wait for dependencies.

Here’s how you might do this in your CircleCI config.yml file:

version: 2.0
jobs:
  build:
    docker:
      - image: your/image_for_primary_container
      - image: postgres:9.6.2-alpine
        environment:
          POSTGRES_USER: your_postgres_user
          POSTGRES_DB: your_postgres_test
    workDir: /your/workdir
    steps:
      - checkout
      - run:
          name: install dockerize
          command: wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && sudo tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
          environment:
            DOCKERIZE_VERSION: v0.3.0
      - run:
          name: Wait for db
          command: dockerize -wait tcp://localhost:5432 -timeout 1m

You can apply the same principle for MySQL:

dockerize -wait tcp://localhost:3306 -timeout 1m

Redis:

dockerize -wait tcp://localhost:6379 -timeout 1m

and other services such as web servers

dockerize -wait http://localhost:80 -timeout 1m

Circleci/postgres:9.6 docker image too slow to start
Racing containers
Problem waiting for postgres with dockerize
Connection issues between docker containers
Using Postgres Docker with Environment Vars
Laravel php artisan mysql connection refused
Step was cancelled
MySQL connection error while migrating to 2.0
Connection refused MySQL
How can I create multiple MySQL databases in the same Docker image?
#2

Do you have any plan to support waiting for services by circle.yml?


#3

I set this up for caching so that builds do not fail if there are connectivity issues for the Dockerize download:

  - restore_cache:
      key: dockerize-{{ arch }}-v0.6.0

  - run:
      name: install dockerize if not found in cache
      command:
        if \[ -f /usr/local/bin/dockerize \]; then
          echo "dockerize found; skipping installation";
        else wget   https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
          && tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
          && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz;
        fi

  - save_cache:
      key: dockerize-{{ arch }}-v0.6.0
      paths:
        - /usr/local/bin/dockerize

I am unsure how to interpolate the $DOCKERIZE_VERSION variable into the cache key so if anyone knows how to do that I am all ears.


#4

Hey @tom, any news on the built-in solution for this? Cheers.


#5

You can also run Docker Compose inside your build container, and specify dependencies between services. This is particularly good if you have several dependencies. (FWIW I think this affects start order rather than doing port testing, but there’s no harm in doing the latter as well).


#6

Are there any plans for implementing this in CircleCI core? Because this is getting to a point where we need to restart 75% of our builds sometimes since the Postgres docker is so slow.

I would really appreciate if you could prioritise this feature.


#7

I’m just a CircleCI user, but I imagine features that have an acceptable workaround would be less likely to be prioritised.

That sounds like an architectural problem that could be fixed within the existing provision. I’d suggest a new post, showing your YAML config and whatever other relevant things you’d need to show readers.


#8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.