Running component tests inside of build process with docker containers (testcontainers)

docker
#1

I have a java application that is build with Gradle inside a Dockerfile and runs during the build some component tests that needs some docker containers. The containers are created via testcontainers.

With this circle.config I got managed to start the containers:

jobs:
  build:
    docker:
      - image: circleci/buildpack-deps
    steps:
      - checkout
      - setup_remote_docker:
          docker_layer_caching: true
      - run:
          name: Copy docker certificates
          # Prepare docker environment for building the jar from inside a docker container
          command: |
            ls /var/run/docker.sock
            cp -R $DOCKER_CERT_PATH ./circle-cert
      - run:
          name: Build+Test
          command: |
            docker build --target=build -t build \
              --build-arg DOCKER_HOST=${DOCKER_HOST} \
              --build-arg DOCKER_CERT_PATH=/circle-cert \
              --build-arg DOCKER_MACHINE_NAME=${DOCKER_MACHINE_NAME} \
              --build-arg DOCKER_TLS_VERIFY=${DOCKER_TLS_VERIFY} \
              .

My Dockerfile:

FROM amazoncorretto:8u202 AS build
ADD . /src/
RUN yum -y install which
WORKDIR /src/
ENV LC_ALL C.utf8
ENV LANG C.utf8
# docker env for component tests
ARG DOCKER_HOST
ENV DOCKER_HOST=$DOCKER_HOST
ARG DOCKER_CERT_PATH
ENV DOCKER_CERT_PATH=$DOCKER_CERT_PATH
ARG DOCKER_MACHINE_NAME
ENV DOCKER_MACHINE_NAME=$DOCKER_MACHINE_NAME
ARG DOCKER_TLS_VERIFY
ENV DOCKER_TLS_VERIFY=$DOCKER_TLS_VERIFY
ARG NO_PROXY
# https://github.com/testcontainers/testcontainers-java/issues/700
ENV TESTCONTAINERS_RYUK_DISABLED true
ADD circle-cert $DOCKER_CERT_PATH

RUN ./gradlew build

When I try to connect to the Postgres DB (jdbc:postgresql://104.196.188.43:32796/xxx) I got a java.net.SocketTimeoutException: connect timed out.

How can I solve my issue?

#2

Where are you attempting to connect to the database? Is this in the docker build step, where gradlew is invoked?

Where is 104.196.188.43 - is that a remote PostgreSQL server? Assuming that my earlier suspicion is correct, it may be worth noting that it is rather unusual to contact external database servers in a build process. I wonder if the contents of that can be brought into the build as a text-based asset?

#3

104.196.188.43 is the external docker host from CircleCI. Where also the database is running that the tests need to access to.

#4

Could I have your comments on this?

#5

In my opinion its very usual to Communicate with a „external“ Database during Build (Test) process. There are component Tests that i want to run.

#6

Well, OK - we can disagree about that, but my wording perhaps led you in the wrong direction. Can you give an expansive answer about whether you can pull a database dump into your CI, spin up a database and then import the data into it? For example, this could be a SQL file in your repo.

Another way of doing this is to maintain a list of migration files (paired with a migration framework) so they can be run for every CI build.

Certainly, you should be able to do what you are asking for, but it will be slow, and may suffer from occasional network outages. If I were coerced into doing that, I’d get a post-build SSH session and try some database connections from the terminal. I don’t think there are many CircleCI firewall outbound blocks, except perhaps the usual SMTP restrictions, so I’d investigate if there are any inbound firewall restrictions on the remote database server.

#7

The tests must also run if I do not use the CI (e.g. with gradle test). So it’s not a solution to put the database into the CircleCI config file (I am correct that this was your suggestion?).
Thanks for your help.

#8

Sort of! I think it should go in your project repo rather than the CircleCI config file. However, read on…

Yes, that is a very good idea. However, your answer suggests to me that your data is static, but that your current test framework presently cannot run tests that need a database.

I happen to have solved this problem recently, with an API project. We run tests locally, and we run them in CircleCI too. In the tests, we run unit tests, database tests and (HTTP) API tests. The basis of being able to do this includes:

  • The application can create its own database structure using a script (migrations)
  • The application can insert some demo data using a script (seeders)
  • The application can run and configure its own database (we use Docker Compose, but there are probably several ways of doing this)
  • Tests that need the database have no order dependency (all tables are truncated in between every test, and each test sets up the initial database state it needs)

We use environment variables in the test bootstrap too, for things like the database host. This has had the happy effect that we have been able to tweak it so we can run our tests on two different major versions of the database (Docker Compose spins up Postgres 9 and Postgres 10).

I am not familiar with Gradle, but if you can do something like the above then you can also run your local tests without the network. For me, that is not only faster, but allows me to work on the move (where there is only a poor internet connection).

#9

Hi Ben. I was going to suggest Mocking the database calls, but you mentioned you are using Docker. Would Docker Compose work to get you access to a database instance for testing? I hope you are not running tests against your production database live.

#10

@halfer but where is your database? How do you create it?

@drazisil no it’s not possible, because we create the resources programmatically. Where is the difference with docker compose? In this case I would start all the container also via the remote docker host and the application must talk to it?

#11

On CircleCI, you have two options: a remote Docker instance of your MySQL/PostgreSQL/etc set up in your config.yml, and connected to through the localhost connection; also you can spin up your own database image using Docker-in-Docker. That is what I prefer, as it is more portable.

So, to answer this question directly, your database is either on the CircleCI build server, or on a remote CircleCI server that is connected to the build server.

In your test bootstrap, or before the tests are started. This can be as simple as running psql -pPassword -uUser < data/create.sql, which contains some CREATE TABLE statements. I use a migrations framework though; since I am using PHP, I use Phinx. However, the Java world is sure to have some good equivalents.

#12

Thats exactly what I try to do. But it’s not possible to talk to the database on the remote docker host.

1 Like
#13

(Clarification of terms: when I said “remote” above, I meant a temporary (throw-away) Docker instance that is part of the CircleCI infrastructure, in the same AWS region. I did not mean an external database outside of the CircleCI infra, which is what I am encouraging you to avoid).

#14

I Talk about the Remote docker Host given by the CircleCI Environment.

#15

Well, OK - if you have a problem, then you will need to give a detailed post about the problem, the configuration you have, and what error(s) you are getting. Good luck!

#16

I already postet all details in my first post.

#17

I don’t think you have. Your first post was about using an external database server, which was not working for you, and which I have discouraged as slow and prone to network failure.

We then started a discussion about whether you could spin up a temporary database within CircleCI infrastructure, which is what I would recommend for CI speed and reliability. However, your first post contains an IP address that you state is an external remote database outside of CircleCI, and so this relates to the original problem, and not the one we were discussing.

I suspect we are just talking past each other at this point, so I will make this my last post. I wish you luck with your CI problem, however you tackle it.

#18

No, the IP was the IP of the CircleCI server where I started the DB.