Testing a multicontainer app (Flask + Redis) with Selenium

Build Environment
1

I am fairly new to CI in general and CircleCI, in particular.

I have a web app that runs 2 docker containers: Flask and Redis. I also have a test suite based on Selenium that runs locally:

  • I spin up the 2-container app using docker-compose up -d
  • then I run browser tests against the docker app, and everything passes

I am trying to achieve the same using CircleCI. I spend hours researching, and below is my config.yml:

# Python CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-python/ for more details
#
version: 2
jobs:
  build:
    docker:
      - image: circleci/python:3.6-buster-browsers

    working_directory: ~/sjaandi_deploy

    steps:
      - checkout

      # Download and cache dependencies
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "requirements.txt" }}
            # no fallback

      - run:
          name: Install Pre-Dependencies
          command: |
            sudo apt-get -y install libc-dev
            sudo apt-get -y install build-essential
            sudo pip install -U pip

      - run:
          name: Install Testing Dependencies
          command: |
            python3 -m venv venv
            . venv/bin/activate
            pip install -r requirements-circleci.txt

      - setup_remote_docker

      - run:
          name: Install Docker Compose
          command: |
            curl -L https://github.com/docker/compose/releases/download/1.19.0/docker-compose-`uname -s`-`uname -m` > ~/docker-compose
            chmod +x ~/docker-compose
            sudo mv ~/docker-compose /usr/local/bin/docker-compose

      - run:
          name: Build And Spin Up Flask App
          command: |
            docker-compose up -d

      - save_cache:
          paths:
            - ./venv
          key: v1-dependencies-{{ checksum "requirements.txt" }}

      # run tests!
      - run:
          name: Run Tests
          command: |
            . venv/bin/activate
            coverage run --source=./flask_app -m pytest tests/ -svv

      - store_artifacts:
          path: test-reports
          destination: test-reports

workflows:
  version: 2
  build_workflow:
    jobs:
      - build

And my Dockerfile is:

FROM circleci/python:3.6-buster

# install dependencies
ADD requirements.txt requirements.txt
RUN sudo apt-get -y install libc-dev
RUN sudo pip install -U pip
RUN sudo pip install -r requirements.txt

# copy flask source
RUN sudo mkdir /flask_app
ADD flask_app /flask_app
RUN sudo mkdir /flask_app/static/users
RUN sudo chown -R circleci:circleci /flask_app/static/users/

ENV FLASK_APP flask_app
ENV FLASK_RUN_HOST 0.0.0.0

CMD ["flask", "run"]

When I push to GitHub, CircleCI starts the build, it goes successfully until the tests, at which point the browser is trying to get 0.0.0.0:5000 (address specified in Dockerfile) and fails the test because the response is 404.

My question is, how do I know at which address my tests can reach out the app running inside a remote Docker container in CircleCI?

Update 1:

I added docker ps at the end of the build, and here is the output:

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                  PORTS                    NAMES
6b9614437251        sjaandideploy_web   "flask run"              1 second ago        Up Less than a second   0.0.0.0:5000->5000/tcp   sjaandideploy_web_1
f7cdf3b7d337        redis               "docker-entrypoint.s…"   1 second ago        Up Less than a second   6379/tcp                 sjaandideploy_redis_1

Update 2:

Also, I examined selenium.driver.page_source after requesting 0.0.0.0:5000 and it has only this: <html><head></head><body></body></html>.

1 Like
2

I was able to solve the problem by changing executor from Docker to machine.

3

You’ve not shown your Docker Compose config, but I suspect that you are trying to expose a container port to your build server. That is disallowed for security reasons - if you could do that then you would have control over the Docker networking stack that is shared with other customers.

One nice solution to this is to run your tests in another container, and just refer to the server by its service name - Docker Compose creates virtual DNS entries automagically, and no port exposure is required for between-container networking.

1 Like
4

Yes I was trying to expose docker containers to the build server. I didn’t try your solution but it makes sense. I was able to solve my problem by using machine executor, which has access to running Docker containers.

5

Great! It’s moderately worth noting that the docs warn the pricing model for Machine executors may change in the future. However, they have been saying that for about two years, so you may be OK! I would expect you would get plenty of advance notice if that were to happen though.

6

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.