[config.yml] How to use circleCI for running tests from a folder containing test.py files

Build Environment
1

Hello there,

I am trying to create a circleCI YAML file for our data-platform.
this is how the basic structure of our repo currently looks like -
data-platform/.circleci/config.yml
data-platform/database_jobs/{all the python feed files}
data-platform/database_jobs/tests/{all the python feed-test files}
feed-test files use pytest for unit-tests

and from my basic understanding of config.yml I came up with this -

version: 2
jobs:
  build:
    docker:
      - image: circleci/python:2.7.14-jessie
        environment:
          TEST_DATABASE_URL: postgresql://root@localhost/circle_test?sslmode=disable
      - image: circleci/postgres:9.6.5-alpine-ram
        environment:
          POSTGRES_USER: root
          POSTGRES_DB: circle_test
          POSTGRES_PASSWORD: ""
    steps:
      - checkout
      - restore_cache:
          key: deps1-{{ .Branch }}-{{ checksum "database_jobs/requirements.txt" }}
      - run:
          name: Install Python dependencies in a venv
          command: |
            virtualenv venv
            . venv/bin/activate
            pip install -r database_jobs/requirements.txt
      - save_cache:
          key: deps1-{{ .Branch }}-{{ checksum "database_jobs/requirements.txt" }}
          paths:
            - venv
  test:
    docker:
      - image: circleci/python:2.7.14-jessie
    steps:
      - checkout
      - run:
          name: Runnning tests
          command: |
            . venv/bin/activate
            py.test -vv database_jobs/tests

workflows:
  version: 2
  build_and_test:
    jobs:
      - build
      - test

Questions:

  1. how to use pytest for testing all the test files located in data-platform/database_jobs/tests/ ?
  2. I have tests that rely on database connections. Do I have to create a circleCI image for testing postgres database OR I will have to create my own database instance on the server(AWS) for running tests?

Error I am getting when trying to run the tests, also it’s not letting me activate my virtual environment:

#!/bin/bash -eo pipefail
. venv/bin/activate
py.test -vv database_jobs/tests
/bin/bash: venv/bin/activate: No such file or directory
Exited with code 1
2

I’m not a Pythonista, but the answer to “how to do X” questions is usually “the same as if you were doing it locally”. Can you run pytest locally to do what you want?

You could use an external VPS provider, but I wouldn’t bother, as that’s going to be slower. You have several options:

  • Add a database image (as you are doing currently) and connect to that
  • Spin up your tests and databases and anything else in a Docker Compose environment under your control
  • Install a database server directly into the build server

Whatever approach you take, you’ll need to initialise the database server with a test database, and run any migrations or SQL import on it before you run your tests.

3

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.