CircleCI: very frustrating experience to get started

ucals · October 24, 2020, 5:10pm

Hi,
I’m new to CircleCI and it has been very frustrating to get a minimal Python/PyTest example working.

According to this documentation, a minimal config.yml example would be something like:

version: 2.1

orbs:
  python: circleci/python@1.2.0
  
workflows:
  main:
    jobs:
      - python/test:
          pkg-manager: pip
          test-tool: pytest

This minimal example runs pip install, correctly installing all dependencies, but fails with ModuleNotFoundError: No module named XXX. My file structure is:

src:
- model.py
tests:
- test_model.py

In my test_model.py, I have import model, which is causing the error. This works locally, but I cannot make it to work on CircleCI. Also, I couldn’t find anywhere in the documentation a simple example of how to add directories to $PATH environment variable. I suspect just adding /home/circleci/project should make it work…

Can anyone help? Thanks!

PS: I noticed that if I navigate to https://app.circleci.com/projects/github/ORG/REPO/master/config I can edit the config file on-the-fly, and the nice editor will show me whether the configuration has errors. However, I could NOT find how to reach this URL through the website UI (i.e. I have to type the URL every time I want to edit it… is there an easier way to do that?)

dsayling · October 27, 2020, 12:26am

@ucals I’m sorry that the documentation around the test job using pip isn’t very clear. This all very much depends on how you plan on setting up your project structure around tests. Because there are different best practices we can’t assume the project structure will be one way. There also so many ways a project can be setup and the packages added to the PYTHONPATH. Here’s a few options, inclduing a quick fix and some suggestions around python packaging.

Quick Fix

Make sure PYTHONPATH is set. Run export PYTHONPATH=$PWD/src as the fastest way to get this going (also assuming pytest is defined in a requirements.txt already - I simply have pytest installed via pip install pytest in my tests below)

circleci@c772bfc9b4c5:~/project$ tree
.
├── src
│   └── main.py
└── tests
    └── test_hello.py

2 directories, 2 files
circleci@c772bfc9b4c5:~/project$ cat src/main.py 

def hello_world():
    print("Hello world")
    return True
circleci@c772bfc9b4c5:~/project$ cat tests/test_hello.py 


import main

def test_hw():
    main.hello_world()
circleci@c772bfc9b4c5:~/project$ echo $PYTHONPATH

circleci@c772bfc9b4c5:~/project$ pytest -q

================================================================== ERRORS ===================================================================
___________________________________________________ ERROR collecting tests/test_hello.py ____________________________________________________
ImportError while importing test module '/home/circleci/project/tests/test_hello.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../.pyenv/versions/3.9.0/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_hello.py:3: in <module>
    import main
E   ModuleNotFoundError: No module named 'main'
========================================================== short test summary info ==========================================================
ERROR tests/test_hello.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1 error in 0.08s
circleci@c772bfc9b4c5:~/project$ export PYTHONPATH=$PWD/src
circleci@c772bfc9b4c5:~/project$ pytest -q
.                                                                                                                                     [100%]
1 passed in 0.01s

I don’t recommend this as a python best practise, but you can get this going by also adding this environment key to your job:

environment:
  PYTHONPATH: /home/circleci/project/src

Other solutions around python best practices

Make `src` a python package

As a python best practise, src should be setup as a python package. You can do that by creating an __init__.py in src, and updating the import in test_hello.py. Notice here I had to change the python path again, but instead of specifying the src directory, I can simply specify the project root. All directories with __init__.py will be discovered as python packages. Now, in test_hello.py we change the import to be from <package> import <module>

circleci@c772bfc9b4c5:~/project$ tree
.
├── src
│   ├── __init__.py
│   └── main.py
└── tests
    └── test_hello.py

2 directories, 3 files
circleci@c772bfc9b4c5:~/project$ cat tests/test_hello.py 
from src import main

def test_hw():
    main.hello_world()
circleci@c772bfc9b4c5:~/project$ echo $PYTHONPATH
/home/circleci/project
circleci@c772bfc9b4c5:~/project$ pytest
============================================================ test session starts ============================================================
platform linux -- Python 3.9.0, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/circleci/project
collected 1 item                                                                                                                            

tests/test_hello.py .                                                                                                                 [100%]

============================================================= 1 passed in 0.01s =============================================================

If the PYTHONPATH is not set, it will fail…

circleci@c772bfc9b4c5:~/project$ echo $PYTHONPATH

circleci@c772bfc9b4c5:~/project$ pytest
============================================================ test session starts ============================================================
platform linux -- Python 3.9.0, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/circleci/project
collected 0 items / 1 error                                                                                                                 

================================================================== ERRORS ===================================================================
___________________________________________________ ERROR collecting tests/test_hello.py ____________________________________________________
ImportError while importing test module '/home/circleci/project/tests/test_hello.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../.pyenv/versions/3.9.0/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_hello.py:1: in <module>
    from src import main
E   ModuleNotFoundError: No module named 'src'
========================================================== short test summary info ==========================================================
ERROR tests/test_hello.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================= 1 error in 0.08s ==============================================================

Avoid setting PYTHONPATH by installing packages

Now, how can we get around this? One way is by turning your project into an installable site package by creating a setup.py. Here, we’ve added a minimal setup.py and then run pip install -e . to install the package ‘src’ into the path. Note, install_requires=[pytest] would make sure pytest is installed.

circleci@c772bfc9b4c5:~/project$ tree
.
├── setup.py
├── src
│   ├── __init__.py
│   └── main.py
└── tests
    └── test_hello.py

2 directories, 4 files
circleci@c772bfc9b4c5:~/project$ cat setup.py 
import setuptools
setuptools.setup(name="tmp-example",version="0.0.1",author="null",author_email="null",url="null",description="null",install_requires=[
          'pytest',
      ])
circleci@c772bfc9b4c5:~/project$ pip install -e .
Obtaining file:///home/circleci/project
Requirement already satisfied: pytest in /home/circleci/.pyenv/versions/3.9.0/lib/python3.9/site-packages (from tmp-example==0.0.1) (6.1.1)
Requirement already satisfied: attrs>=17.4.0 in /home/circleci/.pyenv/versions/3.9.0/lib/python3.9/site-packages (from pytest->tmp-example==0.0.1) (20.2.0)
Requirement already satisfied: iniconfig in /home/circleci/.pyenv/versions/3.9.0/lib/python3.9/site-packages (from pytest->tmp-example==0.0.1) (1.1.1)
Requirement already satisfied: py>=1.8.2 in /home/circleci/.pyenv/versions/3.9.0/lib/python3.9/site-packages (from pytest->tmp-example==0.0.1) (1.9.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in /home/circleci/.pyenv/versions/3.9.0/lib/python3.9/site-packages (from pytest->tmp-example==0.0.1) (0.13.1)
Requirement already satisfied: packaging in /home/circleci/.pyenv/versions/3.9.0/lib/python3.9/site-packages (from pytest->tmp-example==0.0.1) (20.4)
Requirement already satisfied: toml in /home/circleci/.pyenv/versions/3.9.0/lib/python3.9/site-packages (from pytest->tmp-example==0.0.1) (0.10.1)
Requirement already satisfied: pyparsing>=2.0.2 in /home/circleci/.pyenv/versions/3.9.0/lib/python3.9/site-packages (from packaging->pytest->tmp-example==0.0.1) (2.4.7)
Requirement already satisfied: six in /home/circleci/.pyenv/versions/3.9.0/lib/python3.9/site-packages (from packaging->pytest->tmp-example==0.0.1) (1.15.0)
Installing collected packages: tmp-example
  Running setup.py develop for tmp-example
Successfully installed tmp-example
WARNING: You are using pip version 20.2.3; however, version 20.2.4 is available.
You should consider upgrading via the '/home/circleci/.pyenv/versions/3.9.0/bin/python3.9 -m pip install --upgrade pip' command.
circleci@c772bfc9b4c5:~/project$ echo $PYTHONPATH

circleci@c772bfc9b4c5:~/project$ cat tests/test_hello.py 
from src import main

def test_hw():
    main.hello_world()
circleci@c772bfc9b4c5:~/project$ pytest
============================================================ test session starts ============================================================
platform linux -- Python 3.9.0, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/circleci/project
collected 1 item                                                                                                                            

tests/test_hello.py .                                                                                                                 [100%]

============================================================= 1 passed in 0.01s =============================================================

With this project structure, you can now use the following config without setting any environment variables.

      - python/test:
          pkg-manager: pip-dist
          test-tool: pytest

Using an advance packaging tool

It’s highly recommended that you use a higher level packaging tool like pipenv or poetry. poetry is recommended over pipenv.

Poetry

You can setup poetry without a setup.py and just adding a pyproject.toml. Notice this file specifies the packages (src in this case) to create when setting up the environment.

circleci@5ccc46929985:~/project$ tree
.
├── pyproject.toml
├── src
│   ├── __init__.py
│   └── main.py
└── tests
    └── test_hello.py

2 directories, 4 files
circleci@5ccc46929985:~/project$ pytest
bash: pytest: command not found
circleci@5ccc46929985:~/project$ cat pyproject.toml 
# Example toml for integration testing - this is not used by the orb in anyway
[tool.poetry]
authors=["Test"]
name="test"
description="none"
version = "0.0.1"
packages = [
    { include = "src" },
]

[tool.poetry.dependencies]
python = "*"

[tool.poetry.dev-dependencies]
pytest = "*"

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

circleci@5ccc46929985:~/project$ poetry install
Creating virtualenv test-3aSsmiER-py3.9 in /home/circleci/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... (14.7s)

Writing lock file

Package operations: 10 installs, 0 updates, 0 removals

  • Installing pyparsing (2.4.7)
  • Installing six (1.15.0)
  • Installing atomicwrites (1.4.0)
  • Installing attrs (20.2.0)
  • Installing more-itertools (7.2.0)
  • Installing packaging (20.4)
  • Installing pluggy (0.13.1)
  • Installing py (1.9.0)
  • Installing wcwidth (0.2.5)
  • Installing pytest (4.6.11)

Installing the current project: test (0.0.1)
circleci@5ccc46929985:~/project$ poetry run pytest
============================================================ test session starts ============================================================
platform linux -- Python 3.9.0, pytest-4.6.11, py-1.9.0, pluggy-0.13.1
rootdir: /home/circleci/project
collected 1 item                                                                                                                            

tests/test_hello.py .                                                                                                                 [100%]

========================================================= 1 passed in 0.07 seconds ==========================================================

Pipenv

To turn your package into a pipenv project, you simply need to add a Pipfile (but you need the setup.py we created earlier)

circleci@23ea86f1d144:~/project$ tree
.
├── Pipfile
├── setup.py
├── src
│   ├── __init__.py
│   └── main.py
└── tests
    └── test_hello.py

2 directories, 5 files
circleci@23ea86f1d144:~/project$ cat Pipfile 
[packages]
pytest = "*"
tmp-example = {editable = true, path = "."}

circleci@23ea86f1d144:~/project$ pipenv install
Creating a virtualenv for this project…
Pipfile: /home/circleci/project/Pipfile
Using /home/circleci/.pyenv/versions/3.9.0/bin/python3.9 (3.9.0) to create virtualenv…
⠴ Creating virtual environment...created virtual environment CPython3.9.0.final.0-64 in 369ms
  creator CPython3Posix(dest=/home/circleci/.local/share/virtualenvs/project-zxI9dQ-Q, clear=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/circleci/.local/share/virtualenv)
    added seed packages: pip==20.2.3, setuptools==50.3.0, wheel==0.35.1
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

✔ Successfully created virtual environment! 
Virtualenv location: /home/circleci/.local/share/virtualenvs/project-zxI9dQ-Q
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Building requirements...
Resolving dependencies...
✔ Success! 
Updated Pipfile.lock (a4ad01)!
Installing dependencies from Pipfile.lock (a4ad01)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 10/10 — 00:00:03
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
circleci@23ea86f1d144:~/project$ pipenv run pytest
============================================================ test session starts ============================================================
platform linux -- Python 3.9.0, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/circleci/project
collected 1 item                                                                                                                            

tests/test_hello.py .                                                                                                                 [100%]

============================================================= 1 passed in 0.05s =============================================================

I have opened up a few issues on the python orb github page based on your feedback.

Topic		Replies	Views
Local tests passing while circleci brings up module import errors for local file Build Environment circle-yml	1	832	April 23, 2023
[config.yml] How to use circleCI for running tests from a folder containing test.py files Build Environment	2	5909	July 12, 2018
Python tests with coverage step failing in Circle CI, passing locally Build Environment python	4	1745	June 16, 2023
Beginner Question: Specify Input Files in config.yml Build Environment	13	3784	June 18, 2018
No such file or directory when i try to run the pytest in virtual env Build Environment circle-yml	0	1071	April 18, 2022