Python parallelization in 2.0 with circleci?

python

#1

I read this (https://circleci.com/docs/2.0/parallelism-faster-jobs/) but it’s not clear how the circleci tool helps me in Python, where you don’t really use filename globbing to identify and split up tasks. Is there an equivalent for Python/Django that finds or accepts test names and splits them up between containers?


Parallel "stock" Python unit tests with 2.0?
#2

Globbing should still be sufficient for a Python project. All your tests should be in a common directory or have a common naming schema.

If it still doesn’t work for you, I don’t know of any toolsets offhand, but you can just hardcode your test files and split them up by container yourself.


#3

The Python test setup uses class names as they exist inside the files, not filenames themselves. A commandline might be manage.py test mysite.myapp.tests.mytestfile.MyTestClass mysite.myapp.tests.mytestfile.MyTestClass2. How do I apply shell globbing/circleci to get a list like that?


#4

Pytest with xargs worked for me: circleci tests glob **/tests/*.py | circleci tests split --split-by=timings | xargs python -m pytest --reuse-db --junitxml=test_results/results.xml


#5

Thank you, but it seems that approach works only with pytest, and not with the built-in Python/Django test framework. Is there still no available solution that can split at the test level rather than at the file level?


#6