Problem with python test splitting (nose)

Build Environment Running Tests
1

I am trying to get test splitting to work. I am using Django with nosetest as the test runner. Using the default python setup suggested, I get the dreaded:

Error autodetecting timing type, falling back to weighting by name.

I have looked at the output format of junit.xml generated by nose and it looks like this:


<testsuite name="nosetests" tests="222" errors="0" failures="0" skip="0">
<testcase classname="module.tests.integration.api.v1.my_tests.TestGetSerialiserContext" name="my_test_one" time="2.000"/>
<testcase classname="module.tests.integration.api.v1.my_tests.TestGetSerialiserContext" name="my_test_two" time="0.517"/>
<testcase classname="module.tests.integration.api.v1.my_tests.TestGetSerialiserContext" name="my_test_three" time="6.142"/>
...

This seemed promising, and I altered my input so that the parameters passed to circle tests split were in that exact format (using a python script to find the classnames of all my tests).

I changed my invocation to be like:

TESTFILES=$(python manage.py find_test_classes | circleci tests split --split-by=timings --timings-type=classname)

However now when I try running the pipeline I get output like the following:

No timing found for "module.tests.integration.api.v1.my_tests.TestGetSerialiserContext"
No timing found for "module.tests.integration.api.v1.my_tests.TestPostSerialiserContext"
No timing found for "module.tests.integration.api.v1.my_tests.TestPatchSerialiserContext"

etc. for every class.

I have extracted the test timings file (results.json) from the pod and inspected it to see if I can find anything out there. But it looks like this:


{
  "tests": [
    {
      "classname": "module.tests.integration.api.v1.my_tests.TestGetSerialiserContext",
      "file": null,
      "name": "my_test_one",
      "result": "success",
      "run_time": 7.015,
      "message": null,
      "source": "unknown",
      "source_type": "unknown"
    },
    ...
    ]
}

…which looks correct. What is going on, and how can I get my tests to split by timing properly?

2

Hi @fredley,

Do you get the same outcome if you remove --timings-type=classname?

3

Hello again, @fredley!

Please accept my apologies; I wasn’t thinking straight when I sent my previous message.

I just realised that the generated junit.xml doesn’t include the filenames. The CLI expects both filenames and classnames to be present in the timing data produced by the testing suite (see Running Tests in Parallel - CircleCI).

I looked at the Nose documentation but I couldn’t find a way to include the filename field in the XML output :confused: