For us in the end the issue was that CircleCI was expecting relative paths to tests in the xml report, but our report had absolute paths (kudos to Split by timing parallelism does not seem to balance out).
The message that the CircleCi was giving for splitting:
So it looks that internally they store something like
tests/integration/NameOfTest.php
In our xml report it was the absolute path (/root/project/…):
Once we cleaned up the report using the command below to remove the absolute part before storing it as test result/artifact, the timings started to be autodetected:
command: sed -i 's/\/root\/project\///g' tests/_output/integration/result.xml
Here how our test splitting started to look like after:
Note: We are not using flag --timings-type=classname and feeding the list of tests from the file. So the command looks something like this:
command: circleci tests split --split-by=timings --show-counts tests/integration/_circleci_tests_list.txt | xargs -n 1 echo