Circleci tests split with a file

Feedback & Bug Reports
1

We are trying to use the splitting feature of circleci. And while some simple use case works just fine, one that is slightly more involved does not, I’ll explain below.

When we do :

CLASSNAMES=$(circleci tests glob "**/src/test/**/**.java" | grep -v 'spring-cloud-kubernetes-integration-tests' \
                             | xargs grep -l '@Test' \
                             | sed 's/.*src.test.java.//g' | sed 's@/@.@g' \
                             | sed 's/.\{5\}$//' \
                             | circleci tests split --split-by=timings)

this works just fine. But this is a trivial case. In our use case the actual tests are computed in various ways with various data, so I can’t pipe the result directly to “circleci tests split --split-by=timings”. That is: we find the tests in multiple operations, and ultimately get an array… I’ve narrowed the problem down to :

CLASSNAMES=$(circleci tests glob "**/src/test/**/**.java" | grep -v 'spring-cloud-kubernetes-integration-tests' \
                             | xargs grep -l '@Test' \
                             | sed 's/.*src.test.java.//g' | sed 's@/@.@g' \
                             | sed 's/.\{5\}$//')

TEST=$(echo $CLASSNAMES | circleci tests split --split-by=timings)

This one fails to discover the tests with : “Error autodetecting timing type, falling back to weighting by name. Autodetect no matching filename or classname.”.

I also found this page : https://support.circleci.com/hc/en-us/articles/360048786831-Use-test-splitting-with-Python-Django-tests

where is shows that you can persist the tests to a file and then provide that file to “circleci tests split --split-by=timings”, so we tried that also:

CLASSNAMES=$(circleci tests glob "**/src/test/**/**.java" | grep -v 'spring-cloud-kubernetes-integration-tests' \
                                           | xargs grep -l '@Test' \
                                           | sed 's/.*src.test.java.//g' | sed 's@/@.@g' \
                                           | sed 's/.\{5\}$//')

echo $CLASSNAMES > tests.txt
TEST=$(circleci tests split --split-by=timings tests.txt)

The result is the same error : “Error autodetecting timing type, falling back to weighting by name. Autodetect no matching filename or classname.”.

I don’t get it. This should be trivial to achieve, no? What am I doing wrong here?

Thank you.

2

Hi @wind57,

All three examples you shared should indeed yield the same results.

Functionally the circleci tests split command takes in a list of strings (like filename or classname), splits them accordingly (by timing or filesize , etc …, whatever you choose), then passes back out the string values isolated to each container.

If you don’t specify a timing type, and if the test splitter is unable to auto-detect it, the it falls back to weighting by filename.

Now if you’d like to pass a list of classnames to the test splitter, you’ll need to specify the --timings-type=classname flag.

That said, you’ll need to ensure that:

  1. you pass a glob of classnames (rather than filenames).
  2. classnames in said glob exactly match the corresponding strings displayed in the classname field of the tests results.

One way to troubleshoot this is to echo the list of classnames produced.

Let me know if this helps.