No timing data found when splitting by testname, and it does outputs file names instead of expected test names

[henrahmagix]

How can I get circleci tests split --split-by=timings --timings-type=testname to output test names instead of filenames? And correctly find timings data?

I’d like to end up with a computed command like:

rspec --example "..." --example "..." --example="..."

where the --example arguments are compiled from the output of circleci tests split.

---

Currently I’m splitting RSpec tests across 8 parallel nodes by using --timings-type=filename but we have a few files we cannot split up that take a long time to complete, so CircleCI says we have lots of “idle” time.

To reduce the idle time, I’m trying to use --timings-type=testname but it tells me “No timing found” for a lot of files, and then it outputs filenames when I need test names.

The timing data is created by the rspec_junit_formatter gem and uploaded with store_test_results. It’s all correct because we currently have successful splitting by filename, but with lots of idle time remaining, which is what I’m trying to fix here.

Here’s an example of the timings data available on our CircleCI machines that circleci tests split uses:

circleci@ip-<redacted>:~/project$ jq '.tests|.[0:2]' < $CIRCLE_INTERNAL_TASK_DATA/circle-test-results/results.json
[
  {
    "classname": "spec.foo.bar_spec",
    "file": "spec/foo/bar_spec.rb",
    "name": "Foo Bar does foobar",
    "result": "success",
    "run_time": 15.584309
  },
  {
    "classname": "spec.foo.bar_spec",
    "file": "spec/foo/bar_spec.rb",
    "name": "Foo Bar does baz",
    "result": "success",
    "run_time": 3.032515
  }
]

When I ask CircleCI to split the tests by testname, I get:

circleci@ip-<redacted>:~/project$ circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings --timings-type=testname --show-counts --verbose
Read 76 lines of testname(s)
No timing found for "spec/foo/bar_spec.rb"
No timing found for "spec/foo/bar2_spec.rb"
No timing found for "spec/foo/other_spec.rb"
<repeated with remaining 73 filenames>
Bucket 0: assigning 19 testname(s), total weight 11351170
spec/foo/bar_spec.rb
spec/foo/other_spec.rb
<repeated with remaining 17 filenames>
INFO[2025-05-13T15:38:45Z] Clean exit without errors

I’d like to end up with a command like rspec --example "Foo Bar does foobar" --example "Foo Bar does baz" --example="..." where the --example arguments are compiled from the output of circleci tests split, e.g.

circleci tests glob spec/**/*_spec.rb \
  | circleci tests split --split-by=timings --timings-type=testname \
  | sed 's/^.*$/--example "\0"/' \
  > rspec_example_arguments.txt

rspec $(cat rspec_example_arguments.txt | tr '\n' ' ')

Is there a way I can do this please?

Do I need to change what rspec_junit_formatter uploads to CircleCI’s store_test_results? Or is there another argument to circleci tests split to output the "name": keys of the timings data?

Thanks in advance,
Henry =)

[caiolouro]

I’ve had the same problem using Python and, after reaching out to CircleCI support, I realized this wouldn’t work.
I think the problem is tests glob command outputs a list of files, but the tests split --timings-type=testname will be expecting test names, so they won’t match and will say timings are not found. The solution seems to have a custom way of listing test names and pipping that to tests split --timings-type=testname.
For my case, I ended up having to focus on the library that was distributing the test cases amongst CPUs, as CircleCI splitting can really create some hot spots on some nodes.