Test timings can't be found

I’m using RSpec in parallel and splitting by test timings. Recently the timings have stooped working (I’m not sure how far back, but at least the last week. I get output like the following:
TESTFILES=$(circleci tests glob “./spec/**/*_spec.rb” | circleci tests split --split-by=timings --timings-type=classname)
echo $TESTFILES
bundle exec rspec
–format RspecJunitFormatter
–out ./coverage/xml/rspec/rspec.xml
–format progress
– $TESTFILES

No timing found for "spec/api/api_constraints_spec.rb"
No timing found for "spec/application_spec.rb"
...

I have checked the stored test result XML (stored as an artifact for debugging) and they are indeed there, but CircleCI seems to not be finding them. The only thing that I can see that might be causing this is that the paths in the XML file start with ./, which is not present in the warning messages (nor the output of circleci tests glob).

Has something changed that I’ve missed?

I’m seeing the same thing on an Elixir app. Timings/parallelization was working fine until a few weeks ago.

What version of CI are you using? We’re using v2 (not v2.0) and not workflows, and I think that might have something to do with it.

We’re using v2.1 with workflows.

1 Like

We are experiencing the same issue.

Do anyone has a solution for this No timing found issue?
We are experiencing the same issue since last month and for the info, we use version 2.1 with workflow, after investigation and using this validator got a lot of error. We did not change our formatting and it was working fine until last month approx. Dec 6, 2019 is the last time we check after that we got that issue.

We are also trying to figure out why we are getting

No timing found for "tests/integration/reckonhosted/DemoTestSet1Test.php"
No timing found for "tests/integration/reckonhosted/DemoTestSet1_1Test.php"

The command that we use for splitting the tests is the following (debugging one):

#!/bin/bash -eo pipefail
circleci tests split --split-by=timings --timings-type=classname --show-counts tests/integration/_circleci_tests_list.txt | xargs -n 1 echo

I am trying to experiment by removing the ‘–timings-type=classname’, since we are passing the list of path/filename. Guessing that maybe that’s the issue, but not sure.

Nope, this did not help with No timing issue.

We are using
config version: 2.1
workflows:
version: 2

Alright, we actually found the issue from the JUnit report that Cucumber generated by default. As we can see from the doc here the XML as following:


<?xml version="1.0" encoding="UTF-8"?>
<testsuite failures="0" errors="0" skipped="2" tests="2" time="0.05" name="Pending step">
  <testcase classname="Pending step" name="Pending" time="0.05">
    <skipped/>
      <system-out/>
    <system-err/>
  </testcase>
  <testcase classname="Pending step" name="Undefined" time="0.05">
    <skipped/>
    <system-out/>
    <system-err/>
  </testcase>
    <system-out>
      <![CDATA[]]>
    </system-out>
    <system-err>
      <![CDATA[]]>
    </system-err>
</testsuite> 

Using the validator that I mention in the previous comment. We found that some of the tags causing this issue on the circle-ci such as <![CDATA[]]> , <system-out/> , etc.

So we created custom JUnit that makes CircleCI happy about and it works. For the reference of how the XML format should be, please see this StackOverflow threads something like this:

<testsuites>
  <testsuite>
    <properties>
      <property> ... </property>
    </properties>

    <testcase> ... </testcase>
  </testsuite>
</testsuites>

Hope this can be helpful for those have similar issue like us.

I can confirm that RSpec and Karma is producing output similar to your Cucumber output. This sounds like CircleCI isn’t supporting the JUnit XML format properly (rather than three different test suites not supporting the format correctly).