I’m using RSpec in parallel and splitting by test timings. Recently the timings have stooped working (I’m not sure how far back, but at least the last week. I get output like the following:
TESTFILES=$(circleci tests glob “./spec/**/*_spec.rb” | circleci tests split --split-by=timings --timings-type=classname)
echo $TESTFILES
bundle exec rspec
–format RspecJunitFormatter
–out ./coverage/xml/rspec/rspec.xml
–format progress
– $TESTFILES
No timing found for "spec/api/api_constraints_spec.rb"
No timing found for "spec/application_spec.rb"
...
I have checked the stored test result XML (stored as an artifact for debugging) and they are indeed there, but CircleCI seems to not be finding them. The only thing that I can see that might be causing this is that the paths in the XML file start with ./, which is not present in the warning messages (nor the output of circleci tests glob).
Do anyone has a solution for this No timing found issue?
We are experiencing the same issue since last month and for the info, we use version 2.1 with workflow, after investigation and using this validator got a lot of error. We did not change our formatting and it was working fine until last month approx. Dec 6, 2019 is the last time we check after that we got that issue.
I am trying to experiment by removing the ‘–timings-type=classname’, since we are passing the list of path/filename. Guessing that maybe that’s the issue, but not sure.
Nope, this did not help with No timing issue.
We are using
config version: 2.1
workflows:
version: 2
Using the validator that I mention in the previous comment. We found that some of the tags causing this issue on the circle-ci such as <![CDATA[]]> , <system-out/> , etc.
So we created custom JUnit that makes CircleCI happy about and it works. For the reference of how the XML format should be, please see this StackOverflow threads something like this:
I can confirm that RSpec and Karma is producing output similar to your Cucumber output. This sounds like CircleCI isn’t supporting the JUnit XML format properly (rather than three different test suites not supporting the format correctly).
Once we cleaned up the report using the command below to remove the absolute part before storing it as test result/artifact, the timings started to be autodetected: command: sed -i 's/\/root\/project\///g' tests/_output/integration/result.xml
Here how our test splitting started to look like after:
Note: We are not using flag --timings-type=classname and feeding the list of tests from the file. So the command looks something like this: command: circleci tests split --split-by=timings --show-counts tests/integration/_circleci_tests_list.txt | xargs -n 1 echo
The paths in my spec files aren’t absolute, they’re relative to the project root. The only think I can see that may be preventing the timings from being used is the fact that the paths start with ./.
I’ve just tried stripping off the ./ using sed -i 's/\.\///g' ./coverage/xml/rspec/rspec.xml and still get the “No timing found” messages all over the place. I’m running RSpec using:
I ran into a couple of these and it turns out one file was completely commented out and another was a helper file that happened to end in _spec.rb. Might be worth looking into the files that are being reported for similar stuff.
If that doesn’t help, you can check the stored test_results by SSH’ing into one of the containers and checking the file there.
Did anyone figure out how to get this working for rspec?
At a random point in the middle of December (with no circleci changes on our end as far as we can tell), every run of our circleci pipeline has the No timing found for… message for every file in the test suite.
To add to @aharbick’s comment about --timings-type=classname.
--timings-type=classname expects a list of class names not file names to be passed in. So if you are using circleci tests glob you will likely never want to use --timings-type=classname.
To make the point more clearly. circleci tests split simply takes in a list of strings and then tries to match them to the values that were originally in the test results file. So to give an example testcase line from a jUnit XML file
circleci tests split will simply look at the value in classname if you pass in --timings-type=classname. This is why this will never work if you pass in a list of file names. To use --timings-type=classname you want to have a method of creating a list of class names and not actually use the circleci tests glob command. The vast majority of use-cases will only make use of --timings-type=filename which is also the default if you leave it off.