Cucumber JS error for store_test_results

[richrace]

Hi,

I’m using Cucumber JS and trying to correct the autodetection for timings.

CircleCI knows the tests are present and has registered them correctly. In the store_test_results step:

- store_test_results:
    path: packages/e2e-test/test-results.xml

In the step output in CircleCI step

Archiving the following test results
  * /home/circleci/project/packages/e2e-test/test-results.xml

Total size uploaded: 3.0 KiB

It shows 200 tests run and the insights of 10 parallel runs with timings. So far, so good.

But when I run the test the next time, I get this error:

Error autodetecting timing type, falling back to weighting by name. Autodetect no matching filename or classname.  If file names are used, double check paths for absolute vs relative.
Example input file: "features/001-abc/test.feature"
Example file from timings: ""

The Cucumber command is:

CHUNK=circleci tests glob "features/**/*.feature" | circleci tests split --split-by=timings

./../node_modules/.bin/cucumber-js --require steps/index.ts --require-module ts-node/register --retry 5 --no-strict --exit --publish-quiet --format @cucumber/pretty-formatter  --format junit:./test-results.xml ${CHUNK}

An example of the contents of `test-results.xml` file:

<?xml version="1.0"?>
<testsuite failures="0" skipped="0" name="cucumber-js" time="10.494652437999997" tests="2">
  <testcase classname="{713} HO100310 - normal case" name="Resolving one pair of HO100310 exceptions" time="6.6794609959999995">
    <system-out><![CDATA[Given the data for this test is in the PNC................................passed
And "input-message" is received...........................................passed
Given I am logged in as "generalhandler"..................................passed
And I view the list of exceptions.........................................passed
Then I see exception "HO100310 (2)" in the exception list table...........passed
When I open the record for "Bass Barry"...................................passed
And I click the "Offences" tab............................................passed
And I view offence "1"....................................................passed
And I match the offence to PNC offence "1"................................passed
And I return to the offence list..........................................passed
And I view offence "2"....................................................passed
And I match the offence to PNC offence "2"................................passed
And I submit the record...................................................passed
Then the PNC updates the record...........................................passed]]></system-out>
  </testcase>
  <testcase classname="{714} HO100310 - normal case with one offence added in court" name="Resolving one pair of HO100310 exceptions with an offence added in court" time="3.8151914419999984">
    <system-out><![CDATA[Given the data for this test is in the PNC................................passed
And "input-message" is received...........................................passed
Given I am logged in as "generalhandler"..................................passed
And I view the list of exceptions.........................................passed
Then I see exception "HO100310 (3)" in the exception list table...........passed
When I open the record for "Bass Barry"...................................passed
And I click the "Offences" tab............................................passed
And I view offence "1"....................................................passed
And I match the offence to PNC offence "1"................................passed
And I return to the offence list..........................................passed
And I view offence "2"....................................................passed
And I match the offence to PNC offence "2"................................passed
And I return to the offence list..........................................passed
And I view offence "3"....................................................passed
And I match the offence as Added In Court.................................passed
And I submit the record...................................................passed
Then the PNC updates the record...........................................passed]]></system-out>
  </testcase>
</testsuite>`

Any ideas why and how to get it to work?

[richrace]

I’ve tried this: https://support.circleci.com/hc/en-us/articles/360021624194-Test-Summary-Troubleshooting

And I haven’t had any success.

Using this as the path:

- store_test_results:
    path: packages/e2e-test/test-results/results/report.xml

[wyardley]

I haven’t used Cucumber, and it’s been a while since I’ve had to get tests split by timing working, but the link you posted reminded me of one thing:

To split tests by timing, each element in the JUnit XML report should include the following attributes:

• file : file path of the test file which the test case can be found in

From what I can see, the <testcase> elements in your example results have name, time, and classname, but not file.

[richrace]

So I created a script that adds the file after the CircleCI job is finish

The updated report

<?xml version="1.0"?>

<testcase classname=“{713} HO100310 - normal case”

name=“Resolving one pair of HO100310 exceptions” time=“9.299621116”

file=“features/713-HO100310-normal-case/test.feature”>

<![CDATA[Given the data for this test is in the PNC…passed

And “input-message” is received…passed

Given I am logged in as “generalhandler”…passed

And I view the list of exceptions…passed

Then I see exception “HO100310 (2)” in the exception list table…passed

When I open the record for “Bass Barry”…passed

And I click the “Offences” tab…passed

And I view offence “1”…passed

And I match the offence to PNC offence “1”…passed

And I return to the offence list…passed

And I view offence “2”…passed

And I match the offence to PNC offence “2”…passed

And I submit the record…passed

Then the PNC updates the record…passed]]>

<testcase classname=“{714} HO100310 - normal case with one offence added in court”

name=“Resolving one pair of HO100310 exceptions with an offence added in court”

time=“6.340200655” file=“features/714-HO100310-normal-case-added-in-court/test.feature”>

<![CDATA[Given the data for this test is in the PNC…passed

And “input-message” is received…passed

Given I am logged in as “generalhandler”…passed

And I view the list of exceptions…passed

Then I see exception “HO100310 (3)” in the exception list table…passed

When I open the record for “Bass Barry”…passed

And I click the “Offences” tab…passed

And I view offence “1”…passed

And I match the offence to PNC offence “1”…passed

And I return to the offence list…passed

And I view offence “2”…passed

And I match the offence to PNC offence “2”…passed

And I return to the offence list…passed

And I view offence “3”…passed

And I match the offence as Added In Court…passed

And I submit the record…passed

Then the PNC updates the record…passed]]>

And now I get this error on CircleCI:

Autodetected filename timings.
No timing found for "features/003-stop-list/test.feature"
No timing found for "features/011b-nppa-ignored-results/test.feature"
No timing found for "features/013-extra-offence-on-pnc/test.feature"
...

I’m more confused…

[richrace]

Oh, I fixed it

I have split my tests by timing, but why is the split not even?