JUnit output only shows first <failure> tag in a <testcase>

Feedback & Bug Reports
1

Only the first <failure> tag in a <testcase> is getting displayed in the CircleCI UI, even tho more than one <failure> is allowed by the JUnit spec.

Out test runner outputted a file like this:

<?xml version='1.0' encoding='UTF-8'?>
<testsuite name="metabase.bad-test" time="0.004" timestamp="2021-08-03T02:58:16.944423349Z" tests="3" errors="0" failures="2">
<testcase classname="metabase.bad-test" name="failing-test" time="0.004" assertions="2">
<failure>
<![CDATA[
bad_test.clj:6
context 1
expected: 1
  actual: 2
    diff: - 1
          + 2
]]>
</failure>
<failure>
<![CDATA[
bad_test.clj:8
context 1
context 2
expected: {:a 1, :b 2, :c 3}
  actual: {:a 1, :b 2, :c 4}
    diff: - {:c 3}
          + {:c 4}
]]>
</failure>
</testcase>
<testcase classname="metabase.bad-test" name="ok-test" time="0.0" assertions="1">
</testcase>
</testsuite>

However, the Circle CI only shows the first failure for failing-test:

image

This seems like a bug in CircleCI’s UI, or am I missing something?

Thanks!

2

Hi Cam,

Thanks for your comment! Just to re-summarise here what we discussed in your support ticket:

This is expected behaviour to report the first outcome in a testcase. This is in line with other JUnit tooling.

If you think different behaviour might be useful here please vote / comment on the following idea I created:

Thanks!
Al

1 Like
3

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.