JUnit Parser ignore errors in tests

Problem to solve

Gitlab does not show errors in tests

Further details

testsuite example

<?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-failsafe-plugin/xsd/failsafe-test-report.xsd" name="personnel.ForecastListIT" time="777.326" tests="3" errors="1" skipped="0" failures="1">
  <properties>
		<!-- huge prop list -->
  </properties>
  <testcase name="shouldImportLoad" classname="personnel.ForecastListIT" time="98.306">
    <error message="error message" type="org.openqa.selenium.WebDriverException">some more info
</error>
    <system-out><![CDATA[some logs]]></system-out>
    <system-err><![CDATA[some err]]></system-err>
  </testcase>
  <testcase name="shouldSetMiniumOfOperators" classname="personnel.ForecastListIT" time="280.854">
    <failure message="expected:&lt;16&gt; but was:&lt;3&gt;" type="java.lang.AssertionError"><![CDATA[java.lang.AssertionError: expected:<16> but was:<3>
	at personnel.ForecastListIT.shouldSetMiniumOfOperators(ForecastListIT.java:313)
]]></failure>
    <system-out><![CDATA[some logs]]></system-out>
    <system-err><![CDATA[some err]]></system-err>
  </testcase>
  <testcase name="shouldDisplayLoad" classname="personnel.ForecastListIT" time="40.013"/>
</testsuite>

in MR Gitlab show 3 tests 1 failure

but actually there are only 1 successfull test

Proposal

show not only failures, but errors too

What does success look like, and how can we measure that?

i suggest 2 possible looks
Clear: Show 1 failure, 1 error of 3 tests
Simple: Show 2 failure of 3 tests

Simple solution like this :) /gitlab/ci/parsers/junit.rb

        def create_test_case(data)
          if data['failure']
            status = ::Gitlab::Ci::Reports::TestCase::STATUS_FAILED
            system_output = data['failure']
          elsif data['error']
            status = ::Gitlab::Ci::Reports::TestCase::STATUS_FAILED
            system_output = data['error']
          else
            status = ::Gitlab::Ci::Reports::TestCase::STATUS_SUCCESS
            system_output = nil
          end

Links / references

Assignee Loading
Time tracking Loading