Clarify JUnit XML format
Problem to Solve
I would like to request a clarification in the documentation about the supported JUnit XML standard.
The docs states that GitLab support test reports in the JUnit report format. However, JUnit does not seem to have a commonly agreed standard. For example it's stated that "There are several self-proclaimed JUnit formats. Apache Ant doesn't have skipped elements at all and junit5's skipped elements can't have any attributes."
By experimentation with GitLabs Test pane, I found that the Suite column is populated by testcase classname and Name column is populated by testcase name. Meaning this:
<testsuites>
<testsuite name="mySuite" tests="1" failures="1" errors="0" skipped="0">
<testcase classname="myClass" name="myName"/>
<failure type="failure">myFailure</failure>
</testcase>
</testsuite>
</testsuites>
will output Suite: myClass (not mySuite) and Name: myName in Gitlab. The mySuite value does not seem to appear anywhere.
The GitLab docs refer to IBM's standard - JUnit report format XML files - but this does not mention any classname attribute. On the other hand, classname is specified in both junit5 and Apache Ant JUnit (not sure if this one's official).
Proposal
- Better document the supported JUNIT elements in docs
- Create a blog post / example project that shows what is and what might not be supported
- Iterate on how we parse/display unknown fields to better support working by default (here's a raw dump of your extra data?)