GitLab doesn't show any warning regarding an invalid JUnit XML file
Summary
One of our customers (internal) reported that the unit tests are not showing in the pipelines page.
After troubleshooting their XML files, it seems that they uploaded an invalid JUnit XML file. However, xmlstarlet
and Python's xml
library shows that it's a valid XML file. I assume this is the reason why GitLab doesn't show a warning about this.
I created an example of this (based on this example XML from IBM)
$ cat junit.xml
<?xml version="1.0" encoding="UTF-8" ?>
<testsuites id="20140612_170519" name="New_configuration (14/06/12 17:05:19)" tests="225" failures="1262" time="0.001">
Unit
<testsuite id="codereview.cobol.analysisProvider" name="COBOL Code Review" tests="45" failures="17" time="0.001">
<testcase id="codereview.cobol.rules.ProgramIdRule" name="Use a program name that matches the source file name" time="0.001">
<failure message="PROGRAM.cbl:2 Use a program name that matches the source file name" type="WARNING">
WARNING: Use a program name that matches the source file name
Category: COBOL Code Review – Naming Conventions
File: /project/PROGRAM.cbl
Line: 2
</failure>
</testcase>
</testsuite>
</testsuites>
As you can see, the third line contain the string Unit
.
$ xmlstarlet val junit.xml
junit.xml - valid
And xmlstarlet
validates it as a valid XML.
Steps to reproduce
- Look at the branch
broken
in this repository: https://gitlab.com/jdasmarinas/working-junit - Run the job: https://gitlab.com/jdasmarinas/working-junit/-/pipelines/328710171/test_report
Example Project
https://gitlab.com/jdasmarinas/working-junit
- The branch
broken
contains the broken XML. - The branch
main
contains the working XML.
What is the current bug behavior?
GitLab doesn't show any warning and doesn't display any tests.
What is the expected correct behavior?
GitLab should display
Relevant logs and/or screenshots
This is the test report page of the broken XML:
As you can see, it doesn't contain any warning from GitLab. It also doesn't show any test report.
This is the test report page of a working XML:
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)