Junit report displays/counts multiple tests as one test

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Close this issue

Summary

There is some sort of parsing error where multiple test cases with similar names and/or report files that have single testsuites tags with multiple testsuite tags are counted as being one test. I found this under CE 11.11.8 but created an example project.

CASE 1

Resolved with !41918 (diffs)

### Steps to reproduce After I found the issue, i created a minimized example. It seems to be the single varying character isn't enough to look like different tests to the parser, or there are silent parsing failures happening.

To reproduce: Submit this xml as a junit report, and view the output in a merge request.

<?xml version="1.0" encoding="utf-8"?>
<testsuites name="unittests.dll" tests="2" failures="0" time="0.36000510000000024">
<testsuite name="unittests.dll" tests="3" skipped="0" failures="0" errors="0" time="0.36000510000000024" timestamp="2019-09-13T 23:30:12Z" hostname="executor://nunit3testexecutor/">
<testcase classname="A" name="A(.)" file="UnitTests.dll" time="1E-07" />
<testcase classname="A" name="A(!)" file="UnitTests.dll" time="1E-07" />
<testcase classname="A" name="A( )" file="UnitTests.dll" time="1E-07" />
</testsuite>
</testsuites>

Example Project

Here is a merge showing the issue.

Siphonophora/junit-test-parsing-error!1

What is the current bug behavior?

The test widget displays 'Test summary contained no changed test results out of 1 total test'

What is the expected correct behavior?

We should see 'Test summary contained no changed test results out of 3 total test'

CASE 2

Steps to reproduce

In a Typescript monorepo, I activated it in Gitlab CI:

run-tests:
  stage: build
  script:
    - yarn ci_test
  artifacts:
    when: always
    paths:
      - packages/**/test-results.xml
    reports:
      junit:
        - packages/**/test-results.xml

The root level yarn ci_test iterates over the packages in the packages folder, runs yarn test in there and each package now contains a test-results.xml. I get 9 files out of this, most only containing a single test doing nothing for evaluation purposes of this visualization:

 Uploading artifacts...
packages/**/test-results.xml: found 9 matching files 
Uploading artifacts to coordinator... ok            id=509700156 responseStatus=201 Created token=<token>

I specify a top level testsuites name reflecting the package, and testsuite names corresponding to the test file. All this info seems to be dropped from the JUnit test reports view:

A toplevel testsuites tag has a collection of nested testsuite tags. Also these tags are dropped because I only see the separate testcase occurrences in a flat list:

As mentioned earlier, if there are duplicate testcases based on name and classname, these are not listed.

What is the current bug behavior?

I only have a suite called run-tests. This is the name of the stage that produced it. I am not interested in that at all! Once I click the run-tests suite, I also don't see my separate testsuite collections.

What is the expected correct behavior?

Given each file has a toplevel testsuites tag, that enumeration is what I would like to see at the top level. The toplevel set of test suites should be based on the name tag of the toplevel testsuites tag in each archived report file. Two files containing the same top level tag should be joined into a single top level test suite.

There are intermediate testsuite collections, please show these too!

In there, show the separate test cases. Do not anything magic with stripping out duplicates!

CASE 3

Summary

I just found that my JUnit results are not rendered correctly. Also seen that JUnit support is currently discussed so I link this to related issues #299086 (closed) and #324620 (closed). If you consider this a duplicate, feel free to close and reference from the other issues.

Steps to reproduce

I am using Codeception in my test pipeline and it can export JUnit XML report. Here is an example test run:

Here is how gitlab shows them:

I have 4 failures and 34 errors, gitlab only shows 3 failues and 3 errors.

The problem here is that the name property in the JUnit file is considered unique by the gitlab parser. However in my project there are many tests with the same function name in different classes. Also the same test is executed with different input data so multiple instance of class-name can exist with different value in feature. See the following extraction from JUnit XML as an example:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="acceptance" tests="11" assertions="154" failures="0" errors="0" time="26.059122">
    <testcase file="/app/tests/acceptance/comm/NewsletterCest.php" name="testUnsubscribeNewsletter" class="tests\acceptance\comm\NewsletterCest" feature="test unsubscribe newsletter | 1" assertions="4" time="1.031930"/>
     <!-- --- 8< --- -->
  </testsuite>
  <testsuite name="acceptancejs" tests="140" assertions="1375" failures="4" errors="34" time="3490.386799">
    <testcase file="/app/tests/acceptancejs/ord/method_of_payment/BillPaymentMethodCest.php" name="testShopOrderWithPaymentMethod" class="tests\acceptancejs\ord\method_of_payment\BillPaymentMethodCest" feature="test shop order with payment method | 0,0,0,&quot;without_popup&quot;" assertions="17" time="26.190466"/>
    <testcase file="/app/tests/acceptancejs/ord/method_of_payment/BillPaymentMethodCest.php" name="testShopOrderWithPaymentMethod" class="tests\acceptancejs\ord\method_of_payment\BillPaymentMethodCest" feature="test shop order with payment method | 0,0,0,&quot;with_popup&quot;" assertions="11" time="9.214829"/>
    <testcase file="/app/tests/acceptancejs/ord/method_of_payment/BillPaymentMethodCest.php" name="testShopOrderWithPaymentMethod" class="tests\acceptancejs\ord\method_of_payment\BillPaymentMethodCest" feature="test shop order with payment method | 1,0,0,&quot;with_popup&quot;" assertions="11" time="9.220477"/>
    <testcase file="/app/tests/acceptancejs/ord/method_of_payment/BillPaymentMethodCest.php" name="testShopOrderWithPaymentMethod" class="tests\acceptancejs\ord\method_of_payment\BillPaymentMethodCest" feature="test shop order with payment method | 0,1,0,&quot;with_popup&quot;" assertions="11" time="9.532151"/>
    <testcase file="/app/tests/acceptancejs/ord/method_of_payment/BillPaymentMethodCest.php" name="testShopOrderWithPaymentMethod" class="tests\acceptancejs\ord\method_of_payment\BillPaymentMethodCest" feature="test shop order with payment method | 0,0,1,&quot;with_popup&quot;" assertions="4" time="28.927803">
      <error type="Facebook\WebDriver\Exception\TimeoutException">BillPaymentMethodCest: Test shop order with payment method | 0,0,1,"with_popup"
Facebook\WebDriver\Exception\TimeoutException: Waited for 20 secs but text 'Vielen Dank für Ihre Bestellung.' still not found
[...]
</error>
    </testcase>
     <!-- --- 8< --- -->
  </testsuite>
</testsuites>

Results of GitLab environment info

I am using Gitlab 13.1.4-ce.0.

Edited Aug 28, 2025 by 🤖 GitLab Bot 🤖