Add Support for Duplicate test names & classes in JUnit report feature

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Problem to solve

As a developer, when I have a JUnit report file that has duplicate test name & class combinations, I want to be able to see all of the entries on the JUnit report viewing screen, so that the test counts match up, and I can continue to use programatically generated test names and classes in my JUnit report file.

Right now we are using a combination of classname and name to create keys for our hash that we load the JUnit report into: https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/reports/test_case.rb#L26. Because of this, if there are multiple entries in one JUnit report file with the same name and classname, the report will count them individually, but once it is parsed and loaded into GitLab, they will be squashed into one entry. This leads to test counts that seem incorrect on the test page and prevents 1:1 mapping of test report file to GitLab UI from being 100% accurate.

Intended users

User experience goal

I want to be able to see all of the entries on the JUnit report viewing screen, so that the test counts match up.

Proposal

Append to the test name (n) for every instance of a duplicate found for display on the Test Summary and Unit Test report.

Options discussed

There are many different approaches we could take to solve this problem, the following were discussed before 3. was picked.

  1. A boring one might be to take the line number in the JUnit report and use that as part of the composite key for the hash.
  2. Display an * by the test that was duplicated with a note on the page that * indicates multiple test names were found but only one displayed.
  3. Append to the test name (n) for every instance of a duplicate found.

Further details

This was initially reported in another issue about jest tests counts not matching up.

Permissions and Security

Documentation

  • Remove any notes about why duplicate tests are not displayed correctly from the existing docs.

Availability & Testing

What does success look like, and how can we measure that?

Acceptance Criteria

  • When there are duplicate instances of a test with the same suite/test name combo in separate junit files the total count of tests should match up between screens of the JUnit report.

What is the type of buyer?

This feature most applies to the Software Developer and is an enhancement to a ~"GitLab Core" feature.

Is this a cross-stage feature?

No.

Links / references

Reproduction Steps

From #231478 (closed)

Summary

When I navigate between the list of jobs and the detail of a jobs on the Test Report I see two different counts of tests which makes me question the accuracy of any of the data.

This bug is affecting only the jest job.

Steps to reproduce

  1. Navigate to a project with 10s of thousands of tests like gitlab.
  2. Navigate to a pipeline that ran all the tests
  3. Click on the test report tab
  4. Note the number of tests
  5. Click on a job

Example Project

https://gitlab.com/gitlab-org/gitlab/-/pipelines/169603204/test_report

What is the expected correct behavior?

Numbers match.

What is the current bug behavior?

The numbers are different, see the images below.

Relevant logs and/or screenshots

Screenshot_2020-07-23_15.32.01

Screenshot_2020-07-23_15.32.09

Possible fixes

(If you can, link to the line of code that might be responsible for the problem)

Edited by 🤖 GitLab Bot 🤖