Skip to content

Sort Test Reports by Status, Duration Descending

Problem to solve

As a dev team lead, I want to know which tests take the longest to run, so I can task the team to find efficiencies resulting in safer/faster fixes and reduced runner minutes (saving $$).

Intended users

User experience goal

The user should be able to get data about which tests run the slowest in the GitLab UI.

Proposal

Within each Job sort the Junit report on the pipeline by status and then time descending. This will surface the slowest tests to the top, but not before failed ones. The first sorted test should be the slowest failed test if there are failures, if there are no failures, the first sorted test should be the slowest passed test.

We should validate that statuses are sorted in this order:

  1. Errored
  2. Failed
  3. Passed
  4. Skipped

Track how often users click on the Reports Tab so we can measure the impact of this change.

This can be done independently or alongside Sorting by Status (Failed, Skipped, Passed) which is covered by #217514 (closed).

Tests_in_a_job

backend

We need to introduce a sorting mechanism for our TestSuite object. This could be done by 2 options:

  1. We sort the data by default once we finish to process a JUnit report
  2. We introduce the concept of data ordering for our TestSuite object, (similar to what we did for our attachment feature)

Previous Proposal
Add a "See slowest tests" button to the JUnit report. This will download a .csv of the tests sorted by duration (slowest/longest at the top).

Track clicks on the button in snowplow so we can measure what % of customers who see the page download the file.

Further details

This was discussed in a team Think Big / Think Small session as the minimal change to understand if customers value knowing which tests are slowest in their test jobs.

Permissions and Security

Documentation

TBD if we need a docs update.

Availability & Testing

What does success look like, and how can we measure that?

Success Measures

  • We will know this feature is successful if the views of the JUnit Test Report has increased by 10% in 30 days after release of the feature on Gitlab.com

Acceptance Criteria

  • Default sort is by status, time and consistent between browsers
  • There is tracking of views of the JUnit report accessible in Sisense

What is the type of buyer?

This feature is valuable to the team lead but this is a test so will be available in GitLab Core .

Is this a cross-stage feature?

Links / references

Edited by James Heimbuck