Sort Test Reports by Status, Duration Descending
Problem to solve
As a dev team lead, I want to know which tests take the longest to run, so I can task the team to find efficiencies resulting in safer/faster fixes and reduced runner minutes (saving $$
).
Intended users
User experience goal
The user should be able to get data about which tests run the slowest in the GitLab UI.
Proposal
Within each Job sort the Junit report on the pipeline by status and then time descending. This will surface the slowest tests to the top, but not before failed ones. The first sorted test should be the slowest failed test if there are failures, if there are no failures, the first sorted test should be the slowest passed test.
We should validate that statuses are sorted in this order:
Errored
Failed
Passed
Skipped
Track how often users click on the Reports Tab so we can measure the impact of this change.
This can be done independently or alongside Sorting by Status (Failed, Skipped, Passed) which is covered by #217514 (closed).
backend
We need to introduce a sorting mechanism for our TestSuite object. This could be done by 2 options:
- We sort the data by default once we finish to process a
JUnit
report - We introduce the concept of data ordering for our
TestSuite
object, (similar to what we did for our attachment feature)
Track clicks on the button in snowplow so we can measure what % of customers who see the page download the file.
Previous Proposal
Further details
This was discussed in a team Think Big / Think Small session as the minimal change to understand if customers value knowing which tests are slowest in their test jobs.
Permissions and Security
Documentation
TBD if we need a docs update.
Availability & Testing
What does success look like, and how can we measure that?
Success Measures
- We will know this feature is successful if the views of the JUnit Test Report has increased by 10% in 30 days after release of the feature on Gitlab.com
Acceptance Criteria
- Default sort is by status, time and consistent between browsers
- There is tracking of views of the JUnit report accessible in Sisense
What is the type of buyer?
This feature is valuable to the team lead but this is a test so will be available in GitLab Core .