Historic Unit Test insights
Problem to solve
Delaney and Sasha do not have an easy way to get historical data about unit tests like if they are getting slower, flakier and when they were created/last changed/last passed.
Without this context it is hard to point at why the build is feeling/getting slower and what use cases may to be blame for that. Without that context it's not hard to imagine Delaney and Sasha starting to skip tests and not write tests for new code.
Intended users
Further details
Some use cases taken from discussion in another issue include but are not limited to:
- Which tests/jobs are getting slower
- Which tests/jobs are getting flakier (failing more often)
- When a test last passed
- When a test was last changed
- When a test was introduced
Proposal
Outcomes
Sasha When unit tests fail on an MR, I want to get historical context (see use cases) about failed tests, so that I don't feel like i'm stabbing in the dark to fix the build.
Delaney When I hear that builds are slowing down, I want to get historical context (see use cases) about failing unit tests, so that I can confidently coach my team on how to make it better.
At the project level highlight trends of tests that are running slower, failed tests that took the longest, failed tests that are getting flakier
At the MR level highlight failed tests that took the longest, failed tests that are getting flakier and provide context for failed tests (when introduced, last changed, last passed).
Permissions and Security
Documentation
Testing
What does success look like, and how can we measure that?
What is the type of buyer?
Links / references
This issue originated in discussion of #23257 (closed) which was deemed a duplicate of #24792 (closed) which was assigned to a milestone earlier.