History of last 10 executions in MR View
Problem(s) to solve
- Today when a test fails it is very hard to determine quickly if the test has a history of failure indicating it could be flakey or if it is the first time it has failed.
This problem was originally defined at &1875 (closed):
GitLab CI/CD shows job log information, so you can dig into why tests fail, but it requires humans reading a bunch of text. Most languages/frameworks are able to output JUnit XML (or xUnit, NUnit, PHPUnit, etc.)
The user needs a way to more efficiently digest JUnit-style XML information so that they can find out why tests fail quicker.
Intended users
Further details
Use Cases/outcomes
For Sasha and Devon
- When a test fails, I can easily see within that
CI viewTest Summary the history of the test, so I have the context to make a judgement call on if the test is flakey and can be skipped or needs fixed. - When a test run completes, I can easily within that
CI viewTest Summary the historical run times of passed tests, so that I can optimize the parallelization of my CI file to run tests faster. ** *MVC version but we can extend this later to auto optimize or at least suggest *
Proposal
- Show history of the test for the current branch and executions that happened in the target branch, maybe something like this that would show the history of a test from FB pipeline2, FB pipeline 1, Pipeline 2, and Pipeline 1 going backwards in time.
graph TB
Pipeline2 --> featureBranchPL1
subgraph "Feature Branch"
featureBranchPL1(FB Pipeline 1)
featureBranchPL1 --> featureBranchPL2[FB Pipeline 2]
end
subgraph "Default Branch"
Pipeline1[Pipeline 1] --> Pipeline2[Pipeline 2]
Pipeline2 --> Pipeline3[Pipeline 3]
end
This issue will focus on the scope of:
A historical view which showcases unit test information across successful pipelines for the default branch .
Preferably this view will be flexible and thus usable beyond just Junit. It is, however, acceptable for an MVC to start off with a single format if need be.
Permissions and Security
Documentation
Testing
What does success look like, and how can we measure that?
Acceptance Criteria
- A user can click into a test case to see the last n runs of the test (or at least what we have
artifactsdata for) within the CI view to see pass/fail/skip and click for further details of that test run like how long the test took and the error trace. - A user can click into a test case to see the average, min and max times for successful test runs of that test.
Links / references
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.