MR fail on web performance degredation
Problem to solve
For some teams a set level of web performance is critical for their application. Today when that performance degrades there is not an easy way for them to fail an MR or be alerted to changes that will have negative customer outcomes.
Intended users
Use Cases
- I don't want web performance to decrease more than a set value from the current value on master eg; It cannot decrease more than 1 point from /master.
- I don't want web performance to decrease more than a percentage value from the current value on master eg; it cannot decrease more than 2% from /master.
- I don't want web performance to decrease below a set threshold eg; It cannot fall below 50
Proposal
- Add a two new variables
MR_FAILURE_THRESHOLD
andMR_FAILURE_FLOOR
that can be passed through the .gitlab-ci.yml.-
MR_FAILURE_THRESHOLD
will work similar to how degradation threshold works, the MR will not be mergeable if theTotal Score
metric decreases by more than that value. The CI Linter should not allow this value to be less thanDEGRADATION_THRESHOLD
. -
MR_FAILURE_FLOOR
- If theTOTAL_SCORE
goes below this value the MR should not be mergeable. If this threshold is met but the decrease is less than than the value defined inDEGRADATION_THRESHOLD
the MR widget will be displayed.
-
- TBD - should both failure thresholds be allowed in the .gitlab-ci.yml file at the same time?
- TBD - should a code owner be able to override this failure?
Permissions and Security
Documentation
- Update the documentation to detail setup and usage of the new variables.
Testing
What does success look like, and how can we measure that?
Measures of Success
- We expect this to increase the number of users interacting with the Web Performance MR Widget by 20% within 60 days of release on gitlab.com
What is the type of buyer?
A Team Lead is the one likely responsible for leading quality norms for the team and who needs to enforce this kind of feature. This maps to the GitLab Starter tier.
Links / references
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.