MR fail on web performance degredation

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Problem to solve

For some teams a set level of web performance is critical for their application. Today when that performance degrades there is not an easy way for them to fail an MR or be alerted to changes that will have negative customer outcomes.

Intended users

Use Cases

  • I don't want web performance to decrease more than a set value from the current value on master eg; It cannot decrease more than 1 point from /master.
  • I don't want web performance to decrease more than a percentage value from the current value on master eg; it cannot decrease more than 2% from /master.
  • I don't want web performance to decrease below a set threshold eg; It cannot fall below 50

Proposal

  • Add a two new variables MR_FAILURE_THRESHOLD and MR_FAILURE_FLOOR that can be passed through the .gitlab-ci.yml.
    • MR_FAILURE_THRESHOLD will work similar to how degradation threshold works, the MR will not be mergeable if the Total Score metric decreases by more than that value. The CI Linter should not allow this value to be less than DEGRADATION_THRESHOLD.
    • MR_FAILURE_FLOOR - If the TOTAL_SCORE goes below this value the MR should not be mergeable. If this threshold is met but the decrease is less than than the value defined in DEGRADATION_THRESHOLD the MR widget will be displayed.
  • TBD - should both failure thresholds be allowed in the .gitlab-ci.yml file at the same time?
  • TBD - should a code owner be able to override this failure?

Permissions and Security

Documentation

  • Update the documentation to detail setup and usage of the new variables.

Testing

What does success look like, and how can we measure that?

Measures of Success

  • We expect this to increase the number of users interacting with the Web Performance MR Widget by 20% within 60 days of release on gitlab.com

What is the type of buyer?

A Team Lead is the one likely responsible for leading quality norms for the team and who needs to enforce this kind of feature. This maps to the ~"GitLab Starter" tier.

Links / references

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.

Edited by 🤖 GitLab Bot 🤖