Update slowness thresholds based on actual reports count
What does this MR do and why?
These new thresholds are based on actual reports number:
➜ ~/Code/GitLab/quality/engineering-productivity/team git:(main) ✗ [gcp: | k8s: triage-ops/default] $ GITLAB_API_TOKEN=$(cat ~/gitlab/token-rymai.com) ruby scripts/unhealthy_test_issues_statistics.rb --health-problem-type slowness --max-issues 4000
................................................................................................................................................................................................................................................................................................................................................................................................................Fetched 400 issues
................................................................................................................................................................................................................................................................................................................................................................................................................Fetched 800 issues
................................................................................................................................................................................................................................................................................................................................................................................................................Fetched 1200 issues
................................................................................................................................................................................................................................................................................................................................................................................................................Fetched 1600 issues
................................................................................................................................................................................................................................................................................................................................................................................................................Fetched 2000 issues
................................................................................................................................................................................................................................................................................................................................................................................................................Fetched 2400 issues
................................................................................................................................................................................................................................................................................................................................................................................................................Fetched 2800 issues
........................................................................................................................................................................................................................................................................................................................
P75: 77.0
P90: 521.5000000000014
P95: 2176.8999999999996
P99: 6099.0
{:number=>3112.0, :sum=>997974, :variance=>968074.8993080013, :standard_deviation=>983.9079729873121, :min=>1, :max=>6229, :mean=>320.6857326478149, :mode=>1, :median=>15.0, :range=>6228.0, :q1=>2.0, :q2=>15.0, :q3=>77.0}
| # of flakiness reports | Percentile | Label |
|---|---|---|
| 1-520 | <= P90 | slowness4 |
| 521-2176 | P90 < x <= P95 | slowness3 |
| 2177-6098 | P95 < x <= P99 | slowness2 |
| 6099+ | > P99 | slowness1 |
This follows the same strategy as used for flakiness thresholds.
Screenshots or screen recordings
These are strongly recommended to assist reviewers and reduce the time to merge your change.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Rémy Coutable