Add response time to severity table in the docs
What does this MR do?
We'd like to have a standard in place for what severity to assign response time issues. This is a first pass. These numbers are pretty high, but this will allow us to appropriately focus on only the worst performing endpoints and bring them in line.
To help me come up with this initial suggestion, I looked at the most recent 10k results: https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k
Sorting them by TTFB P90, I identified four ranges I thought were appropriate to represent S1-S4:
| Severity | Response time |
|---|---|
| S1 | Above 4000ms |
| S2 | Between 2000ms and 4000ms |
| S3 | Between 1000ms and 2000ms |
| S4 | Between 500ms and 1000ms |
I thought this first stab was clean because the ranges for each severity double as the severity rises. (Nerd alert)
For the most recent 10k run (2020-02-25), here are the tests which would have issues and their respective severity levels:
All the other tests have TTFB P90 below 500ms which we are currently considering as no action needed. Over time, we can change the limits to tighten standards and iterate towards better performance. As we do, I imagine we will drop that 500ms floor as well.
Proposal
| ~bug context | Severity | Endpoint Response time |
|---|---|---|
| ~performance | ~S1 | Above 9000ms to timing out |
| ~performance | ~S2 | Between 2000ms and 9000ms |
| ~performance | ~S3 | Between 1000ms and 2000ms |
| ~performance | ~S4 | Between 500ms and 1000ms |
Closes gitlab-org/quality/performance#95 (closed)
Author's checklist
-
Follow the Documentation Guidelines and Style Guide. -
If applicable, update the permissions table. -
Link docs to and from the higher-level index page, plus other related docs where helpful. -
Apply the documentation label.
Review checklist
All reviewers can help ensure accuracy, clarity, completeness, and adherence to the Documentation Guidelines and Style Guide.
1. Primary Reviewer
-
Review by a code reviewer or other selected colleague to confirm accuracy, clarity, and completeness. This can be skipped for minor fixes without substantive content changes.
2. Technical Writer
-
Optional: Technical writer review. If not requested for this MR, must be scheduled post-merge. To request for this MR, assign the writer listed for the applicable DevOps stage.
3. Maintainer
-
Review by assigned maintainer, who can always request/require the above reviews. Maintainer's review can occur before or after a technical writer review. -
Ensure a release milestone is set. -
If there has not been a technical writer review, create an issue for one using the Doc Review template.