Improve performance of Group List API under load into next tier
The TTFB (Time to First Byte) of the Group List API is over our performance target under load:
* Environment: 1k
* Environment Version: 16.10.0-pre `aba3cbb5b8a`
* Option: 60s_20rps
* Date: 2024-03-11
* Run Time: 1h 26m 0.21s (Start: 04:40:28 UTC, End: 06:06:28 UTC)
* GPT Version: v2.14.0
NAME | RPS | RPS RESULT | TTFB AVG | TTFB P90 | REQ STATUS | RESULT
---------------------------------------------------------|------|--------------------|------------|-----------------------|----------------|---------
api_v4_groups | 20/s | 19.39/s (>14.40/s) | 514.22ms | 903.51ms (<500ms) | 100.00% (>99%) | FAILED¹²
- Latest results and metrics - https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k
* Environment: 25k
* Environment Version: 16.10.0-pre `aba3cbb5b8a`
* Option: 60s_500rps
* Date: 2024-03-11
* Run Time: 1h 30m 45.78s (Start: 05:03:53 UTC, End: 06:34:39 UTC)
* GPT Version: v2.14.0
NAME | RPS | RPS RESULT | TTFB AVG | TTFB P90 | REQ STATUS | RESULT
---------------------------------------------------------|-------|----------------------|-----------|----------------------|----------------|---------
api_v4_groups | 500/s | 377.13/s (>360.00/s) | 1215.52ms | 1595.24ms (<500ms) | 100.00% (>99%) | FAILED¹²
Results before fix from #421572 (comment 1799166044)
* Environment: 1k
* Environment Version: 16.9.0-pre `b3c9b11e892`
* Option: 60s_20rps
* Date: 2024-02-06
* Run Time: 1h 32m 40.43s (Start: 04:31:46 UTC, End: 06:04:27 UTC)
* GPT Version: v2.13.0
❯ Overall Results Score: 95.58%
NAME | RPS | RPS RESULT | TTFB AVG | TTFB P90 | REQ STATUS | RESULT
---------------------------------------------------------|------|--------------------|-----------|-----------------------|----------------|---------
api_v4_groups | 20/s | 16.39/s (>16.00/s) | 1089.01ms | 1253.41ms (<1000ms) | 100.00% (>99%) | FAILED¹²
Older results (16.3.0-pre)
* Environment: 10k
* Environment Version: 16.3.0-pre `609852eb5b9`
* Option: 60s_200rps
* Date: 2023-08-10
* Run Time: 1h 40m 31.71s (Start: 04:58:27 UTC, End: 06:38:59 UTC)
* GPT Version: v2.13.0
NAME | RPS | RPS RESULT | TTFB AVG | TTFB P90 | REQ STATUS | RESULT
---------------------------------------------------------|-------|----------------------|-----------|-----------------------|----------------|---------
api_v4_groups | 200/s | 196.27/s (>160.00/s) | 223.48ms | 265.70ms (<200ms) | 100.00% (>99%) | FAILED²
- Last successful run was at https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k?version_id=a4fdc173c17e413500e3e8f44d9b2715db5d2604, when response time was
155.52ms
, Environment Version: 16.3.0-pre10431006956
Test Details
Testing was done on our 10k Reference Architecture environment with our lab condition GitLab Performance Tool pipeline. The project being tested is a copy of gitlabhq (tarball can be found here). GitLab Performance Tool tests information is listed at Current test details page.
The latest GitLab Performance pipeline results can always be found here. Through this page full Server Metrics can be found via the Metrics Dashboard
link on that page.
As per our performance targets this endpoint's TTFB metric is above the target of 1000 ms which is severity3. Task is to improve the endpoint's performance into next tier.
Edited by Nailia Iskhakova