Get single project API degrades with increased RPS
Summary
Get single project API significantly slows down with increased RPS. It's especially crucial for the current 50k architecture reference: https://gitlab.com/gitlab-org/quality/performance/issues/102
Results against 50k environment(1000 RPS, duration 60 sec):
Environment: 50k (12.3.5-ee 9dbaa740018)
Scenario: 60s_1000rps
Date: 2019-10-10
Run Time: 2303.62s (Start: 13:40:27 UTC, End: 14:18:50 UTC)
NAME | DURATION | P95 | RPS | SUCCESS RATE | RESULT
---------------------------------------------------------|----------|------------|----------------------|----------------|-------
api_v4_projects_project | 60.0s | 1457.64ms | 765.2/s (>800.00/s) | 100.00% (>95%) | Failed
Grafana: http://35.196.33.84/-/grafana/d/tVdO8YIWz/server-performance?orgId=1&refresh=10s
Compare with results against 10k environment(200 RPS, duration 60 sec):
Environment: 10k (12.3.5-ee 9dbaa740018)
Scenario: 60s_200rps
Date: 2019-10-11
Run Time: 1692.72s (Start: 01:42:19 UTC, End: 02:10:32 UTC)
NAME | DURATION | P95 | RPS | SUCCESS RATE | RESULT
---------------------------------------------------------|----------|------------|----------------------|----------------|-------
api_v4_projects_project | 60.0s | 97.90ms | 193.3/s (>160.00/s) | 100.00% (>95%) | Passed
Grafana: http://34.73.165.75/-/grafana/d/tVdO8YIWz/server-performance?orgId=1 (To be able to see it, please sign up)
Steps to reproduce
- Check out the Performance Toolkit
- Run the specific test with the
run-k6
command. For example against the 50k environment you would run this following from the project root:ACCESS_TOKEN="" k6/run-k6 -e environments/50k.json -s scenarios/60s_1000rps.json -t tests/api/api_v4_projects_project.js
, whereACCESS_TOKEN
is a valid GitLab Personal Access Token for the specified environment(50k in this case). The token should come from a User that has admin access for the project(s) to be tested and have API and read_repository permissions. - If you're seeking to run the test against your own environment the Toolkit's documentation has details on how to achieve this.
What is the current bug behavior?
The API response time increases under high load.
What is the expected correct behavior?
That the API response time is improved.
Edited by Nailia Iskhakova