Merge Request Commits API substantially degrades with high commits count
Summary
With our increased performance testing efforts we've started to identify slow areas in GitLab and raising them as issues. One such area is the Merge Request Commits API where we've found its time to respond significantly slows if the number of commits is high (6000+). Other API endpoints in the same area don't show the same degradation:
Environment: http://10k.testbed.gitlab.net
Version: 12.0.2-ee ef76b54fc1e
NAME | RESULT | DURATION | P95 | RPS_COUNT | RPS_MEAN
-----------------------------------------------------|--------|----------|------------|-----------|-------------
api_v4_projects_merge_requests_merge_request | Passed | 30.0s | 146.37ms | 5415 | 180.499432/s
api_v4_projects_merge_requests_merge_request_changes | Passed | 30.0s | 131.92ms | 5432 | 181.066221/s
api_v4_projects_merge_requests_merge_request_commits | Passed | 30.0s | 9965.99ms | 649 | 21.633277/s
api_v4_projects_merge_requests_merge_request_notes | Passed | 30.0s | 225.73ms | 5293 | 176.432752/s
You can also see the latest results here - https://gitlab.com/gitlab-org/quality/performance/wikis/Benchmarks/Latest/10k.
We've also raised an issue for the equivalent web page as it itself takes a significantly long time to render (more than the API time alone) but that may be impacted by the API in part.
The above is in order of total number of projects.
Since the API only returns 20 results via pagination the assumption would be it would take the same time since it's capped but with the above it appears that the background process is still parsing all projects.
Steps to reproduce
- Check out the Performance Toolkit
- Run the specific test with the
run-k6
command. For example against the 10k environment you would run this following from the project root:./run-k6 -e environments/10k.json -o options/60s_200rps.json -t tests/api/api_v4_projects_merge_requests_merge_request_commits.js
. - If you're seeking to run the test against your own environment the Toolkit's documentation has details on how to achieve this.
What is the current bug behavior?
The API will slow exponentially depending on the number of commits in the MR.
What is the expected correct behavior?
That the API responds quickly like the other MR endpoints utilising accepted techniques such as pagination.