Add configurable setting for "Project Jobs API endpoint"
What does this MR do and why?
Raised from this issue: #395702 (closed)
This is an initial proof of concept MR to allow the Project Jobs API endpoint rate limit to be configurable, instead of the hard-coded value of 600
calls per minute.
For initial verification purposes, the UI setting has been placed under User and IP rate limits, but can likely be moved to a more suitable location.
A database migration has been added to define the rate limit setting, which can then be configured via UI or via Rails Console with:
::Gitlab::CurrentSettings.update!(project_jobs_api_rate_limit: <int>)
Screenshots or screen recordings
Proof of Concept of the setting added to the UI:
How to set up and validate locally
- Deploy a GDK instance.
- Go to Admin Area > Settings > Network
- Under User and IP rate limits, confirm that the value for Maximum authenticated requests to
project/:id/jobs
per minute is set to the default of 600. - Reduce this value to a small value for testing, such as
10
, then select Save changes. - Open a Rails console and confirm the setting has been saved to the database with:
::Gitlab::CurrentSettings.project_jobs_api_rate_limit
- Locate a project, and copy the Project ID for reference.
- Create a Personal Access Token, then save this value for reference.
- Run the following bash script against the GDK instance to create calls to the rate-limited endpoint. It should change from
200
responses to429
to indicate the API is being rate-limited:
TOKEN=<token>
ID=<id>
URL="<url>"
for i in {1..20}
do
echo -n "Attempt $i: "
curl -i --header "PRIVATE-TOKEN: $TOKEN" "$URL/api/v4/projects/$ID/jobs" -o /dev/null -w '%{http_code}\n' -s
done
Example Output:
Attempt 1: 200
Attempt 2: 200
Attempt 3: 200
Attempt 4: 200
Attempt 5: 200
Attempt 6: 200
Attempt 7: 200
Attempt 8: 200
Attempt 9: 200
Attempt 10: 200
Attempt 11: 429
Attempt 12: 429
Attempt 13: 429
....
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Ben King