Rate limit the repository compare API
What does this MR do and why?
GET /projects/:id/repository/compare has no rate limit of its own; only the general API throttles apply. This MR adds one, behind the compare_api_rate_limit feature flag (off by default), configured by the project_repositories_compare_limit and project_repositories_compare_unauthenticated_limit application settings (stored in the rate_limits jsonb column, so no migration).
Both settings default to 0, which disables the limit, so GitLab Self-Managed and GitLab Dedicated enforce nothing on upgrade or on flag removal (same approach as !271345 (merged) and !271347 (merged)). GitLab.com gets 2,000 requests per minute for each authenticated user and 100 per minute for each IP address for unauthenticated requests, set through the settings API before the flag is enabled.
The check runs first in the endpoint, before the target project is resolved, so throttled requests do no Gitaly or database work. Over the limit the response is 429 Too Many Requests with Retry-After, like the other API rate limits.
This follows !261912 (merged) (file blame rate limit) file for file.
Rollback: disable compare_api_rate_limit, or set the settings back to 0.
Before enabling on GitLab.com
The GitLab.com authenticated value of 2,000 was set from observed production traffic: it sits below the highest per-user rate the route has handled without incident, and above every other current integration. Details can be found here: https://gitlab.com/gitlab-org/gitlab/-/work_items/635884#note_4209147412. Set both values through the settings API before enabling the flag; GitLab.com reads the code default (0) for any setting it has not stored. The remaining rate limit process steps (stakeholders, change request, GitLab.com rate limits docs) are tracked in the rollout issue, #639297. The flag is off and the defaults are 0, so merging doesn't change any limit anywhere.
References
- Related to https://gitlab.com/gitlab-org/gitlab/-/work_items/635884
- Rollout issue: #639297
- Precedent: !261912 (merged)
Detailed context for reviewers and AI agents
Shape
- Two Labkit rules in
supported_rate_limits.rb:project_repositories_compare(characteristicuser) andproject_repositories_compare_unauthenticated(characteristicip), period 1 minute, limits read from the settings. check_compare_rate_limit!inlib/api/repositories.rbmirrorscheck_blame_rate_limit!inlib/api/files.rb, wrapped in the flag check the waygroup_and_project_delete_rate_limitis.- Settings:
ApplicationSetting.rate_limits_definition,application_settings_helper.rbvisible attributes, JSON schema, settings API docs. No admin UI, like the other flagged 19.6 limits. descdeclares the 429 and describes the limit;doc/api/openapi/openapi_v3.yamlis regenerated.
Choices to confirm
- Per user, not per user and project. A single caller spreading requests over many projects shares one bucket. This is the
group_apishape rather than the blame one; the trade-off is that the authenticated default has to accommodate legitimate multi-project CI, which is why the number is a setting. - Unauthenticated GitLab.com value lower than authenticated. Anonymous callers are the cheaper tier on the web compare page too (see !268855 (merged)).
- Flag actor is
current_user, matchinggroup_and_project_delete_rate_limit, so we can enable for ourselves first and then by percentage of users.
Docs
doc/api/repositories.md, doc/api/settings.md (history and two attribute rows), doc/rate_limits/_index.md, and a new section in doc/rate_limits/api/projects.md.
How to set up and validate locally
-
In the Rails console, enable the flag and lower the limits:
Feature.enable(:compare_api_rate_limit) ApplicationSetting.current.update!(project_repositories_compare_limit: 2, project_repositories_compare_unauthenticated_limit: 1) -
Request a comparison three times within a minute as an authenticated user:
curl --include --header "PRIVATE-TOKEN: <token>" \ --url "http://gdk.test:3000/api/v4/projects/<project id>/repository/compare?from=<base>&to=<head>"The third request returns
429withRetry-After: 60andRateLimit-Name: project_repositories_compare. -
Request a public project's comparison twice without a token. The second request returns
429withRateLimit-Name: project_repositories_compare_unauthenticated. -
Disable the flag (
Feature.disable(:compare_api_rate_limit)) and repeat. Nothing is limited. -
Re-enable the flag, set both settings back to
0(the default), and repeat. Nothing is limited.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.