Add cross-project per-user and per-IP archive rate limit
What does this MR do and why?
The existing project_repositories_archive rate limit is scoped per project and user (5/min), so a caller spread across N projects gets N separate buckets and is never capped. The Oct 21 Kibana data showed a top authenticated user peaking at 584 req/min across projects and a top anonymous IP at 432 req/min across 432 projects.
This MR adds two cross-project rate limits that run alongside the existing per-project limit:
| Rule | Scope | Default | Period |
|---|---|---|---|
repositories_archive |
per user (authenticated) | 200/min | 1 minute |
repositories_archive_unauthenticated |
per IP (unauthenticated) | 100/min | 1 minute |
Both thresholds are backed by application settings (project_repositories_archive_limit, project_repositories_archive_unauthenticated_limit) and tunable via the application settings API without a deploy. Set to 0 to disable.
This is MR 2 of the Phase 1 work for https://gitlab.com/gitlab-org/gitlab/-/work_items/635883, stacked on !269223 (merged).
How the two limits interact
The cross-project limit runs first. If it throttles, the per-project check is skipped. If it passes, the per-project limit runs as before. A user downloading from 40 distinct projects per minute at the per-project cap of 5/min each would hit the 200/min cross-project limit. The 584 req/min and 432 req/min callers from the Kibana data are throttled immediately.
Changes
application_setting.rb: newproject_repositories_archive_limit(default 200) andproject_repositories_archive_unauthenticated_limit(default 100) inrate_limits_definition.supported_rate_limits.rb: newrepositories_archive(%i[user]) andrepositories_archive_unauthenticated(%i[ip]) asstaged_ruleentries gated by thecross_project_archive_rate_limitfeature flag.repository_archive_rate_limiter.rb: checks cross-project limit first (by user or IP), then per-project. In the API,too_many_requests!raises an exception that halts execution. In the controller,performed?guards the second check.- Specs: module spec (3 cases), controller spec (cross-project 429), API spec (cross-project 429, anonymous 429), staged rule
.enforced?and.limit_forspecs. - Docs:
doc/api/settings.mdanddoc/rate_limits/_index.mddocument both new settings and the feature flag.
Feature flag
The cross-project limits use a dedicated cross_project_archive_rate_limit feature flag (gitlab_com_derisk). While disabled, requests over the limit are logged but not blocked, giving observability before enforcement.
- Rollout issue: #640144
The existing archive_rate_limit flag still gates whether any archive rate limiting runs at all.
Trade-offs
- Bypass-header callers remain exempt (out of scope for Phase 1, per the weight breakdown).
References
- Issue: https://gitlab.com/gitlab-org/gitlab/-/work_items/635883
- Rollout issue: #640144
- Stacked on: !269223 (merged)
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist.