Add cross-project per-user and per-IP archive rate limit

What does this MR do and why?

The existing project_repositories_archive rate limit is scoped per project and user (5/min), so a caller spread across N projects gets N separate buckets and is never capped. The Oct 21 Kibana data showed a top authenticated user peaking at 584 req/min across projects and a top anonymous IP at 432 req/min across 432 projects.

This MR adds two cross-project rate limits that run alongside the existing per-project limit:

Rule Scope Default Period
repositories_archive per user (authenticated) 200/min 1 minute
repositories_archive_unauthenticated per IP (unauthenticated) 100/min 1 minute

Both thresholds are backed by application settings (project_repositories_archive_limit, project_repositories_archive_unauthenticated_limit) and tunable via the application settings API without a deploy. Set to 0 to disable.

This is MR 2 of the Phase 1 work for https://gitlab.com/gitlab-org/gitlab/-/work_items/635883, stacked on !269223 (merged).

How the two limits interact

The cross-project limit runs first. If it throttles, the per-project check is skipped. If it passes, the per-project limit runs as before. A user downloading from 40 distinct projects per minute at the per-project cap of 5/min each would hit the 200/min cross-project limit. The 584 req/min and 432 req/min callers from the Kibana data are throttled immediately.

Changes

  • application_setting.rb: new project_repositories_archive_limit (default 200) and project_repositories_archive_unauthenticated_limit (default 100) in rate_limits_definition.
  • supported_rate_limits.rb: new repositories_archive (%i[user]) and repositories_archive_unauthenticated (%i[ip]) as staged_rule entries gated by the cross_project_archive_rate_limit feature flag.
  • repository_archive_rate_limiter.rb: checks cross-project limit first (by user or IP), then per-project. In the API, too_many_requests! raises an exception that halts execution. In the controller, performed? guards the second check.
  • Specs: module spec (3 cases), controller spec (cross-project 429), API spec (cross-project 429, anonymous 429), staged rule .enforced? and .limit_for specs.
  • Docs: doc/api/settings.md and doc/rate_limits/_index.md document both new settings and the feature flag.

Feature flag

The cross-project limits use a dedicated cross_project_archive_rate_limit feature flag (gitlab_com_derisk). While disabled, requests over the limit are logged but not blocked, giving observability before enforcement.

The existing archive_rate_limit flag still gates whether any archive rate limiting runs at all.

Trade-offs

  • Bypass-header callers remain exempt (out of scope for Phase 1, per the weight breakdown).

References

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

Edited by Gloria Odipo

Merge request reports

Loading
Loading