Enforce the per-project Git throttle overrides through Labkit

What does this MR do?

Hooks the stored per-project Git throttle overrides up to Labkit so they actually throttle. The overrides and the settings UI to set them landed in !259138 (merged), so this is only the enforcement layer.

It adds three Labkit rules in their own limiter, rack_request_git_project. They run alongside the three instance-wide Git rules on the same request, and Labkit rejects if either trips. A request has to stay under both the instance-wide limit and the project's. This is an extra ceiling, not a replacement: if it replaced the instance-wide counter, traffic to an overridden project would stop counting against the actor's instance-wide budget, and owning N projects would buy N budgets.

It needs a separate limiter rather than extra rules in the general one. Within a limiter a claim expresses an exclusion and cannot express co-firing, so a second counter needs a second limiter. Protected paths already works this way.

ClassifiedRequest exposes one identity fact per throttle: unauthenticated_git_http_project, authenticated_git_http_project and authenticated_git_lfs_project. Each holds that project's override key for that throttle, or nil. Each rule presence-gates on its own fact, so it is inert for a project with no override of that throttle, which is almost every request. There is one fact per throttle rather than one shared fact because a project overriding only LFS must keep sharing the instance-wide counters for both HTTP throttles.

Resolving a limit that varies per request needs the request. Labkit's SDK supports this through rule_context, which it passes to arity-1 limit and period callables.

Path resolution:

  • Wiki and design repositories resolve to the parent project, so the override cannot be bypassed by pushing to the wiki remote.
  • Git HTTP keeps serving a project's pre-rename paths (Gitlab::RepoPath.find_project follows redirects), so the loader keys overrides by the project's RedirectRoute paths too. All of a project's paths, current and pre-rename, resolve to a single counter key, so a rename cannot buy a second budget.
  • Organization-scoped paths (/o/<org>/group/project.git) have their prefix stripped with the same helper routing uses to constrain the organization path.

Performance: the override set is held in process behind a one-minute TTL rather than read per request. Gitlab::ProcessMemoryCache is deliberately not used. Its DupCoder Marshals the value on every read inside the store's global monitor, which I measured at ~357us per read at the row cap versus 0.58us for a frozen reference. The refresh is single-flighted behind a mutex, and threads that lose that race serve the stale set rather than queueing. Database Lab plans at the 10,000 row ceiling put a refresh at roughly 160ms, which every Git request in the process would otherwise wait on once a minute. The resolved path is memoized on the Rack env so the route regex runs once per request.

Verified end to end on a live instance with every Labkit cohort flag enabled:

  • An overridden project returns 429 after its limit with Ratelimit-Name: throttle_unauthenticated_git_http_project.
  • A real git clone fails with remote: Retry later.
  • A project with no override is unaffected at the instance limit.

This is behind the project_git_throttle_overrides feature flag, disabled by default.

Relevant issues

Related to #629903

Edited by John Cai

Merge request reports

Loading
Loading