Add labkit rate limit adapter for cohort 4 keys

Max Woolfrequested to merge
28811-labkit-rate-limit-cohort-4 into master

What this MR does

Adds the labkit rate-limit adapter path for cohort 4 of the Gitlab::ApplicationRateLimiter → Labkit::RateLimit migration.

Cohort 4 is the sole IncrementPerActionedResource caller — the EE git-abuse flow's "unique project downloads" rate limits — driven from ee/app/services/users/abuse/git_abuse/base_throttle_service.rb. This cohort migrates the keys:

Key Subclass Characteristics count_distinct
unique_project_downloads_for_application ApplicationThrottleService [user] :project_id
unique_project_downloads_for_namespace NamespaceThrottleService [user, namespace] :project_id

Both keys count the distinct number of projects a user downloads in a window (SADD/SCARD), rather than the number of download calls (INCR). Threshold and interval for the namespace key are per-namespace (read from namespace_settings.unique_project_download_limit{,_interval_in_seconds}), not application-wide.

This is the first cohort to use labkit's set-mode rules and the first cohort to use labkit's rule_context: for per-call config. Both prerequisites are now upstream:

  • Labkit::RateLimit::Rule#count_distinct: (labkit-ruby v2.1.0, !292 (merged))
  • Labkit::RateLimit::Limiter#check(..., rule_context:) (labkit-ruby v2.2.0, !300 (merged))

The Gemfile pin moves from ~> 2.0.0 to ~> 2.2 and Gemfile.lock / Gemfile.next.lock resolve to gitlab-labkit 2.2.1.

Mechanism

Spec entries opt in via two new fields

unique_project_downloads_for_application: {
  ...
  count_distinct: :project_id,
  overrides_via_rule_context: true,
  flag_scope: :cohort_4,
}
  • count_distinct: :project_id flips the labkit Rule from INCR to SADD/SCARD. The named slot is not in characteristics: (the bucket key), and the adapter populates it explicitly from resource.id rather than via class-routing — hence the _id suffix to avoid collision with class-routed AR slots.
  • overrides_via_rule_context: true opts the entry out of the shadow_or_enforce? override short-circuit. Instead, the labkit Rule is built with one-arity callables on limit: / period: that read from the per-call rule_context: hash. A nil/missing key falls back to the registered threshold/interval, so behaviour is identical when no override is supplied.

Strategy compatibility gate

dispatch_to_labkit now admits IncrementPerActionedResource in addition to IncrementPerAction, but only when the spec is set-mode (LabkitAdapter.set_mode?). An ad-hoc INCR-mode key called with a resource: argument would otherwise diverge silently from the legacy SADD/SCARD counter.

Peek-then-check is set-safe

BaseThrottleService#initialize calls rate_limited?(peek: true) (to suppress duplicate alert emails); #execute calls it without peek. The labkit peek is read-only (SCARD without SADD), so the Redis SET only accumulates members on the non-peek path. Covered by an explicit integration regression test:

'does not SADD on peek when both peek and check share the same call site'

Feature flags

Two wip flags created in config/feature_flags/wip/, both group: group::networking and incident management, default off:

Flag Purpose
rate_limiter_use_labkit_cohort_4 run labkit path alongside legacy (shadow)
rate_limiter_use_labkit_cohort_4_enforce let labkit's decision win over legacy

Rollout work item: gitlab-org/gitlab#601342 (with shadow/enforce flip steps, Q1–Q7 pass criteria, rollback playbook).

The adapter uses Feature.current_request as the actor, which resolves to a per-call UUID for non-request callers; the rollout playbook is binary on/off only, not percentage.

Tests

156 examples added / updated across:

  • spec/lib/gitlab/application_rate_limiter/labkit_adapter_spec.rb: .set_mode?, opt-in path through .shadow_or_enforce?, .run! SADD semantics, .run_peek! not requiring resource_id, threshold_override propagation via rule_context, fallback to registered values when overrides are nil.
  • spec/lib/gitlab/application_rate_limiter_spec.rb: revised IncrementPerActionedResource dispatch coverage; the gate now distinguishes INCR-mode keys (stay on legacy) from set-mode keys (route to labkit with resource_id + overrides forwarded).
  • ee/spec/lib/ee/gitlab/application_rate_limiter/labkit_adapter/supported_rate_limits_spec.rb: registry shape for the two cohort 4 entries, end-to-end SADD routing for both, peek-then-check integration regression.

Migration context

Screenshots or screen recordings

Not applicable.

How to set up and validate locally

  1. Stop GDK and run bundle install to pick up labkit-ruby 2.2.1.
  2. In a Rails console with the cohort 4 flag enabled: 
    Feature.enable(:rate_limiter_use_labkit_cohort_4)
user = User.first; project = Project.first
Gitlab::ApplicationRateLimiter.throttled?(
  :unique_project_downloads_for_application,
  scope: user, resource: project, threshold: 5, interval: 600
)
  3. Gitlab::Redis::RateLimiting.with { |r| r.keys("labkit:rl:applimiter_unique_project_downloads_for_application*") } should show one SET key with the project id as a member.

MR acceptance checklist

This checklist encourages you to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

🤖beep boop

Merge request reports