Simplify Rate Limiting Configuration
<!-- AI-Sessions
dir: ~/.claude/projects/-Users-samwiskow-projects-rl-ai-project/
5e78f498-82cd-4814-a905-1beb51d908dc.jsonl (2026-06-24)
-->
## Summary
As GitLab.com has evolved, we've introduced rate limits and throttling measures at many layers to enhance security and performance of our platform, ensuring availability and user satisfaction are at a desired level. These measures have often been reactionary, and we have not had a consistent strategy for defining and enforcing these limits.
This lack of consistency can lead to confusion for our users and loss of productivity for our engineers, as well as difficulty when we wish to define and implement new limits. This aims to introduce a transparent strategy for these limits; firstly in order to enable our engineers to introduce sensible policies to enforce our availability goals, and secondly allow us transparency to expose these to our users.
Related Design Document: [Unified Rate Limiting Architecture](https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/unified_rate_limiting/) — the single canonical design for unifying application-level rate limiting through [labkit](https://gitlab.com/gitlab-org/labkit) in three phases. This supersedes the earlier [Simplifying Rate Limiting Configuration](https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/rate_limiting_simplification/) roadmap, which is kept only as historical context (its phase numbering is obsolete).
---
The purpose of this epic is to link all rate-limiting simplification epics in one place and track them against the canonical [Unified Rate Limiting Architecture](https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/unified_rate_limiting/) phases.
## Strategy & rollout context
This is the engineering parent for the rate-limiting program. The strategy, exec approval, and rollout sequencing live one layer up:
* :link: [gitlab-org&22423](https://gitlab.com/groups/gitlab-org/-/work_items/22423) — exec SSOT epic for the rate-limit tiered-limits changes (strategy and exec approval).
* :link: [gitlab-org&22475](https://gitlab.com/groups/gitlab-org/-/work_items/22475) — master rollout plan for this program (child of [gitlab-org&22423](https://gitlab.com/groups/gitlab-org/-/work_items/22423)).
## Canonical phases & crosswalk
The canonical [Unified Rate Limiting Architecture](https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/unified_rate_limiting/) defines three phases. The table below maps them to the deprecated numbering used by the older [Simplifying Rate Limiting Configuration](https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/rate_limiting_simplification/) doc, so older links and references still make sense.
| Canonical (Unified doc) | Deprecated (old doc) | Status |
|---|---|---|
| **Phase 1** — Application-Level Unification (labkit) | "Phase 2 — application" | In progress |
| **Phase 2** — Externalized Configuration (YAML/protobuf) | "Phase 3 — interface" | Not started |
| **Phase 3** — Dynamic External Service | — | Not started |
| (Edge network — completed May 2025) | "Phase 1 — edge network" | Completed (history) |
## Overview
**Completed history: Edge Network and Bypass Configuration** — Completed May 2025
This was the first wave of simplification work, numbered "Phase 1" under the deprecated doc. It is complete and retained here as context; it sits outside the three canonical phases.
* :white_check_mark: [gl-infra&1434](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/1434) — Simplify Bypass Configuration
* :white_check_mark: [gl-infra&1452](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/1452) — Standardize Cloudflare WAF Configuration
**Phase 1: Application-Level Unification (labkit)** — In Progress
All application rate limiting goes through a single API in `labkit-ruby`, so every limit is defined, counted, and observed the same way. Existing configuration (ApplicationSettings, env vars, hardcoded defaults) keeps working — the caller resolves its own configuration and passes it in, so there are no breaking changes for self-managed.
This work was paused in July 2025 because the [GATE](https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/new_auth_stack/) auth architecture was expected to change where enforcement lives. GATE has since been defined, and the strategic decision is to **"go first"**: the unified rate limit configuration interface defined here becomes the contract that GATE and future services conform to — not the other way around.
The authoritative execution plan for Phase 1 lives in the agentic implementation epic:
* :hourglass_flowing_sand: [gl-infra&2021](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/2021) — **Rate Limits: Agentic Implementation Plan** (source of truth for current scope, stages, and status)
Phase 1 is structured as three stages in [gl-infra&2021](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/2021):
1. **Stage 0 — Test Coverage Baseline:** comprehensive coverage of existing `Gitlab::RackAttack` and `Gitlab::ApplicationRateLimiter` behavior, as the safety net before any rate limiting code is changed.
2. **Stage 1 — labkit-ruby rate limit API and identifier design:** the unified API in `gitlab-org/labkit-ruby` that all rate limiting in GitLab flows through. The call site name + request identifier + rules contract established here is the foundation everything else depends on.
3. **Stage 2 — Migrate application rate limiting to labkit:** feature-flagged migration of `ApplicationRateLimiter` (clean switch) and a new rack middleware alongside `Rack::Attack` (parallel-run, log mode first), then consistent response headers and default rate limits for new endpoints.
Prior framing for this work — kept for historical context; superseded by [gl-infra&2021](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/2021)'s labkit-first approach (these were numbered "Phase 2" under the deprecated doc):
* [gl-infra&1503](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/1503) — Simplify application level configuration
* [gl-infra&1511](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/1511) — Simplify RackAttack configuration
**Phase 2: Externalized Configuration (YAML/protobuf)** — Not started
Labkit loads configuration that overrides the application-provided defaults. The format follows a protobuf schema with YAML as the serialization, so the same files load the same way across `labkit-ruby`, `labkit-go`, and the services that consume them. This lets a rule change roll out without a full build and deploy of the application. To be scoped after Phase 1 lands its labkit-ruby contract.
**Phase 3: Dynamic External Service** — Not started
An external service returns rate limit rules dynamically, per-request, based on the identifier. This is how we get per-customer, per-tier, and per-namespace customization without maintaining static configuration files for each case, and near-instant rollout of a change without a deployment. The identifier designed in Phase 1 is the lookup key for this service.
## Progress
```mermaid
flowchart LR
01[Design Document] --> A1
subgraph edge["Edge Network and Bypass - Complete (history)"]
A1[Simplify Bypass Configuration] --> A2
A2[Standardize Cloudflare WAF Configuration]
end
subgraph phase-1["Phase 1: Application-Level Unification - &2021"]
S0[Stage 0: Test Coverage Baseline] --> S1
S1[Stage 1: labkit-ruby Rate Limit API + Identifier] --> S2
S2[Stage 2: Migrate ApplicationRateLimiter + new Rack middleware]
end
subgraph phase-2[Phase 2: Externalized Configuration]
C1[YAML/protobuf config file] --> C2
C2[Redis-backed web-UI rules]
end
subgraph phase-3[Phase 3: Dynamic External Service]
D1[External rule service] --> D2
D2[Per-customer / per-tier rules]
end
A2 ==> S0
S1 ==> C1
C1 ==> D1
classDef complete fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px
classDef progress fill:#fff8e1,stroke:#ff8f00,stroke-width:2px
classDef transition fill:none,stroke:#9c27b0,stroke-width:3px,stroke-dasharray: 5 5
class 01,edge,A1,A2 complete
class phase-1,S0,S1,S2 progress
class phase-2,C1,C2,phase-3,D1,D2 transition
```
**Key**
**Green** — Complete | **Yellow** — In Progress | **Dotted line** — yet to be scoped
<!-- STATUS NOTE START -->
## Status 2026-07-01
The rate-limiting migration's final track — moving the request-throttling layer onto the new framework — landed this week ([gitlab!239466](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/239466) merged 2026-07-02), and the first attempt to switch it on ran in shadow (observe-only) mode on staging. A latent bug made the new middleware error on nearly every request, but its fail-open design meant no request was ever blocked or slowed and the code never ran in production; the flags were switched back off, the fix merged the same day ([gitlab!243638](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/243638)), and the rollout will retry once it deploys. No customer impact, no production behaviour change, and no asks for leadership.
:clock1: **total hours spent this week by all contributors**: 34
**Outcomes this week**
- **Stage 2b middleware merged — the last migration track is now in the codebase.** [gitlab!239466](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/239466) (mwoolf, "Add Labkit::RateLimit shadow + enforce middleware for Rack::Attack") merged 2026-07-02 with approvals from hmerscher + reprazent, closing out the multi-week reprazent design review. It ships the request-throttle migration as a shadow-then-enforce middleware gated by six per-cohort `wip` flags. **All flags default off, so it is dormant on merge** — deployed to `.com` with no behaviour change until a cohort is explicitly enabled. Tracks [Stage 2b #28852](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28852).
- **Feature-flag removal is live on `.com`.** [gitlab!239802](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/239802) (merged 2026-06-24) has ridden the deploy train through production, so every migrated ApplicationRateLimiter cohort is now permanently on the new framework with the temporary rollout flags gone. [#28876](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28876) stays open only until the final close-outs below land.
**In flight**
- **[Stage 2b #28852](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28852) — shadow rollout retry.** The first shadow enablement was attempted on staging on 2026-07-03 and immediately surfaced a nil-rule crash in the middleware (details under Risks). It was contained to staging, the fix [gitlab!243638](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/243638) (mwoolf, approved reprazent) merged the same day, and the plan is to re-enable shadow per cohort once it deploys, confirm the divergence metric emits, and hold under the go/no-go threshold for 24h before any enforce. Tracked on [gitlab#602804](https://gitlab.com/gitlab-org/gitlab/-/issues/602804).
- **Config-from-registry [#29054](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/29054)** progressed: mwoolf's [gitlab!240022](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/240022) ("Resolve rate limit threshold/interval from labkit registry") came **out of Draft** and is ready for review; hmerscher's alternative [gitlab!242446](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/242446) remains Draft.
- **Stage 2b design follow-ups** spun out of the merged middleware and are in triage: [#28853](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28853) (config evolution — callables/precedence/static), [#29052](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/29052) (refine rule action semantics: limit/log/skip), [#29319](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/29319) (deprecate `GITLAB_THROTTLE_DRY_RUN` in favour of the `log` action), [#29320](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/29320) (model the RackAttack user allowlist as an explicit skip rule).
- **[labkit-ruby!315](https://gitlab.com/gitlab-org/ruby/gems/labkit-ruby/-/merge_requests/315)** (mwoolf, Lua-script extraction) — no movement again this week; still open, still waiting on a reviewer.
**Risks & decisions**
- **Contained incident (staging only, no user impact):** the first Stage 2b shadow enablement raised a `NoMethodError` on nearly every request — the middleware read `result.rule.name` on fail-open/unmatched results, which carry a nil rule. The middleware's fail-open guard meant `Rack::Attack` stayed the sole enforcer throughout, so **no request was blocked or delayed and the flags were never enabled in production**. The visible symptoms were an absent divergence metric and log/Sentry volume, which is what prompted disabling the flags. Root cause and fix in [gitlab!243638](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/243638); post-mortem on [gitlab#602804](https://gitlab.com/gitlab-org/gitlab/-/issues/602804).
- **Lesson / action items from the post-mortem:** the unit specs used a fake result that always carried a rule, so the real SDK's nil-rule path was never exercised — the primary gap. Follow-ups: test the middleware against the real SDK result contract; add a canary that alerts when the middleware records block calls but the divergence metric is empty (a blind gate); surface the guarded-exception rate as a metric; and verify an enforced block returns a real 429 before any enforce flag is turned on.
- **Risk (carry-over):** the [regex-timeout fail-open #28882](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28882) still has no movement since 2026-05-08 and continues to gate a safe Stage 2b *enforce*.
**Asks**
- None this week. The Stage 2b retry and config-from-registry work are team-internal and not blocked on leadership.
**Upcoming reviews / approval needs**
- [gitlab!240022](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/240022) — config-from-registry, now out of Draft (mwoolf); needs reviewers.
- [labkit-ruby!315](https://gitlab.com/gitlab-org/ruby/gems/labkit-ruby/-/merge_requests/315) — Lua-script extraction (mwoolf); stalled ~3 weeks, needs a reviewer.
**Next week**
- Deploy the nil-rule fix ([gitlab!243638](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/243638)) and retry the Stage 2b shadow rollout per cohort on staging; confirm the divergence metric emits and sits under the go/no-go threshold for 24h before any enforce.
- Land the post-mortem action items (real-SDK-contract tests and the blind-gate canary) so the retry is observable.
- Close out the migration: confirm [#28876](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28876) and close the superseded [gitlab!236795](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/236795) and the dead cohort-5 adapter [gitlab!236515](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/236515) — both still open, carried from last week.
- Progress config-from-registry ([gitlab!240022](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/240022) / [gitlab!242446](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/242446) / [#29054](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/29054)).
- Resolve the regex-timeout risk ([#28882](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28882)) that gates enforcement.
**Phase 2 stage status — start vs end of week** (start = state at the prior roll-up, 2026-06-26)
| Stage | Issue | Start of week | End of week |
|-------|-------|---------------|-------------|
| ApplicationRateLimiter → Labkit (full migration) | [#28808](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28808) | 🔄 In verification — enforcing on `.com`; FF-removal merged and deploying | 🔄 In verification — FF-removal live on `.com`; all cohorts permanently always-on |
| Feature-flag cleanup | [#28876](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28876) | 🔄 [gitlab!239802](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/239802) merged 2026-06-24 (deploying; `.com` pending) | ✅ Code live on `.com`; issue open only pending close of superseded [gitlab!236795](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/236795) |
| Stage 2b — RackAttack migration | [#28852](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28852) | 🔄 In review — reframed to explicit ordered rules; [gitlab!239466](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/239466) 1 approval | 🔄 Middleware [gitlab!239466](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/239466) merged 2026-07-02 (2 approvals), deployed dormant; shadow rollout attempted on staging → nil-rule crash (no prod/user impact), fix [gitlab!243638](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/243638) merged same day; retry pending deploy |
| Config from registry / static Limiters | [#29054](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/29054) | 🔄 In flight — hmerscher assigned; 2nd MR [gitlab!242446](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/242446) (draft) | 🔄 In flight — [gitlab!240022](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/240022) out of Draft (mwoolf); [gitlab!242446](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/242446) still Draft (hmerscher) |
_Copied from https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/1534#note_3510373104_
<!-- STATUS NOTE END -->
epic