Parent epic: [gitlab-com/gl-infra#2021](https://gitlab.com/groups/gitlab-com/gl-infra/-/work_items/2021) (Phase 2: Rate Limiting Simplification) Repo: [gitlab-com/runbooks](https://gitlab.com/gitlab-com/runbooks) Target file: [`dashboards/rate-limiting/main.dashboard.jsonnet`](https://gitlab.com/gitlab-com/runbooks/-/blob/master/dashboards/rate-limiting/main.dashboard.jsonnet) # [Spec 12] runbooks: Update Rate Limiting Overview dashboard for labkit metrics ## Spec ### Problem Statement [required] The "Rate Limiting: Overview" dashboard at [`dashboards/rate-limiting/main.dashboard.jsonnet`](https://gitlab.com/gitlab-com/runbooks/-/blob/master/dashboards/rate-limiting/main.dashboard.jsonnet) is the operator-facing view of every rate-limiting subsystem in the SaaS stack. Today it has six sections — Cloudflare, HAProxy, Workhorse, RackAttack, ApplicationRateLimiter — but **no section for the labkit-based rate limiter** that is now the canonical migration target for application-level rate limiting (Phase 2, [#2021](https://gitlab.com/groups/gitlab-com/gl-infra/-/work_items/2021)). This is already a problem in production: Cohort 1 ([#28803](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28803), closed 2026-05-05) is at 100% rollout on .com, so five `ApplicationRateLimiter` keys are already running through `Labkit::RateLimit` and emitting the labkit metrics defined by [#28798](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28798) — but operators looking at the overview dashboard see no panels for them. The signal is invisible at the dashboard layer until queried directly in Grafana Explore. A second gap surfaces during the Stage 2b parallel run ([#28852](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28852), [Spec 9]): the labkit rack middleware will run alongside RackAttack in shadow mode for protected paths, and the success criterion for promoting beyond shadow is divergence between the two systems' decisions. There is no panel today that lets an operator compare labkit `action="block"` against `gitlab_rack_attack_events_total{event_type="throttle"}` side-by-side. A third gap is forward-looking: once the migration is fully complete, the legacy `gitlab_application_rate_limiter_throttle_utilization_ratio` histogram becomes redundant. The dashboard's existing ApplicationRateLimiter panel uses a histogram-bucket-subtraction trick to compute the throttled rate (`rate(bucket{le="+Inf"}) - rate(bucket{le=~"1|1\\.0"})`); with `gitlab_labkit_rate_limiter_calls_total{action="block"}` we get the same signal as a direct counter. The histogram retirement should be sequenced with the dashboard switchover so we don't drop the panel before its replacement is in place, and so we don't pay the histogram's `peek × throttle_key × feature_category × bucket` cardinality cost forever. ### Non-Goals [required] - **The auto-generated SLI dashboard** at `dashboards/labkit-rate-limit/main.dashboard.jsonnet` — that file is produced by `make generate` from the metrics-catalog service registration in [#28832](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28832) ([Spec 11]). This spec only edits the bespoke `dashboards/rate-limiting/main.dashboard.jsonnet`. The new section's intro panel links out to the SLI dashboard; it does not duplicate the SLI threshold/burn-rate panels. - **Defining or changing the SLI threshold** — owned by [#28832](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28832). - **Adding or modifying labkit metric emission** — owned by [#28798](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28798) (closed) for the four current metrics. If a new metric is needed (e.g. utilization-ratio histogram), it must be specified and merged in labkit-ruby first; this issue does not introduce metrics. - **Per-request log enrichment** — owned by [#28799](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28799). - **Migrating any ApplicationRateLimiter call sites** — owned by Stage 2a Cohort issues ([#28808](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28808) umbrella, with [#28803](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28803), [#28809](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28809), [#28810](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28810), [#28811](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28811), [#28812](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28812) as cohorts). - **Migrating any RackAttack throttles** — owned by [#28852](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28852) ([Spec 9]). - **Cohort 1 feature-flag cleanup** — owned by [#28876](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28876). - **labkit-go observability** — separate iteration; `gitlab_labkit_rate_limiter_*` metrics emitted by labkit-go are out of scope for the panels this spec adds. - **Dashboard alerting** — operator-facing visualisation only; alerting on the labkit-rate-limit SLI is owned by [#28832](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28832)'s metrics-catalog service definition. ### Acceptance Criteria [required] The work breaks into three phases. **Phases 1 and 2 ship together in the same MR.** Phase 3 is a follow-up MR gated on a documented precondition; the spec captures that gate so the work is not lost. #### Phase 1 — Add the Application::Labkit Rate Limiter section **Scenario 1: Section header and intro panel exist** - Given: the current `dashboards/rate-limiting/main.dashboard.jsonnet` with six sections - When: the MR is merged and the dashboard is regenerated by the runbooks pipeline - Then: the dashboard has a seventh section with `row.new(title='Application::Labkit Rate Limiter')`, positioned at the bottom of the dashboard *after* `Application::ApplicationRateLimiter`. (Earlier drafts suggested inserting between `Application::Rack Attack` and `Application::ApplicationRateLimiter` for narrative reasons, but every existing panel uses absolute `gridPos.y` coordinates and an in-the-middle insert requires re-flowing all subsequent `y` values for no functional gain. Appending preserves the legacy-systems-above / migration-target-below ordering.) The section's first panel is a markdown intro that names the section, links the parent epic [#2021](https://gitlab.com/groups/gitlab-com/gl-infra/-/work_items/2021), links the auto-generated SLI dashboard at `dashboards/labkit-rate-limit/main.dashboard.jsonnet` (owned by [#28832](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28832)), and lists the four metrics the section queries with one-line descriptions. - The dashboard's `tags` array now includes `'labkit-rate-limit'` alongside the existing `'rate-limit'`, `'rack-attack'`, `'application-rate-limiter'`, etc. **Scenario 2: Calls timeseries panel surfaces rate × rule × action** - Given: a labkit-emitting limiter is active in production (Cohort 1 is at 100%, so this holds today) - When: an operator opens the dashboard for `env=gprd` - Then: the section contains a timeseries panel titled "Labkit rate limiter calls (per-second, by rule and action)" whose query uses the recording-rule variant produced by [#28832](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28832) (Spec 11): ```promql sum by (rate_limiter, rule, action) ( sli_aggregations:gitlab_labkit_rate_limiter_calls_total:rate_5m{env="$environment"} ) ``` The legend renders as `{{rate_limiter}} / {{rule}} / {{action}}`. Three actions render as distinct colours: `allow` (green), `log` (yellow), `block` (red). The `rule="unmatched"` series (emitted by the evaluator when no rule matches, always paired with `action="allow"`) is **not** filtered out — operators must be able to see traffic that no rule covered. Querying the recording-rule variant rather than the raw counter is much cheaper on Mimir; it reads empty until [runbooks!10605](https://gitlab.com/gitlab-com/runbooks/-/merge_requests/10605) lands and `make generate` produces the recording-rule files (per @reprazent's review on [runbooks!10606](https://gitlab.com/gitlab-com/runbooks/-/merge_requests/10606)). **Scenario 3: Errors are owned by the auto-generated SLI dashboard** - Given: the labkit error counter is incremented (e.g. Redis cluster instability) - When: an operator opens the dashboard - Then: this dashboard does **not** render its own errors timeseries; instead, the section's intro markdown links out to the auto-generated SLI dashboard at `dashboards/labkit-rate-limit/main.dashboard.jsonnet` (owned by [#28832](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28832)) which shows errors in proper SLO / burn-rate context. Errors are rare given Redis reliability and don't warrant duplicated real-estate on the bespoke overview dashboard. (Earlier drafts of this spec had a dedicated errors timeseries here; removed per @reprazent's review on [runbooks!10606](https://gitlab.com/gitlab-com/runbooks/-/merge_requests/10606).) **Scenario 4: Configured limits and periods rendered as a single table** - Given: the labkit gauges `gitlab_labkit_rate_limiter_limit` and `gitlab_labkit_rate_limiter_period_seconds` are set per matched rule (Cohort 1 sets them today) - When: an operator opens the dashboard - Then: the section contains a single Grafana **table** panel titled "Labkit rate limits and periods" with one row per `(rate_limiter, rule)` pair and three value columns: `Rate limiter` / `Rule` / `Limit` / `Period (seconds)`. The two underlying queries are issued in `format=table, instant=true` mode and combined via Grafana's `merge` + `organize` transformations (mirrors the existing pattern at [`dashboards/wiz-runtime-sensor/global-wiz-sensor.dashboard.jsonnet`](https://gitlab.com/gitlab-com/runbooks/-/blob/master/dashboards/wiz-runtime-sensor/global-wiz-sensor.dashboard.jsonnet) lines 94–117): - Limit: `max by (rate_limiter, rule) (gitlab_labkit_rate_limiter_limit{env="$environment"})` - Period: `max by (rate_limiter, rule) (gitlab_labkit_rate_limiter_period_seconds{env="$environment"})` - `max` aggregation matches the multiprocess gauge mode (`:max`) used by labkit-ruby. (Earlier drafts of this spec had two side-by-side stat panels; combined into one table per @reprazent's review on [runbooks!10606](https://gitlab.com/gitlab-com/runbooks/-/merge_requests/10606) — stat panels don't scale once the migration adds many `(rate_limiter, rule)` pairs.) #### Phase 2 — Comparison panel for the parallel-run **Scenario 5: Comparison panel exists, reads empty until Stage 2b ships, and is explicitly scoped to enforced-vs-enforced parity** - Given: the new section - When: the MR is merged - Then: the section contains a final timeseries panel titled "Labkit `block` vs RackAttack `throttle` (enforced parity, parallel-run validation)" plotting two series — the labkit-side rack limiters introduced by Stage 2b ([#28852](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28852)) named `rack_request` and `rack_request_protected_paths`: ```promql # series A — labkit enforced (action="block") rate, scoped to the rack-side limiters Stage 2b introduces sum(rate( gitlab_labkit_rate_limiter_calls_total{ env="$environment", rate_limiter=~"rack_request|rack_request_protected_paths", action="block" }[$__rate_interval] )) # legend: "labkit block" # series B — RackAttack enforced (event_type="throttle") rate sum(rate( gitlab_rack_attack_events_total{env="$environment", event_type="throttle"}[$__rate_interval] )) # legend: "rack_attack throttle" ``` - The panel description explicitly states two things: 1. **What it compares**: enforced decisions only — labkit `action="block"` against RackAttack `event_type="throttle"`. These are the two cases where each system actually returns 429 to the caller. 2. **What it does NOT compare**: shadow / dry-run decisions (labkit `action="log"` vs RackAttack `event_type="track"`), bypass decisions (labkit's `bypass_header_present` rule emitting `action="log"` vs RackAttack `event_type="safelist"`), and RackAttack `event_type="blocklist"` (which has no labkit equivalent). Shadow-side and dry-run divergence is tracked separately via the `rate_limit_shadow_divergence` log line + counter introduced by [#28852](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28852); operators investigating divergence should consult both. - The panel description also states: this panel will read empty for the labkit series until the Stage 2b shadow rollout emits these limiters in production. **Empty panel is the expected state at merge time** — documented in the panel description so future readers don't think the dashboard is broken. #### Phase 3 — Retire the histogram once the migration is complete (gated follow-up) This phase is **gated** and not landed in the same MR as Phases 1 + 2. It executes only when **both** preconditions hold; the spec captures the gate so the work is not lost. **Preconditions (verified in the follow-up MR description):** - All Cohort issues — Cohort 1 [#28803](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28803), Cohort 2 [#28809](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28809), Cohort 3 [#28810](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28810), Cohort 4 [#28811](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28811), Cohort 5 [#28812](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28812), umbrella [#28808](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28808) — are CLOSED. - All migration feature flags removed (Cohort 1: [#28876](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28876); equivalent for Cohorts 2–5). **Scenario 6: ApplicationRateLimiter throttled-rate panel switched to labkit counter** - Given: the existing panel "ApplicationRateLimiter rate of throttled requests by key" using `rate(gitlab_application_rate_limiter_throttle_utilization_ratio_bucket{le="+Inf"}) - rate(...{le=~"1|1\\.0"})` - When: the follow-up MR lands (preconditions hold) - Then: the panel's query is replaced with `sum by (rate_limiter, rule) (rate(gitlab_labkit_rate_limiter_calls_total{env="$environment", action="block"}[$__rate_interval]))`. The panel is renamed to "Labkit-replaces-ApplicationRateLimiter throttled rate" and its description records the cutover date. **Scenario 7: Histogram emission removed from ApplicationRateLimiter** - Given: `Gitlab::ApplicationRateLimiter` still emits `gitlab_application_rate_limiter_throttle_utilization_ratio` after Cohort cutover - When: the follow-up MR lands (preconditions hold) - Then: the histogram registration and `observe` call are removed from `lib/gitlab/application_rate_limiter.rb` in the gitlab-org/gitlab repo. The runbooks-side dashboard panel switched in Scenario 6 has been deployed to production at least 7 days prior, so we have rolling-window confidence the new panel renders correctly. The cardinality cost reclaimed is captured in the MR description (`peek × throttle_key × feature_category × bucket-count`). **Scenario 8: ApplicationRateLimiter section rendered or removed cleanly** - Given: the histogram is no longer emitted - When: a user opens the dashboard - Then: either the entire `Application::ApplicationRateLimiter` section is removed (because every panel in it queried the histogram and is now redundant with the labkit section), OR the section retains an explicit migration note and points users at the labkit section. The follow-up MR description states which option was chosen and why. **No panel reads "No data" indefinitely** — that is a regression. #### Edge / failure scenarios across all phases **Scenario 9: Dashboard renders cleanly when labkit metrics have not yet been scraped** - Given: a hypothetical environment where labkit is not yet emitting (e.g. a fresh cell, or the scrape endpoint is briefly unavailable) - When: an operator opens the dashboard - Then: the new section renders without Jsonnet/Grafana errors, every panel reads "No data" without runtime errors, and the markdown intro panel is unchanged. The dashboard is not broken by the absence of data. **Scenario 10: New `rate_limiter` / `rule` label values appear without a code change** - Given: a future Cohort or Stage 2b adapter introduces a new `Labkit::RateLimit::Limiter.new(name: "<new>")` value - When: that limiter starts emitting metrics in production - Then: existing panels surface the new label values automatically — queries are `by (rate_limiter, rule, action)`-aggregated and nothing in the dashboard hardcodes label values. **No dashboard MR is required to onboard new limiters or rules.** (The comparison panel in Scenario 5 is the explicit exception: it filters on `rate_limiter=~"rack_request|rack_request_protected_paths"` because that scope is intentionally narrow for parallel-run validation, and is updated under [#28852](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28852) if Stage 2b adds further rack-side limiters.) ### Security Considerations [required] - **Bypass risk**: none introduced. This issue is read-only observability; no rate-limit decisions, configuration, or code paths are modified. - **Config injection**: no admin or user input flows through the dashboard. PromQL is statically defined in jsonnet at build time. - **Exposure of limit values**: the `gitlab_labkit_rate_limiter_limit` and `gitlab_labkit_rate_limiter_period_seconds` gauges are already emitted to Prometheus in production today (per [#28798](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28798)). The dashboard surfaces what is already collected; it does not widen the trust boundary. Per the Phase 2 epic, dashboards remain gated by existing Grafana access controls (production engineering / infra / on-call). No new access path. - **Cardinality cost**: Phase 1 + 2 add queries against existing metrics and **do not** change the underlying metric emission; cardinality is unchanged. Phase 3 *reduces* cardinality by retiring the histogram (`peek × throttle_key × feature_category × [0.25, 0.5, 0.75, 1.0]` buckets) once its replacement is proven equivalent. - **Privilege escalation via config changes**: not applicable. ### Rollout & Backwards Compatibility [required] **Self-managed / Dedicated / Cells**: the runbooks repo and its generated Grafana dashboards apply only to GitLab.com (gprd, gstg, ops). Self-managed, Dedicated, and Cells use their own observability stacks; this dashboard change has no impact on them. The labkit metrics emitted by gitlab-rails are visible to self-managed Prometheus operators via the same metric names, but the bespoke dashboard is GitLab.com-only. **Phase 1 + 2 rollout**: pure dashboard config change. Merged to runbooks `master`; the dashboard pipeline regenerates and deploys to `dashboards.gitlab.net` automatically. **No feature flag** — dashboard changes are not user-facing in the application sense and have no runtime risk other than the dashboard itself. Rollback is `git revert` the runbooks MR. **Phase 3 rollout (the gated follow-up)**: 1. **Step a (runbooks)**: switch the ApplicationRateLimiter throttled-rate panel to query `gitlab_labkit_rate_limiter_calls_total{action="block"}`. Deploy to `dashboards.gitlab.net`. **Wait at least 7 days** to confirm the new panel renders the same shape as the histogram-subtraction panel did (cross-checked via Grafana Explore on overlapping data). 2. **Step b (gitlab-rails)**: remove the histogram emission from `lib/gitlab/application_rate_limiter.rb`. This is a metric-removal, not a behaviour change — the application's rate-limiting logic is unchanged. The 7-day stagger is the rollback window: if the labkit-side panel turns out to undercount or miscount, we revert step (a) before doing step (b) and the histogram is still emitted. **Rollback**: - Phase 1 + 2: `git revert` the runbooks MR; the dashboard regenerates without the section. No data is lost; the labkit metrics keep being emitted. - Phase 3: revert the runbooks panel switch first, then re-add the histogram emission to gitlab-rails if the runbooks revert is insufficient. The runbooks revert is fast; the gitlab-rails revert takes a deployment cycle. ### Validation Loop / Verification Process [required] **Local commands that must pass before requesting human review:** ```bash cd ~/projects/runbooks # 1. jsonnet formatted, dashboards regenerate cleanly, jsonnet tests pass. make verify-fmt # jsonnetfmt parity check make generate # rebuilds all generated dashboards make test # runs the runbooks jsonnet test suite # 2. Diff inspection git diff --stat dashboards/ # confirms only main.dashboard.jsonnet (Phase 1+2) # or main.dashboard.jsonnet + ApplicationRateLimiter section (Phase 3) ``` PromQL syntactic validation is not performed at the `Makefile` layer in `gitlab-com/runbooks` (the repo lints recording rules via `validate-prom-rules` and Mimir rules via `validate-mimir-rules`, not bespoke dashboard PromQL). Dashboard queries are validated by Prometheus at panel-render time during the live-data verification step below. **Live-data verification (post-merge, before declaring done):** For Phases 1 + 2, the operator opens the deployed dashboard for `env=gprd` and confirms: - The new "Application::Labkit Rate Limiter" section is visible at the expected position. - The "Labkit rate limiter calls" timeseries shows non-zero traffic on the Cohort 1 `rate_limiter` label values currently in production. (If no series renders, the metrics emission or scraping is broken — escalate.) - The errors panel reads ~0 in steady state. The `limit` and `period_seconds` tables show the configured limits per `(rate_limiter, rule)` pair. - The comparison panel reads `0` for the labkit series and the existing rate for the `rack_attack` series — the expected state pre-Stage 2b. - Each panel's "Test panel" / "Open in Explore" action returns a successful PromQL evaluation. For Phase 3, the verification is the **7-day soak** comparing labkit-side and histogram-side panels on a per-`rate_limiter` / per-`throttle_key` basis; rates must agree to within ±5% over a 7-day window. **Evidence to post to the MR**: screenshots of the new section in `dashboards.gitlab.net/dashboards/...` for Phase 1 + 2; the 7-day comparison Grafana panel link for Phase 3. ### Observability [optional] This *is* the observability spec — it adds the operator-facing view. The metric definitions themselves are owned upstream: - Counter `gitlab_labkit_rate_limiter_calls_total{rate_limiter, rule, action}` — labkit-ruby `lib/labkit/rate_limit/metrics.rb`, populated by the Evaluator per check (one of `allow|block|log`; `rule="unmatched"` when no rule matched). - Counter `gitlab_labkit_rate_limiter_errors_total{rate_limiter}` — labkit-ruby, populated on Redis exceptions before any rule resolves (so no `rule` label). - Gauge `gitlab_labkit_rate_limiter_limit{rate_limiter, rule}` — labkit-ruby, multiprocess mode `:max`, populated when a rule matches. - Gauge `gitlab_labkit_rate_limiter_period_seconds{rate_limiter, rule}` — same as above. For incident triage, the section's intro markdown panel includes: - Link to the SLI dashboard at `dashboards/labkit-rate-limit/main.dashboard.jsonnet` for SLO / burn-rate context (owned by [#28832](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28832)). - Link to [#2021](https://gitlab.com/groups/gitlab-com/gl-infra/-/work_items/2021) for the migration cohort status — answers "which `rate_limiter` values exist today?". - Link to [#28799](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28799) (or its merged MR once shipped) describing the per-request log fields that match each panel's labels for ad-hoc Kibana / Elasticsearch correlation. ## References - Epic: [gitlab-com/gl-infra#2021](https://gitlab.com/groups/gitlab-com/gl-infra/-/work_items/2021) - Upstream metrics emission (closed; defines the four metrics this dashboard plots): [#28798](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28798) - Metrics-catalog service registration / SLI dashboard owner: [#28832](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28832) ([Spec 11]) - Stage 2b RackAttack migration (consumer of the comparison panel): [#28852](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28852) ([Spec 9]) - Cohort 1 (currently emitting in production): [#28803](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28803) (closed) - Cohort 2–5 (will populate panels as they roll out): [#28809](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28809), [#28810](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28810), [#28811](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28811), [#28812](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28812) - Cohort 1 FF cleanup (precondition for Phase 3): [#28876](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28876) - Per-request log enrichment (cross-referenced from the intro panel): [#28799](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/28799) - Existing dashboard target file: [`dashboards/rate-limiting/main.dashboard.jsonnet` in gitlab-com/runbooks](https://gitlab.com/gitlab-com/runbooks/-/blob/master/dashboards/rate-limiting/main.dashboard.jsonnet)