Enforce Free plan MCP Server throttles

Summary

Switches Free-plan MCP Server traffic from log-only counting to enforcement: requests over 60 a minute get rejected with a 429. Second of three issues; follows #631323 and precedes #631325. Tracking issue #629883 (closed), epic gitlab-org&22800, milestone 19.5.

• DRI: @terrichu
• Team Slack channel: #g_agent_execution

Note

Feature flag controls · Feature flag lifecycle

What the flags do

Flag	Effect when enabled
rate_limiter_mcp_limits_free_enforce	burst_mcp_traffic_per_user_free_plan (the :limit rule) starts matching instead of its _log twin: Free-plan MCP traffic over 60 a minute gets rejected with a 429

The Premium and Ultimate _enforce flags stay off; those plans keep logging only until #631325.

Prerequisites

• #631323 completed and the burst_mcp_traffic_per_user_free_plan_log counters reviewed, so the 60-a-minute limit is known not to reject normal Free-plan traffic
• rate_limiter_mcp_limits_free_info still enabled (from #631323); the :limit rule needs it true alongside _enforce

What could go wrong

• Matching switches from the _log rule's counter key to the :limit rule's counter key for Free-plan traffic; still one rate-limiting Redis key per requester per minute, same as during observe.
• Labkit fails open: if rule evaluation errors, the request still goes through.
• Free-plan users sending more than 60 MCP requests a minute start getting 429s. 60 a minute is the tightest limit of the three plans, so this is the plan most likely to surface a client that polls too aggressively.

Rollout

Per incremental rollout process, run in #production and cross-post to the team channel. No percentage step here: the actor is Feature.current_request, so a partial flip would reject a user on some requests and let the same user through on others, at the same request rate. Enforcement is only coherent at 0% or 100%. The flag goes straight from off to fully on.

Non-production:

/chatops gitlab run feature set rate_limiter_mcp_limits_free_enforce true --dev --pre --staging --staging-ref

Production:

/chatops gitlab run feature set rate_limiter_mcp_limits_free_enforce true

Verify

Grafana: the rate-limiting detail dashboard, Application::Labkit Rate Limiter row. The tier-aware panels added by gitlab-com/runbooks!11499 (merged) read rate_limiter="rack_request" with rule=~".*_traffic_per_.*", so the MCP rules show up as their own series with no dashboard change.

Enforcement is live, and result="block" is the 429 count. There is no dedicated per-throttle 429 counter, and gitlab_labkit_rate_limiter_enforced_total does not exist in gprd.

sum by (rule, result) (sli_aggregations:gitlab_labkit_rate_limiter_rule_evaluations_total:rate_5m{env="gprd", rate_limiter="rack_request", rule=~"burst_mcp_traffic_per_user_free_plan"})

Expect block to appear and the _log twin to fall away. The block rate should sit at or below the log rate measured in #631323 — materially higher means the flip changed which requests match, not just what happens to them.

Fail-open stays flat:

sum(gitlab_component_errors:rate_5m{env="gprd", type="rate-limiting", component="rate_limiter_checks"})

Kibana: on pubsub-rails-inf-gprd*, json.rate_limit_state carries one <limiter>:<rule>:<result> string per counted rule.

json.path:"/api/v4/mcp" and json.rate_limit_state:*burst_mcp*

A rejected request never reaches Grape, so a 429 produces no rate_limit_state entry. Use Kibana for what still gets through and Prometheus for what is blocked.

Roll back if: the fail-open rate rises, redis-cluster-ratelimiting saturation moves, or the block rate exceeds the log-mode share from #631323.

A flag flip does not reach every Puma worker at once, so give it a minute before reading counters. Observed on a GDK: straight after disabling an enforce flag, a 65-request run split 30 on the enforcing rule and 35 on the _log twin; re-run once the flag cache had settled it was 65 on the twin and nothing on the enforcing rule. A mixed first minute after each flip is expected, not a bug.

Runbook: Tier-aware throttles (labkit).

Rollback

Production:

/chatops gitlab run feature set rate_limiter_mcp_limits_free_enforce false

Non-production:

/chatops gitlab run feature set rate_limiter_mcp_limits_free_enforce false --dev --pre --staging --staging-ref

Cleanup

Remove per cleaning up:

/chatops gitlab run feature delete rate_limiter_mcp_limits_free_enforce --dev --pre --staging --staging-ref --production

Hold off until #631325 is also stable. The YAML removal for rate_limiter_mcp_limits_free_enforce happens together with the other five MCP flags in one cleanup MR, tracked on #631325.