Add rate limit logging to budget cap mutations

What does this MR do and why?

Adds rate limit logging to Mutation.upsertUserBudgetCapOverrides and Mutation.upsertFlatUserCap.

Both mutations share one set of counters. Each call is counted per user in 3 windows (minute, hour, day). The log line also records the root namespace.

A call over a threshold is logged as Budget cap rate limit exceeded with throttled: false in additional_details, and still succeeds. The call is only rejected when the enforce_budget_cap_rate_limits feature flag (gitlab_com_derisk, actor: root namespace) is enabled. The flag is off by default.

The check runs after authorization, so unauthorized calls are not counted.

The mutations use the rate limit concern from !267552 (merged), with their own keys and enforce flag.

Thresholds

Window Threshold
minute 150
hour 1,000
day 20,000

The thresholds are based on production usage.

The daily window is a fixed window. If enforcement is enabled, a caller over the daily threshold is rejected until the next UTC day.

Rollout

  1. Merge with the flag off. Calls over a threshold are logged.
  2. Review the logs and gitlab_labkit_rate_limiter_rule_evaluations_total{rule=~"limit_budget_cap_updates_.*", result="block"} for 1 week.
  3. Adjust any threshold that a legitimate caller would exceed.
  4. Decide on enforcement in the rollout issue. Rollback: disable the flag.

Self-managed: after the gitlab_com_derisk flag is removed, the same thresholds apply to self-managed instances. No normal instance gets near them.

References

How to set up and validate locally

  1. In ee/lib/ee/gitlab/application_rate_limiter/labkit_adapter/supported_rate_limits.rb, set the budget_cap_update_per_minute limit to 2 and restart Rails.

  2. As a group owner on a SaaS namespace with GitLab Credits (GITLAB_SIMULATE_SAAS=1), run this mutation 3 times in GraphiQL:

    mutation {
      upsertFlatUserCap(input: { namespacePath: "your-group", flatUserCap: 100, flatUserCapEnabled: true }) {
        errors
      }
    }
  3. The third call still runs, and log/application_json.log contains Budget cap rate limit exceeded with additional_details exceeded_rate_limits: 'budget_cap_update_per_minute', throttled: false.

  4. In the Rails console, run Feature.enable(:enforce_budget_cap_rate_limits).

  5. Run the mutation again. It returns the throttled error.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

Edited by Roy Zwambag

Merge request reports

Loading
Loading