[FF] rate_limit_job_cancel -- Rate limit job cancellations per user

Summary

Roll out the feature currently behind the rate_limit_job_cancel feature flag.

  • DRI: @srajadas
  • Team Slack channel: #g_pipeline-execution

Note

Process and guidance live in the docs — this issue is just the commands and a place to track the rollout. "Rolling out" means incrementally enabling the flag on GitLab.com to validate stability — it is not the same as releasing the feature, which happens when the flag is removed. Feature flag controls · Feature flag lifecycle

What could go wrong?

Legitimate heavy users of job cancellation (scripts that cancel many jobs) get HTTP 429 once they exceed 5 cancels per job per minute, or the per-project limit. Watch the 429 rate on the jobs cancel endpoint and the Job cancel rate limit exceeded log message.

Events

Feature Flag events are only logged by default for feature flags marked for the current or future milestones. To enable while the feature flag is active, see https://docs.gitlab.com/development/feature_flags/#logging

Rollout

Run all production /chatops in #production and cross-post the results to #g_pipeline-execution. Background: incremental rollout process, feature actors.

Non-production

/chatops gitlab run feature set rate_limit_job_cancel 50 --actors --dev --pre --staging --staging-ref
/chatops gitlab run feature set rate_limit_job_cancel true --dev --pre --staging --staging-ref

Production — percentage rollout (wait ≥15 min between steps, watch dashboards):

/chatops gitlab run feature set rate_limit_job_cancel 25 --actors

Or target specific actors instead:

/chatops gitlab run feature set --project=gitlab-org/gitlab,gitlab-org/gitlab-foss rate_limit_job_cancel true
/chatops gitlab run feature set --group=gitlab-org,gitlab-com rate_limit_job_cancel true
/chatops gitlab run feature set --user=srajadas rate_limit_job_cancel true

Before global rollout

Confirm the relevant gotchas before going to 100% — see enabling a feature for GitLab.com:

Cleanup

Remove the flag once deemed stable — see cleaning up. Track it here, or open a follow-up Feature Flag Cleanup issue. Remove the flag and its YAML definition from the codebase, then:

/chatops gitlab run release check https://gitlab.com/gitlab-org/gitlab/-/merge_requests/259438 19.5
/chatops gitlab run feature delete rate_limit_job_cancel --dev --pre --staging --staging-ref --production

Rollback

/chatops gitlab run feature set rate_limit_job_cancel false                                         # production
/chatops gitlab run feature set rate_limit_job_cancel false --dev --pre --staging --staging-ref     # non-production
/chatops gitlab run feature delete rate_limit_job_cancel --dev --pre --staging --staging-ref --production  # remove entirely