Rate limit pipeline schedule management actions
What does this MR do and why?
Adds a per-user rate limit to pipeline schedule management, counted inside the services so the REST API, the GraphQL mutations and the UI share it:
| Key | Scope | Limit |
|---|---|---|
pipeline_schedule_management |
user | pipeline_schedule_management_limit_per_user setting, default 300 per minute |
Creating, updating and deleting a schedule and taking ownership of one all
count toward the same allowance, so alternating actions cannot multiply it.
The setting lives in the existing rate_limits JSONB column, so there is no
migration; 0 disables the limit.
There is no per-project limit: with an equal default it could never fire before the per-user one, as pointed out in !267727 (comment 4204326151). The default of 300 was sized from GitLab.com traffic with headroom. The rollout issue re-checks it against two weeks of logs before the flag is enabled.
The check is a shared concern, Ci::PipelineScheduleRateLimitable, used by
CreateService, UpdateService, TakeOwnershipService and DestroyService.
As in Ci::RetryPipelineService, the attempt is counted before the permission
check so a rejected call still costs an attempt, but a caller without
permission gets the access error, not the throttle. A throttled call returns a
ServiceResponse with reason: :rate_limited. The REST endpoints answer 429
with a Retry-After header, the mutations return the message in errors (no
mutation code change was needed), and the controller shows it as a flash alert.
The REST delete endpoint also maps the service reason the way the trigger token
endpoint does: 403 for :forbidden and 400 with the standard message prefix
otherwise, as suggested in
!267321 (comment 4179362396).
The :forbidden branch is unreachable behind authorize! today and is covered
by a stubbed spec.
In the schedules UI, the delete and take-ownership handlers showed a generic
alert whenever the mutation returned errors and reported a bare error to
Sentry, so a throttled user would never have seen the throttle message. They
now show the mutation's message, as the create and update form already did, and
skip Sentry for these expected service errors, as raised in
!267727 (comment 4189611537).
The play handler is unchanged.
Behind the rate_limit_pipeline_schedule_management feature flag, off by
default, so no changelog entry.
This MR builds on the destroy service introduced in !267321 (merged), which has merged.
References
Related to https://gitlab.com/gitlab-org/gitlab/-/work_items/636908
Rollout issue: #639007
Pipeline retry precedent: !254367 (merged)
Screenshots or screen recordings
One new field in Admin > Settings > CI/CD > Continuous Integration and Deployment: Maximum pipeline schedule changes per user.
How to set up and validate locally
-
Enable the flag and lower the limit in a Rails console:
Feature.enable(:rate_limit_pipeline_schedule_management) ApplicationSetting.current.update!(pipeline_schedule_management_limit_per_user: 1) -
Create a schedule, then update it within a minute. The second call returns
429withRetry-After: 60.curl --request POST --header "PRIVATE-TOKEN: <token>" \ --data "description=a&ref=refs/heads/main&cron=0 1 * * *" \ "http://gdk.test:3000/api/v4/projects/<project_id>/pipeline_schedules" curl --request PUT --header "PRIVATE-TOKEN: <token>" --data "description=b" \ "http://gdk.test:3000/api/v4/projects/<project_id>/pipeline_schedules/<schedule_id>" -
On Build > Pipeline schedules, make two changes within a minute (for example, create a schedule and then delete it). The second shows the "requested too many times" message.
log/application_json.logrecordsPipeline schedule management rate limit exceeded.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.