Log child pipelines that would exceed the per-user pipeline rate limit
What does this MR do and why?
Child pipelines are excluded from the pipeline creation rate limits. That keeps normal fan-out working on the per-commit limit (pipelines_create, 25/min per project, user and commit), but it also means child pipelines don't count toward the per-user limit (pipelines_created_per_user, 2,000/min). The proposed change is to keep them excluded from the per-commit limit and count them toward the per-user one. This MR measures who that would affect before it's enforced.
It adds a log-only experiment behind the ci_log_child_pipeline_rate_limit derisk flag:
- A shadow rule,
pipelines_created_per_user_with_children, copies the per-user limit and uses the Labkit:logaction, so it never blocks. Chain::Limit::RateLimitcounts every pipeline on the shadow key, child pipelines included. Policy, excluded Duo workflow and lint pipelines are still skipped, sharing one exclusion check with the real limit.- When a child pipeline goes over the shadow limit, it logs
Child pipeline would exceed the per-user pipeline rate limitwith the user, namespace, root namespace, project, commit, parent pipeline and plan. The IDs use the Labkit field names (gl_user_id,gl_project_id,gl_root_namespace_id, ...). - The real counters and the pipeline outcome don't change.
Why a shadow key
Calling throttled? on the real key for child pipelines would increment it, using up the user's real allowance before anything is turned on. A separate key gives the combined count without touching the real one.
The shadow rule uses Labkit's :log action. In this code path it doesn't change behaviour: LabkitAdapter.run! returns result.exceeded?, so throttled? returns true once the shadow key is over, the same as with :limit. What keeps this from blocking is that the shadow path only ever logs. :log is there so that:
gitlab_labkit_rate_limiter_rule_evaluations_totalrecords it asresult="log"rather thanresult="block", which gives a "would have blocked" count without parsing logs.- Labkit reports
block?as false, so the rule still can't block if the adapter ever switches to it. - It's clear from the rule definition that this is a shadow.
Labkit doesn't log who went over, so the chain step logs that itself. While the flag is on, each pipeline creation makes one extra Redis increment.
References
- Rollout issue: #637468
Screenshots or screen recordings
Not applicable, no UI changes.
How to set up and validate locally
-
In the Rails console, set a low per-user limit and turn the flag on:
ApplicationSetting.current.update!(pipeline_limit_per_user: 3) Feature.enable(:ci_log_child_pipeline_rate_limit) -
In a project, push a
.gitlab-ci.ymlwith a parent job that triggers three or more child pipelines (trigger: include: child.yml). -
Run the pipeline. All child pipelines are created.
-
Check
log/application_json.logforChild pipeline would exceed the per-user pipeline rate limit.
Example line from a local GDK run (per-user limit 3, a parent with 5 child pipelines that each trigger a grandchild). All 10 trigger jobs succeeded and 9 lines like this were logged:
{
"severity": "INFO",
"time": "2026-11-02T18:21:27.568Z",
"correlation_id": "b8be926fe5c1062e8a441ab0888cb05f",
"meta.feature_category": "continuous_integration",
"meta.caller_id": "Ci::CreateDownstreamPipelineWorker",
"meta.root_caller_id": "Ci::InitialPipelineProcessWorker",
"class_name": "Gitlab::Ci::Pipeline::Chain::Limit::RateLimit",
"gl_namespace_id": 1,
"gl_root_namespace_id": 1,
"gl_project_id": 60,
"gl_user_id": 1,
"commit_sha": "b033a70fd34962d98ca44aac3792dd6046fe76c7",
"parent_pipeline_id": 2348,
"subscription_plan": "default",
"message": "Child pipeline would exceed the per-user pipeline rate limit"
}MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.