Limit the size added by CI/CD variable expansion
What does this MR do and why?
Expanding CI/CD variable references has no bound on the size of the result. This MR limits the bytes that expansion adds to 10 MiB.
The limit applies in two places:
- Variable collections. This is one job's variables during pipeline creation. At the limit, expansion stops and the job gets a variable error that names the variable. This is the same path used for circular references.
ExpandVariables. This is used for keywords likeenvironmentandresource_group. At the limit, the value stays unexpanded. Its callers do not handle errors, so raising there would fail the request.
When file and raw references are left for the runner to expand, the reference text stays in the value, so only its length counts toward the limit.
The limit is a constant, so it applies to every variable source and every instance. 10 MiB is about five times the environment size a job can receive on most Linux runners. Plain variables over 128 KiB can't reach a job there, so the limit can only affect file type variables, plain variables that reference large file type variables, or values used in rules. The ci_variable_expansion_size_limit beta flag, enabled by default, turns the limit off if that blocks valid pipelines.
Resolves https://gitlab.com/gitlab-org/gitlab/-/issues/605500
How to set up and validate locally
-
Set
Gitlab::Ci::Variables::Collection::MAX_EXPANDED_SIZEto10. -
Add this to
.gitlab-ci.yml:variables: A: "$B$B" B: "abcdef" test: script: echo -
Run a pipeline. Creation fails with
expanded variables exceed the limit of 10 B at variable `A`. -
Disable
ci_variable_expansion_size_limitand run it again. The pipeline is created.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.