Merge `stages:` across includes
Problem to solve
When including a template with stages, the behavior is not as one would expect: the
stages key from the included template overwrites rather than merges with the including
.gitlab-ci.yml. So, if you have a
.gitlab-ci.yml which defines the following stages:
stages: - build - deploy - test build_job: stage: build ...
And then include a file that defines a new compliance stage:
stages: - compliance compliance_job: stage: compliance
The pipeline will fail because the resulting pipeline only has a
compliance stage, rather than a
compliance stage. Specifically, these (very common) scenarios are problematic in the current implementation:
If the include defines a
stageskey, existing pipelines will break, as YAML validation will fail if pipelines define jobs that use stage names not defined in the include.
If the include does not define a
stageskey (or does not define
stagekey for its jobs) pipelines will fail YAML validation if the developer pipelines define a
The intended users for this iteration are GitLab system administrators and users responsible for implementing enforcement of governance policies for CI pipelines within GitLab.
Allow the inclusion of stages from enforced templates in an additive manner, so that stages in an enforced template are combined with stages defined in other pipelines.
Referring to the example above, the resulting file would end up with
compliance stages rather than only
compliance. This behaves much more as you'd expect if you include a template that introduces a new
Permissions and Security
This should follow existing permissions for includes.
We should run the manual CI job
package-and-qa on any MR for this feature as this end-to-end test should validate if Auto DevOps is working properly.
There are edge cases in how stages can be defined that should be carefully validated. Specifically, those edge cases are
- If no
stagesare defined in
.gitlab-ci.yml, then the
deployare allowed to be used as job’s stage by default.
- If a job doesn’t specify a
stage, the job is assigned the
The implementation of this proposal should consider that existing pipelines that fall into the above two edge cases may be broken by the inclusion of a required
What does success look like, and how can we measure that?
The implementation of this proposal is successful if a user is able to add additional stages via an include and apply it to existing pipelines with varied existing configurations without requiring any changes to existing
.gitlab-ci.yml configurations in individual developer repositories.
What is the type of buyer?
Links / references
Group-level required pipelines: gitlab-ee#11343