Merge `stages:` across includes
Problem to solve
When including a template with stages, the behavior is not as one would expect: the stages
key from the included template overwrites rather than merges with the including .gitlab-ci.yml
. So, if you have a .gitlab-ci.yml
which defines the following stages.
stages:
- build
- deploy
- test
build_job:
stage: build
...
And then include a file that defines a new compliance stage:
stages:
- compliance
compliance_job:
stage: compliance
The pipeline will fail because the resulting pipeline only has a compliance
stage, rather than a build
, deploy
, test
, and compliance
stage. Specifically, these (very common) scenarios are problematic in the current implementation:
-
If the include defines a
stages
key, existing pipelines will break, as YAML validation will fail if pipelines define jobs that use stage names not defined in the include. -
If the include does not define a
stages
key (or does not definestage
key for its jobs) pipelines will fail YAML validation if the developer pipelines define astages
key.
Intended users
The intended users for this iteration are GitLab system administrators and users responsible for implementing enforcement of governance policies for CI pipelines within GitLab.
Proposal
Allow the inclusion of stages from enforced templates in an additive manner, so that stages in an enforced template are combined with stages defined in other pipelines.
Referring to the example above, the resulting file would end up with build
, deploy
, test
, and compliance
stages rather than only compliance
. This behaves much more as you'd expect if you include a template that introduces a new compliance
stage.
Permissions and Security
This should follow existing permissions for includes.
Documentation
Testing
We should run the manual CI job package-and-qa
on any MR for this feature as this end-to-end test should validate if Auto DevOps is working properly.
There are edge cases in how stages can be defined that should be carefully validated. Specifically, those edge cases are
- If no
stages
are defined in.gitlab-ci.yml
, then thebuild
,test
anddeploy
are allowed to be used as job’s stage by default.- If a job doesn’t specify a
stage
, the job is assigned thetest
stage.
The implementation of this proposal should consider that existing pipelines that fall into the above two edge cases may be broken by the inclusion of a required stages
key.
What does success look like, and how can we measure that?
The implementation of this proposal is successful if a user is able to add additional stages via an include and apply it to existing pipelines with varied existing configurations without requiring any changes to existing .gitlab-ci.yml
configurations in individual developer repositories.
What is the type of buyer?
Links / references
Relevant conversation thread from gitlab-ee#8429
MRs that originally implements the feature: gitlab-ee!14011 and gitlab-ee!13923 (FE)
Group-level required pipelines: gitlab-ee#11343
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.