Ordered stages across includes

Problem to solve

Further building on https://gitlab.com/gitlab-org/gitlab-ee/issues/12730, some customers have an interest in controlling the ordering of included stages. For example, stage(s) which must be executed before any developer-defined stages and/or stage(s) which must run after developer-defined stages.

Intended users

Automation developers

Further details

If there are multiple includes adding before and after steps, all added steps will be placed at the beginning or the end, but without a guarantee of a certain order within the including pipeline. For example, if a template has its own a job and includes b.yml and c.yml which define stages b and c respectively to be placed at the beginning, the actual order of the pipeline could be sequenced b, c, a or c, b, a.

Proposal

To accommodate this, two keys required_stages_before and/or required_stages_after can be added which would define a list of stages that will be prepended or appended respectively to the stages key in the resulting configuration. This allows for the addition of this feature in a backwards-compatible manner with minimal changes required.

Given the following developer-made .gitlab-ci.yml:

# some existing .gitlab-ci.yml file

stages:
  - build
  - test

And an included template .gitlab-required-ci.yml:

required_stages_before:
  - compliance

required_stages_after:
  - audit_stage

The resulting 'effective' stages: key from the merging of these two files should be rendered as follows:

stages:
  - compliance
  - build
  - test
  - audit_stage

Permissions and Security

There are no changes to the permissions system introduced via this change.

Documentation

Testing

What does success look like, and how can we measure that?

What is the type of buyer?

Links / references