Skip to content

Feature Proposal: Gitlab CI: Parallel stages

Problem to solve

If a project has ci-stages, that do different things, you should obviously seperate them into different stages. Pretty often, these different stages don't have any need to be processed sequentially. This is really unneccessary. It leads to people putting things like documentation-compilation into build stages or analysis-stages, were it really doesn't belong. You should therefore have the option to specify that some stages can run in parallel.

Intended users

Types of users:

  • Developers
  • DevOps
  • Administrators
  • Anyone coming in contact with a more complex CI/CD-Pipeline

Further details

Let me give you an example, how this change could clean CI-pipelines up a lot:

  • You have a linting stage for code-analysis.
  • You have a Build-stage where code gets compiled.
  • You also have a documentation-compilation stage were your asciidoc-documentation gets compiled and then pushed to gitlab-pages.
  • You have a third stage stage that deploys your new changes to a staging environment.

At the moment this pipeline would have to look like this:

(analysis) -> (Build) -> (deploy) -> (compile_doc) -> (pages)

There should therefore be an option to specify that some stages can run in parallel, so it could look like this:

(analysis) -> (build) -> (deploy)
(compile_doc) -> (pages)

And maybe even the option to then still let specific parallel stages decide if later stages should be ran (i.e.: on build failure) so it could be used like this:

(analysis) \
            + - (deploy)
(build)    /

(compile_doc) -> (pages)

This would mean that the static code analysis could be executed in parallel to the build stage, but still have the deploy stage wait for these two to succeed, so that it only deploys the changes if the analysis and the build succeeded. The Documentation could be fully seperated from this.

Proposal

This change could be implemented with something like the following syntax in .gitlab-ci.yml:

stage-flow:
  documentation:
    stages:
    - compile_doc
    - pages
  analysis:
    stages:
    - analysis
  build:
    stages:
    - compile
    - minify
  deploy:
    only_after:
    - analysis
    - build
    stages:
    - deploy

This could result in the above-mentioned pipeline. Using the only_after flag you can could specify the stage-paths that have to successfully finish for one path / stage to run.

What does success look like, and how can we measure that?

Success looks great!

Success could be "measured" by checking the productivity, efficiency and clarity of users CI-pipelines. This proposal would bring a big improvement to either understandabilty or efficiency and therefore productivity of GitLab-CI pipelines.