Feature Proposal: Gitlab CI: Parallel stages
Problem to solve
If a project has ci-stages, that do different things, you should obviously seperate them into different stages. Pretty often, these different stages don't have any need to be processed sequentially. This is really unneccessary. It leads to people putting things like documentation-compilation into build stages or analysis-stages, were it really doesn't belong. You should therefore have the option to specify that some stages can run in parallel.
Intended users
Types of users:
- Developers
- DevOps
- Administrators
- Anyone coming in contact with a more complex CI/CD-Pipeline
Further details
Let me give you an example, how this change could clean CI-pipelines up a lot:
- You have a linting stage for code-analysis.
- You have a Build-stage where code gets compiled.
- You also have a documentation-compilation stage were your asciidoc-documentation gets compiled and then pushed to gitlab-pages.
- You have a third stage stage that deploys your new changes to a staging environment.
At the moment this pipeline would have to look like this:
(analysis) -> (Build) -> (deploy) -> (compile_doc) -> (pages)
There should therefore be an option to specify that some stages can run in parallel, so it could look like this:
(analysis) -> (build) -> (deploy)
(compile_doc) -> (pages)
And maybe even the option to then still let specific parallel stages decide if later stages should be ran (i.e.: on build failure) so it could be used like this:
(analysis) \
+ - (deploy)
(build) /
(compile_doc) -> (pages)
This would mean that the static code analysis could be executed in parallel to the build stage, but still have the deploy stage wait for these two to succeed, so that it only deploys the changes if the analysis and the build succeeded. The Documentation could be fully seperated from this.
Proposal
This change could be implemented with something like the following syntax in .gitlab-ci.yml
:
stage-flow:
documentation:
stages:
- compile_doc
- pages
analysis:
stages:
- analysis
build:
stages:
- compile
- minify
deploy:
only_after:
- analysis
- build
stages:
- deploy
This could result in the above-mentioned pipeline.
Using the only_after
flag you can could specify the stage-paths that have to successfully finish for one path /
stage to run.
What does success look like, and how can we measure that?
Success looks great!
Success could be "measured" by checking the productivity, efficiency and clarity of users CI-pipelines. This proposal would bring a big improvement to either understandabilty or efficiency and therefore productivity of GitLab-CI pipelines.