.pre vs needs: [] breaks encapsulation
The Problem
The .pre
stage is guaranteed to run before any other stage, and is intended to run before any other jobs. Jobs that have needs: []
specified do not wait for any other jobs. This leads to a bit of an "unstoppable force vs. impenetrable shield" issue.
The current behavior is that needs: []
wins out and the job is scheduled in parallel with the .pre
jobs. This behavior is not unreasonable, but part of the rationale of .pre
is to create an encapsulation boundary between a main file and its includes. Ideally, a build file should not have to be aware of .pre
jobs in its includes for safe behavior.
Possible Resolutions
There are two possible solutions to this problem.
needs
Takes Priority
In this scenario, needs
is treated as a power user function and, with the usual safeguards disabled, the author needs to be aware of the entire build process and ensure that all expected dependencies are maintained. The documentation for .pre
and needs
should both probably mention this caveat.
Pros
- Keeping current behavior means no new GitLab code is needed and no changes to how existing pipelines run.
- Simpler mental model for how stages and
needs
work together.
Cons
- Maintainers of
.gitlab-ci.yaml
have to be aware of all changes to included files and how they may impact each job'sneeds
. - The safe behavior of waiting for all
.pre
jobs to finish is not default and hard to specify, especially since waiting for an entire stage is not currently possible.
.pre
Takes Priority
The .pre
stage is treated specially, and thus works as an encapsulation boundary. Since it's not unreasonable for the .gitlab-ci.yaml
file to have final say over how everything is scheduled, a new keyword is provided to allow jobs to run at the same time as jobs in .pre
.
A possible choice for the keyword would be wait_for_pre
with a default value of true
.
Pros
- The safe behavior is the default, but faster behavior is possible when the authors are sure it's safe.
Cons
- Need yet another keyword if running a job at the same time as
.pre
is desired.