Allow for indicating that a predecessor job is optional in a DAG
Problem to solve
We are implementing an MVC for a directed acyclic graph (DAG) execution model in GitLab CI via https://gitlab.com/gitlab-org/gitlab-ce/issues/47063#note_198535521, which introduces the following syntax for marking a job as running after another job, without waiting for its own stage:
deploy: stage: deploy needs: [test] ...
A "smart default" we chose was for anything indicated as needed by a job was in fact needed, and if the job did not exist (likely because it didn't match to the pipeline's
only/except rules), that it would also not run. However, there are some unusual cases where the intent would be:
If job _x_ exists, then wait for it, otherwise run as part of the chosen stage
This is not possible with the current syntax, and this limits usefulness of a whole DAG feature significantly. Due to other limitations teams are often forced to duplicate jobs, this in turn causes snowball effect for DAG, where there is currently no way to express
needs: [A or B]. Allowing to specify job in
needs which might not exist in the given pipeline makes gitlab-ci.yml files more concise.
We can make this easier by introducing an optional
require: false value to the
needs: syntax, which will allow you to specify that the job should go ahead if the job it refers to does not exist:
deploy: stage: deploy needs: - job: test require: false ...