Allow for indicating that a predecessor job is optional in a DAG
Release post
With the implementation of our directed acyclic graph (DAG) in GitLab CI we introduce a syntax that makes a job run only if a different job finish its run (without waiting to its own stage). however if the DAG syntax is combined with rules syntax - which detairmine if a job will gets added to a pipeline there can be a situation where a job is waiting for a job that does not exists in the pipeline (because it did not got added to the pipeline), this will cause the pipeline to fail only on run. In this release we are adding the optional keyword to any DAG job which will process the pipline with the condition that If the optional job exists, then wait for it, otherwise run as part of the chosen stage, this will allow you to saifly combined the very popular rules syntax with the growing popularity of the DAG concept
Problem to solve
We are implementing an MVC for a directed acyclic graph (DAG) execution model in GitLab CI via https://gitlab.com/gitlab-org/gitlab-ce/issues/47063#note_198535521, which introduces the following syntax for marking a job as running after another job, without waiting for its own stage:
deploy:
stage: deploy
needs: [test]
...A "smart default" we chose was for anything indicated as needed by a job was in fact needed, and if the job did not exist (likely because it didn't match to the pipeline's only/except rules), that it would also not run. However, there are some unusual cases where the intent would be:
If job _x_ exists, then wait for it, otherwise run as part of the chosen stage
This is not possible with the current syntax, and this limits usefulness of a whole DAG feature significantly. Due to other limitations teams are often forced to duplicate jobs, this in turn causes snowball effect for DAG, where there is currently no way to express needs: [A or B]. Allowing to specify job in needs which might not exist in the given pipeline makes gitlab-ci.yml files more concise.
Intended users
Further details
Proposal
We can make this easier by allowing you to specify that the job should go ahead if the job it refers to does not exist:
job1:
needs: [a, b, c, { job: d, optional: true }, e, f]The default value, when not explicitly overridden, is false (which is the current behavior today).
Note that this should work in concert with #233876 in cases where both are provided.