Backend: needs: optional: true with include: rules is causing pipeline errors
Existing behavior
Today it's impossible to use needs:
to create a job dependency that points to a job added with include:local:rules
. When the configuration is checked for validity,
GitLab returns undefined need: <job-name>
. this issue is
to improve this behavior.
Summary
When the needs: optional
keywords are used with include: rules
, the pipeline fails with the message "child-job-name
job: undefined need: parent-job-name
". According to our docs, the needs: optional
keyword can be used to "need a job that sometimes does not exist in the pipeline". In this case, if we use the include: rules
keywords to control when a file with jobs in it gets included, the pipeline fails.
Steps to reproduce
- Create a new project.
- Add two yml files:
.gitlab-ci.yml
include:
- local: .hello-world.yml
rules:
- if: $RUN_HELLO_WORLD == "true"
depends-on-hello-world:
needs:
- job: hello-world
optional: true
script:
- echo "This job depends on Hello World!"
.hello-world.yml
hello-world:
script:
- echo "Hello World!"
- Attempt to run the pipeline by sending
RUN_HELLO_WORLD=true
, the pipeline runs. - Attempt to run the pipeline by sending
RUN_HELLO_WORLD=false
, the pipeline fails.
Example Project
Example Project: https://gitlab.com/b_freitas/needs-optional-and-include-with-rules
- Branch that reproduces the bug, using
needs:optional
withinclude:rules:if
- Branch that works, using
needs:optional
withinclude
only (withoutrules:if
underinclude
)
What is the current bug behavior?
The pipeline does not start and we get the following error message: "depends-on-hello-world job: undefined need: hello-world".
What is the expected correct behavior?
If the RUN_HELLO_WORLD variable is set to true
the pipeline should run with the two jobs.
If the RUN_HELLO_WORLD variable is set to false
the pipeline should run with one job (depends-on-hello-world) and ignore the other one (hello-world).
Relevant logs and/or screenshots
Output of checks
This bug happens on GitLab.com
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)
Possible fixes
Steps to include files and run the pipeline;
- include the YAML files (we can't include
.hello-world.yml
because of the rule) - validate all needs independently from
optional
- validate all needs with
optional
I guess we may fix this by using the optional attribute in the 2nd step. So, probably we need to adjust Gitlab::Ci::YamlProcessor
to use optional
.
Implementation
backend | MR |
---|---|
Fix YamlProcessor to allow needing an optional, undefined job | !116335 (merged) |