<!--- Please read this! Before opening a new issue, make sure to search for keywords in the issues filtered by the "regression" or "type::bug" label: - https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression - https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=type::bug and verify the issue you're about to submit isn't a duplicate. ---> ### Summary When a job has `when: on_failure`, it [should run](https://docs.gitlab.com/ee/ci/yaml/#when) when at least one other job in the same pipeline fails. When the job that has `when: on_failure` also has `needs`, the job is unexpectedly **skipped** when other jobs in the same pipeline fail. If the `needs` are removed: the `when: on_failure` job works properly: it runs when other jobs in the same pipeline fail. ### Steps to reproduce <!-- Describe how one can reproduce the issue - this is very important. Please use an ordered list. --> 1. Use a `.gitlab-ci.yml` file like the one shown below 2. Observe that the `build` job fails 3. Observe that the `rollback` job is skipped (The `rollback` job should run because `build` failed.) <summary>Expand to see the.gitlab-ci.yml</summary> <details> <pre> build_job: stage: build script: - exit 1 test_job: stage: test script: - date rollback_job: stage: deploy needs: - job: test_job optional: true - job: build_job optional: true script: - date when: on_failure </pre> </details> ### Proposal The reason is that we are skipping the job if it is a DAG job and needs any `skipped` or `ignored` job; The below condition should be modified to accommodate this scenario for when it occurs. https://gitlab.com/gitlab-org/gitlab/-/blob/13a24803c5569aea2e62b439991b7a99f4334e50/lib/gitlab/ci/status/composite.rb#L35-37 ```ruby if @dag && any_skipped_or_ignored? # The DAG job is skipped if one of the needs does not run at all. 'skipped' ``` ### Example Project This unexpected behavior can be observed in the :globe_with_meridians: **Public** [gitlab-gold/briecarranza/issues/when-on-failure](https://gitlab.com/gitlab-gold/briecarranza/issues/when-on-failure) project. ### What is the current *bug* behavior? <!-- Describe what actually happens. --> A job with `when: on_failure` is skipped when it contains `needs` and at least one job in the pipeline has failed.  ### What is the expected *correct* behavior? A job with `when: on_failure` and `needs` should **run** when at least one other job in in the pipeline has failed.  The screenshot above shows what things should look like. Removing the `needs` altogether permits things to look like the screenshot above. ### Relevant logs and/or screenshots <!-- Paste any relevant logs - please use code blocks (```) to format console output, logs, and code as it's tough to read otherwise. --> ### Output of checks <!-- If you are reporting a bug on GitLab.com, uncomment below --> <!-- This bug happens on GitLab.com --> <!-- /label ~"reproduced on GitLab.com" --> #### Results of GitLab environment info <!-- Input any relevant GitLab environment information if needed. --> <details> <summary>Expand for output related to GitLab environment info</summary> <pre> (For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`) </pre> </details> #### Results of GitLab application Check <!-- Input any relevant GitLab application check information if needed. --> <details> <summary>Expand for output related to the GitLab application check</summary> <pre> (For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:check SANITIZE=true`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true`) (we will only investigate if the tests are passing) </pre> </details> ### Possible fixes <!-- If you can, link to the line of code that might be responsible for the problem. --> ### Possible Workarounds - Remove the `needs` from the `rollback` job completely - This may not be feasible for some environments. - Possibly: move the jobs that _may_ fail to the another stage in the pipeline - I wrote "sometimes" in the issue title because there is one specific set of circumstances I have identified thus far where the presence of `when: on_failure` and optional `needs` do work as expected. See [this example pipeline](https://gitlab.com/gitlab-gold/briecarranza/issues/when-on-failure/-/pipelines/755503330). A few more thoughts on this: Observe that the optional `needs` job fails. The [documentation on `needs:optional`](https://docs.gitlab.com/ee/ci/yaml/#needsoptional) notes: > To need a job that sometimes does not exist in the pipeline, add `optional: true` to the `needs` configuration. That sounds like it's about the absence or presence of the needed job and not about the success or failure of the job. - [ ] Is the thought above right? It is not possible to use `allow_failure` to work around this because we also note in the docs: > - If `allow_failure: true` is set, the job is always considered successful, and later jobs with [`when: on_failure`](https://docs.gitlab.com/ee/ci/yaml/#when) don’t start if this job fails.e