### Release notes Use `needs` in rules to update a job's [`needs`](#needs) for specific conditions. When a condition matches a rule, the job's `needs` configuration is completely replaced with the `needs` in the rule ### Problem to solve <details><summary>Original description</summary> <!-- What problem do we solve? --> * Introduction of Rules and Needs features provide some incredibly powerful tools. * Implementing Needs becomes a task that requires duplicating a massive number of downstream jobs if they can't be dynamically set. A single downstream, with 2 optional presteps, quickly becomes 4 individual jobs that have to be mutually exclusive. ``` deploy release production: extends: - .deploy - .release - .production deploy release production w/ prestep 1: extends: - .deploy - .release - .production - .prestep1 deploy release production w/ prestep 2: extends: - .deploy - .release - .production - .prestep2 deploy release production w/ prestep all: extends: - .deploy - .release - .production - .prestepall ``` Allowing needs to be set with rules conditions can easily cut this down to one job that is much more predictable ``` deploy release production: extends: - .deploy - .release - .production rules: # rules allows for a list of individual rule objects to be evaluated in order, # until one matches and dynamically provides attributes to the job. - if: '$DEPLOY_PRESTEP1 && $DEPLOY_PRESTEP2' needs: ["prestep1:release:production","prestep2:release:production"] when: on_success - if: '$DEPLOY_PRESTEP1' needs: ["prestep1:release:production"] when: on_success - if: '$DEPLOY_PRESTEP2' needs: ["prestep2:release:production"] when: on_success when: manual ``` **Goal:** We will allow rules to take needs which will be the exact set of keywords that we have under job's needs currently, see [doc](https://docs.gitlab.com/ee/ci/yaml/#needs). ### Intended users <!-- Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later. Personas can be found at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/ --> * DevOps Engineer - flexible yet predictable templating and abstraction ### Further details <!-- Include use cases, benefits, and/or goals (contributes to our vision?) --> * Reduce complexity and increase readability by reducing number of jobs. ### What does success look like, and how can we measure that? **Definition of done:** We can apply needs to rules with all the keywords that currently are available under job's needs. </details> <details><summary>Original proposal</summary> ### Proposal Support the following syntax ``` test: rules: - if: '$SOMETING1' needs: ["job1"] - if: '$SOMETING2' needs: ["job2"] ``` When '$SOMETHING1' is true, the `test` job `needs` will change to \["job1"\] When '$SOMETHING2' is true, the `test` job `needs` will change to \["job2"\] When neither '$SOMETHING1' or '$SOMETHING2' evaluates to true, the job wont have any needs defined ``` test: needs: ['lint'] rules: - if: '$SOMETHING1' needs: ["job1"] - if: '$SOMETHING2' needs: ["job2"] ``` When '$SOMETHING1' is true, the `test` job `needs` will change to \["job1"\] When '$SOMETHING2' is true, the `test` job `needs` will change to \["job2"\] When neither '$SOMETHING1' or '$SOMETHING2' evaluates to true, the `needs` will remain on \['lint'\] if '$SOMETHING1' is true: ``` test: needs: ['lint'] rules: - if: '$SOMETHING1' needs: - job: "job1" optional: true - if: '$SOMETHING2' needs: ["job2"] ``` it evaluates to: ``` test: needs: - job: ["job1"] optional: true ``` Rules are evaluated when the pipeline is created, and evaluated in order until the first match **Technical approach:** The job's needs logic will not be copied, but reused under rules needs so in the future any changes done to needs on the job's side, will be reflected under the rules needs and vice versa ### Permissions and Security <!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)? --> N/A </details> [MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/112725) addressing this issue. [FF](https://gitlab.com/gitlab-org/gitlab/-/issues/394769) issue. ### Updated proposal Currently we have [needs on jobs](https://docs.gitlab.com/ee/ci/yaml/#needs) eg ``` build: stage: build script: echo needs: [test] ``` In this issue, we are introducing needs on rules eg ``` lint_job: stage: lint script: 'echo lint_job' needs: [test] rules: - if: $var == null needs: [test_job] ``` or ``` lint_job: stage: lint script: 'echo lint_job' needs: [test] rules: - if: $var == null needs: - job: test_job artifacts: true optional: false ``` If in the above `lint_job` example, if the rule evaluates to true, the job's need gets overridden to `test_job`. (In the later example, test_job attributes are also set: artifacts is set to true and option is set to false, if not specified, the default value is set) **Once specific condition in the rule is met, the needs from the rule override the needs on the job.** We will also allow the following keywords in the needs: artifacts, optional. In this issue, we will not support the following options: 1. https://docs.gitlab.com/ee/ci/yaml/#needsproject -> adding dependencies to download artifacts. 2. https://docs.gitlab.com/ee/ci/yaml/#needspipeline -> provides mirroring pipeline status from an upstream pipeline to a bridge 3. https://docs.gitlab.com/ee/ci/yaml/#needspipelinejob -> also downloading artifacts from a job in its parent pipeline or another child pipeline ### Documentation <!-- See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html Add all known Documentation Requirements here, per https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html#documentation-requirements --> https://docs.gitlab.com/ee/ci/yaml/#permitted-attributes ### Testing <!-- What risks does this change pose? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? See the test engineering process for further help: https://about.gitlab.com/handbook/engineering/quality/test-engineering/ --> * Create a job who's `needs` can be controlled using variable flags. * Toggle different variable flags. Alternative testing: Create yml file with a pipeline and set rules to evaluate to true, push the code. Once the code is pushed, the yml will automatically run and the jobs needs to be evaluated with considerations on job needs. ### What does success look like, and how can we measure that? **Definition of done:** We can apply needs to rules with artifacts, optional keywords. ### Links / references https://gitlab.my.salesforce.com/0016100001beDS0