Introduce nullable input types

Problem statement

CI inputs currently do not accept null. All inputs types require that the value match the type and are non-nullable. However, users have brought up several cases where they would like to use null value.

Original description provided by user

In YAML there is a null type which is used in GitLab CI to indicate the absence of a value for a node - the null value can also be specified explicitly (with node: null) which is often used to clear a non-null value that's been inherited (most often via extends but sometimes also via job overrides). In GitLab CI the null type is also not always equal to the zero-value of the type of the node. So for example the configure needs: [] (where [] is the zero type of a sequence node) is semantically NOT equal to needs: null (or simply omitting needs in a simple job). The former means that the job will not need any previous job and no matter in which stage it is it will run immediately. The latter will also not need any previous job but will wait for the jobs in the previous stages. The problem now for component author is that how do you provide an input to the component users to give the possibility to either specify a custom job to wait for (or a sequence of them) or let the job run in it's stage or execute immediately? Most importantly here though is that as a component author you want to have a default for jobs that's not a surprise if not set which would be the null value - but at the moment there is no possible to solve this with a single input. The needs keyword is not an exception - there are plenty of them and the reason for component authors and users to need null for inputs.

Additional context and details on customer problems

When using pipeline components, we may want to have control over needs: from the outside. This is especially true when not relying completely on DAG. Also, a component user may not have DAG setup etc or may not want/have the stage test.

E.g.

spec:
  inputs:
    needs:
      description: 'Explicitly set needs (['need', 'need']). When left empty, rely on stage dependencies.'
      default: '{}'
---
stages:
  - lint
  - build
  - test
  - deploy

job:
  needs: $[[ inputs.needs ]]
  stage: deploy

(pardon my abuse of global stages in the component)What the user wants/expects, is that to run this component after all previous stages have completed. The traditional pipeline flow. Because a component cannot know which needs there are from previous stages of course. We can let the user define these, as a work-around only, but ideally the component should work 'out of the box'.

The following is impossible with the current limitations. Ideally, we'd set needs to null. This would work, if not that defaults: null is not valid, and default: 'null' is not quite what we expect of course. The first fails on the fact that defaults is not a string, the second fails as 'null' is not a hash or an array.

Currently, we also often use {} to 'unset' items in the gitlab yaml, and I propose we continue this trend for needs.

If needs where to accept {}, which should be interpreted as null, we actually 'undefine' needs, and rely on stages again to set the build order. Obviously, this does require the user to have the correct stages, but that is probably more convenient for them, then to rebuild their entire pipeline to properly have 'needs' (or hack their pipeline to introduce a fake stage, which can then be used to 'need' upon.

All of the above works also for stage: for the same reason ...

User flow discussion

  • the user can specify a list of jobs a component-defined job needs.
    • an empty list is also okay
    • lets ignore that needs supports scalar string values, too
  • the default behavior is to wait for previous stages
  • the user does not need to override the job afterwards
  • the user can use the component as a direct child pipeline - so the user is not able to override the job.

The the user does not need to override the job afterwards point is very important for UX. I see as a component as a blackbox for the user. Ideally, they only need to know the spec (currently, only inputs). Overriding the job after the component include greatly defeats the purpose of a component used as an "interface".

Implementation plan

Usage tracking

  1. Add instrumentation on null and {}

Proposal

Add a nullable option to inputs, available for all inputs types.

spec:
  inputs:
    test_job_needs:
      type: array
      default: null
      nullable: true
---
test_job:
  needs: $[[ inputs.test_job_needs ]]
  script: ls
Edited by 🤖 GitLab Bot 🤖