Backend: Allow needs to use the parallel keyword

Problem to solve

Definition of parents jobs for the pipeline DAG using the needs: keyword is not very flexible. We are limited to a set of fixed job names. This means the DAG cannot be combined with Matrix jobs - either for the current job being defined, or for the parent jobs to be specified by needs:.

User experience goal

The user should be able to define job dependencies in the pipeline DAG using Matrix job definitions, to streamline the .gitlab-ci.yml file.

Proposal

Since variables are defined as part of a Matrix job, I propose that any job in the needs: subsection of a job definition should support variable expansion. That would also open it to more general use cases, where variables are otherwise defined.

This proposal already has its own issue: #255311 (closed)

This issue will be scoped to allow needs to use the parallel keyword, to be able to do configs like below:

test:
  needs:
    - job: build
      parallel:
        matrix:
          - PLATFORM: ubuntu
            STACK: python
# depends on build[ubuntu, python] only

test:
  needs:
    - job: build
      parallel:
        matrix:
          - PLATFORM: ubuntu
            STACK: [ruby, python]

# depends on build[ubuntu, ruby] and build[ubuntu, python]

out of scope:

test:
  needs:
    - job: build
      parallel:
        matrix:
          - PLATFORM: ubuntu

# depends on build[ubuntu, *]

test:
  needs:
    - job: build
      parallel:
        matrix:
          - PLATFORM: ubuntu
            STACK: []

# depends on build[ubuntu]

Additional comments can be found in the following thread

Availability & Testing

What does success look like, and how can we measure that?

What is the type of buyer?

Is this a cross-stage feature?

Links / references

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.