Skip to content

Backend: Add needs: parallel: matrix enhancements

After further consideration, we plan to be more explicit at what variables are needed by using * See examples below:

linux:build:
  stage: build
  script: echo "Building linux..."
  parallel:
    matrix:
      - PROVIDER: 
          - aws
          - gcp
        STACK:
          - monitoring
        DATABASE:
          - pg
          - my
      - PROVIDER: 
          - gcp

# creates 2 matrices (5 jobs)
# linux:build[aws, monitoring, pg], linux:build[aws, monitoring, my]
# linux:build[gcp, monitoring, pg], linux:build[gcp, monitoring, my]

# linux:build[gcp]

Case 1 - needed gcp with other specified variables and second matrix

linux:rspec1:
  stage: test
  needs:
    - job: linux:build
      parallel:
        matrix:
          - PROVIDER: gcp
            STACK: *
            DATABASE: *
          - PROVIDER: gcp

# depends on 
# linux:build[gcp, *, *] -> linux:build[gcp, monitoring, pg], linux:build[gcp, monitoring, my]
# linux:build[gcp] because of the second matrix

Case 2 - needed gcp without other variables (second matrix)

linux:rspec2:
  stage: test
  needs:
    - job: linux:build
      parallel:
        matrix:
          - PROVIDER: gcp

# depends on linux:build[gcp]
# does not depend on the first matrix since it also has other variables that were not specified in the above example.
Original description

As part of #254821 (closed), the following use cases still need to be implemented:

linux:build:
  stage: build
  script: echo "Building linux..."
  parallel:
    matrix:
      - PROVIDER: 
          - aws
          - gcp 
        STACK:
          - monitoring
          - app1
          - app2
      - PROVIDER: 
          - aws

# creates
# linux:build[aws, monitoring], linux:build[aws, app1], linux:build[aws, app2]
# linux:build[gcp, monitoring], linux:build[gcp, app1], linux:build[gcp, app2]
# linux:build[aws]

Case 1 - all permutations of linux:build that have the aws provider

linux:rspec:
  stage: test
  needs:
    - job: linux:build
      parallel:
        matrix:
          - PROVIDER: aws

# depends on linux:build[aws, *]
# -> linux:build[aws, monitoring], linux:build[aws, app1], linux:build[aws, app2], linux:build[aws]

Case 2 - needing only PROVIDER without stack (1D matrix) - linux:build[aws]

linux:rspec:
  stage: test
  needs:
    - job: linux:build
      parallel:
        matrix:
          - PROVIDER: aws
            STACK: []

# depends on linux:build[aws]
Note:
  • This change should not need a feature flag
  • Documentation updates are needed for this MR
  • More information can be found on the related issue.
Edited by Kasia Misirli