Skip to content

Backend: No pipelines created when matrix var is longer than 114 chars

Summary

After spending hours trying to understand what was going wrong, I came to the conclusion I found a specific edge case where GitLab does not throw any error messages and simply gets bugged.

When these conditions are met:

  • There is a needs relations between two jobs
  • The parent job has a matrix
  • The total number of characters in the matrix variables values is greater than 114

...no pipelines are created, without any notifications of what could have gone wrong.

Steps to reproduce

  1. Create a project with this minimal .gitlab-ci.yml:
Parent job:
  stage: build
  parallel:
    matrix:
      - FOO: "fooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo"
  script:
    - echo "Hello world"

Child job:
  stage: test
  needs: ["Parent job"]
  script:
    - echo "Hello world"
  1. Commit & push -> It creates a pipeline
  2. Add a o to the FOO variable
  3. Commit & push -> no pipelines

When multiple variables are defined, the issue occurs when the sum of the variables characters is above 114 :

Click to expand 
Parent job:
  stage: build
  parallel:
    matrix:
      - FOO: "fooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo"
        BAR: "oooooooooooo"
  script:
    - echo "Hello world"

Child job:
  stage: test
  needs: ["Parent job"]
  script:
    - echo "Hello world"

Example Project

Here is the project : https://gitlab.com/vdsbenoit/needs-matrix

We can see that no pipelines ran for this commit : vdsbenoit/needs-matrix@49ebea1e

What is the current bug behavior?

No pipelines are created

What is the expected correct behavior?

A pipeline should be created

Output of checks

This bug happens on GitLab.com

Possible fixes

I don't have any fixes to this issue. The best workaround I found is to remove the needs instruction while waiting for a fix.

  • Surface the error when no pipeline is created when matrix reaches limit
  • Match job name chars limit

Note

We are a 150 seats GitLab Premium customer. Support on this issue would be much appreciated 🙏

Proposal

Iteration Description Limitations
Step 1: Surface the error Ensure that an error message is surfaced when character limits are exceeded. Does not solve the problem to allow for long matrix section. Users still need to use identified workaround of variable indirection.
Step 2: Make the character limits match Does not solve the problem to allow for long matrix section. Users still need to use identified workaround of variable indirection.
Step 3: Update UI to show variables Add UI elements to expose the variables used, rather than relying on the name. This may require updating the API to contain the information if not already available Does not solve the problem to allow for long matrix section. Users still need to use identified workaround of variable indirection.
Step 4: Allow Custom Matrix Names Exact details to be determined. Allow the user to assign names to the various configurations through the config. This allows for names that are meaningful. When names aren't provided default to names by variable - same behaviour as today. Requires the UI update from step 3 to maintain parity with existing experience of viewing variables, but does fix the issue of being limited in matrix variables.

Questions to Answer

Implementation Table

Group Issue Link
backend 👈 You are here
backend Backend: Match Ci::BuildNeed name limit with jo... (#420669 - closed)
backend / frontend Frontend/Backend: Update UI to show variables (#428165)
backend Backend: Allow Custom Matrix Names (#428169 - closed)
Edited by Mark Nuzzo