Allow grouping/collapsing CI jobs

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

This is related to #16079 (closed) but the resolution for that does not address this use case. I'm currently using 17.6.0-pre on a Premium SAAS plan.

Now that it's possible to have includes with inputs, I've been able to write CI pipelines with parameterised jobs. This makes it far, far easier to express more detailed relationships between a greater number of jobs in a pipeline, which reflects the reality for more complex projects. However, the greater number of jobs does not lead to a great UI. Here's an example (I've changed our job names for privacy).

Screenshot_from_2024-11-07_13-18-49

I would like to be able to express groupings of these jobs so that the UI lets viewers collapse some of them and get the bigger picture. But it does not make sense to just number these jobs, they are not "Rust project A, job 1", "Rust project A, job 2", "Rust project A, job 3" — they are "Rust project A, lint", "Rust project A, build", "Rust project A, test". They are not suitable for matrix jobs - the difference between "lint" and "test" is not just changing an OS image name or compiler flag, the entire script and even job definition is different. Many jobs that should be in groupings cannot be expressed as matrix jobs because they don't appear as siblings.

Here's the example CI config I used in the linked issue, from a monorepo with a bunch of projects of different languages:

/
├─ project-a # C
├─ project-b # Rust
├─ project-c # C
├─ project-d # Rust
┊

The CI config makes heavy use of the new includes-with-inputs system, because most projects of the same language have the same procedure. So the root .gitlab-ci.yml includes each project's CI config eg.

include:
  - local: /project-a/.gitlab-ci.yml
  - local: /project-b/.gitlab-ci.yml
  # etc

A single project might have a config that looks like:

include:
  # Creates project-b/configure-image:
  # Creates project-b/build-image:
  #
  # Those two jobs should NOT be grouped with the others below!
  - local: /.ci/Image build.gitlab-ci.yml
    inputs:
      project-name: project-b
      image-name: rust:1.71.1-slim
  # Creates:
  # - project-b/format
  # - project-b/lint
  # - project-b/audit
  # - project-b/build
  # - project-b/test
  #
  # These jobs SHOULD be grouped, so a person can just see if
  # Project B passed or failed.
  - local: /.ci/Rust.gitlab-ci.yml
    inputs:
      project-name: project-b

Jobs like configure-image and build-image naturally group together across ALL projects (see first column in picture) - it's likely that if one fails, many will. Jobs like format, lint and audit likewise go together within a project. At the moment I have massive vertical columns of format, lint, audit jobs in the check stage, three per project. It would be fantastic if I could just fold them into a project-b/all group by adding a YAML key to the template like ingroup: $[[ inputs.project-name]]/all.

Edited by 🤖 GitLab Bot 🤖