Skip to content

Inline upstream/downstream pipelines expansion

Problem to Solve

Currently it is not possible to show upstream or downstream pipeline information in context of the current pipeline in order to be able to backtrace what triggered which pipeline and/or what other pipelines did it cause to trigger.

Proposal

When projects are connected via triggers or dependencies, we should display a complete pipeline graph of the entire flow, across all projects.

  • For any given project, show all upstream and downstream projects in the same pipeline graph, clearly indicating which jobs belong to the pipeline in focus. e.g. put external jobs

Note, this is not about showing artifact, image, submodule or other dependencies. It's about what pipelines cause other pipelines to run.

Solution

Logic of prototype from https://gitlab.com/gitlab-org/gitlab-ee/issues/2122#note_117374817 is still applicable

Specs:

Upstream

Loading:

upstream-loading

Expanded:

upstream-expanded

Downstream

Loading:

downstream-loading

Multiple + one is expanded

multiple-downstreams

Single + expanded:

single-downstream

Notes

  • Upstream/downstream pipelines are expandable+collapsable showing their pipeline graph inline
  • Inline upstream/downstream pipeline graphs are positioned along the Y-axis according to the position of their project position in the upstream/downstream lists from which they are expanded from.
  • Only one downstream can be expanded at a time (if you expand another one, the initial one will be collapsed/replaced)
  • Expanded upstream pipelines do not show downstreams of their own which are non related to the current pipeline
  • Upstream/downstream loading states temporarily replace the pipeline status icon with a spinner
  • Scrolling stays unaffected, except for positioning expanded/collapsed content into view.

FE notes:

  1. Fetch the main endpoint pipelines/{id}.json
  2. Look for triggered_by for the upstream
  3. Look for triggered for the downstream pipelines
  4. Look for triggered_by.path and triggers[idx].path it only misses the .json Frontend will start polling on click for each upstream/downstream
  • it’s not possible to have any upstream/downstream expanded by default bc we won’t have the data. the user has to click on it, we are all ok with this 👍
  • We will poll the endpoints for both upstream & downstream pipelines
  • Once the user clicks to collapse, we will stop polling. if the user clicks again we fetch again and show a loading again

Documentation

This would good to document at https://docs.gitlab.com/ee/ci/pipelines.html#pipeline-graphs, perhaps adding a new section about seeing upstream and downstream pipelines.

Documentation blurbs

  • General topic

Larger projects, especially those adopting a micro-services architecture, often have a set of interdependent components that form the complete product. When working with these projects, it is important to be able to build, test, and deploy both the individual parts as well as the whole. For example when developers are working on a low level service, it is important to ensure downstream components are not negatively impacted by changes.
With Cross Project Pipelines, GitLab will have first class support for managing and running CI/CD pipelines across projects. Users will be able to easily link a projects CI/CD stages into the full pipeline, as well as view current status.

  • Specific documentation blurb

In a multi-project scenario, it is very important for developers to access some basic details of related projects when viewing upstream and downstream links. Clicking on the project you're interested in, it will be expanded to a more comprehensive view that shows stages and jobs, as well as further dependencies. You can iterate on it in order to get the full view of the complete pipeline.

What does success look like, and how can we measure that?

  • The user is able to see which pipelines caused the current pipeline to run (all the way upstream to the original pipeline that set everything in motion).
  • The user is able to see which downstream pipelines are caused by the current pipeline, up until the last generation of every downstream.

Links / references

Edited by Jason Yavorska