Docs: Clarify baseline pipeline selection for MR report comparisons

Problem to solve

The Metrics Reports documentation states that "GitLab compares the metrics from the feature branch to the values from the target branch" but does not explain the mechanism for selecting the specific pipeline on the target branch.

This gap also affects Code Quality, Test Reports, and other MR report comparison features that rely on the same comparison_base_pipeline logic.

Specifically, the documentation does not address:

  1. Which pipeline on the target branch is selected for comparison
  2. What pipeline status is required (if any) for a pipeline to be considered a valid baseline
  3. How the target commit is determined - the method uses a fallback cascade of three SHAs (target_sha from merge request pipelines, diff_base_sha, diff_start_sha)
  4. How pipeline strategy affects baseline resolution - projects using merge request pipelines resolve through target_sha, while branch pipelines fall through to the merge-base commit

Without this information, users cannot reliably determine whether their CI/CD configuration will produce consistent baselines for MR report comparisons.

Further details

Source code reference (verified identical on v18.5.0-ee and v18.9.0-ee):

The audience is CI/CD administrators and pipeline authors who configure report artifacts and need to understand what makes a valid comparison baseline - particularly in self-managed environments where pipeline configurations vary.

Proposal

Add a "How baseline comparison works" section to the Metrics Reports documentation (and cross-reference from Code Quality and Test Reports pages) that explains:

  • The SHA cascade fallback strategy (target_shadiff_base_shadiff_start_sha)
  • That the most recently created pipeline (by ID) matching the SHA and target branch ref is selected
  • That no pipeline status filter is applied - any pipeline with matching SHA/ref can serve as the baseline
  • The practical difference between merge request pipelines and branch pipelines for baseline availability
  • That the head (feature branch) pipeline must be in a completed state with report artifacts for comparison to trigger, while the base pipeline has no such requirement
  • Guidance on ensuring a baseline is always available (e.g., ensuring target branch pipelines run and produce report artifacts)

Who can address the issue

Technical writers for the CI/CD or Testing documentation areas. The source code references above provide the implementation details to verify accuracy.

Other links/references

  • #343065 - Related: better error messages for metric reports (covers UX when comparison fails)
  • #27313 (closed) - Previously addressed for security reports specifically (DAST); non-security reports use a different code path
Edited by 🤖 GitLab Bot 🤖