Improve confusing coverage-check approval behavior with merged results pipelines

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Problem to solve

Improve the coverage-check approval rule behavior when using merged results pipelines to only require approval when the merge request actually reduces code coverage, rather than requiring approval based on the initial branch divergence point.

As a developer using merged results pipelines with coverage-check approval rules, I want the approval requirement to be based on whether my merge request actually reduces code coverage when merged, so I can avoid unnecessary approvals and compute resource usage when coverage hasn't actually decreased.

Current behavior: When using merged results pipelines with coverage-check approval rules, approval is required based on comparing the merged result coverage to the coverage at the point where the source branch originally diverged from the target branch. This can result in approval being required even when:

  • The merge request doesn't actually reduce coverage compared to the current target branch
  • The target branch coverage has decreased after the source branch was created
  • The merged result would maintain or improve coverage compared to the current target branch state

Example scenario:

  1. Branch created from main when coverage is 68%
  2. Main branch coverage decreases to 67%
  3. MR created from the branch (still at 68% coverage)
  4. Merged results pipeline runs and shows coverage decrease (comparing 67% merged result to 68% divergence point)
  5. Approval required despite the MR not actually reducing coverage below current main

Intended users

User experience goal

The user should be able to use merged results pipelines with coverage-check approval rules and only be required to get approval when their merge request would actually reduce the project's test coverage compared to the current state of the target branch.

Proposal

Modify the coverage-check approval rule logic for merged results pipelines to:

  1. Compare against current target branch coverage instead of divergence point coverage
  2. Use the most recent coverage data from the target branch when evaluating approval requirements
  3. Only require approval when the merged result coverage is lower than the current target branch coverage

Implementation approach:

  • When a merged results pipeline completes, compare the coverage result against the most recent coverage from the target branch
  • If target branch coverage is not available (no recent pipeline with coverage), fall back to current behavior
  • Update the approval rule evaluation service to handle this comparison logic
  • Ensure the UI clearly indicates what coverage values are being compared

User journey:

  1. Developer creates branch from main (coverage: 68%)
  2. Main branch coverage decreases to 67% due to other changes
  3. Developer creates MR with merged results pipeline
  4. Pipeline runs and produces 67% coverage (same as current main)
  5. System compares 67% (merged result) vs 67% (current main) = no decrease
  6. Approval rule remains optional instead of required

Further details

Benefits:

  • Reduces unnecessary approval overhead for teams using merged results pipelines
  • Saves compute resources by avoiding unnecessary pipeline reruns/rebases
  • Provides more intuitive behavior that matches user expectations
  • Aligns with the stated purpose of coverage-check rules (prevent coverage reduction)

Use cases:

  • Teams using merged results pipelines for integration testing
  • Projects with frequent main branch updates that affect coverage
  • Organizations wanting to maintain coverage standards without approval friction

Current workarounds:

  • Rebasing branches to update the divergence point (requires additional pipelines / compute minutes)
  • Manually approving rules that shouldn't require approval

Permissions and Security

This change affects the evaluation logic for coverage-check approval rules but doesn't change the permission model:

  • No impact to members with no access (0) - they cannot see coverage or approval status
  • No impact to Guest (10) members - they can see coverage but cannot approve
  • No impact to Reporter (20) members - they can see coverage but cannot approve
  • Positive impact to Developer (30) members - fewer unnecessary approval requirements
  • Positive impact to Maintainer (40) members - fewer unnecessary approvals to review
  • Positive impact to Owner (50) members - fewer unnecessary approvals to review

The change maintains existing security boundaries and only affects when approval is required, not who can approve.

Documentation

Documentation updates needed:

Availability & Testing

Risks:

  • Changes to approval rule evaluation logic could affect existing workflows
  • Need to ensure backward compatibility for projects not using merged results pipelines
  • Performance impact of fetching current target branch coverage data

Test coverage needed:

  • Unit test changes:
    • Coverage comparison logic for merged results pipelines
    • Approval rule evaluation with different coverage scenarios
    • Fallback behavior when target branch coverage unavailable
  • Integration test changes:
    • End-to-end approval workflow with merged results pipelines
    • Coverage data retrieval and comparison
    • UI display of coverage comparison values
  • End-to-end test changes:
    • Complete MR workflow with coverage-check rules and merged results pipelines
    • Cross-browser testing for coverage display in MR widget

Available Tier

  • Premium and Ultimate (coverage-check approval rules are a Premium feature)

Feature Usage Metrics

Track usage through:

  • Number of coverage-check approval rules using merged results pipelines
  • Frequency of approval requirement changes (required → optional) after pipeline completion
  • User interactions with coverage-related approval rules

What does success look like, and how can we measure that?

Success metrics:

  • Reduction in unnecessary coverage-check approvals for merged results pipelines
  • Decreased time-to-merge for MRs that don't actually reduce coverage
  • Improved user satisfaction with coverage approval workflow

Acceptance criteria:

  • Coverage-check approval rules correctly compare merged results against current target branch coverage
  • Approval is only required when merged result coverage is lower than current target branch
  • Existing behavior is preserved for non-merged-results pipelines
  • UI clearly shows what coverage values are being compared
  • Documentation accurately reflects the new behavior

Is this a cross-stage feature?

Yes, this affects multiple areas:

  • Verify stage: Pipeline execution and coverage reporting
  • Create stage: Merge request approval workflows
  • Plan stage: Project settings and approval rule configuration

Coordination needed with:

  • Pipeline execution team for coverage data handling
  • Merge request team for approval rule evaluation
  • Frontend team for UI updates showing coverage comparison

What is the competitive advantage or differentiation for this feature?

  • Provides more intelligent and intuitive coverage-based approval workflows
  • Reduces friction in advanced CI/CD setups while maintaining quality standards
  • Demonstrates GitLab's sophisticated understanding of complex pipeline scenarios
  • Improves developer experience for teams using advanced GitLab features

Links / references

Edited by 🤖 GitLab Bot 🤖