Skip to content

Measure impact of internal maintenance policy extension pilot

Background

We are extending our bug fix maintenance policy from supporting one previous release to three previous releases (&971 (closed)). From the epic, the following were the intended impacts of the effort.

Customer Impact: Currently only 5% of SM ARR is on the current stable version, meaning 95% of customers don't benefit from bug fixes under our existing policy. Extending to n-2 coverage will enable 45% of SM ARR to benefit from bug fixes - improving quality for an order of magnitude more customers with relatively little effort.

Operational Benefits: This change also eliminates the need for ad-hoc backport requests and approvals, reducing engineering effort and streamlining release workflows.

Objective

Measure the impact of this policy change to ensure it delivers improved quality and reliability for significantly more customers, while confirming operational benefits are achieved.

Success Criteria

  • Improve customer experience and quality coverage
    • Applicable fixes that get backported to n version (before) get backported to n-1 and n-2 versions (Increase percentage of SM ARR benefiting from bug fixes from 5% to target of 45%)
    • Reduce customer requests for exceptional backports to older versions
    • Decrease support escalations for known bugs in n-1 and n-2 versions
  • Confirm operational benefits
    • Reduce release management effort on backport request reviews and approvals
    • Preserve patch release quality across all supported versions

Measurements to Track

Customer impact

  • Measure adoption as percentage of bug fixes backported to release versions previously requiring backport requests
    • % of bug fixes backported to current (N) branch that have been backported to older N-1 branch
      • These are bug fixes that are applicable to maintained versions that would be published in the next patch for the previous (n-1) release (5 weeks and 7 weeks after release)
      • This would require a backport request prior to extension of the maintenance policy
      • This (N-1) is the release used by Dedicated
    • % of bug fixes backported to current (N) branch that have been backported to older N-2 branch
      • These are bug fixes that are applicable to maintained versions that would be published in the next patch for the previous (n-2) release (9 weeks and 11 weeks after release)
      • This would require a backport request prior to extension of the maintenance policy
  • Fewer customer tickets

Release management workflow efficiency

  • Number of backport requests
    • Number of backport requests
      • This shows the savings in operational overhead as another benefit of the maintenance policy extension. We want this number to be low.
    • Number of backport requests related to support tickets
    • Number of backport requests labeled with high severity or customer impacting
      • These show the quality improvement of releases by directly measuring support load related to bug fixes. We want this number to be low.

Measurements and highlights

Before pilot

#21187 (comment 2693654842)

After pilot

#21187 (comment 2693758317)

Feedback from release management after internal rollout

#21187 (comment 2761077273)

Feedback from GitLab team members after internal rollout

This has been captured by the internal survey #21508

Exit Criteria

  • Baseline measurements collected
  • Pilot measurements collected
Edited by Jenny Kim