Skip to content

Ensure merge SHA is available during Pipelines for merged results life period

TODO

  • Remove the feature flag depend_on_persistent_pipeline_ref

Status

This feature is currently behind depend_on_persistent_pipeline_ref feature flag, which is disabled by default.

We'll evaluate this feature in %12.4 and remove it if it's confirmed as deemed stable.

Timeline Action
Oct 9th 3:00 AM UTC The feature flag depend_on_persistent_pipeline_ref is enabled on all projects on production
Fri Oct 4 07:15:43 UTC 2019 The feature flag depend_on_persistent_pipeline_ref is enabled on gitlab-com/www-gitlab-com and gitlab-org/gitlab on production

Problem

When user uses Pipelines for merged results, pipelines run on a (prospective) merge commit instead of a feature branch. This temporary merge commit is generated at refs/merge-requests/:iid/merge and it's overwritten per 1. Source branch is advanced 2. Target branch is advanced. This causes the following problems in a development flow, especially in a busy repo:

  1. A developer pushes a new commit to a merge request, which targets "master".
  2. The prospective merged result is renewed. The SHA is SHA-A.
  3. A pipeline for merged result is created and run on the merged results (SHA-A).
  4. The other MR has been merged into master branch i.e. master branch is advanced.
  5. The prospective merged result is renewed. The SHA is SHA-B.
  6. The pipeline ran for SHA-A is invalidated (User sees fatal: reference is not a tree: in job log) thus failed
  7. The developer cannot test code in the merge requests.

See https://gitlab.slack.com/archives/C0SFP840G/p1568030407243300 for the actual problem in www-gitlab-com

This is a crucial problem that developer cannot really see if the new code passes tests or not.

Workaround

  1. Make the MR WIP to trigger detached merge request pipelines. This pipeline runs on source branch thus correctly finishes regardless of the target advanced case.
  2. Use merge train. Although, merge train is used when a maintainer decided to merge something and it cannot be used in the development flow.

Proposal

Create a refs/pipelines/:iid ref per pipeline

  • The system ensures a corresponding pipeline ref exists when build status transits to running.
  • Runners fetch code from the pipeline ref instead of source branch, refs/head or refs/merge.
  • The system cleans up the pipeline ref when pipeline finished i.e. no more active jobs.

/cc @ogolowinski @jlenny @oswaldo @tmaczukin @tkuah

Edited by Shinya Maeda