Fast-forward merge support for merge trains (#4911) · Epics · Epics · GitLab.org

Fast-forward merge support for merge trains

*This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.*  Promoted from https://gitlab.com/gitlab-org/gitlab/-/issues/35628+ ### Release notes Fast-forward merges are a common and highly popular merge method. Merge Trains can help support some of the greater challenges related to frequent rebase and re-running pipelines when changes to the main branch occur. Previously, these two methods were incompatible. In this release, you can now use Fast-forward merge with Merge Trains which means no merge commits will be created and all merges that are part of the Merge Train are fast-forwarded permitting merge when the branch is fast-forwarded. ### Problem to solve Merge Trains could help solve a fundamental contention problem of fast forward merges because the CI pipeline must be run everytime the merge request is rebased, and the merge request must be rebased every time `main` changes - which is frequently! This significantly limits the frequency with which merge requests can be merged. ### Proposal | Desired-workflow diagram | |------------------| | ![Overview_FF_MT](/uploads/86e139da8ec760ff4d7109e896766639/Overview_FF_MT.png) | ## Old description <details> <summary>Click to expand!</summary> When a merge request is created or updated, `refs/merge-requests/$iid/merge` will be generated with the merge result. Currently this is always generated using the `merge` merge method, regardless of the project settings. A more accurate merge ref should probably be generated https://gitlab.com/gitlab-org/gitlab/issues/26996. But the merge-ref is not the whole story. The merge ref is ephemeral and does not impact the contents of the branch. In order to actually complete the merge, the merge train also needs to rebase the actual feature branch, and then merge. | **Fast-forward merge** | **Merge Train** | | ------ | ------ | | - User creates a merge request `awesome-feature` from the tip of `master` branch. - Someone merged the other merge request into `master`. Now `master` branch is advanced. - Now the `awesome-feature` branch is outdated thus it cannot be merged. - User pulls the latest `master` branch, rebases `awesome-feature` on the `master` manually, and push it to the remote repository. - Once the pipeline passed, `awesome-feature` can be merged manually. | - User creates a merge request `awesome-feature` from the tip of `master` branch. - User added the merge request on merge train. - Someone merged the other merge request into `master`. Now `master` branch is advanced. - Merge Train reconstruct the cascading-refs and re-run pipelines automatically. - Once the pipeline passed, `awesome-feature` is merged automatically. | ## Current problem **Merge Trains with Fast Forward Merge** 1. User creates a merge request `awesome-feature` from the tip of `master` branch. 1. User added the merge request on merge train. 1. Someone merged the other merge request into `master`. Now `master` branch is advanced. 1. Now the `awesome-feature` branch is outdated thus it cannot be merged. 1. Merge Train tries to reconstruct the cascading-refs but the merge request is not mergeable thus *the merge request is dropped from the merge train*. ## Proposal **Fast-forward with merge train** When the fast-forward merge (--ff-only) setting is enabled in the project settings & merge trains are enabled, no merge commits will be created and all merges that are part of the merge train are fast-forwarded, which means that merging is only allowed if the branch can be fast-forwarded. **Rebase with merge train** When a fast-forward merge with merge train is not possible, the application will try to rebase - this will rebase the entire merge train. Rebasing on merge train should happen automatically and shouldn't require a manual action from users. NOTE: "Semi-linear history merge requests" should be also supported ### Project settings > General > Merge request section ⭐️ [SEE DESIGN PROPOSAL](https://gitlab.com/gitlab-org/gitlab/issues/35628/designs/ff-merge-train--settings.png) * Add a new line of copy to the `Fast-forward merge` option explaining if it activated together with merge trains, merge is only allowed if the branch can be fast-forwarded * *Once merge trains are enabled, merging is only allowed if the branch can be fast forwarded* * Add a link to the documentation page: NEW SECTION ON DOCS, NEEDS TO BE CREATED cc @marcia * Update the copy for `merge pipelines` and add link to merge trains documentation * *Enables merge train by default. Merge pipelines will try to validate the post-merge result prior to merging* * Add a link to the documentation page: https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/ ### Merge request page ⭐️ [SEE DESIGN PROPOSAL](https://gitlab.com/gitlab-org/gitlab/issues/35628/designs/ff-merge-train--system-notes.png) Fast-forward with merge train **fails**: * A message should be added to the `system notes` in the merge request. **Rebase with merge train **fails**:** * A message should be added to the `system notes` in the merge request. **Fast-forward / Rebase with merge train **succeeds**:** * A message should be added to the `system notes` in the merge request. * @dosuken123 to check: An existing rebase/ff-merge feature already provides some system notes for the usage, so we might not need to add extra ones in merge train context. ### Workflow 1 1. User adds an MR to a merge train. 1. Merge Train creates a new pipeline and validates the MR. 1. Merge Train rebases the MR onto the latest master. 1. If succeed, continue. 1. If failed, drops the MR from the train. A message should be added to the `system notes` in the merge request. 1. Merge Train merges the MR with Fast Forward option. 1. If succeed, the process complete. 1. If failed, drops the MR from the train. A message should be added to the `system notes` in the merge request. Users might want to bypass a merge train exceptionally, and merge immediately instead. This path should also be supported. ### Workflow 2 1. User adds an MR to a merge train by clicking "Start/Add merge train" button (and the merge train takes care of the rebase + ff-merge) 1. Note: Applicable on **Add/Start Merge Train /When Pipeline Succeeds** states. 1. User wants to merge an MR immediately by clicking "Merge" button (e.g. There is an urgent patch needs to be merged asap). 1. If the MR is not on top of the latest master, users have to rebase the MR at first by clicking "Rebase" button. (i.e. How ff-merge option works today) ⭐️ [SEE DESIGN PROPOSAL](https://gitlab.com/gitlab-org/gitlab/issues/35628/designs/ff-merge-train.png) On the MR widget, the dropdown should be updated when rebasing manually is an option for the following criteria: **If the feature branch is NOT on the latest master:** * `Start/Add merge train` button (primary) * `Rebase` dropdown menu option (should substitute the option `merge immediately` ## Documentation Yes, we will need to make updates to the documentation. ## Links / references - https://gitlab.com/gitlab-org/gitlab/-/issues/26996 is a pre-requisite; it allow users to use 'Pipelines for Merged Results' with a fast forward strategy. Per [current docs](https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_results/#prerequisites) on pipelines for merged results: > Prerequisites To enable pipelines for merge results: > > You must have maintainer permissions. You must be using GitLab Runner 11.9 or later. You must not be using fast forward merges yet. To follow progress, see #58226. </details> ### Proposed workaround: While I have not implemented it yet, knowing this is 3+ months away, I would expect I could implement a workaround as follows, but would love insight from others on what they assess is most efficient etc.: * Have the MR pipeline build still publish the image as second-to-last step tagged at that commit hash. * As last step, either trigger "main" with that commit tag as a variable (probably not ideal because of race condition/lack of persistence), or commit that tag in a manifest/config somewhere (since we know the only way to get work into "main" is for this pipeline to run), and then when "main" runs it will now have the "publication commit" and final commit hashes available. * This would replace a complete (mostly cached, but still) rebuild-and-tag on "main" that we currently use which costs minutes on our pipeline every merge and enables a stray commit to "main" to release a viable image that was never actually tested. * "main" pipeline now simplifies to pull image@hash-pub, re-tag image@hash-pub => image@hash-main, and push image@hash-main. # Epic Progress Report  <details><summary>Issues (total: 17)</summary> <details><summary>Next 1-3 releases (1)</summary> | :checkered_flag: | Iteration | References | Status / ETA | DRI | Impact | | --- | --- | --- | --- | --- | --- | | :hourglass_flowing_sand: | gitlab-org/gitlab#423893+ | gitlab-org/gitlab#423893 | %Next 1-3 releases | | | </details> <details><summary>13.7 (1)</summary> | :checkered_flag: | Iteration | References | Status / ETA | DRI | Impact | | --- | --- | --- | --- | --- | --- | | :x: | gitlab-org/gitlab#284121+ | gitlab-org/gitlab#284121 | %13.7 | | | </details> <details><summary>14.7 (1)</summary> | :checkered_flag: | Iteration | References | Status / ETA | DRI | Impact | | --- | --- | --- | --- | --- | --- | | :x: | gitlab-org/gitlab#282422+ | gitlab-org/gitlab#282422 | %14.7 | | | </details> <details><summary>16.1 (2)</summary> | :checkered_flag: | Iteration | References | Status / ETA | DRI | Impact | | --- | --- | --- | --- | --- | --- | | :x: | gitlab-org/gitlab#413105+ | gitlab-org/gitlab#413105 | %16.1 | | | | :x: | gitlab-org/gitlab#390937+ | gitlab-org/gitlab#390937 | %16.1 | | | </details> <details><summary>16.2 (1)</summary> | :checkered_flag: | Iteration | References | Status / ETA | DRI | Impact | | --- | --- | --- | --- | --- | --- | | :x: | gitlab-org/gitlab#298541+ | gitlab-org/gitlab#298541 | %16.2 | | | </details> <details><summary>16.4 (7)</summary> | :checkered_flag: | Iteration | References | Status / ETA | DRI | Impact | | --- | --- | --- | --- | --- | --- | | :hourglass_flowing_sand: | gitlab-org/gitlab#422724+ | gitlab-org/gitlab#422724 | %16.4 | | | | :hourglass_flowing_sand: | gitlab-org/gitlab#422483+ | gitlab-org/gitlab#422483 | %16.4 | | | | :x: | gitlab-org/gitlab#422197+ | gitlab-org/gitlab#422197 | %16.4 | | | | :hourglass_flowing_sand: | gitlab-org/gitlab#421588+ | gitlab-org/gitlab#421588 | %16.4 | | | | :hourglass_flowing_sand: | gitlab-org/gitlab#418822+ | gitlab-org/gitlab#418822 | %16.4 | | | | :x: | gitlab-org/gitlab#413104+ | gitlab-org/gitlab#413104 | %16.4 | | | | :hourglass_flowing_sand: | gitlab-org/gitlab#282442+ | gitlab-org/gitlab#282442 | %16.4 | | | </details> <details><summary>No milestone (2)</summary> | :checkered_flag: | Iteration | References | Status / ETA | DRI | Impact | | --- | --- | --- | --- | --- | --- | | :x: | gitlab-org/gitlab#287809+ | gitlab-org/gitlab#287809 | | | | | :x: | gitlab-org/gitlab#282424+ | gitlab-org/gitlab#282424 | | | | </details> <details><summary>Backlog (2)</summary> | :checkered_flag: | Iteration | References | Status / ETA | DRI | Impact | | --- | --- | --- | --- | --- | --- | | :hourglass_flowing_sand: | gitlab-org/gitlab#282426+ | gitlab-org/gitlab#282426 | %Backlog | | | | :x: | gitlab-org/gitlab#35628+ | gitlab-org/gitlab#35628 | %Backlog | | | </details> </details> Use epic gitlab-org&4911 on issues to add them to this list. Generated at <code>2023-09-13 04:15:44 UTC</code> by https://gitlab.com/gitlab-org/ci-cd/ops-eng-managers/headway/-/jobs/5073870639.

epic