Proposal: Use the merge-train to sync security changes into the GitLab canonical repo

Context

One of the final steps of a security release is to update the GitLab canonical repository with the security changes. When the update fails, a merge request is automatically opened on GitLab canonical to bring the security changes, while this merge request is automated it is subject to long GitLab pipelines, merge request failures, and code owner approvals. Until those are handled, the merge request is time-consuming and requires a release manager intervention.

Proposal Use the merge train to sync security changes into the GitLab canonical repo.

To reduce the release manager toil when syncing repositories, the merge train can be used to update the GitLab canonical repository after the security release is published, this is the same strategy used to keep GitLab security repositories up to date while the security release is in progress. The viability of this solution will be tested on #19361 (closed)

To use the merge train on the GitLab project:

  • The GitLab repository sync should be handled independently of the sync_remotes job. This job should handle the rest of the repositories
  • A dedicated job should be added to handle the sync of the GitLab project, this should be executed on the final steps of the security release.
  • The new job will trigger a pipeline schedule on the merge train repo, the pipeline should kick the merge train to sync changes to the GitLab canonical repository.
  • The merge train has a retry built-in mechanism and will attempt to sync 5 times before failing.
  • If it fails, the pipeline schedule should automatically kick another merge train in the next x minutes
  • A mechanism should be configured to detect if the merge train is needed, and if it is not, disable the merge train.

Considering the GitLab repository traffic, the merge train could have higher chances of success if it is triggered at the EOD in AMER time or at the beginning of the day in APAC.

Pros:

  • Reduces the release manager's workload associated with syncing the GitLab canonical project.
  • Removes the merge request requirement and the problems associated with it (long pipelines, code owner approvals, etc).
  • Prevents release managers from pushing to master.

Cons:

Implementation details

Testing results

There were two opportunities to test the execution of the merge train to sync security changes back to the GitLab canonical repository, on both attempts, the merge train successfully synced the repositories on the first attempt 🎉

Details:

Follow-ups

Edited by Mayra Cabrera