Detect stale replica reads in MergeRequest#reload_diff

Sincheol (David) Kimrequested to merge
583576-mr-shows-zero-commits-and-empty-changes-tab into master

What does this MR do and why?

Merged MRs intermittently show zero commits and empty changes tab after merge. The latest_merge_request_diff is overwritten with an empty diff (state: 'empty', commits_count: 0, head_commit_sha: nil).

There seems to be a race condition between MergeService and RefreshService caused by how the Gitaly post-receive hook triggers database reads from a potentially stale replica.

  1. Gitaly post-receive hook runs in a separate web request with its own DB session — not the MergeWorker session
  2. The WAL recorded for PostReceiveWorker is from a replica (no writes in the web request)
  3. This replica WAL could predate lock_mr
  4. UpdateMergeRequestsWorker inherits this stale WAL
  5. post_merge_manually_merged finds the MR as opened (stale replica), tries mark_as_merged, but it fails silently because the primary already has merged
  6. No write succeeds → session stays on replica
  7. reload_merge_requests queries the same stale replica → finds MR as opened
  8. reload_diff creates an empty diff because source branch is gone and target has the merge commit

The broken MR diff from the issue shows:

  • mark_as_merged at 2025-12-09 18:10:03 UTC
  • Empty diff created at 2025-12-09 18:10:09 UTC (6 seconds later)
  • state: 'empty', commits_count: 0, head_commit_sha: nil, start_commit_sha = merge commit SHA

Source branch deletion is not required to trigger this bug — it just makes the symptom worse (head_commit_sha: nil). Even with the source branch present, reload_diff produces an empty diff because the target branch already contains the source commits after merge.

To validate this theory, MergeRequest#reload_diff now checks the primary state before proceeding. If the primary state differs from the replica state, it logs a warning and returns early preventing an empty diff from being written on top of a valid merged diff. This is behind the mr_refresh_use_primary feature flag.

References

Screenshots or screen recordings

Before After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #583576

Edited by Sincheol (David) Kim

Merge request reports