[FF] merge_request_refresh_skip_diff_commit_preload -- skip the unused post-merge diff commit preload in RefreshService

Summary

Adds the flag merge_request_refresh_skip_diff_commit_preload (project actor, type gitlab_com_derisk, default disabled, milestone 19.3) to skip an expensive, unused preload in MergeRequests::RefreshService#post_merge_manually_merged.

That method eagerly preloads the diff commit graph - latest diff, its commits, each commit's metadata, and each metadata row's commit author and committer - for every open merge request targeting the pushed branch. On gitlab-org/gitlab that is roughly 2,784 merge requests on every push to master. Nothing on this code path reads the preloaded data.

The cost comes from MergeRequestDiffCommit belongs_to :merge_request_commits_metadata, which has an instance-dependent association scope. Rails evaluates that scope lambda once per diff commit during preload and builds a relation object each time, adding almost no queries but enormous CPU and allocations.

Locally, on 100 open merge requests each with a 30-commit diff, on a 2-commit push: RefreshService CPU drops from 8.034s to 0.046s and allocations from 655k to 69k objects (about 99% CPU reduction).

Production evidence: three UpdateMergeRequestsWorker jobs on gitlab-org/gitlab on 2026-08-20, all ordinary small merges to master, different authors, spanning two releases, each showed 311-343s wall time, 306-339s CPU, about 1.8 GB and 4.07 million objects, but only 2.5-4.2s of database time. The near-identical fingerprint across unrelated pushes is the signature of a fixed traversal that does not depend on push contents.

Introduced by !251113 (merged). DRI: @marc_shaw. Team Slack channel #g_code_review.

What could go wrong?

The only consumer of the preloaded graph is MergeRequestDiff#head_commit_sha, which falls back to reading commit rows when the head_commit_sha column is NULL. That column has been populated since GitLab 8.4 (January 2016), so only pre-2016 diffs can reach this path, and a spec covers that case with the flag both on and off.

Without the preload, that fallback is actually faster - it runs an indexed query with LIMIT 1 instead of loading and sorting every commit row in Ruby.

The realistic failure mode is a merge request that was manually merged into the target branch not being detected and closed - it would stay open. There is no data change and nothing to repair.

Monitoring

  • Kibana, index pubsub-sidekiq-inf-gprd*: json.class : "UpdateMergeRequestsWorker" and json.meta.project : "gitlab-org/gitlab". Watch json.cpu_s, json.duration_s, and json.mem_total_bytes fall. Before the change these sit around 300s CPU and 1.8 GB on master pushes.
  • urgent-cpu-bound and urgent-ci-pipeline Sidekiq shard apdex and CPU saturation on https://dashboards.gitlab.net
  • UpdateMergeRequestsWorker exception rate.

Rollout

  1. Non-production:

    /chatops run feature set merge_request_refresh_skip_diff_commit_preload true --dev --pre --staging --staging-ref

    Leave 24 hours.

  2. Production, gitlab-org/gitlab first - it is the project showing the pathology and gives an immediate before/after on continuous master push traffic:

    /chatops run feature set --project=gitlab-org/gitlab merge_request_refresh_skip_diff_commit_preload true

  3. Percentage of actors, then global, waiting at least 15 minutes between steps:

    /chatops run feature set merge_request_refresh_skip_diff_commit_preload 25 --actors
    /chatops run feature set merge_request_refresh_skip_diff_commit_preload 100 --actors
    /chatops run feature set merge_request_refresh_skip_diff_commit_preload true

Rollback - set the flag back to false; the next push takes the old path:

/chatops run feature set merge_request_refresh_skip_diff_commit_preload false

Cleanup

Once stable, remove the flag, the preload_latest_diff_commit call site guard, and the YAML definition, then:

/chatops run feature delete merge_request_refresh_skip_diff_commit_preload --dev --pre --staging --staging-ref --production

The preload_latest_diff_commit scope on MergeRequest will then have no callers left and can be deleted along with its specs in spec/models/merge_request_spec.rb.