Skip the post-merge diff commit preload in RefreshService

post_merge_manually_merged preloads the full diff-commit graph for every open merge request targeting the pushed branch - about 2,784 of them on every push to gitlab-org/gitlab master - so that the handful the push actually merges can avoid a lazy load. Removing the preload cuts RefreshService CPU by 99.5% on a local fixture (8.034s to 0.044s on a 2-commit push).

Two things read the graph, and both are cheap without it. PostMergeService reads the commits of each merge request it marks as merged, which costs one extra query per merged merge request instead of a preload across every candidate. The head_commit_sha NULL fallback for pre-8.4 diffs is the other, and it is faster without the preload. Both are covered by new specs.

Behind merge_request_refresh_skip_diff_commit_preload (project actor, default off).

Detailed context for AI agents

Mechanism

preload_latest_diff_commit preloads latest_merge_request_diff -> merge_request_diff_commits -> merge_request_commits_metadata -> [commit_author, committer], against every open merge request targeting the pushed branch.

Almost nothing on this path reads that graph. The filter immediately after it checks diff_head_sha and merge_request_diff.state, both columns already loaded by preload_project_and_latest_diff. diff_head_commit resolves through project.commit_by(oid: head_commit_sha), a Gitaly lookup keyed on a column.

The one real consumer is PostMergeService, which reaches the graph through MergeRequestMetricsService#merge -> Analytics::MergeRequestMetricsCalculator#productivity_data -> MergeRequestDiff#commits. That is EE-only and runs once per merge request the push actually marks as merged, typically none or one. Tracing every call to MergeRequestDiff#merge_request_diff_commits across a push with 250 bystander merge requests and 10 that the push merges gives 10 reads, all of them from the metrics calculator. So the preload was loading the graph for every candidate to save a lazy load for the few that need it, and dropping it trades a preload across thousands of merge requests for one query per merged merge request. The other reads in that trace come from ReloadDiffsService writing new diffs, which no preload can serve because those diffs did not exist when it ran.

It is expensive out of all proportion to its query count. MergeRequestDiffCommit belongs_to :merge_request_commits_metadata carries an instance-dependent scope, ->(diff_commit) { where(project_id: diff_commit.project_id) }. Rails evaluates that lambda once per record during preload and builds a relation object each time - nearly free in queries, brutal in CPU and allocations.

The single consumer is MergeRequestDiff#head_commit_sha falling back to reading commit rows when the column is NULL. That column has been populated since GitLab 8.4 (January 2016). Without the preload the fallback is faster, because commit_shas runs an indexed LIMIT 1 query when the association is not loaded instead of loading every commit row and sorting in Ruby.

Measurements

Local bench: 100 open merge requests, each with a 30-commit diff (3,000 diff commits), 2-commit push.

Isolating the preload levels on the relation load:

preload	cpu	queries	allocations
none	0.012s	4	20,513
+ diff commits	0.238s	4	204,737
+ commits and metadata	7.998s	5	586,619
+ author and committer	8.000s	7	606,789

The metadata level alone accounts for 7.76s of the 8.0s, and adds one query.

Full RefreshService on the same fixture:

	before	after
cpu	8.034s	0.044s
allocations	654,974	68,646
queries	20	17

Cost of the change

Dropping the preload turns two reads that used to come from memory into lazy loads. Neither changes behaviour; both are extra queries.

Per merge request the push marks as merged: one extra query. PostMergeService reads that merge request's commits through the metrics calculator described above. Tracing a push with 250 bystander merge requests and 10 that the push merges gives exactly 10 of these. A real push merges none or one.

Per merge request that merely targets the branch, when its latest diff has a NULL head_commit_sha: one extra query. This is the one worth watching, because it scales with the number of candidates rather than the number of merges. diff_head_sha falls back to reading the commit rows, which the preload used to serve. Measured on a 250 merge request fixture with every head_commit_sha nulled out:

	queries	cpu
head_commit_sha present	16	0.064s
head_commit_sha NULL	266	0.350s

The fallback stays an indexed LIMIT 1 lookup, so it is one query per merge request with no further fan-out, but at scale that is a query per open merge request on the branch.

head_commit_sha has been written on every diff since GitLab 8.4 (January 2016), so a NULL requires an open merge request whose latest diff was created before then and has never been regenerated in the decade since. Expected to be zero on GitLab.com. Confirm before widening the rollout:

SELECT COUNT(*)
FROM merge_requests mr
JOIN merge_request_diffs d ON d.id = mr.latest_merge_request_diff_id
WHERE mr.state_id = 1 AND d.head_commit_sha IS NULL;

If that is not near zero, the fix is to filter post-merge candidates in SQL on latest_merge_request_diff_id with an explicit head_commit_sha IS NULL branch rechecked in Ruby. Benchmarked at 250 merge requests it returns an identical result set and costs 2.3x less CPU and 2.8x fewer allocations than today's Ruby filter, so it is a strictly better follow-up rather than a fallback.

Everything else the trace surfaced (MergeRequestDiff#set_count_columns and #save_commits) comes from ReloadDiffsService writing new diffs during reload_merge_requests. Those diffs do not exist when the preload runs, so no preload ever served them and dropping it changes nothing there.

Production evidence

Two UpdateMergeRequestsWorker jobs on gitlab-org/gitlab, 2026-08-20, both ordinary 2-commit merge-train merges to master, different authors:

• jid 18d1950b138cebd3abe821ed - duration 331s, cpu_s 328, mem 1.795 GB, 4.074M objects
• jid d7a1dea34b341a42fd74c25d - duration 343s, cpu_s 339, mem 1.797 GB, 4.075M objects
• jid 56d64468b3b28af82b215dc6 - duration 311s, cpu_s 306, mem 1.797 GB, 4.075M objects

All three at cpu/wall ~0.99, with db_duration_s of only 3.7s, 2.5s and 4.2s, gitaly_calls 16 and db_main_write_count 17 in every case. db_main_count is 87 in all three and db_main_replica_count is 2362 / 2363 / 2358.

The third job is worth calling out: it landed at 11:19 UTC on a later release, and the change it was merging was a 10-file spec tidy-up (+297/-326). A near-identical fingerprint across three different pushes, three different authors and two releases is the signature of a fixed traversal that does not depend on push contents.

Related context

• Same worker and shard as INC-12753 (gitlab-com/gl-infra/production#22645 (closed)), but a different trigger: this cost is fixed per push rather than driven by push size.
• Independent of merge_request_refresh_batched_commit_lookup (!248701 (merged)), which changes whether a target-branch merge request needs its diff reloaded and does not touch this path.
• The three-level nesting was added in September 2025 by 0c4740bca8e3 and f8a12c900c70, for MergeRequest#recent_commits. That method uses a separate preload_commits_metadata helper, so this call site inherited the fan-out without needing it.
• After this change preload_latest_diff_commit has no callers left. It still has its own spec coverage in spec/models/merge_request_spec.rb, so removing the scope is left as a follow-up rather than bundled here.

Tests

• New spec asserting RefreshService does not call preload_latest_diff_commit. Verified red before the change, green after.
• New spec: a merge request whose diff has a NULL head_commit_sha - the pre-8.4 shape the preload existed to serve - is still detected and closed as manually merged.
• spec/services/merge_requests/refresh_service_spec.rb: 101 examples, 0 failures.
• spec/models/merge_request_spec.rb preload scope block: 6 examples, 0 failures.
• Local flag-validation suite for !248701 (merged): 41 examples, 0 failures.
• Rubocop clean on both changed files.

Rollout

Behind merge_request_refresh_skip_diff_commit_preload, project actor, gitlab_com_derisk, default off. Enabling it skips the preload.

The change removes work that is provably unread on this path, and the one legacy fallback it could affect is pinned by a new spec, so the flag is here for a fast rollback on a hot worker rather than because the behaviour is in doubt. Rollback is setting the flag back to false - the next push takes the old path, and there is nothing to repair.