[Feature flag] Rollout of diff_stitcher_single_file_lookahead

What to do

Enable diff_stitcher_single_file_lookahead on a test group, check three pages (listed below), watch the canaries, then widen.

  • Flag: diff_stitcher_single_file_lookahead
  • Type: gitlab_com_derisk, default off
  • Actor: the repository container (a project in the common case), matching the tree_entries_filesystem_sort precedent in the same class
  • Introduced by !257263 (merged)

What the flag gates

  • Gitlab::GitalyClient::DiffStitcher#each holds each streamed patch until the next one arrives, so single_file? can report whether the stream held exactly one patch.
  • On: the first file of a multi-file Gitaly-streamed diff no longer auto-expands.
  • Off: master's behaviour. single_file? counts patches seen so far, so the first patch of any stream reads as single and that file auto-expands.
  • The gitlab-generated collapse fix in the same merge request is NOT gated. expand_diff? short-circuits on generated before it consults single_file?, so turning this flag off does not reintroduce #572763 (closed).

Blast radius

This is wider than merge request diffs.

  • DiffStitcher is built in Gitlab::GitalyClient::CommitService#call_commit_diff, which serves both diff and diff_from_parent.
  • Commit views and compare views stream through it unconditionally.
  • Merge request diffs stream through it only when:
    • ignore_whitespace_change is set (?w=1, "Show whitespace changes" turned off, or an anonymous visitor, since DiffHelper#hide_whitespace? is true for those), or
    • the persisted diffs were purged.

Expected change (not a regression)

  • In multi-file Gitaly-streamed diffs the first file stops auto-expanding.
  • A large first file, over the viewer's 1MB collapse limit, now renders collapsed like the files after it.

How to verify correctness

This change fails silently rather than loudly. The risk is a patch dropped, duplicated or reordered, and nothing currently counts files rendered per diff. So the check is manual.

  • Independent detector: the diff file count header comes from a separate DiffStats RPC, not from the stitcher. Compare it against the rendered file list. A mismatch means a dropped or duplicated patch, without trusting the code under test.
  • After enabling for a group, check these pages:
    • A multi-file commit.
    • A compare view.
    • A merge request with ?w=1 where two or more files match a gitlab-generated pattern.

Canaries that should not move

  • gitaly_duration_s and request latency on diff endpoints should stay flat. Holding one patch should not change time to first byte, because decorate! materialises the whole collection before streaming starts.
  • If gitaly_duration_s drops, treat that as instrumentation breaking rather than a win. An earlier rejected approach used Enumerator#next, which would have moved the stream read into a Fiber where RequestStore is invisible, silently losing the timings recorded in the ensure of Gitlab::GitalyClient::Call#instrument_stream. This design avoids that, and a drop would mean the property regressed.

Errors to watch

  • Sentry for anything raised from lib/gitlab/gitaly_client/diff_stitcher.rb or lib/gitlab/git/diff_collection.rb.
  • The Error streaming diffs log line from RapidDiffs::StreamingResource.
  • 5xx rates on the diff and compare endpoints.

Known behaviour difference

  • If the gRPC stream fails part way through, the consumer receives one fewer patch before the exception than it would with the flag off, because the last patch is still held.
  • This only matters to a caller that rescues and keeps partial results. Not considered a blocker.

Rollout steps

  1. Enable the flag for a test group and run the page checks above.
  2. Watch the canaries and error signals for a full day of normal traffic.
  3. Widen gradually (for example a few more groups, then a small percentage of actors, then larger percentages), repeating the checks at each step.
  4. Enable by default once the ladder completes with no regressions, and change the flag definition to default on.
  5. Remove the flag and the off-path code in a follow-up merge request.

Rollback

  • Disable the flag.
  • The gitlab-generated fix stays in place, since it is ungated.