Resolve diff note line code for files larger than 1 MB

What does this MR do and why?

POST /projects/:id/merge_requests/:merge_request_iid/discussions returns HTTP 400 (Note {line_code: ["can't be blank", "must be a valid line code"]}) for every comment on a text file larger than 1 MB — including a comment on line 1.

Creating a DiffNote validates presence of line_code, which is resolved by unfolding the diff via Gitlab::Diff::LinesUnfolder. The guard old_line > @blob.lines.size uses Gitlab::BlobHelper#lines, which returns [] when the blob is not viewable? — and viewable? is !large? && text_in_repo? with large? = size > 1 MB. So for any file over 1 MB, @blob.lines.size is 0, the guard always bails (old_line > 0), the line is never unfolded, line_code stays blank, and the request 400s.

This is essentially pre-existing: large? uses the real blob size, so @blob.lines is empty regardless of the diff blob fetch limit.

The fix counts newline bytes in @blob.data (the already-fetched, possibly size-capped data) instead of using @blob.lines. This:

• works for files over 1 MB (the blob data is available even when BlobHelper#lines is empty);

• matches git's LF-based line numbering and #blob_lines (both split @blob.data on \n), unlike BlobHelper#lines, which additionally splits on \r/\r\n and keeps a trailing empty element (over-counting);

• uses an allocation-free String#count byte scan — O(n) time, O(1) memory, and ~3.4× faster than materializing a line array:

  approach	~time / MB	allocation
  data.count("\n") (chosen)	~0.13 ms	none
  data.lines.size	~0.46 ms	~N line strings + array
  each_line.count	~0.50 ms	Ruby-level enumeration

The end_with?("\n") term adds the final unterminated line so the count equals String#lines.size for files without a trailing newline.

Bounded processing (viewable-size cap)

Both the guard and #blob_lines read the data through a single blob_data helper that caps it at the viewable size (Gitlab::BlobHelper::MEGABYTE, 1 MB). The diff blob fetch is already size-capped (512 KB by default), so this is a no-op on the normal path, but it guarantees we never count or parse more than a viewable-sized chunk even if the blob was fully loaded upstream via load_all_data! (e.g. during highlighting on the diff-rendering path). This keeps unfolding bounded (~28 ms for a fully-loaded 1 MB blob) instead of scaling with an arbitrarily large loaded blob.

Slicing at the byte cap can split a multibyte UTF-8 character on the 1 MB boundary, leaving an invalid-UTF-8 tail. String#count("\n") raises ArgumentError ("invalid byte sequence in UTF-8") on such a string, which would turn the intended 400 → 201 fix into a 500 for a fully-loaded > 1 MB blob. So blob_data scrubs the trailing partial character after slicing. This only runs on the > 1 MB branch (blobs within the cap return unchanged), and scrub is a no-op on valid data — @blob.data is always valid UTF-8 or binary via EncodingHelper#encode!, so the invalid encoding only ever exists as an artifact of our own boundary slice.

This does not regress any other unfolder consumer (note creation, draft-note publishing, or the render-time CollectionUnfolder): for files ≤ 1 MB the cap is a no-op and behavior is unchanged; for files > 1 MB every consumer previously bailed unconditionally (@blob.lines empty), so the capped path can only do the same or better.

Scope

This resolves comments whose line is within the fetched blob window (line 1 and the common case where changes/comments are near the hunk). A comment far beyond the size-capped fetch on a large file still bails — resolving that requires loading more of the blob (a memory trade-off), left for a follow-up.

Feature flag

Gated behind the gitlab_com_derisk flag unfold_diff_note_large_blob (default disabled). A single check in Gitlab::Diff::LinesUnfolder covers both consumers (note creation and render-time CollectionUnfolder).

• Enabled — the fix above: byte-scan line count + 1 MB-capped, scrubbed blob_data.
• Disabled — byte-for-byte the previous behavior: the guard uses @blob.lines.size (bails for > 1 MB blobs) and #blob_lines reads @blob.data uncapped.

Rationale for gating: LinesUnfolder also runs at render time (context expansion around notes on every diff view), and the byte scan counts \n only (vs BlobHelper#lines also splitting on \r) — so a kill-switch lets us disable instantly on GitLab.com if needed.

Rollout issue: #605432

References

• Bug report: #604785
• Reproduced with kballon-bug-report/zd731611_mr_discussion_400!1 (closed) (a ~1 MB long-file).

Screenshots or screen recordings

Not applicable — backend/API-only change.

How to set up and validate locally

1. Enable the flag: in a Rails console, Feature.enable(ob).
2. Create a text file larger than 1 MB on the target branch, change a line near the top on a source branch, and open a merge request.
3. Comment on line 1 via the API:

   POST /projects/:id/merge_requests/:merge_request_iid/discussions

4. With the flag enabled the discussion is created (201efore this change) the request returns 400`.

Automated coverage (both flag states, plus the multibyteses):

bundle exec rspec \
  spec/requests/api/discussions_spec.rb \
  spec/lib/gitlab/diff/lines_unfolder_spec.rb

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.