Keep streamed HTML tasks short in the chunk writer

What does this MR do and why?

The chunk writer used for generic HTML streaming started near its largest chunk size, so the render balancer almost never ran, and each 100 ms timed flush parsed everything that had arrived in one long task. This affects every streamed page that uses this writer: blame today, and Rapid Diffs until !253645 lands, after which Rapid Diffs no longer uses it. This MR starts the writer at the smallest chunk size, routes timed flushes through the balancer, serialises writes, joins pending chunks once per flush, and waits for the final drain on close. No UI change.

Measurements

Local 1,000-file merge request (998 streamed files, about 85k elements), Overview tab then Changes tab, headless 1440x900, no extensions, master and the branch served by the same local stack, 3 runs each, medians. Measured on Rapid Diffs alone on master (this writer's current consumer):

Browser Blocked before Blocked after Longest task before Longest task after Last file before Last file after
Chromium 151 748 ms 772 ms (within noise) 319 ms 239 ms 5.0 s within noise
Firefox 152 2,518 ms 3,197 ms 1,024 ms 328 ms 5.7 s within noise

The longest task drops by about a quarter in Chromium and by two thirds in Firefox. Firefox total blocked time rises because more, shorter flushes each end in master's full file-browser re-render. !253736 (merged) removes that re-render, so this MR should merge after it.

Reproduction steps

  1. Open the blame page on a large file. Confirm streaming still completes and rows appear progressively.
  2. Open a large MR, go to the Changes tab, and record a Performance profile during streaming.
  3. Confirm no task takes longer than about 250 ms during streaming.

References

Split out of !253645 (original umbrella MR, now the complete-file streaming MR).

Sibling MRs from the same investigation:

  • !253645 - Insert complete diff files while streaming Rapid Diffs
  • !253736 (merged) - Stop re-rendering the file browser per streamed diff file
  • !253737 (closed) - Cut redundant per-file scans while streaming Rapid Diffs
  • !253738 (closed) - Skip the web streams polyfill when the browser can pipe natively
  • !253740 (closed) - Render identical diff file header buttons once per request

Feedback thread: #596236

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Stanislav Lashmanov

Merge request reports

Loading
Loading