Stop the notebook text diff at the render timeout
What does this MR do and why?
The notebook render timeout (1.5 s on web, 10 s in Sidekiq) now stops the diff process that computes a notebook (.ipynb) text diff, instead of covering only the notebook transform. IpynbDiff.diff returns a lazy Diffy::Diff, so the diff process ran after the timeout, and Open3.capture3 waits for the process to exit even when the timeout interrupts it. The change is behind its own feature flag, stop_notebook_text_diff_at_timeout (gitlab_com_derisk, default off); disabling it restores the current behavior.
Design notes
IpynbDiff::InterruptibleTextDiff (gems/ipynbdiff/lib/ipynb_diff/interruptible_text_diff.rb) is a Diffy::Diff subclass that overrides only #diff. It writes the same temporary files, runs the same binary (diff_bin) with the same options through Process.spawn in its own process group, reads standard output to the end, and waits for the exit status. Standard error goes to /dev/null; Diffy also discards it.
Thread.handle_interrupt(Object => :never)defers asynchronous interrupts (such as the oneTimeoutraises) everywhere except the read and the wait. When an interrupt stops the read or the wait,ensurekills the process group and reaps the process, so adiff_binwrapper cannot leave children running.- End of output only means that the process closed standard output, so the class waits for the exit status.
diffexits with 0 (no differences) or 1 (differences). Any other status, or a signal, raisesIpynbDiff::DiffErrorinstead of returning empty or partial text.DiffFilelogsIPYNB_DIFF_FAILEDand shows the raw diff, and the failure is not cached. - The output is the same as
Diffy::Diff: same binary, arguments, encoding handling (force_encoding('ASCII-8BIT')for output that is not valid UTF-8), andDiffy's own#eachand formats. The class supports whatIpynbDiffpasses (string sources, empty diffs allowed) and raisesArgumentErrorforsource: 'files'orallow_empty_diff: false. IpynbDiff.diff(..., interruptible: true)selects it. The default staysDiffy::Diff.
DiffFile. With the flag, render_notebook_diff passes interruptible: true and calls notebook_diff.diff inside Gitlab::RenderTimeout.timeout. A timeout goes to the existing rescue Timeout::Error (IPYNB_DIFF_TIMEOUT log, ipynb_semantic_diff_timeouts_total), and the file shows the raw diff, as for a transform timeout today. Without the flag, nothing changes.
Alternatives rejected.
- Move the text diff inside
Timeoutand keepOpen3.capture3:capture3joins the process inensure, so the call returns only whendiffexits, and its reader threads printIOError: stream closed in another thread. - Compute the text diff in Ruby with
diff-lcs, soTimeoutcan interrupt it: its hunks can differ from GNUdiff, which moves line positions and the line codes of existing notebook discussions, and it is slower for large inputs.
Measured locally (synthetic notebook pairs, macOS diff):
| Case | Diffy::Diff |
InterruptibleTextDiff |
|---|---|---|
| 476 KB with 40 PNG outputs | 0.036 s | 0.037 s, same text |
| 570 KB with 40 PNG outputs | 0.039 s | 0.038 s, same text |
| 327 KB, every line changed | 0.243 s | 0.236 s, same text |
0.5 s Timeout around a 3 s diff |
returns after 3.29 s | returns after 0.51 s, no process left |
Specs. gems/ipynbdiff/spec/ipynb_diff/interruptible_text_diff_spec.rb compares the text with Diffy::Diff for both include_diff_info values and for text that is not valid UTF-8, checks that the temporary files are removed (also after an interrupt), that a failing or killed diff raises, that unsupported options raise, and that an interrupt kills and reaps the process and the processes a wrapper started. spec/lib/gitlab/diff/rendered/notebook/diff_file_spec.rb checks the same through the render timeout, and that a failed diff shows the raw diff and is not cached.
Out of scope. The highlight pass of the rendered diff (DiffFile#highlighted_diff_lines) still runs outside the render timeout, and neither this MR nor the cache MR caches it. It is a follow-up.
Stacked MR, sibling, ownership
- Stacked on !263925 (merged) (target branch
635044-cache-notebook-diff-render), which adds the render cache behindcache_notebook_diffs. The two flags are independent, so either change can be rolled back alone. This one can roll out first. - Sibling: !263894 (merged) stops the metadata and stats endpoints from rendering notebooks, behind
reduce_notebook_diff_rendering. It changesapp/helpers/diff_helper.rb,app/models/merge_request_diff.rbandlib/gitlab/diff/file_collection/base.rb, and no file of this MR, so the merge order does not matter. ipynbdiffis a path gem (gems/ipynbdiff), so the app uses the change without a release or aGemfile.lockchange.- Ownership:
Gitlab::Diff::Rendered::Notebookandgems/ipynbdiffbelong to MLOps. The diff endpoints that call them belong to Code Review (code_review_workflow), so Code Review should co-review. - No changelog entry: the flag is off by default.
- No database changes.
References
- Related to https://gitlab.com/gitlab-org/gitlab/-/work_items/635044
- Builds on: !263925 (merged)
- Feature flag rollout issue: https://gitlab.com/gitlab-org/gitlab/-/work_items/635151
- Sibling: !263894 (merged)
Screenshots or screen recordings
No UI change.
How to set up and validate locally
The timeout is hard to reach from the UI, so this check uses the Rails console (gdk rails console).
-
Build two notebooks whose text diff takes a few tenths of a second:
nb = ->(lines) { Gitlab::Json.generate(cells: [{ cell_type: 'code', metadata: {}, outputs: [], source: lines }], metadata: {}, nbformat: 4, nbformat_minor: 5) } from = nb.((1..60_000).map { |i| "a#{i * 7919 % 100_003}\n" }) to = nb.((1..60_000).map { |i| "b#{i * 104_729 % 100_019}\n" }) -
Time the text diff under a 0.05 s timeout, with and without
interruptible:[false, true].map do |interruptible| text_diff = IpynbDiff.diff(from, to, interruptible: interruptible) started = Time.now Timeout.timeout(0.05) { text_diff.diff } rescue Timeout::Error [interruptible, (Time.now - started).round(2)] endExpect:
falsereturns only when thediffprocess ends (about 0.2 s on a MacBook),truereturns after 0.05 s. -
Run
Feature.enable(:stop_notebook_text_diff_at_timeout), then open a merge request that changes a notebook. Expect: the Changes tab shows the rendered notebook as before.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.