Parse only the new part of each chat log snapshot

What does this MR do and why?

Duo Agentic Chat history now loads with far less CPU and memory on long chats. Each request re-parsed a full chat log snapshot per checkpoint thread, the main cost behind slow getWorkflowLatestCheckpoint requests. The output stays the same because the skipped entries are already in the history.

Design notes

ChannelValuesReconstructor#channel_history (in ee/lib/gitlab/duo_workflow/channel_values_reconstructor.rb) folds the ui_chat_log blobs across a workflow's whole checkpoint chain. Each thread starts with a compaction snapshot that holds the full log so far. Before this MR, every snapshot was inflated and parsed in full on every request.

A new private class, SnapshotParser, inflates each snapshot into one of two reused buffers. When the snapshot's text starts with the previous snapshot's text minus its closing ], and the next byte is ,, it parses only "[" + the rest. An identical snapshot parses as []. Anything else, like a trimmed log or a rewritten entry, is parsed whole, as before.

Output can't change: every entry of the previous snapshot is already in the history when the next snapshot arrives (seeded or matched by #unseen_entries), so #unseen_entries would drop those entries anyway.

A delta that decodes to null or false replaces the history. The parser then forgets the previous snapshot (snapshots.reset unless history.is_a?(Array)), so the next snapshot is parsed whole. Specs cover both values. A snapshot that directly follows such a delta, or a Hash delta, now seeds the history. Master dropped that snapshot, and after a Hash raised NoMethodError on the next delta.

Malformed input still raises as it did before. The tail path is taken only when the previous snapshot held at least one entry, and when the text after the comma isn't blank. So [a] followed by [a,], or [] followed by [,a], falls back to a full parse, which raises CorruptBlobError.

Both inflate buffers are binary strings, like the result of Zlib::Inflate.inflate. A UTF-8 buffer would make Gitlab::Json::StreamValidator reject invalid UTF-8 bytes that .decode accepts.

A new private class, SeenMessageIds, keeps the set of seen message_ids and only indexes entries added since the last snapshot, instead of rebuilding the set from the whole history on every snapshot. It starts over if the history object changes.

Zlib::Inflate#finish after inflate(data, buffer:) returns an invalid object that segfaults on use (reproduced on Ruby 3.3.11, zlib gem 3.2.3). The code checks finished? and never calls finish. That pattern passed under GC.stress and GC.compact.

The tail slice stops one byte before the closing ]. A slice that reaches the end of the buffer shares it, and the next inflate then copies the whole buffer (measured: 66 MB extra over 10 inflates of a 4.4 MB buffer, zero when stopping one byte short).

Intentional divergence: Gitlab::Json.safe_parse limits (for example 100,000 total elements) now apply to each parsed tail, not to the whole snapshot. Each parse stays bounded, and the history already grows past those limits through deltas.

No feature flag: the output is identical and the change is local to one method, like the previous performance change to this file in !258386 (merged).

Out of scope: dropping the JSON stream validator is tracked in #631640 (closed).

Production prefix hit rate and benchmark

Measured on Database Lab on 2026-10-01, with an exact byte comparison of consecutive snapshots inside the clone (a PL/pgSQL zlib inflater; no data left the clone).

Snapshots created since 2026-09-10: 99.5% of 2,434 snapshot pairs (99.2% of bytes) extend the previous snapshot, across 65 random chat workflows. The 11 misses differ within the first 80 bytes, which matches a log trimmed at compaction.

Snapshots created before 2026-09-10: 50.6% of 776 pairs. These misses differ near the end of the log. The cause was not found. Those rows leave the 30-day partitions around 2026-10-10.

Workflow 7993706 (the large chat workflow behind the parent issue): 326 of 326 pairs.

Fixture CPU before CPU after Objects before Objects after Malloc before Malloc after
Shaped like workflow 7993706 (98% prefix hits) 2.51 s 0.30 s 7.40M 265k 892 MB 36 MB
65% prefix misses 2.37 s 1.57 s 7.30M 4.64M 852 MB 599 MB

Compare ratios, not seconds: local CPU is slower than production CPU by a different factor. Output matched the code on master for all five benchmark fixtures (trimmed history, in-place rewrites, entries without message_id).

References

  • Issue: #631639 (closed)
  • Parent: #631606
  • Sibling MR !258940 (merged) also changes channel_history (it adds from: to resume a cached history). Whichever MR lands second must keep both: start the fold from from and keep SnapshotParser. A resumed fold has no previous snapshot, so its first snapshot is parsed whole, which is correct.

Screenshots or screen recordings

No UI change.

How to set up and validate locally

  1. In GDK with Duo Agentic Chat working, open the chat panel in a project (for example http://gdk.test:3000/gitlab-org/gitlab-test) and send at least four prompts in one chat, so the workflow has several checkpoint threads.
  2. Find the workflow id: in rails console, run Ai::DuoWorkflows::Workflow.order(:id).last.id (or use the id shown in the chat URL).
  3. On master, in rails console, run:
w = Ai::DuoWorkflows::Workflow.find(WORKFLOW_ID)
Digest::SHA256.hexdigest(Gitlab::Json.dump(w.channel_message_history(w.latest_checkpoint_header, 'ui_chat_log')))
  1. Check out this branch, restart rails-web (gdk restart rails-web), and run the same snippet in a new console. Expect the same digest.
  2. Reload the chat panel. Expect the full history to show, in the same order as before.
Specs

bundle exec rspec ee/spec/lib/gitlab/duo_workflow/channel_values_reconstructor_spec.rb covers prefix hits, an identical snapshot, the gateway's ", " separators, a number the next snapshot extends, a rewritten entry, deltas that replace the history, malformed JSON at the junction, invalid UTF-8, and a truncated snapshot.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Eduardo Bonet

Merge request reports

Loading
Loading