Add API read path for incremental checkpoint blobs (phase 2)

What does this MR do?

Phase 2 of incremental checkpoint blobs: read persisted checkpoints back from the slim p_duo_workflows_checkpoint_headers table plus the incremental blobs, without touching the full p_duo_workflows_checkpoints table (which is being retired — #605653 (closed)).

New read path

  • GET .../workflows/278964/checkpoints/by_thread_ts?thread_ts=… — a new internal getter keyed by the durable langgraph thread_ts (not the checkpoints-table integer id, which disappears with that table). Behind the duo_workflow_read_incremental_checkpoints flag, it loads the latest header for the thread_ts (partition-pruned by workflow_created_at) and rebuilds channel_values from the blobs onto the slim header. No p_duo_workflows_checkpoints read.
  • trace_channel_values reads the latest header + blobs when the read flag is on.
  • accumulated_blobs_for bounds workflow_created_at (the dedicated partition key from !244403 (merged)), pruning to a single daily partition.
  • The legacy GET /:checkpoint_id no longer reconstructs — the full row still carries channel_values during dual-write, so it needs none.

Reconstruction

channel_values are folded from blobs by ChannelValuesReconstructor (compaction ⇒ replace, conversation ⇒ append), one current_thread group at a time. With AIGW self-contained blob groups (!6179 (closed)) each group re-seeds every channel as a full snapshot at group start, so a group reconstructs on its own.

Dependencies / stacking

Targets feature/checkpoint-headers-read-base = master era + !244403 (merged) (blob repartition by workflow_created_at) + !244914 (merged) (headers table + shadow write). Retarget to master once both merge.

  • AIGW companion (required before enabling the FF): the gateway must call the new by_thread_ts endpoint instead of the integer-id route. Tracked as an ai-assist follow-up.
  • Child MR !243813 (graphql read) is stacked on this branch and re-rebases after this update.

Feature flag

duo_workflow_read_incremental_checkpoints (wip, default off). The read path is a no-op until enabled; writes (headers + blobs) already shadow-write behind the workflow's incremental_checkpoints_enabled.

Testing

Model, internal-API (by_thread_ts + legacy), trace, and reconstructor specs updated and green.

Database

All read queries are keyed by workflow_id + workflow_created_at, so each prunes to the single daily partition of the target table and uses an index scan (no sequential scans, no cross-partition scans). Plans captured on the GDK DB (workflow_id = 123, created_at = 2026-07-20):

checkpoint_header_for / latest_checkpoint_header / ancestor_thread_ts (all against p_duo_workflows_checkpoint_headers):

SELECT * FROM p_duo_workflows_checkpoint_headers
WHERE workflow_id = 123 AND workflow_created_at = '2026-07-20 12:00:00'
  AND thread_ts = 'ts-1' ORDER BY id ASC;
 Sort  (cost=3.20..3.21 rows=1 width=188)
   Sort Key: id
   ->  Index Scan using p_duo_workflows_checkpoint_headers_20260720_workflow_id_idx
         on p_duo_workflows_checkpoint_headers_20260720
         Index Cond: (workflow_id = 123)
         Filter: (workflow_created_at = '2026-07-20 12:00:00+00' AND thread_ts = 'ts-1')

accumulated_blobs_for (against p_duo_workflows_checkpoint_blobs):

SELECT * FROM p_duo_workflows_checkpoint_blobs
WHERE workflow_id = 123 AND current_thread = 0
  AND workflow_created_at = '2026-07-20 12:00:00'
  AND thread_ts IN ('ts-1','ts-2') ORDER BY id ASC;
 Index Scan using p_duo_workflows_checkpoint_blobs_20260720_pkey
   on p_duo_workflows_checkpoint_blobs_20260720
   Index Cond: (workflow_created_at = '2026-07-20 12:00:00+00')
   Filter: (thread_ts = ANY('{ts-1,ts-2}') AND workflow_id = 123 AND current_thread = 0)

Each plan targets one _20260720 partition only. No new migrations — the tables and partition scheme come from the base branch (!244403 (merged), !244914 (merged)).

Edited by Eduardo Bonet

Merge request reports

Loading