Add internal by_thread_ts checkpoint read endpoint (phase 2, 2/3)

What does this MR do?

Part 2/3 of the phase-2 incremental checkpoint read path (split out of !242398 (closed)). Stacked on !247133 (merged) (the read engine).

Adds GET .../workflows/:id/checkpoints/by_thread_ts?thread_ts=… — a new internal getter keyed by the durable langgraph thread_ts (not the checkpoints-table integer id, which disappears when that table is retired, #605653 (closed)). Behind duo_workflow_read_incremental_checkpoints it loads the latest header for the thread_ts (partition-pruned by workflow_created_at) and rebuilds channel_values from the blobs via the engine. No p_duo_workflows_checkpoints read.

Includes the regenerated openapi_v3.yaml / token-scope docs for the endpoint.

Dependencies

  • Requires !247133 (merged) (read engine + model methods + FF). Retarget to master once that merges.
  • AIGW companion (required before enabling the FF): the gateway must call this by_thread_ts endpoint instead of the integer-id route.

Testing

Internal-API request specs (by_thread_ts + legacy) updated and green.

Manual QA

Compares the new by_thread_ts blob-reconstruction path against the legacy full-table GET .../checkpoints list, per thread_ts.

Setup (GDK rails console) — the endpoint needs an ai_workflows OAuth token from the trusted Duo Workflow application (a plain PAT does not work, because ai_workflows is an OAuth-only scope):

# bin/rails runner
u = User.find(1)
t = Ai::DuoWorkflows::CreateOauthAccessTokenService.new(
  current_user: u, organization: u.organizations.first
).execute[:oauth_access_token]
puts t.plaintext_token   # 2h TTL

Use a workflow that has both paths written (incremental_checkpoints_enabled plus blobs). Requests go through Workhorse (the endpoint requires the Workhorse JWT, which nginx→workhorse injects for the normal GDK URL).

TOKEN=<token from above>
BASE=http://gdk.test:3000/api/v4/ai/duo_workflows/workflows/<id>/checkpoints

# legacy list (full table)
curl -s -H "Authorization: Bearer $TOKEN" "$BASE?per_page=100" -o /tmp/list.json

# per thread_ts: value mismatches on shared channels, plus key-set diffs
for TS in $(jq -r '.[].thread_ts' /tmp/list.json); do
  curl -s -H "Authorization: Bearer $TOKEN" "$BASE/by_thread_ts?thread_ts=$TS" -o /tmp/by.json
  jq -S --arg ts "$TS" '.[]|select(.thread_ts==$ts)|.checkpoint.channel_values' /tmp/list.json > /tmp/L.json
  jq -S '.checkpoint.channel_values' /tmp/by.json > /tmp/R.json
  echo "$TS | val-mismatch=$(jq -rn --slurpfile l /tmp/L.json --slurpfile r /tmp/R.json '[($l[0]|keys)[] as $k|select(($r[0]|has($k)) and ($l[0][$k]!=$r[0][$k]))|$k]|join(","))') | only-in-full=$(jq -rn --slurpfile l /tmp/L.json --slurpfile r /tmp/R.json '(($l[0]|keys)-($r[0]|keys))|join(","))') | only-in-recon=$(jq -rn --slurpfile l /tmp/L.json --slurpfile r /tmp/R.json '(($r[0]|keys)-($l[0]|keys))|join(","))')"
done

Result (workflows 623 and 624, 93 checkpoints total): val-mismatch is empty for every checkpoint — each blobbed channel reconstructs to the exact full-table value. The paths are not byte-identical by design:

  • only-in-full: scalar channels (approval, goal, namespace, last_human_input, branch:to:*) are not blobbed, so the reconstruction omits them.
  • only-in-recon: __start__, the ephemeral langgraph input channel, is dropped from the full checkpoint after consume but persists in the accumulated blobs.

The gateway must supply scalars from its own state and tolerate the extra __start__.

Related: #604677 (closed)


Restructure — epic &22939

Part of the phase-2 read-path epic. Each consumer is gated by its own flag in addition to the master kill switch.

Code still gates on the shared duo_workflow_read_incremental_checkpoints flag today; repointing to the per-consumer flag is deferred until the restructure lands.

Edited by Eduardo Bonet

Merge request reports

Loading
Loading