Fix latest/first checkpoint lookup for incremental-only workflows

Eduardo Bonetrequested to merge
612586-verify-whether-internal-checkpoint-listing-ap into master

What

WorkflowPresenter#latest_checkpoint/#first_checkpoint back the GraphQL latestCheckpoint/firstCheckpoint fields, which the Duo Workflow Service fetches as its dominant resume source on every normal graph invocation (_workflow_config["latest_checkpoint"]/["first_checkpoint"], checked first in aget_tuple before any other fallback). Both read the legacy checkpoints table unconditionally.

duo_workflow_write_incremental_only stops writing that table, so once it's on for a workflow both fields return null, and the Duo Workflow Service treats an established workflow as brand new — a full resume break, not an edge case.

How

Fall back to checkpoint_headers when the workflow reconstructs from blobs (Workflow#reconstruct_from_blobs_for_graphql?), reusing the existing dw_read_blobs_graphql flag rather than adding a new one — it already gates the presenter's channel_values reconstruction, so the object lookup and the content it returns stay in lockstep instead of risking two flags that must both be flipped together.

Added Workflow#earliest_checkpoint_header and gave #latest_checkpoint_header a checkpoint_ns: argument. WorkflowEventType/WorkflowCheckpointEventPresenter needed no changes — verified they already work against a CheckpointHeader-backed object, since they only touch attributes/associations both Checkpoint and CheckpointHeader share.

Database

Workflow#latest_checkpoint_header/#earliest_checkpoint_header query p_duo_workflows_checkpoint_headers filtered by workflow_id, workflow_created_at (the partition key, always supplied by the checkpoint_headers association scope), and checkpoint_ns, ordered by thread_ts/id. Checked against the dev DB: partition pruning via workflow_created_at keeps every real call scoped to one partition, and execution stays sub-millisecond both for a light workflow (9 headers, index scan) and the heaviest workflow in the dev DB (158 headers, seq scan chosen by the planner over the smaller partition-local row set). Full plans and query text: see the ~"database::review pending" thread on this MR.

QA

  1. Enable the following feature flags
Feature.enable(:duo_workflow_incremental_checkpoints)
Feature.enable(:duo_workflow_read_incremental_checkpoints)
Feature.enable(:dw_read_blobs_graphql)

  1. Start a conversation with duo chat, one or two questions

  2. Restart AIGW and DWS

gdk restart duo-workflow-service gitlab-ai-gateway
  1. Continue the conversation

You should see in the logs data being loaded from p_duo_workflows_checkpoint_headers, not p_duo_workflows_checkpoints

tail -f log/development.log log/graphql_json.log | grep -i --line-buffered -E "duo_workflows_checkpoint|latestCheckpoint|firstCheckpoint|p_duo_workflows_checkpoints|p_duo_workflows_checkpoint_headers"

References

Edited by Eduardo Bonet

Merge request reports