Send the checkpoint channel membership is enabled
Problem
The GitLab backend cannot filter the checkpoint blob fold to live channels when duo_workflow_write_incremental_only is on.
The fold is a union of every channel ever written, because blobs are an append-only log. A blob log cannot express a channel deletion. gitlab-org/gitlab#613975 (closed) needs a channel membership list to filter that fold.
The backend stores this membership in channel_keys on p_duo_workflows_checkpoint_headers (added in gitlab-org/gitlab!250189 (merged)). It derives the value from channel_values.keys in Ai::DuoWorkflows::CreateCheckpointService. In incremental-only mode, the gateway drops channel_values from the payload before it reaches the backend. Every header written in that mode gets channel_keys = NULL.
There is no legacy full row in this mode, so the read path has no fallback either.
Why the backend cannot derive it
duo_workflow_service/checkpointer/gitlab_workflow.py (around line 1560, added by !6519 (merged)) strips channel_values from the skeleton payload:
if write_incremental_only:
payload["checkpoint"] = {
key: value
for key, value in checkpoint.items()
if key != "channel_values"
}The backend cannot rebuild the membership from what remains. channel_versions keeps channels consumed since the last checkpoint, so its key set is a superset of the live membership (see the spec case "when a channel has a version but no value" in ee/spec/services/ai/duo_workflows/create_checkpoint_service_spec.rb). The blob set is also wrong, because each step blobs only the channels that changed, not the full membership.
Proposal
Add the live channel membership to the incremental-only payload explicitly. For example, send a channel_keys field with list(checkpoint["channel_values"].keys()), computed before the skeleton drops the values. The backend then stores that list directly instead of deriving it.
The alternative, keeping channel_values in the payload, would defeat the purpose of incremental-only mode.
Sequencing
This change must land and deploy before duo_workflow_write_incremental_only is enabled anywhere. The flag is off today, so no production data is affected yet.
The backend counterpart is gitlab-org/gitlab#617794 (closed), which accepts the new field. It has to ship first, so an older backend keeps working while the gateway rolls out.
gitlab-org/gitlab!250826 (merged) assumes the membership is present, and falls back to the dual-written legacy row when it is missing. That fallback has no legacy row to read in incremental-only mode.