Add incremental checkpoint blob storage for Duo Workflows
What does this MR do and why?
Implements the Rails backend for incremental checkpoint blob storage in Duo Workflows.
Instead of sending the full channel_values state with every checkpoint, agents can now send only the changed channels as channel_blobs. The gateway reconstructs full state by walking the parent_ts chain and accumulating blobs per channel (last-write-wins per channel key).
This is the Rails side of the implementation. The gateway side is in gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!5672 (merged).
Implementation details
- New table
duo_workflows_checkpoint_blobs: stores per-channel versioned blobs keyed by(workflow_id, thread_ts, channel, version). Up to 1 MiB of base64-encoded binary data per blob. Ai::DuoWorkflows::CheckpointBlobmodel: usesBulkInsertSafeandSyncWorkflowAttributesconcerns.CreateCheckpointService: acceptschannel_blobsparam and writes checkpoint + blobs atomically in a transaction.Workflow#accumulated_blobs_for(checkpoint): walks theparent_tschain (bounded by workflow step count), fetches all blobs for the chain in a single query, returns them oldest-first.- API:
POST /checkpointsaccepts optionalchannel_blobsarray;GET /checkpoints/:idreturnschannel_blobs(accumulated blobs when feature flag is on, or the checkpoint's own blobs when off). - Feature flag
duo_workflow_incremental_checkpoints(wip, default off) gates all blob writing and accumulated blob resolution.
MR acceptance checklist
- This MR has the correct label and milestone
- I have evaluated the MR acceptance checklist
- Feature flag added (
duo_workflow_incremental_checkpoints, wip, default off) - Migration uses
disable_ddl_transaction!+add_concurrent_foreign_key - Database dictionary entry added
- All tests pass (88 examples, 0 failures)
- RuboCop: 0 offenses
- OpenAPI docs regenerated
Related
- Gateway MR: gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!5672 (merged)
- POC reference: !238008 (closed)
- Work item: #596714 (closed)