[FF] dw_read_blobs_trace — Reading workflow trace from incremental blobs

Summary

Roll out the workflow trace download (GET trace.jsonl), gated by the dw_read_blobs_trace feature flag. Part of the incremental checkpoint read path (epic &22939).

This consumer is additionally gated by the master kill switch duo_workflow_read_incremental_checkpoints (604687) — it reconstructs from blobs only when both flags are on.

Note

Process and guidance live in the docs — this issue is just the commands and a place to track the rollout. "Rolling out" means incrementally enabling the flag on GitLab.com to validate stability — it is not the same as releasing the feature, which happens when the flag is removed. Feature flag controls · Feature flag lifecycle

What could go wrong?

Rollout

Run all production /chatops in #production and cross-post to #g_agent-execution.

Non-production

/chatops gitlab run feature set dw_read_blobs_trace 50 --actors --dev --pre --staging --staging-ref
/chatops gitlab run feature set dw_read_blobs_trace true --dev --pre --staging --staging-ref

Production — percentage rollout (wait ≥15 min between steps, watch dashboards):

/chatops gitlab run feature set dw_read_blobs_trace <percentage> --actors

Or target specific actors instead:

/chatops gitlab run feature set --project=gitlab-org/gitlab,gitlab-org/gitlab-foss dw_read_blobs_trace true
/chatops gitlab run feature set --group=gitlab-org,gitlab-com dw_read_blobs_trace true
/chatops gitlab run feature set --user=<gitlab-username-of-dri> dw_read_blobs_trace true

Before global rollout

Confirm the relevant gotchas before going to 100% — see enabling a feature for GitLab.com:

  • Docs + version history updated
  • Breaking changes announced, if any
  • Change management issue opened, if required
  • External API consumers handled with a fail-open mechanism, if applicable

Cleanup

Remove the flag once deemed stable. Remove the flag and its YAML definition, then:

/chatops gitlab run release check <merge-request-url> <milestone>
/chatops gitlab run feature delete dw_read_blobs_trace --dev --pre --staging --staging-ref --production

Rollback

/chatops gitlab run feature set dw_read_blobs_trace false                                         # production
/chatops gitlab run feature set dw_read_blobs_trace false --dev --pre --staging --staging-ref     # non-production