### Summary Let an external system **subscribe to a Duo flow's lifecycle over a webhook** — so it can trigger a flow through the API and then receive pushed updates (started, progress, completed, failed) instead of polling. No custom code inside GitLab required. **Reference implementation:** https://gitlab.com/gitlab-org/gitlab/-/merge_requests/246849 — a working end-to-end proof of concept of the full idea. The issues below break it into shippable pieces; the MR stays as the reference to split from. **Demo:** [](https://youtu.be/w2iPc4hjY74) ### Where this comes from (the Challenge) This idea came out of the GitLab **DAP Extensibility challenge** (gitlab-org#22652) — an internal challenge exploring how to make external agents and tools first-class participants in the Duo Agent Platform, so events in outside systems can start a flow and get the result back with no human babysitting the run. This epic takes the "trigger a flow via API, listen to changes via webhook" primitive from that exploration and turns it into a deliverable feature. ### Why (product context) Duo runs a flow once and shows it on different **surfaces** through small adapters — today Slack and GitLab Duo notes. Those adapters live inside GitLab's codebase, so **an external product cannot add one for itself**. An external client _can_ already start a flow over the REST API, but after that it's blind to what happens. The only ways to follow along today don't fit external, server-to-server tools: - **Polling** the workflow status — wasteful and laggy. - **A live GraphQL subscription** (what our own web UI uses) — a WebSocket tied to the user who owns the flow. A stateless, multi-tenant backend service has no user session to hold a socket as, and can't see flows it didn't personally start. So any external integration (for example third-party incident-management or ticketing tools, CI jobs, or internal bots) that wants "kick off a Duo flow and get the result back" has to build a lot of plumbing that basically already exists internally. ### What we're building Reuse GitLab's **existing webhook system** instead of inventing a parallel one: - Turn on a capability flag, `duo_flow_callback_enabled`, on a normal **project or group webhook**. - When triggering a flow, reference that webhook by id via `callback_hook_id`. - GitLab then `POST`s a small JSON payload to that webhook every time the flow changes state. Under the hood this is a **third messaging adapter** (alongside Slack and Duo notes) — the generic one that removes the "must write code inside GitLab" limitation. It deliberately is **not** an event subscription (`*_events`) and is not part of `TriggerableHooks`: it's a capability flag, so it never fires from ordinary webhook event dispatch. ### What a client receives Four lifecycle events: `flow.started`, `flow.progress`, `flow.completed`, `flow.failed`. Payload envelope: ```json { "object_kind": "duo_workflow", "version": "1", "event": "flow.completed", "event_id": "flow.completed-2722", "project": { "id": 17, "path_with_namespace": "group/project", "web_url": "…" }, "workflow": { "id": 2722, "status": "finished", "web_url": "…" }, "client_reference": "run-abc123", "message": "Done!" } ``` - `message` on completion (agent's final answer), `error` on failure, `client_reference` echoed back for correlation, `event_id` for de-duplication, `version` for the payload contract. - Delivery reuses `WebhookDeliveryWorker` (retry/backoff) + `WebHookService` (signing, SSRF checks, delivery logging). **Terminal events are retried until delivered; progress is best-effort.** ### How it works (runtime path, for engineers) ``` flow state change → CallbackWorker (picks the adapter via AdapterRegistry) → lifecycle hook (on_flow_completed, …) → Webhook adapter builds the JSON payload → WebhookDeliveryWorker → WebHookService signs + POSTs → client endpoint ``` ### Permissions - **Turning the flag on a webhook:** Maintainer+ on the project, or Owner on the group (the standard `admin_web_hook` permission). - **Triggering a flow that references a hook:** anyone who can already create the flow. The hook was approved by a Maintainer/Owner when it was configured, and "is this hook in scope for this project?" is the security boundary — we don't re-check webhook-admin rights at trigger time. ### Triggering auth (scope decision) Starting a flow from outside uses a **user personal access token (PAT)** — a seat-holding user. This is fine for now and is a valid long-term path too. (A service-account / "act on behalf of a mapped user" model may make more sense eventually, but it's a separate, larger piece and is **not required** for this epic.) ### Maturity stages Release stages are tracked as **child epics** that collect and prioritize their own issues (mirroring the Duo Slack rollout epic): - **Experimental: release requirements** (child epic) — build the trigger→callback path behind a feature flag, then gate, document, and enable per group before declaring experimental (user-PAT triggering; no SLA). Holds the first concrete issues. - **GA: release requirements** (child epic) — flag removed / on by default; hardening, audit events, payload-versioning policy, security & observability review, reference example. ### Out of scope / dependencies - Service-account / composite-identity triggering (a separate effort; user PAT is used here). - Adding new surfaces/adapters beyond the generic webhook one.