## Product context Duo agents run on one of two execution engines: a CI runner (heavyweight — it can check out and change code) or Workhorse (lightweight, in-app, API-only, no project needed). Today every surface that starts or resumes a session must know which engine to call. This change builds a single entry point: hand it a session and what just happened (the user gave input, or approved/denied a tool), and it picks the right engine. The first user of this is approving a Duo's tool call from Slack. The longer-term goal is two feature-complete, interchangeable engines behind one door, so the engine becomes a per-run choice the caller (or platform) makes — not something each surface reimplements. ## Why Duo flows started from a server-side surface — a Slack mention, an `@GitLabDuo` comment, a webhook — run on CI today. The Workhorse runtime (https://gitlab.com/gitlab-org/gitlab/-/work_items/628429) removes that limit for API-only flows: no runner, no project, the run acts as the user. Slack is the first surface to use it. It will not be the last. The `@GitLabDuo` mention assistant, webhooks and a future generic "GitLab Duo" entry all want the same thing: run an API-only flow on Workhorse, and hand coding work over to Duo Developer on CI. The same flow may run on Workhorse for Slack and on the client's WebSocket for web chat. So the runtime is a decision the **caller** makes, per surface, and it must be made in one place that every caller can reach. Without that place, each surface wires Workhorse on its own. !254733 already does this for Slack: a driver and a turn worker that only the messaging adapter can call, with the flow-to-runtime mapping as a list inside the adapter. Good enough to dogfood; not a shape the next surface can reuse. ## What this issue delivers One entry point to start a run on an existing session, with the runtime named by the caller: ```ruby Ai::DuoWorkflows::ExecuteRunService.new(workflow, event:, runtime: nil) ``` - `event` is a goal on the first turn and an approval decision on a continuation. A resume is another run through the same entry. - `runtime` is `:ci` or `:workhorse`. On a continuation it may be omitted; the session's recorded runtime is used. - The flow's `coding_environment` is checked against the runtime. `full` on Workhorse is an error before anything is created. - The session's container may be a Project or a Namespace. `:ci` still needs a Project; `:workhorse` does not. - `:ci` mints tokens and calls today's `StartWorkflowService` / `ResumeWorkflowService`. `:workhorse` enqueues the Workhorse turn worker. And, so the Workhorse runtime is reusable beyond Slack: - The turn worker takes a session and an event, not a Slack bundle, and lives under `Ai::DuoWorkflows`. - The worker does not deliver results. A turn that ends in `input_required` publishes `WorkflowInputRequiredEvent`; `CallbackWorker` delivers on every runtime, as it does for `finished` today. A connection lost mid-turn drops the session; `WorkflowFailedEvent` (https://gitlab.com/gitlab-org/gitlab/-/work_items/628437) tells the surface. - The session records the runtime when a run starts. ## Done when - `ExecuteRunService` exists and starts or continues a run on an existing session for `:ci` and `:workhorse`. With no `runtime` on a continuation, it uses the session's recorded runtime. A Slack session on `developer/v1` (flag off) that pauses for a governance approval resumes on `:ci` through it. - A flow with `coding_environment: full` started on `:workhorse` fails with a clear error before anything is created. - The Workhorse turn worker is callable with only a session and an event. Nothing in it references a messaging bundle or delivers to a surface. - `require_input` publishes an event and `CallbackWorker` delivers the answer for Workhorse runs. No inline delivery remains in the turn worker. - The session's last runtime is available for `:ci` and `:workhorse`. It is *derived*, not stored: `Workflow#last_runtime` returns `:ci` if a workload exists, else infers from the flow's coding environment. No column is added (`duo_workflows_workflows` is Siphon-replicated to ClickHouse, so a column needs a matching `siphon_` column). A real column — needed for true mid-session engine switching — is deferred to https://gitlab.com/gitlab-org/gitlab/-/work_items/629005. - Slack routes through `ExecuteRunService` behind `slack_duo_api_flow`. With the flag off, the old path is untouched. The adapter no longer holds a list of Workhorse flows. - Slack approvals (https://gitlab.com/gitlab-org/gitlab/-/work_items/628438) resume through `ExecuteRunService` with an approval event, on `:ci` and on `:workhorse`. - With `:workhorse`, the Slack session is created on the user's default Duo namespace, not the `duo-workspace` project. The session page link, the sessions list and the quota check work for that session. - `@GitLabDuo` mentions, flow triggers and code review behave exactly as before. - On `:workhorse`, no workspace project is materialized: `DefaultProjectFlowResolver` skips workspace-project and service-account resolution when the resolved runtime is `:workhorse` (the session is already stored on the default Duo namespace; the leftover project lookup only feeds flow-config/service-account resolution). <details> <summary><strong>Proposal</strong> — for whoever implements this</summary> ### Shape ``` surface (AppMentionedService, later: notes, webhook) decides: flow, runtime, container → adapter.with_lifecycle_hooks do |ctx| CreateWorkflowService # session; accepts Project or Namespace ExecuteRunService(workflow, event:, runtime:) end ExecuteRunService validates event vs session state (goal on first run; approval when awaiting one) validates flow coding_environment vs runtime records workflow.runtime ├─ :ci → mint tokens (WorkflowContextGenerationService), then │ StartWorkflowService / ResumeWorkflowService as they are. │ ~40 lines. Not an extraction; see below. └─ :workhorse → Ai::DuoWorkflows::WorkhorseTurnWorker.perform_async(workflow_id, event) └─ ServerSideExecutionService (Igor's client, !254712) session state machine finish / drop / stop / require_tool_call_approval / require_input → events → CallbackWorker → adapter # one delivery owner ``` ### Event shape Two kinds, keyed off the session's awaiting-state rather than enumerated per surface: - `{ type: :input, text: }` — the run's input. Valid when the session is `created` (first turn) **or** `input_required` (a free-text reply continuing the turn). Named `input`, not `goal` or `prompt`, because this field historically carries structured payloads too (for example `code_review/v1` passes an MR iid, unpacked downstream) — a neutral name keeps that escape hatch honest instead of implying human prose. - `{ type: :approval, approved:, message: }` — a decision on a pending ask. Valid when the session `awaiting_approval?`, covering both `tool_call_approval_required` and `plan_approval_required`. One type: tool and plan approval carry the identical payload (`approved` + optional `message`), and the REST resume endpoint already treats them as one (`human_approval` / `human_message`). `valid_for?` reads the session state, so adding a new awaiting-state does not mean editing the event class. Whether an `input` is the opening instruction or a reply is derived from `workflow.created?` — the caller always passes the existing session, so the role is not encoded in the type. The canonical shape stays the plain hash (it crosses a Sidekiq boundary as JSON). Each runtime translates it in one method: | Runtime | Translation | |---|---| | `:ci` | `human_approval:` / `human_message:` → `--approval` / `--rejection-reason` (`ResumeWorkflowService`); `input` → the run's goal / `DUO_WORKFLOW_GOAL` | | `:workhorse` | protojson `approval: { approval: {} }` or `{ rejection: { message: } }`; `input` → the turn's `goal` (`ServerSideExecutionService`) | Optional and **not built here**: a `context:` key for `additional_context` envelopes (attachments, trigger/resource context). It is additive and optional everywhere today, so it can be added later without breaking callers. One known gap: the CI path recomputes `additional_context` on resume; the Workhorse path currently sends none. ### Delivery: one owner, on every runtime !254733 delivers inline from the turn worker because `require_input` publishes no event. Igor also prefers not to route through the event store inside one system. The trade-off: - Inline: no event needed; but the worker knows the messaging adapter, so only messaging surfaces can use it, and the CI path (event-driven) and the Workhorse path (inline) become two delivery owners for the same adapter. !254733's review already found the first bug this causes: an approval pause claimed as a delivery. - Event-driven: one more Sidekiq job per turn end, which is noise next to an LLM turn; `CallbackWorker` stays the only owner; the worker is reusable by any caller. This issue takes the event-driven side. The one thing the session cannot see is a connection dying mid-turn with the session left `running`; the worker drops it and `WorkflowFailedEvent` does the rest. ### Runtime on the session A `runtime` attribute set when a run starts. Deriving from `workflows_workloads` is an acceptable stopgap for CI. Web chat and the CLI may record `:client` at the `:ws` attach; optional here, cheap, and it stops those sessions being invisible. ### Why the CI path is not extracted here The CI start code (tokens, identity link, membership, branch, `StartWorkflowService`) lives in `ExecuteWorkflowService#build_start_workflow_params` and is shared by triggers, vulnerability flows, notes and the API. Extracting it for one caller means either a copy or a second create path. The extraction is the job of the consolidation issue (https://gitlab.com/gitlab-org/gitlab/-/work_items/629005), once a third runtime makes the strategy worth its cost. What `:ci` does here is smaller: `ExecuteWorkflowService` creates *and* starts, so there is no "start on an existing session" for CI today. The entry mints the two tokens the same way and calls `StartWorkflowService` or `ResumeWorkflowService`. That is the ~40 lines. Identity link and membership stay in the adapter for now. ### Moving a session between runtimes later `workflow.runtime` is the runtime of the last run. A later `ExecuteRunService(workflow, event:, runtime: :other)` at a turn boundary is the same session, a new run, and the attribute updates. Not built here, but two constraints are worth knowing now: a Namespace-backed session cannot move to `:ci` while CI needs a Project; and once sessions have several runs, the history wants a runs table — `workflows_workloads` is already that table for CI. ### Where today's pieces go | Today (!254733) | Becomes | |---|---| | `Ai::Messaging::ExecuteServerSideFlowService(bundle:)` | gone; the adapter creates the session and calls `ExecuteRunService` | | `Ai::Messaging::ServerSideTurnWorker(workflow_id)` delivers inline | `Ai::DuoWorkflows::WorkhorseTurnWorker(workflow_id, event)`; holds the connection, reports a dead connection, delivers nothing | | `Adapters::Base#server_side_flows` list | gone; `runtime:` on the bundle, set by the surface | | `Adapters::Base#provision_service_account` | stays for `:ci` until https://gitlab.com/gitlab-org/gitlab/-/work_items/629005 moves it | ### Sequencing !254733 merges as it is. The reshape (worker moves, delivery removed, driver dissolved) follows in this issue and merges only once `WorkflowInputRequiredEvent` exists. The event is Jannik's, as a follow-up to !255163. The approval MR (!254849) wires against the final worker signature, so comments 1 and 2 on !254733 should be agreed before it lands. ### Guard rails - Build next to the old path. `StartWorkflowService` and `ExecuteWorkflowService` are not changed. - Only Slack routes through the new entry. - Naming: `ExecuteRunService` says what it does. `RunService` and `RunWorkloadService` exist; avoid another bare `Run*Service`. Runtime keys name *where* the run happens: `:ci`, `:workhorse`, later `:client`. ### Background - Investigation and plan: https://gitlab.com/gitlab-org/gitlab/-/work_items/602541. - Adapter history: orchestrator → "caller resolves" (`d115c6d1`) → `trigger` grew identity linking and membership (`c182c4d7`, `c9b85c50`). The guarantee stays; `:ci` callers still cannot forget, because the adapter still does it for them until consolidation. - Direction: sessions are durable, runs come and go, a session has one or more runs on possibly different runtimes. This issue records the runtime and puts the choice in one place; it does not build moving sessions. </details> ## Not in this issue - Extracting the CI path into a runtime class, and moving provisioning out of the adapter — https://gitlab.com/gitlab-org/gitlab/-/work_items/629005. - Migrating flow triggers, code review or `@GitLabDuo` onto the new entry — https://gitlab.com/gitlab-org/gitlab/-/work_items/629005. - Fire-and-forget execution. The Workhorse endpoint holds the connection for the turn; the worker blocks by design. ## Depends on / unblocks - Depends on: !254712 (Workhorse client); `WorkflowInputRequiredEvent` (Jannik, follow-up to !255163) before inline delivery is removed. Coordinates with !254733 (Slack wiring), !254849 (approvals). - Unblocks: Slack approvals resuming a session; the `@GitLabDuo` mention assistant on Workhorse; https://gitlab.com/gitlab-org/gitlab/-/work_items/629005.