Start MCP agent sessions on Workhorse behind a flag

What does this MR do and why?

With mcp_duo_agent_sessions_on_workhorse on, start_duo_session runs a foundational agent the way web chat and Slack do: on GitLab itself, as the user, without a CI job. Until now every agent session over MCP waited for a runner and ran as a service account, although agents never run in CI anywhere else. Orbit still needs a repository checkout, so it stays in CI, as does every agent started through the REST API directly.

Part of #631699, stacked on !259677 (merged). Answering a Workhorse session with send_duo_session_input needs a policy change of its own and comes in the next MR, so with the flag on a session can be started and read but not yet answered.

what changes on the start path

POST /ai/duo_workflows/workflows takes the Workhorse branch only when all of these hold: the request was dispatched by the MCP handler (mcp_dispatched_request?), start_workflow is set, the definition is a foundational chat agent, ExecuteRunService.runtime_for resolves it to :workhorse (coding environment none, from !259677 (merged)), and the flag is on for the user.

On that branch:

• The session is created by CreateWorkflowService as before, so the agentic chat checks (access_duo_agentic_chat, agent enabled for the namespace, Ultimate where required) and the credit check still run.
• The first turn starts with ExecuteRunService.new(workflow, event: { type: 'input', ... }, runtime: :workhorse), which enqueues Ai::Messaging::ServerSideTurnWorker. The worker calls POST .../workflows/:id/execute with the user's own OAuth token, and that endpoint authorizes read_duo_workflow again. The session has no messaging callback context, so the worker delivers nothing and the MCP client reads the answer with get_duo_session.
• enforce_start_workflow_service_account is skipped. It refused agents whenever enforce_composite_identity_for_api_started_workflows was on, because agents have no service account; a Workhorse turn acts as the user and needs none.
• The session gets the privileges the agent declares on Ai::FoundationalChatAgent, which are what web chat gives every agent: read and write GitLab plus start flows, with only read pre-approved. With DEFAULT_PRIVILEGES the model would also be offered file, command and git tools, and on /execute Workhorse answers every one of those with action cannot be executed by this client. apply_flow_declared_privileges reads them only for a Workhorse session, so with the flag off an agent's session in CI keeps the defaults. The values still go through clamp_client_privileges, so group tool rules apply.
• environment stays ambient, which is what Slack runs on Workhorse and keeps MCP sessions out of the web chat history.
• The route presents workload: null. The MCP wrapper has no current user of its own, so it cannot evaluate the flag; it reads a missing workload as "started without a CI job" and says so, with a 15 second poll hint instead of 120.

If ExecuteRunService refuses the start, the route returns its message as a 422 and nothing is enqueued. The created session is left behind, as it is when StartWorkflowService fails.

things for the reviewer to weigh

• Identity. The session runs as the user, the way Slack and web chat run on Workhorse, so it skips the composite identity check from #601901 on purpose. That was settled in the #631699 thread: the user decides in their MCP client which tools run without asking. Marking send_duo_session_input as destructive and showing the pending tool call in get_duo_session, so that choice is an informed one, are follow-ups.
• Worker capacity. ServerSideTurnWorker holds a Sidekiq thread for the whole turn, with concurrency_limit -> { 50 } shared with Slack. This MR does not give MCP its own limit; that is an open question on #631699.
• Workhorse lock release. Reading the Workhorse code, the per-workflow lock (2 hour TTL) looks like it is released with the request context, which is already cancelled when the caller hangs up (runner.go, release path). If so, a turn killed mid-way would leave the session returning 409 until the TTL expires. I have not reproduced it; it would affect Slack the same way.

References

Part of #631699. Stacked on !259677 (merged). Related: !257062 (merged), #628435 (closed), #628429 (closed), #629005.

Screenshots or screen recordings

The start_duo_session text for a Data Analyst session:

Before	After (flag on)
Session 99 started. It runs in a CI job, which can take a few minutes to start. Poll get_duo_session with workflow_id=99 in 120 seconds.	Session 99 started. It runs without a CI job. Poll get_duo_session with workflow_id=99 in 15 seconds.

How to set up and validate locally

steps and what I got in GDK

GDK serves the main checkout, which is a few days behind master, and its databases do not have master's newest migrations. So I ran this branch's two commits (this MR and !259677 (merged)) on top of that checkout instead: cherry-picked onto 0a80f055 together with af87792a (default_runtime_for) and served from a second Puma on port 3999. The changed lines are identical to the branches. The turn itself ran through GDK's own Sidekiq, Workhorse, DWS and AI Gateway.

1. Enable the flag for the user and create an OAuth token with the mcp scope:

   Feature.enable(:mcp_duo_agent_sessions_on_workhorse, User.find_by_username('root'))

2. Start a Data Analyst session over MCP:

   curl -s -X POST "http://gdk.test:3000/api/v4/mcp" -H "Authorization: Bearer $TOKEN" \
     -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
     -d '{"jsonrpc":"2.0","id":"1","method":"tools/call","params":{"name":"start_duo_session","arguments":{"project_id":"gitlab-org/gitlab-test","agent":"analytics_agent/v1","goal":"How many merge requests were merged in this project in the last 30 days?"}}}'

   What I got:

   {
     "content": [{ "type": "text", "text": "Session 1027 started. It runs without a CI job. Poll get_duo_session with workflow_id=1027 in 15 seconds. View it at http://gdk.test:3000/gitlab-org/gitlab-test/-/automate/agent-sessions/1027" }],
     "structuredContent": { "workflow_id": 1027, "status": "created", "workload_id": null, "poll_after_seconds": 15, "web_url": "http://gdk.test:3000/gitlab-org/gitlab-test/-/automate/agent-sessions/1027" },
     "isError": false
   }

3. Watch the session. It went from created to running to input_required within about 20 seconds. log/sidekiq.log shows Ai::Messaging::ServerSideTurnWorker for 1027 done: 16.140713 sec. No CI pipeline was created, and the session has no workload:

   w = Ai::DuoWorkflows::Workflow.find(1027)
   [w.workloads.count, w.last_runtime, w.environment, w.agent_privileges, w.pre_approved_agent_privileges, w.service_account_id]
   # => [0, :workhorse, "ambient", [2, 3, 7], [2], nil]

4. Read the answer with get_duo_session and {"workflow_id":1027}. What I got: status input_required, turn_complete: true, and the agent's answer, "No merge requests were merged in the last 30 days in the Gitlab Test project ...", with the GLQL query behind it.

5. Check what stays in CI:

   • With the flag disabled for the user, the same request started session 1028 in CI (workload 548, "It runs in a CI job ...", 120 second poll).
   • With the flag enabled, an orbit_agent/v1 session (1029) also went to CI (workload 549), with the full clone.

6. send_duo_session_input on 1027 is still refused ("Session 1027 is not waiting for input, or you cannot resume it ..."). That is the next MR.

7. Specs:

   bin/rspec ee/spec/requests/api/mcp/handlers/call_tool_spec.rb \
     ee/spec/services/mcp/tools/duo_workflows/start_duo_session_service_spec.rb
   bin/rspec ee/spec/requests/api/ai/duo_workflows/workflows_spec.rb -e 'started outside MCP'

   The four Workhorse examples in call_tool_spec.rb fail against the route on master; the two CI examples pass on both.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.