Reuse MCP connections and avoid payload copies in duoworkflow

Igor Drozdovrequested to merge
refactor/duoworkflow-connection-reuse into master

What does this MR do and why?

This MR cuts setup latency, connection churn and memory use in the Workhorse Duo Workflow package (workhorse/internal/ai_assist/duoworkflow). It is not meant to change behaviour.

  • Shared transport for external MCP servers. Each session built its own restricted HTTP transport and never closed it. No connection was reused across sessions, and idle connections stayed open until they timed out. Sessions now share one transport whose idle-connection limits are sized for that.
  • Concurrent MCP setup. Workhorse connected to each MCP server and listed its tools one server after another before the session could start, so session start took the sum of every server's round trips. Servers are now set up concurrently. Their tools are merged in server-name order, so the tool list no longer depends on map iteration order.
  • Fewer copies of HTTP action payloads. The request body was copied into a buffer, and the response body was copied again into the proto string field, so a response near the 4 MB limit was held in memory twice. Now the request body is read directly from the action, and the response is buffered in a strings.Builder, whose String() doesn't copy.
  • Pooled marshaling buffer for the ndjson transport. It kept one buffer per run, which stayed at its largest size for the whole run. It now uses the buffer pool the WebSocket transport already uses, and marshals outside the write lock.

Notes for reviewers:

  • Different users' requests to the same external MCP host can now share connections. Headers are still set per request by each session's own round tripper, and no session has a cookie jar, so no credentials are shared. This would only matter for a server that ties authentication to the connection, which MCP servers don't do.
  • Setup now runs outside the request goroutine, where net/http no longer catches panics. Each setup goroutine recovers a panic and reports it as an MCP initialization error, so the session continues as it already does when an MCP server fails.
  • Connecting to the DWS gRPC service per session is out of scope. Pooling those connections needs per-call metadata for the user agent and client name, plus a decision on load balancing long-lived streams, so it will be a separate MR.

References

Screenshots or screen recordings

Not applicable: no UI changes.

How to set up and validate locally

  1. Configure an external MCP server for Duo Agent Platform. Start a few agentic chat sessions in a row.
  2. In the Workhorse logs, check that each session still lists the MCP tools and that MCP tool calls still succeed.
  3. Configure two or more MCP servers. Check that session start is no longer delayed by the sum of the servers' response times, and that the tool order sent to Duo Workflow Service stays the same across sessions.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports