Skip to content

Implement MCP client that uses Gitlab MCP server

Igor Drozdovrequested to merge
id-workhorse-mcp-client-implementaion into master

What does this MR do and why?

MCP client is implemented in Workhorse. Workhorse is a proxy in front of Gitlab Rails and it already implements communication with Duo Workflow Service:

Diagram

sequenceDiagram
    participant User
    participant Workhorse
    participant GitLabRails as GitLab Rails
    participant DuoWorkflowService as Duo Workflow Service

    %% Initial Setup Phase
    User->>Workhorse: Establish WebSocket connection
    Note over Workhorse,User: Bidirectional communication
    Workhorse->>GitLabRails: Auth request
    GitLabRails->>Workhorse: DWS and MCP server configurations
    
    %% Service Connections
    Workhorse->>GitLabRails: Init session with MCP Server
    GitLabRails->>Workhorse: List of available tools
    Workhorse->>DuoWorkflowService: Establish gRPC connection
    Note over Workhorse,DuoWorkflowService: Bidirectional communication
    
    %% Tool Execution Flow
    User->>Workhorse: Input that may require MCP tool call
    Workhorse->>DuoWorkflowService: Propagate message
    DuoWorkflowService->>Workhorse: RunMCPTool action
    Workhorse->>GitLabRails: Handle action and call MCP tool
    GitLabRails->>Workhorse: Response
    Workhorse->>DuoWorkflowService: Send MCP tool response
    DuoWorkflowService->>Workhorse: Agent response
    Workhorse->>User: Agent response

Why

  • It enables support for self managed customers
  • It enables support for customers with IP restrictions
  • It enables support for airgapped customers

If we implement an MCP client in Duo Workflow Service, we'll likely encounter the issues we used to have with direct http requests from DWS to Rails:

  • Self-managed customers may have closed networks or .com customers may have IP restrictions configured: in this case, a request from DWS is rejected and it's not easy to specify an IP in allowlist because DWS IP is dynamic.
  • With Workhorse proxy we establish a gRPC connection with DWS and due to bidirectional communication over the established channel, the issue with incoming requests is solved.

MCP Server Configuration

It's implemented in a way that Gitlab Rails may specify configuration for any number of Streamable HTTP MCP servers:

{
  server_name: {
    URL: <server-url>,
    Headers: <headers-send-on-each-request>,
    Tools: <list-of-supported-tools> # empty means that all tools will be listed
  }
}

GitLab configuration is hard-coded, while the list may also contain other server configurations For example,

{
  gitlab: {
    # URL is skipped and automatically resolved in Workhorse 
    Headers: {
      Authorization: "Bearer #{gitlab_token}"
    },
    Tools: GITLAB_ENABLED_TOOLS
  },
  context7: {
    URL: "https://mcp.context7.com/mcp",
  }
}

Or the list can be extended by user provided configurations on instance/namespace/project/user levels. Enabling such feature .com will require AppSec review, but one of the very likely use cases: instance admins of self-managed instances can configure access to their own closed MCP servers. Enabling such feature is out of scope of this MR, we just initially design the feature in a way that it's possible in the future.

Additionally, Tools version is available: it filters the list of available tools.

Migration plan

Currently, Gitlab server configuration narrows the list to a single tool (get_issue). Additionally, the list is sent as x-gitlab-enabled-mcp-server-tools header. The idea is:

  • Extend the list of Tools and now they are passed as MCP tools to Duo Workflow Service and in x-gitlab-enabled-mcp-server-tools header
  • Duo Workflow Service reads x-gitlab-enabled-mcp-server-tools header and excludes the specified tools from the list of its own defined tools, so the specified MCP tools can be used instead
  • We gradually extend the list with more tools with/without a feature flag to eventually mostly use MCP Server tools

A problem that is not yet solved in this MR: some of the tools should be pre-approved and some of them should require approval. We can have an additional PreApprovedTools option in MCP server configuration to mark some tools as pre-approved.

How to test it

The implementation is behind mcp_client feature flag.

  • Enable mcp_client feature flag
  • Compile Workhorse. From gitlab folder: cd workhorse && make && gdk restart gitlab-workhorse && gdk tail -f gitlab-workhorse
  • In Duo Workflow Service, manually remove get_issue (or any other) tool from the list: duo_workflow_service/workflows/chat/workflow.py and run it as poetry run duo-workflow-service. In gitlab folder make sure that config/gitlab.yml contains:
development:

  ...

  duo_workflow:
    service_url: 0.0.0.0:50052
    secure: false
  • Visit gdk.test:3333 (instead of gdk.test:3000) to have Web Agentic Chat available
  • Ask Agentic Chat to explain some issue
  • An MCP tool get_issue should be called

Relates to #561296

Edited by Martin Wortschack

Merge request reports