Spike: Establish connection between GitLab remote MCP server and Agent Platform

Problems to solve

In order to use our own remote MCP server as outlined in &18759 for Duo Agent Platform functionality, we need to prove out how the connection, including how authentication should look like.

Context

Below is the diagram showing an existing tool (create_issue) calling flow.

sequenceDiagram
    participant Client as Client Application
    participant Server as Duo Workflow Service
    participant TokenAuth as Token Authority
    participant HTTPFactory as HTTP Client Factory
    participant DirectClient as Direct HTTP Client
    participant GitLabAPI as GitLab API

    Note over Client, GitLabAPI: Authentication Setup Phase
    
    Client->>+Server: Request with user credentials
    Server->>+TokenAuth: Create JWT token with scopes
    TokenAuth->>TokenAuth: Sign JWT with private key
    TokenAuth-->>-Server: JWT token + expiration
    
    Server->>+HTTPFactory: get_http_client(base_url, gitlab_token)
    Note over HTTPFactory: base_url matches DUO_WORKFLOW_DIRECT_CONNECTION_BASE_URL
    HTTPFactory->>+DirectClient: Create DirectGitLabHttpClient(base_url, gitlab_token)
    DirectClient->>DirectClient: Store gitlab_token for auth
    DirectClient-->>-HTTPFactory: Direct client instance
    HTTPFactory-->>-Server: HTTP client instance
    Server-->>-Client: Service ready with authenticated client

    Note over Client, GitLabAPI: API Request Phase
    
    Client->>+Server: Tool call (e.g., CreateIssue)
    Server->>Server: Validate project URL/ID
    Server->>Server: Prepare request data
    
    Server->>+DirectClient: apost(path, body)
    DirectClient->>DirectClient: Build request headers:<br/>Authorization: Bearer {gitlab_token}<br/>Content-Type: application/json
    DirectClient->>DirectClient: Get session from connection pool
    DirectClient->>+GitLabAPI: HTTP POST /api/v4/projects/{id}/issues<br/>with Bearer token
    
    alt Successful Authentication
        GitLabAPI->>GitLabAPI: Validate Bearer token
        GitLabAPI->>GitLabAPI: Check token permissions/scopes
        GitLabAPI->>GitLabAPI: Process request (create issue)
        GitLabAPI-->>DirectClient: 201 Created + issue data
        DirectClient->>DirectClient: Parse JSON response
        DirectClient-->>Server: {"created_issue": response}
    else Authentication Failure
        GitLabAPI-->>-DirectClient: 401 Unauthorized
        DirectClient-->>-Server: Exception with auth error
        Server-->>Client: {"error": "Authentication failed"}
    end
    
    Server-->>-Client: Final response

Desired Outcome

A PoC that shows that Agentic Chat and Remote Development flows can use GitLabs remote MCP server instead of internal tools to interact with GitLab.

Plan of attack

High level sequence 
sequenceDiagram
    autonumber
    participant MCPManager
    participant MCPClient
    participant MCPServer as MCP Server

    Note over MCPManager,MCPServer: Initialization Phase
    activate MCPManager
    MCPManager->>+MCPClient: initialize(config) / 
    MCPClient->>MCPServer: initialize request (JSON-RPC)
    MCPServer-->>MCPClient: initialize response (capabilities)
    MCPClient-->>MCPServer: notifications/initialized
    MCPClient-->>-MCPManager: initialization complete

    Note over MCPManager,MCPServer: Tool Discovery Phase
    MCPManager->>+MCPClient: list_tools()
    MCPClient->>MCPServer: tools/list request
    MCPServer-->>MCPClient: tools/list response (available tools)
    MCPClient-->>-MCPManager: McpTool[] (cached)

    Note over MCPManager,MCPServer: Operation Phase
    rect rgb(200, 220, 250)
        note over MCPManager,MCPServer: Normal protocol operations
        MCPManager->>+MCPClient: call_tool(name, args)
        MCPClient->>MCPServer: tools/call request
        MCPServer-->>MCPClient: tools/call response (result)
        MCPClient-->>-MCPManager: McpToolResult

        Note over MCPManager,MCPClient: Health Check Operations
        MCPManager->>+MCPClient: health_check()
        MCPClient->>MCPClient: list_tools() (internal)
        MCPClient-->>-MCPManager: health status

        Note over MCPManager,MCPClient: Error Recovery
        alt Connection Failed
            MCPManager->>MCPClient: reconnect_client()
            MCPClient->>MCPClient: close() existing connection
            MCPClient->>MCPServer: re-initialize
            MCPServer-->>MCPClient: new connection established
            MCPClient-->>MCPManager: reconnection status
        end
    end

    Note over MCPManager,MCPServer: Shutdown Phase
    MCPManager->>MCPClient: close()
    MCPClient-->MCPServer: Disconnect (HTTP connection close)
    MCPClient-->>MCPManager: shutdown complete
    Note over MCPManager,MCPServer: Connection closed
  • Introduce new tool, suffix with MCP, e.g. create_issue_mcp
  • Finalise authentication method with MCP server, might require changes in GitLab monolith.
  • Error handling logic.
  • Feature flag to switch to MCP-server-backed tool for testing and rollout.

Questions

References

Edited by Tan Le