Investigate and fix HTTP-type MCP server configuration for direct connections

Summary

While implementing default OAuth scope to mcp for dynamic client registration in !208967 (merged), testing revealed that the simplified HTTP-type MCP server configuration doesn't work as expected and requires additional changes.

Current Issue

The following configuration was attempted but didn't succeed:

{
  "mcpServers": {
    "GitLab-GDK-http": {
      "type": "http",
      "url": "https://gdk.test:3443/api/v4/mcp",
      "disabled": false
    }
  }
}

This configuration should allow direct HTTP connections to the GitLab MCP server without relying on the mcp-remote tool, but currently fails.

Background

From the discussion in !208967 (merged):

  • The mcp-remote library is experimental and doesn't receive consistent updates
  • We want to move away from requiring mcp-remote as a client connection library
  • The current MCP API only allows mcp-remote connections to use http as the transport protocol
  • Direct HTTP-type connections should be supported but currently don't work

Expected Behavior

Users should be able to configure MCP servers with direct HTTP connections using the simplified configuration above, without needing the mcp-remote wrapper.

Investigation Areas

  1. Transport Protocol Support: Verify that the MCP API properly handles direct HTTP requests from MCP clients
  2. Authentication Flow: Ensure OAuth flow works correctly for direct HTTP connections
  3. Rate Limiting: Check if rate limiting issues affect direct connections differently than mcp-remote connections
  4. CORS/Headers: Verify proper CORS headers and request handling for direct HTTP connections

Acceptance Criteria

  • Direct HTTP-type MCP server configuration works without mcp-remote
  • OAuth authentication flow completes successfully for direct connections
  • Rate limiting doesn't prevent legitimate direct HTTP connections
  • Documentation is updated with working direct HTTP configuration examples
  • Tests are added to verify direct HTTP connection functionality

Related