Implement Core MCP Server with OpenAPI Tool Generation

Overview

Implement the core MCP server functionality that can dynamically generate MCP tools from OpenAPI specifications and execute HTTP requests.

Current Code Analysis

Dependencies Issues:

Cargo.toml only has rmcp = "0.2.1" but code imports rig::completion::ToolDefinition
Missing: reqwest, serde_json, thiserror, tokio
Current code uses rig types instead of rmcp types

Current Architecture Problems:

Wrong Tool Definition: Using rig::completion::ToolDefinition instead of rmcp's #[tool] macro
No MCP Server: No ServerHandler implementation
Static Implementation: No dynamic tool registration from OpenAPI specs
Incomplete HTTP Integration: HTTP client exists but not integrated with MCP

New Architecture Design

flowchart TD
    subgraph MCP["MCP Server Layer"]
        SH["ServerHandler<br/>- get_info()<br/>- Tool routing"]
        TR["Tool Registry<br/>- Dynamic Registration<br/>- Tool metadata"]
        HC["HTTP Client<br/>- Execute Requests<br/>- Handle Auth"]
    end
    
    subgraph PROC["Processing Layer"]
        OS["OpenAPI Spec<br/>- Load spec<br/>- Parse paths<br/>- Extract operations"]
        TG["Tool Generator<br/>- Convert ops to tools<br/>- Generate descriptions<br/>- Handle parameters"]
        RT["Request Transformer<br/>- Params → HTTP<br/>- Response → MCP<br/>- Error handling"]
    end
    
    subgraph EXT["External"]
        API["OpenAPI Definition<br/>YAML/JSON"]
        HTTP["HTTP APIs<br/>REST endpoints"]
        CLIENT["MCP Client<br/>AI Assistant"]
    end
    
    API --> OS
    OS --> TG
    TG --> TR
    TR --> SH
    SH --> RT
    RT --> HC
    HC --> HTTP
    CLIENT <--> SH

Migration Plan: From Current Code to MCP Architecture

Phase 1: Foundation Migration (Days 1-2) ✅ COMPLETE

Task 1: Fix Dependencies ✅ COMPLETE
- Update Cargo.toml with proper dependencies
- Remove rig dependency, add missing crates
- Update imports throughout codebase
Task 2: Create Core MCP Types ✅ COMPLETE
- Create OpenApiServer struct implementing ServerHandler
- Define tool parameter/response types
- Create error handling structures
Task 3: Refactor OpenAPI Parser ✅ COMPLETE
- Move from single path parsing to full spec parsing
- Create OpenApiSpec struct to hold parsed data
- Update parameter extraction for MCP format

Phase 2: Dynamic Tool System (Days 3-4) ✅ COMPLETE

Task 1: Tool Registry Implementation ✅ COMPLETE
- Create ToolRegistry to hold dynamic tools
- Implement tool registration from OpenAPI operations
- Create tool metadata storage
Task 2: Dynamic Tool Generation ✅ COMPLETE
- Convert convert_path_to_tool_definition to generate rmcp tools
- Create tool factory that generates async tool functions
- Handle different parameter types (path, query, body)
Task 3: MCP Server Structure ✅ COMPLETE
- Implement ServerHandler trait for OpenApiServer
- Create get_info() method with server metadata
- Set up tool routing and dispatch

Phase 3: HTTP Integration (Days 5-6) ✅ COMPLETE

Task 1: HTTP Client Integration ✅ COMPLETE
- Refactor HTTP client to work with MCP tool calls
- Create request builder from tool parameters
- Handle authentication (basic implementation)
Task 2: Request/Response Transformation ✅ COMPLETE
- Convert MCP tool calls to HTTP requests
- Transform HTTP responses to CallToolResult
- Handle different content types and error cases
Task 3: Error Handling ✅ COMPLETE
- Map HTTP errors to proper MCP error codes
- Validate tool parameters with comprehensive constraints checking
- Handle OpenAPI spec validation
- Added regex pattern validation for string parameters
- Enhanced HTTP error categorization with specific context

Phase 4: Testing & Integration (Days 7-8) ✅ COMPLETE

Test Strategy: Swagger Petstore API

Why Petstore: The Swagger Petstore is the canonical example OpenAPI specification with:

✅ Comprehensive Operations: CRUD operations across multiple resources (Pet, Store, User)
✅ Parameter Variety: Path parameters (/pet/{petId}), query parameters (findByStatus?status=available), request bodies
✅ Live Endpoint: https://petstore.swagger.io/v2 for real integration testing
✅ Well-Documented: Standard example used across OpenAPI tooling
✅ Error Cases: Proper HTTP status codes and validation responses

Petstore Operations to Test:

# Pet Operations
GET /pet/findByStatus?status=available,pending,sold
GET /pet/findByTags?tags=tag1,tag2
GET /pet/{petId}
POST /pet                    # Add new pet
PUT /pet                     # Update existing pet
POST /pet/{petId}           # Update pet with form data
DELETE /pet/{petId}         # Delete pet

# Store Operations  
GET /store/inventory        # Get inventory
POST /store/order          # Place order
GET /store/order/{orderId} # Get order
DELETE /store/order/{orderId} # Cancel order

# User Operations
POST /user                 # Create user
GET /user/{username}       # Get user
PUT /user/{username}       # Update user
DELETE /user/{username}    # Delete user

Integration Test Architecture ✅ COMPLETE

Following rmcp-actix-web patterns, the test structure is:

tests/
├── common/
│   ├── mod.rs                    # Shared test utilities ✅
│   ├── petstore_spec.json        # Test OpenAPI spec ✅
│   └── openapi_server.rs         # OpenAPI server test helpers ✅
├── test_with_js.rs              # JavaScript client integration tests ✅
├── test_with_python.rs          # Python client integration tests ✅
├── test_with_js/                # JS client files ✅
│   ├── package.json
│   ├── client.js             # SSE client
│   └── streamable_client.js  # HTTP client
├── test_with_python/            # Python client files ✅
│   ├── pyproject.toml
│   └── client.py             # Multi-mode client
└── snapshots/                   # insta snapshot files ✅
    ├── js_sse_client_responses.snap
    └── python_sse_client_responses.snap

Key Adaptations from rmcp-actix-web:

Calculator service → OpenApiServer with Petstore spec ✅
Basic tool validation → OpenAPI tool generation and validation ✅
Same transport patterns → SSE and StreamableHttp testing with OpenAPI tools ✅
Same client patterns → JS/Python clients testing Petstore operations ✅

Key Architectural Changes:

From: Static rig::ToolDefinition → To: Dynamic rmcp #[tool] methods ✅
From: Single path parsing → To: Full OpenAPI spec processing ✅
From: Manual tool creation → To: Automatic tool generation from specs ✅
From: No server → To: Full MCP ServerHandler implementation ✅

File Structure After Migration:

src/
├── main.rs              # Server startup and CLI (deferred to issue #3)
├── server.rs            # MCP ServerHandler implementation ✅
├── openapi_spec.rs      # OpenAPI parsing and spec handling ✅
├── tool_registry.rs     # Dynamic tool registration ✅
├── tool_generator.rs    # Generate MCP tools from OpenAPI ✅ + 4 unit tests
├── http_client.rs       # HTTP request execution ✅
├── error.rs            # Error types and handling ✅
├── lib.rs              # Public API ✅
└── tests/              # Integration tests ✅ (multi-language clients complete)
    ├── common/
    │   ├── mod.rs                ✅
    │   ├── petstore_spec.json    ✅
    │   └── openapi_server.rs     ✅
    ├── test_with_js.rs           ✅
    ├── test_with_python.rs       ✅
    ├── test_with_js/             ✅
    │   ├── package.json
    │   ├── client.js
    │   └── streamable_client.js
    ├── test_with_python/         ✅
    │   ├── pyproject.toml
    │   └── client.py
    └── snapshots/                ✅

Progress Summary

✅ Completed Components

Dependencies & Build System: All required crates added and compiling
Error Handling: Comprehensive OpenApiError with rmcp integration + enhanced validation
OpenAPI Parsing: Full specification parsing with OpenApiSpec
Tool Registry: Dynamic tool management with validation and statistics
Tool Generator: Advanced tool creation with parameter type handling + regex validation + 4 Petstore unit tests
MCP Server Structure: Complete ServerHandler implementation with tool routing
HTTP Client Integration: Full-featured HTTP client with request/response handling
Request/Response Transformation: Enhanced parameter extraction and response formatting
Parameter Validation: Comprehensive constraint checking with regex patterns
Multi-Language Client Tests: Complete JS and Python client integration tests with OpenAPI Petstore tools ✅

✅ Current Architecture

OpenApiServer {
    spec_url: String,
    registry: ToolRegistry {
        tools: HashMap<String, ToolMetadata>,
        operations: HashMap<String, OpenApiOperation>,
        spec: Option<OpenApiSpec>
    },
    http_client: HttpClient,
    base_url: Option<String>
}

📊 Current Capabilities

✅ Dynamic Tool Loading: Load OpenAPI specs and auto-generate MCP tools
✅ MCP Protocol Support: Full ServerHandler implementation
✅ HTTP Request Execution: Real API calls with comprehensive parameter handling
✅ Request Body Support: JSON and form-urlencoded content types
✅ Enhanced Response Formatting: Rich MCP content with request/response details
✅ Error Handling: Comprehensive error management throughout the stack
✅ Parameter Validation: Full constraint validation with regex patterns, enum values, min/max lengths
✅ Unit Testing: 4 Petstore operation tests with snapshot validation
✅ Multi-Language Integration Testing: Complete JS/Python client test suite with RMCP + SSE transport

🎯 Key Milestones Achieved

Milestone 1: Core MCP server with dynamic tool registration ✅
Milestone 2: Working HTTP integration with real API calls ✅
Milestone 3: Complete request/response transformation with body support ✅
Milestone 4: Comprehensive error handling and parameter validation ✅
Milestone 5: Unit testing foundation with Petstore API validation ✅
Milestone 6: Multi-language client integration tests with comprehensive RMCP transport coverage ✅
Current: Production-ready OpenAPI-to-MCP bridge with enterprise-grade error handling and comprehensive test coverage!

Acceptance Criteria

Server can load OpenAPI spec and register tools dynamically ✅
Tools can be listed via MCP protocol ✅
Tools can be invoked and execute actual HTTP requests ✅
Proper error handling and validation ✅
Unit tests for tool generation with Swagger Petstore API ✅
Multi-language client integration tests with Swagger Petstore API ✅
Integration tests passing with Swagger Petstore API ✅

Related Issues

Issue #7 (closed): End-to-End HTTP Request Integration Tests (follow-up)
Issue #3 (closed): CLI Interface Implementation (deferred)

Edited Jul 09, 2025 by Wally The Wobot