Chapter 15: Testing MCP Servers

Introduction

Testing is a critical aspect of building any software and specifically reliable MCP servers. Since the MCP ecosystem will have many more servers than clients (similar to how there are many more websites than web browsers), robust server testing is essential for ensuring:

• Protocol compliance - Your server correctly implements the MCP specification
• Capability correctness - Tools, resources, and prompts work as advertised
• Error handling - Graceful degradation under failure conditions
• Performance - Acceptable response times under load
• Integration - Compatibility with Claude and other MCP clients

This chapter covers the testing tools and strategies available for MCP server developers, from interactive browser-based testing to automated CI/CD integration.

Testing Philosophy

Why Focus on Server Testing?

The MCP ecosystem follows a similar pattern to web APIs:

• Many servers - Each organization/developer creates servers for their specific data sources
• Few clients - Claude Desktop, IDE integrations, and other standard clients
• Server diversity - Different languages, deployment models, and capabilities
• Client standardization - Clients follow the MCP spec consistently

Just as you would thoroughly test a REST API before deploying it, MCP servers need comprehensive testing to ensure they work correctly with any compliant MCP client.

Testing Pyramid for MCP Servers:

         ┌─────────────────┐
         │  E2E Scenarios  │  ← Full workflows with real clients
         │  (mcp-tester)   │
         └─────────────────┘
       ┌───────────────────────┐
       │  Integration Tests    │  ← Tool/Resource/Prompt testing
       │  (mcp-tester + unit)  │
       └───────────────────────┘
   ┌─────────────────────────────────┐
   │          Unit Tests             │  ← Handler logic, validation
   │          (cargo test)           │
   └─────────────────────────────────┘

Official MCP Inspector

The MCP Inspector is the official visual testing tool provided by Anthropic for interactive debugging and exploration of MCP servers.

What is MCP Inspector?

The MCP Inspector consists of two components:

1. MCP Inspector Client (MCPI) - React-based web UI for interactive testing
2. MCP Proxy (MCPP) - Node.js server that acts as a protocol bridge

Repository: github.com/modelcontextprotocol/inspector

Installation and Usage

# One-off debugging (easiest)
npx @modelcontextprotocol/inspector <mcp-server-command>

# Example: Test a stdio server
npx @modelcontextprotocol/inspector node my-server/index.js

# Example: Test with environment variables
npx @modelcontextprotocol/inspector -- MY_VAR=value node server.js

Features

Interactive Browser Interface:

• Tool Discovery - Browse all available tools and their schemas
• Tool Execution - Call tools with custom arguments from a web form
• Resource Exploration - List and read available resources
• Prompt Testing - Test prompts with various argument combinations
• Real-time Feedback - See immediate responses and errors
• Protocol Visualization - View JSON-RPC messages sent/received

When to Use MCP Inspector:

✅ Good for:

• Interactive exploration of server capabilities
• Manual testing during development
• Debugging tool schemas and responses
• Understanding how tools behave
• Quick smoke tests

❌ Not ideal for:

• Automated testing in CI/CD
• Performance testing
• Comprehensive regression testing
• Testing multiple scenarios
• Batch testing of many tools

Security Note

Important: The MCP Inspector proxy requires authentication by default. When starting the server, a random session token is generated and printed to the console. Always use the latest version (0.14.1+) which includes important security fixes.

Example Session

# Start the inspector for a stdio server
npx @modelcontextprotocol/inspector cargo run --bin my-mcp-server

# Output:
# MCP Inspector is running on http://localhost:5173
# Session token: abc123...
# Open this URL in your browser to start testing

# Browser opens showing:
# - Server info (name, version, capabilities)
# - Tools tab with all available tools
# - Resources tab with available resources
# - Prompts tab with available prompts
# - Logs tab with protocol messages

MCP Server Tester (mcp-tester)

For automated, comprehensive, and CI-ready testing, the PMCP SDK provides the MCP Server Tester (mcp-tester) - a powerful command-line tool specifically designed for testing MCP servers in development and production.

Location: examples/26-server-tester/

Why mcp-tester?

Unlike the interactive MCP Inspector, mcp-tester is designed for:

• ✅ Automated testing - Run comprehensive test suites without manual interaction
• ✅ CI/CD integration - JSON output, exit codes, and scripting support
• ✅ Scenario testing - Define complex multi-step workflows in YAML/JSON
• ✅ Protocol compliance - Validate MCP protocol adherence
• ✅ Performance testing - Measure response times and throughput
• ✅ OAuth support - Test authenticated servers with automatic token management
• ✅ Multi-transport - Test HTTP, HTTPS, WebSocket, and stdio servers

Installation

Option 1: Download Pre-built Binaries (Recommended)

Pre-built binaries are available for Windows, macOS, and Linux from the GitHub Releases page:

# macOS (Apple Silicon)
curl -L https://github.com/paiml/rust-mcp-sdk/releases/latest/download/mcp-tester-macos-aarch64.tar.gz | tar xz
sudo mv mcp-tester /usr/local/bin/

# macOS (Intel)
curl -L https://github.com/paiml/rust-mcp-sdk/releases/latest/download/mcp-tester-macos-x86_64.tar.gz | tar xz
sudo mv mcp-tester /usr/local/bin/

# Linux (x86_64)
curl -L https://github.com/paiml/rust-mcp-sdk/releases/latest/download/mcp-tester-linux-x86_64.tar.gz | tar xz
sudo mv mcp-tester /usr/local/bin/

# Windows (PowerShell)
Invoke-WebRequest -Uri "https://github.com/paiml/rust-mcp-sdk/releases/latest/download/mcp-tester-windows-x86_64.zip" -OutFile "mcp-tester.zip"
Expand-Archive -Path "mcp-tester.zip" -DestinationPath "C:\Program Files\mcp-tester"
# Add C:\Program Files\mcp-tester to your PATH

Option 2: Build from Source

# From the SDK repository
cd examples/26-server-tester
cargo build --release

# The binary will be at target/release/mcp-tester

# Optional: Install globally
cargo install --path .

Quick Start

# Test a local HTTP server (basic)
mcp-tester test http://localhost:8080

# Test with tool validation
mcp-tester test http://localhost:8080 --with-tools

# Test a stdio server
mcp-tester test stdio

# Quick connectivity check
mcp-tester quick http://localhost:8080

Core Testing Commands

1. Full Test Suite

Run comprehensive tests including protocol compliance, capability discovery, and tool validation:

mcp-tester test <URL> [OPTIONS]

# Examples:
mcp-tester test http://localhost:8080 --with-tools --format json
mcp-tester test https://api.example.com/mcp --timeout 60

Options:

• --with-tools - Test all discovered tools with schema validation
• --tool <NAME> - Test a specific tool
• --args <JSON> - Provide custom tool arguments
• --format <FORMAT> - Output format: pretty, json, minimal, verbose
• --timeout <SECONDS> - Connection timeout (default: 30)
• --insecure - Skip TLS certificate verification

Example output (pretty format):

=== MCP Server Test Results ===

✓ Core Tests
  ✓ Connection establishment      (42ms)
  ✓ Server initialization          (158ms)
  ✓ Capability discovery           (23ms)

✓ Protocol Tests
  ✓ JSON-RPC 2.0 compliance        (5ms)
  ✓ MCP version validation         (2ms)
  ✓ Required methods present       (1ms)

✓ Tool Tests
  ✓ Tool discovery (5 tools)       (15ms)
  ✓ search_wikipedia              (234ms)
  ✓ get_article                   (156ms)
  ⚠ get_summary (schema warning)  (89ms)

Summary: 12 passed, 0 failed, 1 warning in 725ms

2. Protocol Compliance Testing

Validate strict protocol compliance:

mcp-tester compliance http://localhost:8080 --strict

# Validates:
# - JSON-RPC 2.0 format
# - MCP protocol version support
# - Required methods (initialize, ping)
# - Error code standards
# - Response structure correctness

3. Tool Discovery and Validation

List and validate tool schemas:

mcp-tester tools http://localhost:8080 --verbose

# Output includes:
# - Tool names and descriptions
# - Input schema validation
# - Schema completeness warnings
# - Missing properties/types

Schema validation warnings:

✓ Found 10 tools:
  • search_wikipedia - Search for Wikipedia articles by query
    ✓ Schema properly defined

  • get_article - Retrieve full Wikipedia article content
    ⚠ Tool 'get_article' missing 'properties' field for object type

  • get_summary - Get a summary of a Wikipedia article
    ⚠ Tool 'get_summary' has empty input schema - consider defining parameters

Schema Validation Summary:
⚠ 3 total warnings found
  - 1 tools with empty schema
  - 2 tools missing 'properties' in schema

4. Resource Testing

Test resource discovery and reading:

mcp-tester resources http://localhost:8080

# Validates:
# - Resource listing
# - URI format correctness
# - MIME type presence
# - Resource metadata
# - Resource content reading

5. Prompt Testing

Test prompt discovery and execution:

mcp-tester prompts http://localhost:8080

# Validates:
# - Prompt listing
# - Description presence
# - Argument schema validation
# - Prompt execution

6. Connection Diagnostics

Troubleshoot connection issues with layer-by-layer diagnostics:

mcp-tester diagnose http://localhost:8080 --network

# Tests in order:
# 1. URL validation
# 2. DNS resolution
# 3. TCP connectivity
# 4. TLS/SSL certificates (for HTTPS)
# 5. HTTP response
# 6. MCP protocol handshake

Example diagnostic output:

=== Layer-by-Layer Diagnostics ===

✓ Layer 1: URL Validation
  - URL: http://localhost:8080
  - Scheme: http
  - Host: localhost
  - Port: 8080

✓ Layer 2: DNS Resolution
  - Resolved to: 127.0.0.1

✓ Layer 3: TCP Connection
  - Connected successfully

✗ Layer 4: HTTP Response
  - Error: Connection refused
  - Possible causes:
    • Server not running
    • Wrong port
    • Firewall blocking connection

Recommendation: Verify server is running on port 8080

Testing OAuth-Protected Servers

The mcp-tester supports interactive OAuth 2.0 authentication with automatic browser-based login and token caching using OpenID Connect discovery.

Interactive OAuth Flow (OIDC discovery)

Auto-Discovery vs Explicit Issuer: If --oauth-issuer is omitted, the tester attempts OIDC discovery from the MCP server base URL (e.g., https://api.example.com/.well-known/openid-configuration). Providing --oauth-issuer explicitly is recommended for reliability, especially when the OAuth provider and MCP server are hosted on different domains.

# Interactive OAuth with automatic browser login (explicit issuer - recommended)
mcp-tester test https://your-oauth-server.com/mcp \
  --oauth-issuer "https://auth.example.com" \
  --oauth-client-id "your-client-id" \
  --oauth-scopes openid,email,profile

What happens:

1. ✅ Tester generates secure PKCE challenge
2. 🌐 Opens your browser to the OAuth provider login page
3. 🔐 You authenticate with your credentials
4. ✅ Tester receives the authorization code via local callback server
5. 🎫 Exchanges code for access token
6. 💾 Caches token locally (~/.mcp-tester/tokens.json) for future requests (unless --oauth-no-cache)
7. 🚀 Automatically injects Authorization: Bearer header into all MCP requests

AWS Cognito Example

mcp-tester test https://your-api.execute-api.us-west-2.amazonaws.com/mcp \
  --oauth-issuer "https://your-pool.auth.us-west-2.amazoncognito.com" \
  --oauth-client-id "your-cognito-client-id" \
  --oauth-scopes openid \
  --with-tools

Subsequent runs reuse cached tokens (no re-authentication needed):

mcp-tester test https://your-api.execute-api.us-west-2.amazonaws.com/mcp \
  --oauth-client-id "your-cognito-client-id" \
  --with-tools
# ← Uses cached token automatically!

Manual Token (Alternative)

If you already have an access token (for example, from a previous OAuth flow or from another tool):

# Pass token directly
mcp-tester test https://your-oauth-server.com/mcp --api-key "YOUR_ACCESS_TOKEN"

# Or via environment variable
export MCP_API_KEY="YOUR_ACCESS_TOKEN"
mcp-tester test https://your-oauth-server.com/mcp

Pro Tip: Copy Token from MCP Inspector

If you’ve authenticated using the official MCP Inspector’s OAuth flow, you can copy the access token from the final step and reuse it in mcp-tester:

1. Run MCP Inspector with OAuth (it will complete the OAuth flow)
2. In the Inspector’s console output or browser developer tools, locate the access token
3. Copy the token value
4. Use it with mcp-tester:

   mcp-tester test $SERVER_URL --api-key "eyJhbGci..." --with-tools
   

This is useful for quickly testing with an already-authenticated session without going through the OAuth flow again in mcp-tester.

Scenario-Based Testing

The most powerful feature of mcp-tester is scenario testing - defining complex, multi-step test workflows in YAML or JSON files.

Why Scenarios?

Scenarios enable:

• ✅ Reproducible tests - Define once, run anywhere
• ✅ Complex workflows - Test multi-step user interactions
• ✅ Data dependencies - Use outputs from one step in later steps
• ✅ Regression testing - Detect breaking changes automatically
• ✅ Documentation - Scenarios serve as executable documentation

Scenario File Structure

name: Wikipedia Server Test               # Required
description: Test search and article retrieval  # Optional
timeout: 60                               # Overall timeout (seconds)
stop_on_failure: true                     # Stop on first failure

variables:                                # Define reusable variables
  test_query: "artificial intelligence"
  test_title: "Artificial intelligence"

setup:                                   # Run before main steps
  - name: Verify server health
    operation:
      type: list_tools

steps:                                   # Main test steps
  - name: Search for articles
    operation:
      type: tool_call
      tool: search_wikipedia
      arguments:
        query: "${test_query}"
        limit: 10
    store_result: search_results        # Store for later use
    assertions:
      - type: success
      - type: array_length
        path: results
        greater_than: 0

  - name: Get first article
    operation:
      type: tool_call
      tool: get_article
      arguments:
        title: "${search_results.results[0].title}"
    assertions:
      - type: success
      - type: exists
        path: content
      - type: contains
        path: content
        value: "${test_query}"
        ignore_case: true

cleanup:                                # Always run, even on failure
  - name: Clear cache
    operation:
      type: tool_call
      tool: clear_cache
    continue_on_failure: true

Running Scenarios

# Run a scenario file
mcp-tester scenario http://localhost:8080 my-test.yaml

# Run with detailed step-by-step output
mcp-tester scenario http://localhost:8080 my-test.yaml --detailed

# Run with JSON output for CI
mcp-tester scenario http://localhost:8080 my-test.yaml --format json > results.json

Automatic Scenario Generation

The mcp-tester can automatically generate test scenarios from your server’s discovered capabilities:

# Generate basic scenario
mcp-tester generate-scenario http://localhost:8080 -o test.yaml

# Generate comprehensive scenario with all tools
mcp-tester generate-scenario http://localhost:8080 -o full_test.yaml \
  --all-tools --with-resources --with-prompts

Generated scenario example:

name: my-server Test Scenario
description: Automated test scenario for server
timeout: 60
stop_on_failure: false

steps:
  - name: List available capabilities
    operation:
      type: list_tools
    store_result: available_tools
    assertions:
      - type: success
      - type: exists
        path: tools

  - name: Test tool: search_wikipedia
    operation:
      type: tool_call
      tool: search_wikipedia
      arguments:
        query: "TODO: query"  # ← Replace with real values
        limit: 10
    timeout: 30
    assertions:
      - type: success

  # ... more generated tests

Workflow:

1. Generate the scenario template
2. Edit to replace TODO: placeholders with real test data
3. Add custom assertions
4. Run the scenario

Operation Types

Tool Call:

operation:
  type: tool_call
  tool: tool_name
  arguments:
    param1: value1

List Operations:

operation:
  type: list_tools      # List all tools
  # OR
  type: list_resources  # List all resources
  # OR
  type: list_prompts    # List all prompts

Resource Operations:

operation:
  type: read_resource
  uri: resource://path/to/resource

Prompt Operations:

operation:
  type: get_prompt
  name: prompt_name
  arguments:
    key: value

Utility Operations:

operation:
  type: wait
  seconds: 2.5
  # OR
  type: set_variable
  name: my_var
  value: some_value

Custom JSON-RPC:

operation:
  type: custom
  method: some.method
  params:
    key: value

Assertion Types

Success/Failure:

assertions:
  - type: success    # Expects no error
  - type: failure    # Expects an error

Value Comparisons:

assertions:
  - type: equals
    path: result.status
    value: "active"

  - type: contains
    path: result.message
    value: "success"
    ignore_case: true

  - type: matches
    path: result.id
    pattern: "^[a-f0-9-]{36}$"  # UUID regex

Existence Checks:

assertions:
  - type: exists
    path: result.data

  - type: not_exists
    path: result.error

Array and Numeric:

assertions:
  - type: array_length
    path: results
    greater_than: 5
    # OR equals: 10
    # OR less_than_or_equal: 20
    # OR between: {min: 5, max: 15}

**Path Expressions (JSONPath-style):**
```yaml
assertions:
  - type: jsonpath
    expression: "result.items[0].id"
    expected: "abc-123"   # Optional: if omitted, only checks presence

  - type: jsonpath
    expression: "data.user.profile.email"  # Dot notation
    expected: "test@example.com"

  - type: jsonpath
    expression: "results[0]"  # Array index

Note: The jsonpath assertion type uses simple path expressions with dot notation and array indexing (e.g., user.items[0].name), not full JSONPath query language. For full JSONPath support with wildcards, filters, and recursive descent, consider using dedicated assertion tools in your CI pipeline.

Numeric Comparisons:

assertions:
  - type: numeric
    path: result.count
    greater_than_or_equal: 100

Variables and Result Reuse

Define variables:

variables:
  user_id: "test_123"
  api_key: "${env.API_KEY}"  # From environment

Store step results:

steps:
  - name: Create item
    operation:
      type: tool_call
      tool: create_item
      arguments:
        name: "Test"
    store_result: created_item  # ← Store result

  - name: Update item
    operation:
      type: tool_call
      tool: update_item
      arguments:
        id: "${created_item.result.id}"  # ← Use stored result

Complete Scenario Example

File: scenarios/user-workflow.yaml

name: Complete User Workflow Test
description: Test user creation, retrieval, update, and deletion
timeout: 120
stop_on_failure: false

variables:
  test_email: "test@example.com"
  test_name: "Test User"

setup:
  - name: Clean up existing test user
    operation:
      type: tool_call
      tool: delete_user
      arguments:
        email: "${test_email}"
    continue_on_failure: true

steps:
  # Step 1: List tools to verify server capabilities
  - name: Verify server has user management tools
    operation:
      type: list_tools
    assertions:
      - type: success
      - type: contains
        path: tools
        value: "create_user"

  # Step 2: Create a new user
  - name: Create test user
    operation:
      type: tool_call
      tool: create_user
      arguments:
        email: "${test_email}"
        name: "${test_name}"
    store_result: new_user
    timeout: 30
    assertions:
      - type: success
      - type: exists
        path: result.id
      - type: equals
        path: result.email
        value: "${test_email}"
      - type: matches
        path: result.id
        pattern: "^[a-f0-9-]{36}$"

  # Step 3: Verify user exists
  - name: Retrieve created user
    operation:
      type: tool_call
      tool: get_user
      arguments:
        id: "${new_user.result.id}"
    assertions:
      - type: success
      - type: equals
        path: result.email
        value: "${test_email}"
      - type: equals
        path: result.name
        value: "${test_name}"

  # Step 4: Update user
  - name: Update user status
    operation:
      type: tool_call
      tool: update_user
      arguments:
        id: "${new_user.result.id}"
        status: "active"
    store_result: updated_user
    assertions:
      - type: success
      - type: equals
        path: result.status
        value: "active"

  # Step 5: Verify update persisted
  - name: Verify user was updated
    operation:
      type: tool_call
      tool: get_user
      arguments:
        id: "${new_user.result.id}"
    assertions:
      - type: equals
        path: result.status
        value: "active"

  # Step 6: List users
  - name: Verify user appears in list
    operation:
      type: tool_call
      tool: list_users
      arguments:
        status: "active"
    assertions:
      - type: success
      - type: array_length
        path: users
        greater_than: 0

cleanup:
  - name: Delete test user
    operation:
      type: tool_call
      tool: delete_user
      arguments:
        id: "${new_user.result.id}"
    continue_on_failure: true
    assertions:
      - type: success

Run the scenario:

mcp-tester scenario http://localhost:8080 scenarios/user-workflow.yaml --detailed

CI/CD Integration

The mcp-tester is designed for seamless CI/CD integration with JSON output, exit codes, and headless operation.

GitHub Actions

name: MCP Server Tests

on:
  push:
    branches: [main, develop]
  pull_request:

jobs:
  test-mcp-server:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Set up Rust
        uses: actions-rust-lang/setup-rust-toolchain@v1

      - name: Build MCP server
        run: cargo build --release

      - name: Start MCP server in background
        run: |
          cargo run --release --bin my-mcp-server &
          SERVER_PID=$!
          echo "SERVER_PID=$SERVER_PID" >> $GITHUB_ENV
          sleep 5  # Wait for server to start

      - name: Build mcp-tester
        working-directory: examples/26-server-tester
        run: cargo build --release

      - name: Run protocol compliance tests
        run: |
          examples/26-server-tester/target/release/mcp-tester \
            compliance http://localhost:8080 \
            --format json \
            --strict \
            > compliance-results.json

      - name: Run tool validation tests
        run: |
          examples/26-server-tester/target/release/mcp-tester \
            tools http://localhost:8080 \
            --test-all \
            --format json \
            > tool-results.json

      - name: Run scenario tests
        run: |
          examples/26-server-tester/target/release/mcp-tester \
            scenario http://localhost:8080 \
            tests/scenarios/smoke-test.yaml \
            --format json \
            > scenario-results.json

      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: test-results
          path: |
            compliance-results.json
            tool-results.json
            scenario-results.json

      - name: Stop server
        if: always()
        run: kill $SERVER_PID || true

GitLab CI

test-mcp-server:
  stage: test
  image: rust:latest

  services:
    - name: my-mcp-server:latest
      alias: mcp-server

  before_script:
    - cd examples/26-server-tester
    - cargo build --release

  script:
    # Run tests against service
    - >
      ./target/release/mcp-tester
      test http://mcp-server:8080
      --with-tools
      --format json
      > test-results.json

    # Check exit code
    - if [ $? -ne 0 ]; then exit 1; fi

  artifacts:
    paths:
      - test-results.json
    when: always

Note: The mcp-tester outputs JSON format (via --format json), not JUnit XML. If your CI system requires JUnit XML reports, you can convert the JSON output using tools like jq or write a custom converter script.

Jenkins

pipeline {
    agent any

    environment {
        SERVER_URL = 'http://localhost:8080'
    }

    stages {
        stage('Build Server') {
            steps {
                sh 'cargo build --release'
            }
        }

        stage('Start Server') {
            steps {
                sh '''
                    cargo run --release &
                    echo $! > server.pid
                    sleep 5
                '''
            }
        }

        stage('Build Tester') {
            steps {
                dir('examples/26-server-tester') {
                    sh 'cargo build --release'
                }
            }
        }

        stage('Run Tests') {
            parallel {
                stage('Protocol Compliance') {
                    steps {
                        sh '''
                            examples/26-server-tester/target/release/mcp-tester \
                                compliance ${SERVER_URL} \
                                --format json \
                                > compliance.json
                        '''
                    }
                }

                stage('Tool Validation') {
                    steps {
                        sh '''
                            examples/26-server-tester/target/release/mcp-tester \
                                tools ${SERVER_URL} \
                                --test-all \
                                --format json \
                                > tools.json
                        '''
                    }
                }

                stage('Scenario Tests') {
                    steps {
                        sh '''
                            examples/26-server-tester/target/release/mcp-tester \
                                scenario ${SERVER_URL} \
                                scenarios/regression.yaml \
                                --format json \
                                > scenario.json
                        '''
                    }
                }
            }
        }
    }

    post {
        always {
            sh 'kill $(cat server.pid) || true'
            archiveArtifacts artifacts: '*.json', allowEmptyArchive: true
        }
    }
}

Docker-based Testing

Dockerfile for testing:

FROM rust:1.75 as builder

WORKDIR /app
COPY . .

# Build server
RUN cargo build --release

# Build tester
WORKDIR /app/examples/26-server-tester
RUN cargo build --release

FROM debian:bookworm-slim

# Install runtime dependencies
RUN apt-get update && apt-get install -y \
    ca-certificates \
    && rm -rf /var/lib/apt/lists/*

# Copy binaries
COPY --from=builder /app/target/release/my-mcp-server /usr/local/bin/
COPY --from=builder /app/examples/26-server-tester/target/release/mcp-tester /usr/local/bin/

# Copy test scenarios
COPY scenarios /scenarios

# Test script
COPY <<EOF /test.sh
#!/bin/bash
set -e

# Start server in background
my-mcp-server &
SERVER_PID=$!

# Wait for server
sleep 5

# Run tests
mcp-tester test http://localhost:8080 --format json > /results/test-results.json
mcp-tester scenario http://localhost:8080 /scenarios/main.yaml --format json > /results/scenario-results.json

# Stop server
kill $SERVER_PID

echo "All tests passed!"
EOF

RUN chmod +x /test.sh

VOLUME /results
CMD ["/test.sh"]

Run tests in Docker:

# Build test image
docker build -t my-mcp-server-tests .

# Run tests
docker run -v $(pwd)/test-results:/results my-mcp-server-tests

# Check results
cat test-results/test-results.json

Pre-commit Hook

Add MCP server testing to your pre-commit workflow:

#!/bin/bash
# .git/hooks/pre-commit

echo "Running MCP server tests..."

# Start server
cargo run --bin my-mcp-server &
SERVER_PID=$!
sleep 3

# Run quick tests
cd examples/26-server-tester
cargo run --release -- test http://localhost:8080 --format minimal

RESULT=$?

# Cleanup
kill $SERVER_PID 2>/dev/null || true

if [ $RESULT -ne 0 ]; then
    echo "MCP server tests failed! Commit aborted."
    exit 1
fi

echo "MCP server tests passed!"
exit 0

Testing Best Practices

1. Test Pyramid Strategy

Unit Tests (Foundation):

#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_tool_input_validation() {
        let tool = MyTool;
        let invalid_input = json!({"missing": "required_field"});

        assert!(tool.validate_input(&invalid_input).is_err());
    }
}
}

Integration Tests (Middle):

# Test individual tools with real server
mcp-tester test http://localhost:8080 --tool search --args '{"query": "test"}'

Scenario Tests (Top):

# Test complete workflows
mcp-tester scenario http://localhost:8080 scenarios/user-workflow.yaml

2. Schema-Driven Testing

Always define complete JSON schemas for your tools:

#![allow(unused)]
fn main() {
// Good - Complete schema
ToolInfo {
    name: "search".to_string(),
    description: Some("Search for items".to_string()),
    input_schema: json!({
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": "Search query",
                "minLength": 1
            },
            "limit": {
                "type": "number",
                "description": "Max results",
                "minimum": 1,
                "maximum": 100,
                "default": 10
            }
        },
        "required": ["query"]
    })
}

// Bad - Empty schema
ToolInfo {
    name: "search".to_string(),
    description: Some("Search for items".to_string()),
    input_schema: json!({})  // ← mcp-tester will warn!
}
}

3. Test Data Management

Use variables for reusable test data:

variables:
  test_user_email: "test@example.com"
  test_date: "2024-01-01"
  api_base_url: "${env.API_BASE_URL}"  # From environment

steps:
  - name: Create user
    operation:
      type: tool_call
      tool: create_user
      arguments:
        email: "${test_user_email}"

4. Comprehensive Assertions

Test success AND content:

assertions:
  # Not enough - only checks for success
  - type: success

  # Better - verify actual data
  - type: success
  - type: exists
    path: result.id
  - type: array_length
    path: result.items
    greater_than: 0
  - type: matches
    path: result.created_at
    pattern: "^\\d{4}-\\d{2}-\\d{2}T"

5. Error Case Testing

Test failure scenarios explicitly:

steps:
  - name: Test invalid input handling
    operation:
      type: tool_call
      tool: create_user
      arguments:
        email: "invalid-email"  # Missing @ symbol
    assertions:
      - type: failure          # Expect this to fail
      - type: exists
        path: error.message
      - type: contains
        path: error.message
        value: "invalid email"
        ignore_case: true

6. Performance Testing

Set appropriate timeouts and measure performance:

steps:
  - name: Fast operation
    operation:
      type: list_tools
    timeout: 5  # Should be fast

  - name: Slow operation (large data processing)
    operation:
      type: tool_call
      tool: process_large_dataset
      arguments:
        size: 10000
    timeout: 120  # Allow more time

7. Idempotent Tests

Use setup/cleanup for consistent test state:

setup:
  # Clean slate before each run
  - name: Delete existing test data
    operation:
      type: tool_call
      tool: cleanup_test_data
    continue_on_failure: true

steps:
  # ... tests ...

cleanup:
  # Always clean up, even on failure
  - name: Remove test artifacts
    operation:
      type: tool_call
      tool: cleanup_test_data
    continue_on_failure: true

8. Versioned Test Scenarios

Maintain scenarios alongside code:

my-mcp-server/
├── src/
│   └── main.rs
├── tests/
│   └── scenarios/
│       ├── v1.0/
│       │   ├── smoke-test.yaml
│       │   └── regression.yaml
│       ├── v1.1/
│       │   ├── smoke-test.yaml
│       │   ├── regression.yaml
│       │   └── new-feature.yaml
│       └── current -> v1.1/
└── Cargo.toml

Troubleshooting

Common Issues

Connection Refused:

# Use diagnostics to identify the problem
mcp-tester diagnose http://localhost:8080 --network

# Common causes:
# - Server not running
# - Wrong port
# - Firewall blocking connection

TLS Certificate Errors:

# For self-signed certificates in development
mcp-tester test https://localhost:8443 --insecure

Timeout Issues:

# Increase timeout for slow servers or cold starts
mcp-tester test $URL --timeout 120

OAuth Authentication Failures:

# Clear cached tokens and re-authenticate
rm ~/.mcp-tester/tokens.json
mcp-tester test $URL --oauth-client-id $CLIENT_ID

Schema Validation Warnings:

# Fix by adding complete schema
input_schema:
  type: object
  properties:
    param1:
      type: string
      description: "What this parameter does"
  required: ["param1"]

Summary

Effective MCP server testing requires a layered approach:

1. Interactive Testing - Use MCP Inspector for exploration and manual debugging
2. Automated Testing - Use mcp-tester for comprehensive, reproducible tests
3. Scenario Testing - Define complex workflows in YAML for regression testing
4. CI/CD Integration - Automate testing in your deployment pipeline

Key Takeaways:

• ✅ More servers than clients - Server testing is critical for ecosystem health
• ✅ MCP Inspector - Official tool for interactive, browser-based testing
• ✅ mcp-tester - Comprehensive CLI tool for automated testing and CI/CD
• ✅ Scenarios - Define reproducible test workflows in YAML/JSON
• ✅ OAuth support - Test authenticated servers with automatic token management
• ✅ Multi-transport - Test HTTP, HTTPS, WebSocket, and stdio servers
• ✅ Schema validation - Catch incomplete tool definitions early
• ✅ CI/CD ready - JSON output, exit codes, and headless operation

Next Steps:

1. Set up MCP Inspector for interactive development testing
2. Create basic smoke test scenario for your server
3. Add mcp-tester to your CI/CD pipeline
4. Build comprehensive regression test suite
5. Add pre-commit hooks for fast feedback

With comprehensive testing in place, you can confidently deploy MCP servers that work reliably with Claude and other MCP clients.