MCPContractTestAutomation

MCP Contract Testing Framework

A robust contract testing framework for REST APIs with Model Context Protocol (MCP) integration, demonstrated using the Petstore Swagger API.

Overview

This framework provides a comprehensive approach to contract testing that leverages the principles of the Model Context Protocol to maintain state, manage context, and validate API responses against OpenAPI/Swagger specifications.

Key Features

• MCP Context Management: Maintains state across test cases, similar to how MCP manages context for AI models
• Automated Contract Generation: Derives contracts from OpenAPI/Swagger specifications
• Schema Validation: Validates API responses against expected schemas
• Contextual Testing: Builds relationships between tests, preserving context for dependent operations
• Test Data Generation: Creates realistic test data based on schema definitions
• Detailed Reporting: Tracks test results with rich metadata for analysis

Project Structure

MCP Contract Testing/
├── config.js                # Framework configuration
├── package.json             # Project dependencies
├── README.md                # This file
└── src/
    ├── contracts/           # Generated contract files
    ├── mcp/                 # MCP integration modules
    │   └── contextManager.js # MCP context management
    ├── tests/               # Test files
    │   ├── pet.test.js      # Pet API tests
    │   ├── store.test.js    # Store API tests
    │   └── user.test.js     # User API tests
    └── utils/               # Utility modules
        ├── apiClient.js     # API client with MCP integration
        ├── contractGenerator.js # Contract generation utilities
        ├── contractValidator.js # Schema validation utilities
        ├── generateContracts.js # Contract generation script
        └── testHelper.js    # Test utilities

Getting Started

Prerequisites

• Node.js 14+
• NPM or Yarn

Installation

1. Clone this repository
2. Install dependencies:

npm install

Generating Contracts

Before running tests, generate contract files from the OpenAPI specification:

npm run generate-contracts

This will create JSON contract files in the src/contracts directory.

Running Tests

Run all tests:

npm test

Run specific test suites:

# Run only pet API tests
npx jest src/tests/pet.test.js

# Run only store API tests
npx jest src/tests/store.test.js

# Run only user API tests
npx jest src/tests/user.test.js

MCP Integration

This framework leverages key concepts from the Model Context Protocol:

Context Management

• Context Creation: Each test suite creates a shared context
• Context Inheritance: Individual tests inherit from shared contexts
• Context Updates: Test results update contexts in real-time
• Context References: Creates relationships between related API operations

Contextual Validation

• Schema Validation: Validates response data against OpenAPI schemas
• Context Consistency: Ensures responses are consistent with established context
• Stateful Testing: Maintains state throughout test execution

Customization

Configuration

Edit config.js to customize the framework behavior:

• API Settings: Base URL, timeouts, authentication
• MCP Settings: Enable/disable features, configure telemetry
• Validation Settings: Schema validation options
• Test Environment: Parallelization, logging

Adding New Tests

1. Create a new test file in src/tests/
2. Import the test helper and context manager
3. Create a shared context for the test suite
4. Write test cases using the test helper
5. Update contexts with test results

Best Practices

• Contextual Testing: Group related API operations in test suites with shared contexts
• Test Isolation: Create new contexts for independent test scenarios
• Context Hierarchies: Use parent-child relationships for contexts when appropriate
• Context References: Create references between related contexts for traceability
• Telemetry: Enable detailed logging for failed tests to aid debugging

Troubleshooting

Known Issues and Solutions

1. Unknown Format Error:

   • Error: unknown format "int32" ignored in schema
   • Solution: Add custom format definitions to Ajv in the ContractValidator constructor.

2. Schema Reference Resolution:

   • Error: can't resolve reference #/definitions/Category from id #
   • Solution: Use SwaggerParser.dereference() instead of SwaggerParser.parse() to properly resolve all references.

3. Open Handles After Tests:

   • Issue: Jest detects open handles after tests complete (e.g., "TLSWRAP").
   • Solution: Implement a cleanup method that closes connections after tests and call it in the afterAll hook.

4. Schema Validation Failures:

   • Issue: External API responses may not strictly conform to the schema.
   • Solution: In testing environments, consider making schema validation more lenient or focusing on critical fields only.

Memory Management

Make sure to properly clean up resources after tests complete by:

1. Implementing a cleanup method on the TestHelper
2. Calling the cleanup method in the afterAll hook of your test files
3. Properly closing any open connections