patronus-mcp-server
The Patronus MCP Server is an implementation of an MCP server for the Patronus SDK, providing a standardized interface for running powerful LLM system optimizations, evaluations, and experiments. It allows initialization with an API key and project settings, and supports both single and batch evaluations.
GitHub Stars
13
User Rating
Not Rated
Favorites
0
Views
30
Forks
3
Issues
0
Installation
Difficulty
IntermediateEstimated Time
10-20 minutes
Requirements
Python 3.7以上pipの最新バージョンInstallation
Installation
Prerequisites
Please specify required software and versions:Python: 3.7 or higher
pip: Latest version
Installation Steps
1. Clone Repository
bash
git clone https://github.com/patronus-ai/patronus-mcp-server.git
cd patronus-mcp-server
2. Create and Activate Virtual Environment
bash
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
3. Install Dependencies
bash
uv pip install -e .
uv pip install -e ".[dev]"
Troubleshooting
Common Issues
Issue: Server won't start Solution: Check Python version and reinstall dependencies.Configuration
Configuration
Basic Configuration
Setting API Key
You can set the API key in an environment variable:bash
export PATRONUS_API_KEY=your_api_key_here
Advanced Configuration
Security Settings
Store API keys in environment variables or secure configuration files
Set appropriate file access permissions
Configuration Example
Environment Variable Setting
bash
export PATRONUS_API_KEY="your-api-key"
Examples
Examples
Running the Server
Here are ways to run the server with an API key:Using Command Line Argument
bash
python src/patronus_mcp/server.py --api-key your_api_key_here
Using Environment Variable
bash
export PATRONUS_API_KEY=your_api_key_here
python src/patronus_mcp/server.py
Executing Single Evaluation
python
from patronus_mcp.server import Request, EvaluationRequest, RemoteEvaluatorConfig
request = Request(data=EvaluationRequest(
evaluator=RemoteEvaluatorConfig(
name="lynx",
criteria="patronus:hallucination",
explain_strategy="always"
),
task_input="What is the capital of France?",
task_output="Paris is the capital of France.",
task_context=["The capital of France is Paris."],
))
response = await mcp.call_tool("evaluate", {"request": request.model_dump()})
Use Cases
Performance evaluation of AI models for specific projects
Batch evaluation of AI systems using multiple evaluation criteria
Experiments and optimizations of AI models using datasets
Retrieving evaluation results through the API for analysis