toolhive-buildkite-plugin

A Buildkite plugin that enables running MCP servers using ToolHive

GitHubスター

5

ユーザー評価

未評価

お気に入り

0

閲覧数

3

フォーク

0

イシュー

1

README
ToolHive Buildkite Plugin

A Buildkite plugin that enables running Model Context Protocol (MCP) servers using ToolHive in your CI/CD pipelines.

Features
  • Automatic ToolHive Installation: Downloads and installs ToolHive if not already available
  • MCP Server Management: Start, manage, and clean up MCP servers during pipeline execution
  • Multiple Server Sources: Support for registry servers, Docker images, and protocol schemes (uvx://, npx://, go://)
  • Flexible Configuration: Customize transport methods, ports, volumes, secrets, and more
Usage

Add the plugin to your pipeline step:

steps:
  - label: "Run with MCP Server"
    command: "your-command-here"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "fetch"  # Server from ToolHive registry
Registry Server Example
steps:
  - label: "Use Fetch MCP Server"
    command: "curl http://localhost:8080/some-endpoint"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "fetch"
          transport: "stdio"
          proxy-port: 8080
Docker Image Example
steps:
  - label: "Use Custom MCP Server"
    command: "your-command"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "my-registry/my-mcp-server:latest"
          transport: "sse"
          volumes:
            - "/host/path:/container/path:ro"
Protocol Scheme Example
steps:
  - label: "Use Python MCP Server"
    command: "your-command"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "uvx://some-python-mcp-package@1.0.0"
          transport: "streamable-http"
          args:
            - "--verbose"
            - "--config=/path/to/config"
With Secrets
steps:
  - label: "Use GitHub MCP Server"
    command: "your-command"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "github"
          secrets:
            - name: "github-token"
              target: "GITHUB_PERSONAL_ACCESS_TOKEN"
            - name: "api-key"
              target: "API_KEY"
Configuration
Required
Option Type Description
server String The MCP server to run (registry name, Docker image, or protocol scheme)
Optional
Option Type Default Description
name String Auto-generated Custom name for the MCP server instance
transport String "" (auto) Transport method: stdio, sse, or streamable-http
proxy-port Integer Random Specific port for the ToolHive proxy
secrets Array [] Secrets to pass to the MCP server
volumes Array [] Volume mounts in format "host-path:container-path[:ro]"
args Array [] Additional arguments to pass to the MCP server
permission-profile String Default Permission profile for the MCP server
toolhive-version String Latest Specific version of ToolHive to download
cleanup Boolean true Whether to clean up the MCP server on exit
mcp-config-file String ./mcp_servers.json Path where to generate the MCP config file
mcp-config-cleanup Boolean true Whether to remove the MCP config file on exit
Secrets Configuration

Secrets are configured as an array of objects with name and target properties:

secrets:
  - name: "secret-name-in-toolhive"
    target: "ENVIRONMENT_VARIABLE_NAME"

The name refers to a secret stored in ToolHive's secret management system, and target is the environment variable name that will be set in the MCP server container.

Note that secrets must be created in ToolHive before they can be used in the plugin.

Volume Configuration

Volumes are specified as strings in Docker volume format:

volumes:
  - "/host/path:/container/path"      # Read-write mount
  - "/host/path:/container/path:ro"   # Read-only mount
Server Types
Registry Servers

Use servers from the ToolHive registry:

# Fetch MCP server
server: "fetch"
# GitHub MCP server
server: "github"
# Filesystem MCP server
server: "filesystem"
Docker Images

Use any Docker image that implements the MCP protocol:

# Custom Docker image
server: "my-registry/my-mcp-server:v1.0.0"
# GitHub Container Registry image
server: "ghcr.io/org/mcp-server:latest"
Protocol Schemes

Use package managers to run MCP servers:

# Python via uv
server: "uvx://python-mcp-package@1.0.0"
# Node.js via npm
server: "npx://node-mcp-package@2.0.0"
# Go module
server: "go://github.com/org/go-mcp-server"
MCP Configuration File Generation

The plugin automatically generates an MCP configuration file that contains connection details for all spawned MCP servers. This makes it easy for MCP clients to discover and connect to available servers.

Configuration File Format

The generated file follows the standard MCP configuration format:

{
  "mcpServers": {
    "fetch-server": {
      "url": "http://localhost:8080/mcp",
      "type": "streamable-http"
    },
    "github-server": {
      "url": "http://localhost:8081/sse#github-server",
      "type": "sse"
    }
  }
}
URL Format Details
  • SSE servers: http://localhost:{port}/sse#{server-name}
  • Streamable HTTP servers: http://localhost:{port}/mcp
  • Type: Either "sse" or "streamable-http"
Environment Variable

The plugin exports BUILDKITE_PLUGIN_TOOLHIVE_MCP_CONFIG_FILE pointing to the generated configuration file:

echo "MCP config file: $BUILDKITE_PLUGIN_TOOLHIVE_MCP_CONFIG_FILE"
cat $BUILDKITE_PLUGIN_TOOLHIVE_MCP_CONFIG_FILE
Usage with MCP Clients
steps:
  - command: |
      # Use the generated MCP configuration with your tools
      my-mcp-client --config $BUILDKITE_PLUGIN_TOOLHIVE_MCP_CONFIG_FILE
      
      # Or read the configuration programmatically
      python -c "
      import json
      import os
      config_file = os.environ['BUILDKITE_PLUGIN_TOOLHIVE_MCP_CONFIG_FILE']
      with open(config_file) as f:
          config = json.load(f)
          print('Available MCP servers:', list(config['mcpServers'].keys()))
      "
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "fetch"
          mcp-config-file: "./my_mcp_config.json"
How It Works
  1. Environment Hook: Checks if ToolHive is available, downloads it if needed
  2. Pre-Command Hook: Starts the specified MCP server with the given configuration
  3. Command Execution: Your pipeline command runs with the MCP server available
  4. Pre-Exit Hook: Stops and removes the MCP server (if cleanup is enabled)
Server Naming

The plugin automatically generates unique server names to avoid conflicts:

  • Uses custom name if provided via the name option
  • Otherwise generates: build-{BUILD_NUMBER}-step-{STEP_KEY}-{SERVER_NAME}
  • Names are normalized (lowercase, special characters replaced with hyphens)
Requirements
  • Docker or Podman container runtime
  • Internet access to download ToolHive (if not already installed)
  • Sufficient permissions to run containers
Troubleshooting
ToolHive Installation Issues

If ToolHive fails to install:

  1. Check internet connectivity
  2. Verify the GitHub releases are accessible
  3. Ensure sufficient disk space
  4. Check file permissions in the installation directory
MCP Server Startup Issues

If the MCP server fails to start:

  1. Check the server logs: thv logs <server-name>
  2. Verify the server configuration
  3. Ensure required secrets are available
  4. Check container runtime (Docker/Podman) status
Port Conflicts

If you encounter port conflicts:

  1. Use the proxy-port option to specify a different port
  2. Check for other services using the same port
  3. Use dynamic port allocation (default behavior)
Examples
Complete Example with All Options
steps:
  - label: "Complex MCP Server Setup"
    command: |
      echo "MCP server is running"
      curl http://localhost:9000/health
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "my-registry/custom-mcp:v2.0.0"
          name: "my-custom-server"
          transport: "sse"
          proxy-port: 9000
          secrets:
            - name: "api-token"
              target: "API_TOKEN"
            - name: "db-password"
              target: "DATABASE_PASSWORD"
          volumes:
            - "./config:/app/config:ro"
            - "./data:/app/data"
          args:
            - "--log-level=debug"
            - "--config=/app/config/server.yml"
          permission-profile: "network"
          toolhive-version: "v0.0.33"
          cleanup: true
          mcp-config-file: "./custom_mcp_config.json"
          mcp-config-cleanup: false
Multiple Steps with Different Servers
steps:
  - label: "Step 1: Use Fetch Server"
    command: "test-fetch-functionality"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "fetch"
          
  - label: "Step 2: Use GitHub Server"
    command: "test-github-integration"
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "github"
          secrets:
            - name: "github-token"
              target: "GITHUB_PERSONAL_ACCESS_TOKEN"
Multiple MCP Servers in One Step

You can run multiple MCP servers in a single step by calling the plugin multiple times:

steps:
  - label: "Use Multiple MCP Servers"
    command: |
      echo "Both servers are now running"
      curl http://localhost:8080/fetch-endpoint
      curl http://localhost:8081/github-endpoint
    plugins:
      - StacklokLabs/toolhive#v0.0.2:
          server: "fetch"
          name: "fetch-server"
          proxy-port: 8080
      - StacklokLabs/toolhive#v0.0.2:
          server: "github"
          name: "github-server"
          proxy-port: 8081
          secrets:
            - name: "github-token"
              target: "GITHUB_PERSONAL_ACCESS_TOKEN"

Important Notes for Multiple Servers:

  • Each server must have a unique name to avoid conflicts
  • Each server should use a different proxy-port if specified
  • All servers will be automatically cleaned up at the end of the step
  • Servers are started in the order they appear in the plugin list
Contributing
  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Test the plugin
  5. Submit a pull request
License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Links