mcp-llms-txt-explorer

The MCP LLMS.txt Explorer is a server designed for exploring and analyzing websites that implement llms.txt files. This tool helps discover compliant websites and provides access to structured data. It features website validation and listing capabilities, making it a valuable resource for developers.

GitHub Stars

66

User Rating

Not Rated

Favorites

0

Views

36

Forks

14

Issues

2

Installation
Difficulty
Intermediate
Estimated Time
10-20 minutes
Requirements
Node.js: 18.0.0以上
pnpm: 最新版
+1 more

Installation

Installation

Prerequisites

Please specify required software and versions:
Node.js: 18.0.0 or higher
pnpm: Latest version
Claude Desktop: Latest version

Installation Steps

1. Clone Repository

bash
git clone https://github.com/thedaviddias/mcp-llms-txt-explorer.git
cd mcp-llms-txt-explorer

2. Install Dependencies

bash
pnpm install

3. Build the Server

bash
pnpm run build

4. Configure Claude Desktop

Edit claude_desktop_config.json to add the MCP server:
json
{
  "mcpServers": {
    "llms-txt-explorer": {
      "command": "node",
      "args": ["/path/to/llms-txt-explorer/build/index.js"]
    }
  }
}

5. Start Server

bash
pnpm start

Troubleshooting

Common Issues

Issue: Server won't start Solution: Check Node.js version and reinstall dependencies. Issue: Not recognized by Claude Desktop Solution: Verify configuration file path and syntax.

Configuration

Configuration

Basic Configuration

Claude Desktop Setup

Edit ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):
json
{
  "mcpServers": {
    "llms-txt-explorer": {
      "command": "node",
      "args": ["/path/to/llms-txt-explorer/build/index.js"]
    }
  }
}

Environment Variables

Set the following environment variables as needed:
bash
export API_KEY="your-api-key"
export DEBUG="true"

Advanced Configuration

Security Settings

Store API keys in environment variables or secure configuration files
Set appropriate file access permissions
Adjust logging levels

Performance Tuning

Configure timeout values
Limit concurrent executions
Set up caching

Configuration Examples

Basic Configuration

json
{
  "mcpServers": {
    "llms-txt-explorer": {
      "command": "node",
      "args": ["/path/to/llms-txt-explorer/build/index.js"]
    }
  }
}

Examples

Examples

Basic Usage

Here are basic usage examples for the MCP server:

Using with Claude Desktop

1Verify MCP Server Startup
Open Claude Desktop and confirm that the configuration has been loaded correctly.
2Execute Basic Commands

   Available tools from this MCP server:
   - check_website: Check if a website has llms.txt files
   - list_websites: List known websites with llms.txt files
   

Programmatic Usage

javascript
// JavaScript example (Node.js)
const { MCPClient } = require('@modelcontextprotocol/client');

const client = new MCPClient();
await client.connect();

// Execute tool
const result = await client.callTool('check_website', {
  domain: 'example.com'
});

console.log(result);

Advanced Examples

Automation Script

bash
#!/bin/bash

Batch processing example

for file in *.txt; do mcp-tool process "$file" done

API Integration

python

Python example

import requests import json def call_mcp_tool(tool_name, params): response = requests.post( 'http://localhost:3000/mcp/call', json={ 'tool': tool_name, 'parameters': params } ) return response.json()

Usage example

result = call_mcp_tool('list_websites', {})

Use Cases

Creating a script to check if a specific website has llms.txt files
Validating the contents of llms.txt files to ensure they are correctly formatted
Retrieving a list of compliant websites to compare AI/LLM implementations
Automating a test script to periodically check for the existence of llms.txt files