WEB-SCRAPING-MCP
MCP Server leveraging crawl4ai for web scraping and LLM-based content extraction (Markdown, text snippets, smart extraction). Designed for AI agent integration.
GitHub Stars
15
User Rating
Not Rated
Favorites
0
Views
22
Forks
6
Issues
1
Installation
Difficulty
IntermediateEstimated Time
10-20 minutes
Installation
You can run this server either locally or using the provided Docker configuration.
This method bundles Python and all necessary libraries. You only need Docker installed on the host machine.
1 Install Docker: Download and install [Docker Desktop](https://www.docker.com/products/docker-desktop/) for your OS. Start Docker Desktop.
2 Clone Repository:
bash
git clone https://github.com/your-username/your-repo-name.git # Replace with your repo URL
cd your-repo-name
3 Create
.env File: Create a file named .env in the project root directory and add your API keys:.env
# Required for the smart_extract tool
GOOGLE_API_KEY=your_google_ai_api_key_here
# Optional, checked by server but not currently used by tools
# OPENAI_API_KEY=your_openai_key_here
# MISTRAL_API_KEY=your_mistral_key_here
4 Build the Docker Image:
bash
docker build -t crawl4ai-mcp-server .
Additional Resources
Author Information
I'm Maitreya Mishra , an AI development intern @ ML research lab Parul University. I specialize in deep learning and Applied AI.
Gujarat , India
2
Followers
31
Repositories
0
Gists
0
Total Contributions
Related MCPs