1
Add it to Claude Code
claude mcp add mcp-crawl4ai -- mcp-crawl4ai2
Make your agent remember this setup
mcp-crawl4ai's config, env vars, and the gotchas you hit — recalled in every future Claude Code, Cursor, and Codex session.
npx conare@latestFree · one command · indexes the sessions already on disk. Set up in the browser instead →
What it does
- Full MCP compliance via FastMCP v3
- Headless Chromium managed as a lifespan singleton
- LLM-optimized output including markdown, cleaned HTML, and plain text
- Support for session-aware workflows and deep site traversal
- Automatic browser setup and Playwright dependency management
Tools 2
crawlPerforms web crawling and scraping tasks with support for deep traversal and structured extraction.close_sessionCloses an active crawling session and cleans up resources.Try it
→Crawl the documentation page at https://example.com/docs and summarize the key features in markdown.
→Extract the pricing table from https://example.com/pricing and format it as a JSON object.
→Perform a deep crawl of https://example.com to find all links related to 'API documentation'.
→Scrape the latest news articles from https://example.com/news and provide a bulleted list of headlines.
Original README from wyattowalsh/mcp-crawl4ai
MCP-Crawl4AI
A Model Context Protocol server for web crawling powered by Crawl4AI
Overview
MCP-Crawl4AI is a Model Context Protocol server that gives AI systems access to the live web. Built on FastMCP v3 and Crawl4AI, it exposes 4 tools, 2 resources, and 3 prompts through the standardized MCP interface, backed by a lifespan-managed headless Chromium browser.
Only 2 runtime dependencies — fastmcp and crawl4ai.
[!TIP] Full documentation site →
Key Features
- Full MCP compliance via FastMCP v3 with tool annotations (
readOnlyHint,destructiveHint, etc.) - 4 focused tools centered on canonical scrape/crawl plus session lifecycle/artifacts
- 3 prompts for common LLM workflows (summarize, extract schema, compare pages)
- 2 resources exposing server configuration and version info
- Headless Chromium managed as a lifespan singleton (start once, reuse everywhere)
- Multiple transports — stdio (default) and Streamable HTTP
- LLM-optimized output — markdown, cleaned HTML, raw HTML, or plain text
- Canonical option groups for extraction, runtime, diagnostics, sessions, rendering, and traversal
- List and deep traversal in one
crawlcontract - Session-aware workflows with explicit session close and artifact retrieval tools
- Auto browser setup — detects missing Playwright browsers and installs automatically
Installation
<details> <summary><strong>pip</strong></summary>pip install mcp-crawl4ai
mcp-crawl4ai --setup # one-time: installs Playwright browsers
uv add mcp-crawl4ai
mcp-crawl4ai --setup # one-time: installs Playwright browsers
docker build -t mcp-crawl4ai .
docker run -p 8000:8000 mcp-crawl4ai
The Docker image includes Playwright browsers — no separate setup needed.
</details> <details> <summary><strong>Development</strong></summary>git clone https://github.com/wyattowalsh/mcp-crawl4ai.git
cd mcp-crawl4ai
uv sync --group dev
mcp-crawl4ai --setup
[!NOT