README.md
High-performance web search service for LLM clients via MCP
Web Search MCP
High-performance web search service for LLM clients via Model Context Protocol. Powered by Camoufox stealth browser.
Features
- 3 Search Engines — Google, Bing, DuckDuckGo with automatic fallback
- Multi-depth Scraping — SERP parsing → full-text extraction → outbound link crawling
- Stealth Browser — Camoufox anti-detection Firefox (GeoIP, Humanize, Locale)
- Auto-scaling Pool — Browser pool auto-scales at 80% utilization, configurable upper limit
- Admin Dashboard — Search analytics, system monitoring, API key management, IP banning
- API Key Auth — Built-in key generation (
wsm_prefix) with call limits and revocation - Dual Output — JSON and Markdown formats
- REST API — Standard HTTP API alongside MCP protocol
Quick Start
Docker (Recommended)
git clone https://github.com/nicepkg/web-search-mcp.git
cd web-search-mcp
# Configure
cp .env.example .env
# Edit .env — set ADMIN_TOKEN
# Launch
docker compose up -d
# Verify
curl http://127.0.0.1:8897/health
Create API Key & Register to Claude Code
# 1. Create an API key via Admin API
curl -X POST http://127.0.0.1:8897/admin/api/keys \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{"name": "claude-code", "call_limit": 10000}'
# Save the returned wsm_xxx key (only shown once)
# 2. Register MCP to Claude Code
claude mcp add-json -s user web-search-fast '{
"type": "http",
"url": "http://127.0.0.1:8897/mcp",
"headers": {"Authorization": "Bearer wsm_your-api-key-here"}
}'
# 3. Restart Claude Code session
Or use the Admin Dashboard at http://127.0.0.1:8897/admin to create keys visually.
MCP Tools
| Tool | Description | Timeout |
|---|---|---|
web_search |
Search the web, returns Markdown results | 25s |
get_page_content |
Fetch and extract content from a URL | 20s |
list_search_engines |
List available engines and pool status | — |
web_search Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
query |
string | required | Search keywords |
engine |
string | duckduckgo |
google / bing / duckduckgo |
depth |
int | 1 |
1=SERP only, 2=SERP+content, 3=SERP+content+outlinks |
max_results |
int | 5 |
Max results (1-20) |
Search Depth
| Depth | Behavior | Description |
|---|---|---|
1 |
SERP parsing | Default. Extracts titles, links, snippets |
2 |
SERP + content | Navigates to each result, extracts page content |
3 |
SERP + content + outlinks | Also crawls outbound links from content |
Authentication Model
| Token Type | Source | Access |
|---|---|---|
ADMIN_TOKEN |
Environment variable | Admin panel API (superuser) |
wsm_ API Key |
Created via Admin panel | MCP / Search API |
ADMIN_TOKENhas superuser access to all endpointswsm_keys can only access MCP and search endpoints (not admin API)- If no
ADMIN_TOKENis set and no API keys exist, all endpoints are open
REST API
Standard HTTP API available alongside MCP.
# GET
curl 'http://127.0.0.1:8897/search?q=python+asyncio&engine=duckduckgo&max_results=5' \
-H 'Authorization: Bearer wsm_your-key'
# POST
curl -X POST http://127.0.0.1:8897/search \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer wsm_your-key' \
-d '{"query": "python asyncio", "engine": "duckduckgo", "depth": 2, "max_results": 5}'
| Parameter | Type | Default | Description |
|---|---|---|---|
q / query |
string | required | Search keywords (1-500 chars) |
engine |
string | duckduckgo |
google / bing / duckduckgo |
depth |
int | 1 |
Scraping depth: 1-3 |
format |
string | json |
json / markdown |
max_results |
int | 10 |
Max results (1-50) |
timeout |
int | 30 |
Timeout in seconds (5-120) |
Admin Dashboard
Access http://127.0.0.1:8897/admin and login with ADMIN_TOKEN.
- Dashboard — Search stats, CPU/memory monitoring, browser pool status, latency charts, engine distribution, success rate
- API Keys — Create/revoke keys with call limits
- IP Bans — Ban/unban IP addresses
- Search Logs — Search history with IP filtering
Engine Status
| Engine | Status | Notes |
|---|---|---|
| DuckDuckGo | Stable | Recommended default, HTML-lite mode |
| Limited | May trigger captcha on some IPs, auto-fallback | |
| Bing | Available | Uses global.bing.com to avoid geo-redirect |
When Google is blocked, automatically falls back: DuckDuckGo → Bing.
Deployment
Endpoints
| URL | Description |
|---|---|
http://127.0.0.1:8897/mcp |
MCP endpoint (Streamable HTTP |
Tools (3)
web_searchSearch the web, returns Markdown resultsget_page_contentFetch and extract content from a URLlist_search_enginesList available engines and pool statusEnvironment Variables
ADMIN_TOKENrequiredToken for accessing the admin panel and API key managementConfiguration
claude_desktop_config.json
{"mcpServers": {"web-search-fast": {"type": "http", "url": "http://127.0.0.1:8897/mcp", "headers": {"Authorization": "Bearer wsm_your-api-key-here"}}}}Try it
→Search for the latest updates on the Model Context Protocol using Google.
→Find and summarize the content of this URL: https://example.com
→Perform a deep search for 'best practices in Python async' and include outbound links.
→List the currently available search engines and their status.