Explore and analyze inspect_ai evaluation logs directly from LLMs
inspect-logs-mcp
MCP server for exploring inspect_ai evaluation logs from Claude Code.
What it does
This MCP server gives Claude Code, Cursor, and other LLMs direct access to your inspect_ai evaluation logs, allowing you to:
- List logs: See all your evaluation runs with metadata
- View summaries: Get detailed evaluation results, scores, and token usage
- Inspect samples: View full conversation history for any sample
- Search & filter: Find logs by task, model, date, or status
- Compare runs: Side-by-side comparison of two evaluation runs
- Aggregate stats: Get statistics across multiple runs
Installation
Currently only source installation is supported.
git clone https://github.com/PranshuSrivastava/inspect-logs-mcp.git
cd inspect-logs-mcp
pip install -e .
Configuration
To use with Claude code in your current directory, run the following command:
claude mcp add --transport stdio inspect-logs inspect-logs-mcp
To use with Claude code in your global configuration, run the following command:
claude mcp add --scope user --transport stdio inspect-logs inspect-logs-mcp
To use with Cursor, Antigravity or other IDEs, add the following to your mcp config file:
{
"mcpServers": {
"inspect-logs": {
"command": "inspect-logs-mcp",
"env": {
"INSPECT_LOG_DIR": "./logs"
}
}
}
}
Environment Variables
| Variable | Description | Default |
|---|---|---|
INSPECT_LOG_DIR |
Directory containing .eval log files |
./logs |
INSPECT_LOGS_MCP_DEFAULT_LIMIT |
Default number of logs limit | 50 |
INSPECT_LOGS_MCP_MAX_LIMIT |
Maximum number of logs limit | 500 |
How it Works
The server reads .eval files (which are ZIP archives containing JSON) using the inspect_ai.log API. All file operations happen in memory - no files are extracted to disk, so your logs directory stays clean.
Requirements
- Python 3.10+
- inspect-ai >= 0.3.70
- mcp >= 1.0.0
Tools (3)
list_logsList all evaluation runs with metadataview_summaryGet detailed evaluation results, scores, and token usage for a specific loginspect_sampleView full conversation history for a specific sample within a logEnvironment Variables
INSPECT_LOG_DIRDirectory containing .eval log filesINSPECT_LOGS_MCP_DEFAULT_LIMITDefault number of logs limitINSPECT_LOGS_MCP_MAX_LIMITMaximum number of logs limitConfiguration
{"mcpServers": {"inspect-logs": {"command": "inspect-logs-mcp", "env": {"INSPECT_LOG_DIR": "./logs"}}}}