LLMBenchmark.ai
Which LLM for which task: benchmark scores and API prices for 400+ models. Free, no key
Reviews
No review yet. Used LLMBenchmark.ai? Be the first to help others choose.What it does
LLMBenchmark.ai is an MCP server, a connector that lets an AI assistant use it as a tool. It can be used hosted (nothing to install) or installed locally.
No capability identified yet from its description.
Facts come from the Official MCP Registry, as declared by the publisher. Capabilities are matched automatically (low confidence). None of this is a test of the tool.
At a glance
How to connect
As declared in the Official MCP Registry · hosted endpoint tested by Electra Index 1d ago (live check)Hosted endpoint
https://api.llmbenchmark.ai/mcpstreamable-http
In Claude Code: claude mcp add --transport http llmbenchmark-ai https://api.llmbenchmark.ai/mcp
Install package
- PYPI
llmbenchmark-mcpv1.0.0 · runs locally (stdio)
MCP settings (Claude Desktop, Cursor and other MCP clients)
{
"mcpServers": {
"llmbenchmark-ai": {
"url": "https://api.llmbenchmark.ai/mcp"
}
}
}Reliability
Re-tested about every 3 days · last check 1d agoObserved endpoint availability: the share of our conclusive checks that got an answer. A request for a key or sign-in counts as an answer (the server is up, its access is protected). Timeouts, unreachable endpoints, server errors and invalid MCP replies count as failures; rate-limited checks and checks during our own incidents are not counted. This is what we observed at each check, not continuous uptime: an outage between two checks may go unseen. The 90-day figure will appear once 90 days of checks exist (checks since 19 Nov 2026). How we measure
Tool changes
Tool tracking since 20 Nov 2026 · test history since 19 Nov 2026No tool added or removed since tracking started. 6 tools in the latest list.
Changes are confirmed by comparing complete tool lists only, read about every 3 days, and dated by the day we saw them. A renamed tool appears as one removal and one addition.
Live check
Electra Index connected to the hosted endpoint 1d agoONLINE LLMBenchmark.ai answered an MCP connection on 19 Nov 2026 and introduced itself as LLMBenchmark.ai 1.0.0 (MCP protocol 2025-06-18).
6 tools offered to AI assistants
recommend_model· Which model should I use for this task? Maps the task text to a category (coding, agentic, reasoning, math, knowledge, web design, or overall when nothing matches) by a fixed keyword map, filters by output price per million tokens, contextget_leaderboard· The top models on one category's board. category is one of overall, coding, agentic, reasoning, math, knowledge, design (see list_categories). limit is 1 to 50. overall is the LLMBenchmark Index: a weighted mean of the Vibe Coding Index, thcompare_models· Two models side by side: score on every category board where both were measured, prices, context. Names are matched loosely: 'claude opus', 'gpt-5' or a slug all work.get_model· One model: maker, release date, prices, context, open weights, and its score and rank on every category board. Name or slug.get_prices· API prices (USD per million tokens), cheapest output price first: the maker's own price when it serves the model on OpenRouter, else the median of providers (price_basis says which), plus the cheapest provider. Filter by model name and/or mlist_categories· The categories recommend_model and get_leaderboard know, the benchmark board behind each, and the task keywords that select it.
What "online" means: the server answered an MCP connection at the time of the check. It is not a guarantee of security, quality or continuous uptime. How we test: we open an MCP session and ask for the tool list, as an AI assistant would. We never run a tool and never send a key. Re-tested about every 3 days. Method
Is this your server? Monitoring Pro: a check every 30 minutes and an email within the hour when it stops answering. Launch offer: $39.99 once, for life.
Facts & evidence
Each fact shows where it was seen and when.| Fact | Value | Evidence | Source | Last confirmed |
|---|---|---|---|---|
| Install package | llmbenchmark-mcp | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| Package registries | pypi | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| Local install (package) | yes | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| Version | 1.0.1 | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| Hosted endpoint | https://api.llmbenchmark.ai/mcp | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| Remote access (hosted) | yes | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| Website | https://llmbenchmark.ai/api | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| Repository | https://github.com/Take-Group/llmbenchmark-mcp | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| Registry status | active | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| MCP server | yes | DECLARED | SRC-001 Official MCP Registry | 1d ago |
Change history
- CHANGED19 Nov 2026local install: no → yes
- VERSION19 Nov 20261.0.0 → 1.0.1
- NEW19 Nov 2026first seen in the index
Is this page accurate?
Flag it and our robot re-checks the official source. Reports never change a score.Is LLMBenchmark.ai still online?
Free alerts · no account▲ 0Do you run LLMBenchmark.ai? Know when it stops answering
For publishers · optionalIts hosted endpoint can be monitored. Monitoring Pro connects every 30 minutes like an AI assistant, emails you within the hour when it stops answering, keeps a 90-day availability history and adds your answer rate to your badge.
Monitoring never changes a score or a rank.
Questions about LLMBenchmark.ai
How do I connect LLMBenchmark.ai?
Use the hosted endpoint https://api.llmbenchmark.ai/mcp in your assistant's MCP settings. The exact settings are in "How to connect" above, as declared in the Official MCP Registry.
What tools does LLMBenchmark.ai offer?
When Electra Index connected on 19 Nov 2026, it listed 6 tools: recommend_model, get_leaderboard, compare_models, get_model, get_prices, list_categories. The full list is in "Live check" above.
Is LLMBenchmark.ai online?
Yes. Its hosted endpoint answered an MCP connection from Electra Index 1d ago.