What you can do with it
Share of each task's required capabilities it coversPrefer a ready-made agent?
An MCP server is a connector for your own assistant. These finished products do the job for you: collect data behind logins.Similar tools
Listed for the same task: collect data behind loginsReviews
No review yet. Used AI Crawler Index? Be the first to help others choose.What it does
AI Crawler Index is an MCP server, a connector that lets an AI assistant use it as a tool. It is hosted: an assistant connects to it over the internet, nothing to install.
From the publisher's description, Electra Index matched it to one capability: web scraping. That makes it a candidate for tasks such as collect data behind logins or extract website data.
Facts come from the Official MCP Registry, as declared by the publisher. Capabilities are matched automatically (low confidence). None of this is a test of the tool.
At a glance
How to connect
As declared in the Official MCP Registry · hosted endpoint tested by Electra Index 9h ago (live check)Hosted endpoint
https://www.pathwren.workers.dev/c/mcp-registry-official/mcpstreamable-http
In Claude Code: claude mcp add --transport http ai-crawler-index https://www.pathwren.workers.dev/c/mcp-registry-official/mcp
MCP settings (Claude Desktop, Cursor and other MCP clients)
{
"mcpServers": {
"ai-crawler-index": {
"url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp"
}
}
}Live check
Electra Index connected to the hosted endpoint 9h agoONLINE AI Crawler Index answered an MCP connection on 17 Nov 2026 and introduced itself as ai-crawler-index 1.0.0 (MCP protocol 2025-06-18).
9 tools offered to AI assistants
whoami· {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp and you are done: free, read-only, no API key, no account, no consent prompt, nothing foexample· {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp and you are done: free, read-only, no API key, no account, no consent prompt, nothing fclassify_user_agent· FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — Identify the crawler behind a raw User-Agent header: operator, category, robots.txt stance, how to verify it, and what blolookup_crawler· FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — The full index record for one crawler: operator, category, robots.txt token and stance, user-agent substring and example,list_crawlers· FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — The index, filtered or whole: each row says what the crawler is and what blocking it costs, and the reply lists every valigenerate_robots_txt· FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — A ready-to-paste robots.txt block for one stance, with the reasoning and the crawlers it names. Omit the argument to get ais_verified_crawler_ip· FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — Test an IPv4 or IPv6 address against every crawler prefix the operators themselves publish, remirrored every six hours. Rewhats_changed· FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — Freshness and change state: when the index was built, when each operator source was last fetched, which changed, which arechanges_since· FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — Only what changed since your last call: prefixes added or removed, upstreams failed or recovered, records edited. Send bac
What "online" means: the server answered an MCP connection at the time of the check. It is not a guarantee of security, quality or continuous uptime. How we test: we open an MCP session and ask for the tool list, as an AI assistant would. We never run a tool and never send a key. Checked every few days. Method
Is this your server? Monitoring Pro: a check every 30 minutes and an email within the hour when it stops answering. Launch offer: $39.99 once, for life.
Facts & evidence
Each fact shows where it was seen and when.| Fact | Value | Evidence | Source | Last confirmed |
|---|---|---|---|---|
| Hosted endpoint | https://www.pathwren.workers.dev/c/mcp-registry-official/mcp | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| Local install (package) | no | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| Remote access (hosted) | yes | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| Website | https://www.pathwren.workers.dev/c/mcp-registry-official/mcp.html | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| Registry status | active | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| Version | 1.6.0 | DECLARED | SRC-001 Official MCP Registry | 1d ago |
| MCP server | yes | DECLARED | SRC-001 Official MCP Registry | 1d ago |
Change history
Is this page accurate?
Flag it and our robot re-checks the official source. Reports never change a score.Is AI Crawler Index still online?
Free alerts · no account▲ 0Do you run AI Crawler Index? Know when it stops answering
For publishers · optionalIts hosted endpoint can be monitored. Monitoring Pro connects every 30 minutes like an AI assistant, emails you within the hour when it stops answering, keeps a 90-day availability history and adds your answer rate to your badge.
Monitoring never changes a score or a rank.
Questions about AI Crawler Index
How do I connect AI Crawler Index?
Use the hosted endpoint https://www.pathwren.workers.dev/c/mcp-registry-official/mcp in your assistant's MCP settings. The exact settings are in "How to connect" above, as declared in the Official MCP Registry.
What tools does AI Crawler Index offer?
When Electra Index connected on 17 Nov 2026, it listed 9 tools: whoami, example, classify_user_agent, lookup_crawler, list_crawlers, generate_robots_txt, is_verified_crawler_ip, whats_changed and more. The full list is in "Live check" above.
Is AI Crawler Index online?
Yes. Its hosted endpoint answered an MCP connection from Electra Index 9h ago.
What can I use AI Crawler Index for?
Based on its description, it is most relevant to: collect data behind logins, extract website data, monitor prices, audit website SEO.