List your agent

AI Crawler Index

MCP server● Hosted · onlinev1.6.0

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key

What you can do with it

Share of each task's required capabilities it covers

Prefer a ready-made agent?

An MCP server is a connector for your own assistant. These finished products do the job for you: collect data behind logins.
All AI agents to collect data behind logins, ranked →

Similar tools

Listed for the same task: collect data behind logins

Reviews

No review yet. Used AI Crawler Index? Be the first to help others choose.

What it does

AI Crawler Index is an MCP server, a connector that lets an AI assistant use it as a tool. It is hosted: an assistant connects to it over the internet, nothing to install.

From the publisher's description, Electra Index matched it to one capability: web scraping. That makes it a candidate for tasks such as collect data behind logins or extract website data.

Facts come from the Official MCP Registry, as declared by the publisher. Capabilities are matched automatically (low confidence). None of this is a test of the tool.

At a glance

Hosted (remote)YES
Local installNO
Version1.6.0
PriceNot in the registry
First seen26 Sept 2026
Last checked1d ago

How to connect

As declared in the Official MCP Registry · hosted endpoint tested by Electra Index 9h ago (live check)

Hosted endpoint

  • https://www.pathwren.workers.dev/c/mcp-registry-official/mcp streamable-http

In Claude Code: claude mcp add --transport http ai-crawler-index https://www.pathwren.workers.dev/c/mcp-registry-official/mcp

MCP settings (Claude Desktop, Cursor and other MCP clients)

{
  "mcpServers": {
    "ai-crawler-index": {
      "url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp"
    }
  }
}

Live check

Electra Index connected to the hosted endpoint 9h ago

ONLINE AI Crawler Index answered an MCP connection on 17 Nov 2026 and introduced itself as ai-crawler-index 1.0.0 (MCP protocol 2025-06-18).

9 tools offered to AI assistants

  • whoami · {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp and you are done: free, read-only, no API key, no account, no consent prompt, nothing fo
  • example · {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp and you are done: free, read-only, no API key, no account, no consent prompt, nothing f
  • classify_user_agent · FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — Identify the crawler behind a raw User-Agent header: operator, category, robots.txt stance, how to verify it, and what blo
  • lookup_crawler · FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — The full index record for one crawler: operator, category, robots.txt token and stance, user-agent substring and example,
  • list_crawlers · FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — The index, filtered or whole: each row says what the crawler is and what blocking it costs, and the reply lists every vali
  • generate_robots_txt · FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — A ready-to-paste robots.txt block for one stance, with the reasoning and the crawlers it names. Omit the argument to get a
  • is_verified_crawler_ip · FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — Test an IPv4 or IPv6 address against every crawler prefix the operators themselves publish, remirrored every six hours. Re
  • whats_changed · FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — Freshness and change state: when the index was built, when each operator source was last fetched, which changed, which are
  • changes_since · FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — Only what changed since your last call: prefixes added or removed, upstreams failed or recovered, records edited. Send bac

What "online" means: the server answered an MCP connection at the time of the check. It is not a guarantee of security, quality or continuous uptime. How we test: we open an MCP session and ask for the tool list, as an AI assistant would. We never run a tool and never send a key. Checked every few days. Method

Is this your server? Monitoring Pro: a check every 30 minutes and an email within the hour when it stops answering. Launch offer: $39.99 once, for life.

Facts & evidence

Each fact shows where it was seen and when.
FactValueEvidenceSourceLast confirmed
Hosted endpointhttps://www.pathwren.workers.dev/c/mcp-registry-official/mcpDECLAREDSRC-001 Official MCP Registry1d ago
Local install (package)noDECLAREDSRC-001 Official MCP Registry1d ago
Remote access (hosted)yesDECLAREDSRC-001 Official MCP Registry1d ago
Websitehttps://www.pathwren.workers.dev/c/mcp-registry-official/mcp.htmlDECLAREDSRC-001 Official MCP Registry1d ago
Registry statusactiveDECLAREDSRC-001 Official MCP Registry1d ago
Version1.6.0DECLAREDSRC-001 Official MCP Registry1d ago
MCP serveryesDECLAREDSRC-001 Official MCP Registry1d ago

Change history

Baseline
No change detected yet. History builds up as the robots revisit this server.

Is this page accurate?

Flag it and our robot re-checks the official source. Reports never change a score.

Do you run AI Crawler Index? Know when it stops answering

For publishers · optional

Its hosted endpoint can be monitored. Monitoring Pro connects every 30 minutes like an AI assistant, emails you within the hour when it stops answering, keeps a 90-day availability history and adds your answer rate to your badge.

Monitor AI Crawler IndexLaunch offer $39.99 once, for life instead of $9.99/month · All plans

Monitoring never changes a score or a rank.

Are you the publisher of AI Crawler Index?

Free · show your listing

An independent listing builds trust with your buyers. Announce it, and add the badge to your website or README.

Announce it on LinkedIn
Listed on Electra IndexListed on Electra Index
<a href="https://electraindex.com/mcp/ai-crawler-index?utm_source=badge&utm_medium=share&utm_campaign=listing"><img src="https://electraindex.com/api/badge/ai-crawler-index/card" width="260" height="72" alt="Listed on Electra Index"></a>

Questions about AI Crawler Index

How do I connect AI Crawler Index?

Use the hosted endpoint https://www.pathwren.workers.dev/c/mcp-registry-official/mcp in your assistant's MCP settings. The exact settings are in "How to connect" above, as declared in the Official MCP Registry.

What tools does AI Crawler Index offer?

When Electra Index connected on 17 Nov 2026, it listed 9 tools: whoami, example, classify_user_agent, lookup_crawler, list_crawlers, generate_robots_txt, is_verified_crawler_ip, whats_changed and more. The full list is in "Live check" above.

Is AI Crawler Index online?

Yes. Its hosted endpoint answered an MCP connection from Electra Index 9h ago.

What can I use AI Crawler Index for?

Based on its description, it is most relevant to: collect data behind logins, extract website data, monitor prices, audit website SEO.