List your agent

AI Crawler Permission Check

MCP server● Hosted · onlinev0.1.0

Check a site's robots.txt rules for AI crawlers (GPTBot, ClaudeBot, PerplexityBot and more)

What you can do with it

Share of each task's required capabilities it covers

Prefer a ready-made agent?

An MCP server is a connector for your own assistant. These finished products do the job for you: collect data behind logins.
All AI agents to collect data behind logins, ranked →

Similar tools

Listed for the same task: collect data behind logins

Reviews

No review yet. Used AI Crawler Permission Check? Be the first to help others choose.

What it does

AI Crawler Permission Check is an MCP server, a connector that lets an AI assistant use it as a tool. It is hosted: an assistant connects to it over the internet, nothing to install.

From the publisher's description, Electra Index matched it to one capability: web scraping. That makes it a candidate for tasks such as collect data behind logins or extract website data.

Facts come from the Official MCP Registry, as declared by the publisher. Capabilities are matched automatically (low confidence). None of this is a test of the tool.

At a glance

Hosted (remote)YES
Local installNO
Version0.1.0
PriceNot in the registry
First seen21 Nov 2026
Last checked3h ago

How to connect

As declared in the Official MCP Registry · hosted endpoint tested by Electra Index 3h ago (live check)

Hosted endpoint

  • https://opencurrent.ai/mcp/robots streamable-http

In Claude Code: claude mcp add --transport http ai-crawler-permission-check https://opencurrent.ai/mcp/robots

MCP settings (Claude Desktop, Cursor and other MCP clients)

{
  "mcpServers": {
    "ai-crawler-permission-check": {
      "url": "https://opencurrent.ai/mcp/robots"
    }
  }
}

Reliability

Re-tested about every 3 days · last check 3h ago
Insufficient observations1 conclusive check in 30 days; a figure is shown from 10
1 / 0Answers / failures, 30 days
No key neededAccess status (latest answer)
Observed test history · one mark = one check
21 Nov21 Nov
AnsweredAnswered, key requiredNo valid answer

Observed endpoint availability: the share of our conclusive checks that got an answer. A request for a key or sign-in counts as an answer (the server is up, its access is protected). Timeouts, unreachable endpoints, server errors and invalid MCP replies count as failures; rate-limited checks and checks during our own incidents are not counted. This is what we observed at each check, not continuous uptime: an outage between two checks may go unseen. The 90-day figure will appear once 90 days of checks exist (checks since 21 Nov 2026). How we measure

Live check

Electra Index connected to the hosted endpoint 3h ago

ONLINE AI Crawler Permission Check answered an MCP connection on 21 Nov 2026 and introduced itself as robots-ai-check 0.1.0 (MCP protocol 2025-06-18).

2 tools offered to AI assistants

  • check_ai_crawlers · For a domain (and optional path), whether each known AI crawler/agent user-agent is allowed by robots.txt, with the deciding rule.
  • check_user_agent · Whether a given crawler user-agent token may fetch a path on a domain according to robots.txt.

What "online" means: the server answered an MCP connection at the time of the check. It is not a guarantee of security, quality or continuous uptime. How we test: we open an MCP session and ask for the tool list, as an AI assistant would. We never run a tool and never send a key. Re-tested about every 3 days. Method

Is this your server? Monitoring Pro: a check every 30 minutes and an email within the hour when it stops answering. Launch offer: $39.99 once, for life.

Facts & evidence

Each fact shows where it was seen and when.
FactValueEvidenceSourceLast confirmed
Hosted endpointhttps://opencurrent.ai/mcp/robotsDECLAREDSRC-001 Official MCP Registry3h ago
Local install (package)noDECLAREDSRC-001 Official MCP Registry3h ago
Remote access (hosted)yesDECLAREDSRC-001 Official MCP Registry3h ago
Websitehttps://opencurrent.ai/mcp/robotsDECLAREDSRC-001 Official MCP Registry3h ago
Registry statusactiveDECLAREDSRC-001 Official MCP Registry3h ago
Version0.1.0DECLAREDSRC-001 Official MCP Registry3h ago
MCP serveryesDECLAREDSRC-001 Official MCP Registry3h ago

Change history

  • NEW21 Nov 2026
    first seen in the index

Is this page accurate?

Flag it and our robot re-checks the official source. Reports never change a score.

Do you run AI Crawler Permission Check? Know when it stops answering

For publishers · optional

Its hosted endpoint can be monitored. Monitoring Pro connects every 30 minutes like an AI assistant, emails you within the hour when it stops answering, keeps a 90-day availability history and adds your answer rate to your badge.

Monitor AI Crawler Permission CheckLaunch offer $39.99 once, for life instead of $9.99/month · All plans

Monitoring never changes a score or a rank.

Are you the publisher of AI Crawler Permission Check?

Free · show your listing

An independent listing builds trust with your buyers. Announce it, and add the badge to your website or README.

Announce it on LinkedIn
Listed on Electra IndexListed on Electra Index
<a href="https://electraindex.com/mcp/ai-crawler-permission-check?utm_source=badge&utm_medium=share&utm_campaign=listing"><img src="https://electraindex.com/api/badge/ai-crawler-permission-check/card" width="260" height="72" alt="Listed on Electra Index"></a>

Questions about AI Crawler Permission Check

How do I connect AI Crawler Permission Check?

Use the hosted endpoint https://opencurrent.ai/mcp/robots in your assistant's MCP settings. The exact settings are in "How to connect" above, as declared in the Official MCP Registry.

What tools does AI Crawler Permission Check offer?

When Electra Index connected on 21 Nov 2026, it listed 2 tools: check_ai_crawlers, check_user_agent. The full list is in "Live check" above.

Is AI Crawler Permission Check online?

Yes. Its hosted endpoint answered an MCP connection from Electra Index 3h ago.

What can I use AI Crawler Permission Check for?

Based on its description, it is most relevant to: collect data behind logins, extract website data, monitor prices, audit website SEO.