List your agent

Crawl Census

MCP server● Hosted · onlinev1.0.0

Ask before you fetch: will this domain serve your crawler, refuse it, or charge it?

What you can do with it

Share of each task's required capabilities it covers

Prefer a ready-made agent?

An MCP server is a connector for your own assistant. These finished products do the job for you: collect data behind logins.
All AI agents to collect data behind logins, ranked →

Similar tools

Listed for the same task: collect data behind logins

Reviews

No review yet. Used Crawl Census? Be the first to help others choose.

What it does

Crawl Census is an MCP server, a connector that lets an AI assistant use it as a tool. It is hosted: an assistant connects to it over the internet, nothing to install.

From the publisher's description, Electra Index matched it to one capability: web scraping. That makes it a candidate for tasks such as collect data behind logins or extract website data.

Facts come from the Official MCP Registry, as declared by the publisher. Capabilities are matched automatically (low confidence). None of this is a test of the tool.

At a glance

Hosted (remote)YES
Local installNO
Version1.0.0
PriceNot in the registry
First seen26 Sept 2026
Last checked1d ago

How to connect

As declared in the Official MCP Registry · hosted endpoint tested by Electra Index 2h ago (live check)

Hosted endpoint

  • https://crawlcensus.com/mcp streamable-http

In Claude Code: claude mcp add --transport http crawl-census https://crawlcensus.com/mcp

MCP settings (Claude Desktop, Cursor and other MCP clients)

{
  "mcpServers": {
    "crawl-census": {
      "url": "https://crawlcensus.com/mcp"
    }
  }
}

Live check

Electra Index connected to the hosted endpoint 2h ago

ONLINE Crawl Census answered an MCP connection on 17 Nov 2026 and introduced itself as crawl-census 1.0.0 (MCP protocol 2025-06-18).

7 tools offered to AI assistants

  • scan_site · Run a live AI-accessibility audit of a domain: robots.txt policy for every tracked AI crawler, live user-agent probes, JavaScript-free readability, structured data and llms.txt. Returns a score out of 100 with per-check detail.
  • site_report · Return the most recent stored audit for a domain without triggering a new scan. Faster and free of load on the target site.
  • census_stats · Corpus-level statistics: how many measured domains block each AI crawler, mean access score, llms.txt adoption.
  • crawl_preflight · Decide whether a crawler may fetch a list of domains before spending requests on them. Works for any crawler token, not only the ones this census tracks: an unrecognised agent is resolved from each domain's stored robots.txt rather than ref
  • agent_profile · What this census measures and publishes about one AI crawler: how often it is disallowed in robots.txt, how often live requests carrying its user agent are refused at the network edge whatever robots.txt says, whether its operator documents
  • census_facts · Every headline finding from the census as discrete, dated records rather than prose. Each carries its value, unit, denominator, measurement date, the page it comes from and a ready-made citation line, plus the caveats that apply to all of t
  • submit_domains · Queue domains the census has not measured yet so a later crawl_preflight can answer them. This closes the loop crawl_preflight starts: anything it returns as unknown with measurable true is worth submitting, and the reply names any that wer

What "online" means: the server answered an MCP connection at the time of the check. It is not a guarantee of security, quality or continuous uptime. How we test: we open an MCP session and ask for the tool list, as an AI assistant would. We never run a tool and never send a key. Checked every few days. Method

Is this your server? Monitoring Pro: a check every 30 minutes and an email within the hour when it stops answering. Launch offer: $39.99 once, for life.

Facts & evidence

Each fact shows where it was seen and when.
FactValueEvidenceSourceLast confirmed
Hosted endpointhttps://crawlcensus.com/mcpDECLAREDSRC-001 Official MCP Registry1d ago
Local install (package)noDECLAREDSRC-001 Official MCP Registry1d ago
Remote access (hosted)yesDECLAREDSRC-001 Official MCP Registry1d ago
Websitehttps://crawlcensus.com/for-crawlersDECLAREDSRC-001 Official MCP Registry1d ago
Repositoryhttps://github.com/taylorsmithgg/crawl-census-clientDECLAREDSRC-001 Official MCP Registry1d ago
Registry statusactiveDECLAREDSRC-001 Official MCP Registry1d ago
Version1.0.0DECLAREDSRC-001 Official MCP Registry1d ago
MCP serveryesDECLAREDSRC-001 Official MCP Registry1d ago

Change history

Baseline
No change detected yet. History builds up as the robots revisit this server.

Is this page accurate?

Flag it and our robot re-checks the official source. Reports never change a score.

Do you run Crawl Census? Know when it stops answering

For publishers · optional

Its hosted endpoint can be monitored. Monitoring Pro connects every 30 minutes like an AI assistant, emails you within the hour when it stops answering, keeps a 90-day availability history and adds your answer rate to your badge.

Monitor Crawl CensusLaunch offer $39.99 once, for life instead of $9.99/month · All plans

Monitoring never changes a score or a rank.

Are you the publisher of Crawl Census?

Free · show your listing

An independent listing builds trust with your buyers. Announce it, and add the badge to your website or README.

Announce it on LinkedIn
Listed on Electra IndexListed on Electra Index
<a href="https://electraindex.com/mcp/crawl-census?utm_source=badge&utm_medium=share&utm_campaign=listing"><img src="https://electraindex.com/api/badge/crawl-census/card" width="260" height="72" alt="Listed on Electra Index"></a>

Questions about Crawl Census

How do I connect Crawl Census?

Use the hosted endpoint https://crawlcensus.com/mcp in your assistant's MCP settings. The exact settings are in "How to connect" above, as declared in the Official MCP Registry.

What tools does Crawl Census offer?

When Electra Index connected on 17 Nov 2026, it listed 7 tools: scan_site, site_report, census_stats, crawl_preflight, agent_profile, census_facts, submit_domains. The full list is in "Live check" above.

Is Crawl Census online?

Yes. Its hosted endpoint answered an MCP connection from Electra Index 2h ago.

What can I use Crawl Census for?

Based on its description, it is most relevant to: collect data behind logins, extract website data, monitor prices, audit website SEO.