AI crawler access report

Can AI crawlers reach aicost.tools?

AI crawler access report · checked 2026-11-11 · log in to re-run
100%
aicost.tools · Cloudflare
aicost.tools lets all 8 tracked AI crawlers through.
Open to AI crawlersCloudflare8/8 bots allowed
AI crawlers allowed100%
GPTBot
OAI-SearchBot
ChatGPT-User
ClaudeBot
Claude-User
PerplexityBot
Google-Extended
Applebot-Extended

Per-bot results

GPTBotrobots.txt allows this bot, no challenge detectedAllowed
OAI-SearchBotrobots.txt allows this bot, no challenge detectedAllowed
ChatGPT-Userrobots.txt allows this bot, no challenge detectedAllowed
ClaudeBotrobots.txt allows this bot, no challenge detectedAllowed
Claude-Userrobots.txt allows this bot, no challenge detectedAllowed
PerplexityBotrobots.txt allows this bot, no challenge detectedAllowed
Google-Extendedrobots.txt allows this bot, no challenge detectedAllowed
Applebot-Extendedrobots.txt allows this bot, no challenge detectedAllowed

Origin fingerprint

ASN: AS13335 (CLOUDFLARENET - Cloudflare, Inc., US)
Bot protection: Cloudflare (passive)

What this means for aicost.tools

On aicost.tools, no AI crawlers are blocked, and 8 can reach the site - GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot, Google-Extended, Applebot-Extended. The origin resolves to Cloudflare.

aicost.tools scores higher than 24% of the 25 sites checked here so far; the average AI access score is 90/100.

Keep aicost.tools open to AI

Nothing to unblock - every tracked AI crawler can reach aicost.tools. Re-check after any CDN, WAF or robots.txt change, since AI-bot blocking is often switched on by default after a security tweak.

AI crawler access FAQ

Does AI crawler access guarantee aicost.tools gets cited?

No. Access is necessary, not sufficient - it removes the blocker. Whether aicost.tools gets cited still depends on content, structured data and authority.

How is this measured?

We parse aicost.tools's public robots.txt and fingerprint its origin (ASN and bot-protection vendor), then score the share of 8 tracked AI crawlers that can reach it. No private data or IP addresses are shown.

Check your own site free

Run the same per-bot check on any URL, then sign up free to have ChatGPT Search and Perplexity actually fetch it - the real-world test - plus monitoring and exports.

Run a free check

AI crawler access on Cloudflare

We have checked 14 other sites on Cloudflare. The Cloudflare hub aggregates per-bot block rates and average access scores across every Cloudflare site checked here.

Does Cloudflare block AI crawlers?