Amazon Made 600M Products Invisible to ChatGPT. Your Cloudflare Might Be Doing the Same Thing.

Since July 1, 2025, Cloudflare blocks AI crawlers by default on every site. If you haven't touched your bot settings since then, here's the blunt version: ChatGPT can't see your products. Neither can Perplexity. Neither can Claude. You did nothing wrong and you're still invisible.

And you're not alone - sites blocking AI crawlers now outnumber those blocking Googlebot by 7 to 1 (Cloudflare). Most of them have no idea it's happening to them.

Stay in the loop

Get news and updates about GEO, AI search and new features. Unsubscribe anytime.

Email address Subscribe Also create my free GEOlikeaPro account - run an AI-visibility audit on your site

Check yours right now - takes 30 seconds

1. Cloudflare Dashboard → select your domain
2. Security → Bots
3. Look for the "AI Crawlers" toggle
4. If it says Block - that's the default, and your store is invisible to AI search right now

Checked it? Good. Now let me explain why this matters and exactly what to do about it.

The numbers that should worry you

Source: Cloudflare crawl-to-referral data and industry breakdown

Look at the split. PerplexityBot at 88:1 and OAI-SearchBot at 401:1 are search crawlers - they send real customers to your store. GPTBot at 1,700:1 and ClaudeBot at 73,000:1 are training crawlers - they take your content and give you almost nothing back.

The default Cloudflare block kills both without asking. That's the mistake. You want to block one and allow the other, deliberately.

The setup I'd run for e-commerce (copy-paste this)

Allow (these send you customers):

• OAI-SearchBot - ChatGPT Search & Shopping
• ChatGPT-User - when users ask ChatGPT to browse your site
• PerplexityBot - Perplexity citations
• ClaudeBot - Claude web search (now on all plans)
• Google-Extended - Google AI Overviews & Gemini

Block (these take your content for training):

• GPTBot - OpenAI model training
• CCBot - Common Crawl
• anthropic-ai - Anthropic model training
• Bytespider - ByteDance

That keeps you visible in ChatGPT Shopping (50M queries/day), Perplexity, Claude, and AI Overviews - channels that convert 1.3x to 6x higher than non-branded organic search - while still blocking the 80% of AI crawl traffic that's pure training and returns zero referral value. You give up nothing that pays you and keep everything that does.

"But what about bandwidth costs?"

Fair question, so here's the honest answer. When Read the Docs blocked all AI crawlers, their bandwidth dropped 75% - 800 GB/day down to 200 GB/day - saving $1,500/month (Search Engine Journal).

But Read the Docs is a documentation site. They don't sell products through AI search. You do. The selective setup above blocks the bandwidth-heavy training crawlers and keeps the search crawlers that actually drive revenue - you get most of the savings without amputating the channel.

One more thing: Perplexity cheats

Block PerplexityBot in robots.txt and Perplexity may still read your content anyway. Cloudflare caught them running stealth, undeclared crawlers that rotate IPs and spoof real-browser user agents (Cloudflare investigation), and delisted them as a verified bot over it.

So if you genuinely need Perplexity out, robots.txt won't get you there. Use Cloudflare's WAF rules or AI Crawl Control - anything less is theater.

---

Do this right now

Step 1: Open Cloudflare → Security → Bots. Screenshot your current settings before you change anything.

Step 2: Switch from blanket block to selective - allow the search crawlers, block the training ones.

Step 3: Check back in 2 weeks. If you're on GEOlikeaPro, run an SOV audit before and after - you'll see the move land in your AI visibility scores.

See where you stand - free to run.