Why This Matters for Search & AI Visibility
AI crawlers do not execute interactive CAPTCHA or browser Turnstile challenges. A 403 response at the CDN edge makes your entire domain invisible to generative search engines.
Is Cloudflare or My Firewall Blocking AI Search Bots?
Many CDN and WAF configurations—including Cloudflare Bot Fight Mode—treat AI search crawlers as unverified scrapers, serving JS challenges or HTTP 403 Forbidden responses to GPTBot, ClaudeBot, and PerplexityBot even when Googlebot passes cleanly.
Live Domain Diagnostic
GoCrawl Technical Advisory
How to Fix It Permanently
Create an explicit WAF Skip / Allow rule for verified AI search user-agents (OAI-SearchBot, GPTBot, ClaudeBot, PerplexityBot) and verify parity across all five user-agents.