# GoCrawl (gocrawl.tech) > High-speed Technical SEO and AI Search (GEO) crawler, interactive website grader, MCP server, and agency deliverable suite for macOS, Windows, Linux, and the web. ## Core Platform - [Instant Website & AI Search Grader](https://gocrawl.tech/): Audit any domain across 6 technical SEO and AI Search dimensions - [Interactive SEO & AI Search Questions Hub](https://gocrawl.tech/q): 20 live diagnostic question pages with instant domain checks - [Free SEO & AI Search Micro-Tools](https://gocrawl.tech/tools): AI Bot Parity Checker, 3-File /llms.txt Generator, Keyword Alignment Grader, and Redirect Tracer - [Full Knowledge Base (/llms-full.txt)](https://gocrawl.tech/llms-full.txt): Single-file technical reference and 20-question answer digest ## Interactive Diagnostic Questions - [Is ChatGPT Blocked From Crawling My Website?](https://gocrawl.tech/q/is-chatgpt-blocked-from-my-website): Test live if OpenAI's GPTBot and OAI-SearchBot can crawl your website or if Cloudflare, WAF rules, or robots.txt are blocking ChatGPT Search citations. - [Why Is Perplexity or Claude Not Citing My Website?](https://gocrawl.tech/q/why-is-perplexity-not-citing-my-website): Diagnose why PerplexityBot and ClaudeBot fail to cite your domain. Check AI crawler parity, /llms.txt presence, and structured entity grounding. - [Is Cloudflare or My Firewall Blocking AI Search Bots?](https://gocrawl.tech/q/is-cloudflare-blocking-ai-bots): Run a simultaneous multi-agent probe across Googlebot, GPTBot, OAI-SearchBot, ClaudeBot, and PerplexityBot to detect WAF and CDN bot blocking. - [What Should My robots.txt Look Like for AI Search Engines?](https://gocrawl.tech/q/robots-txt-for-ai-search-engines): Audit your live robots.txt for AI search crawler access and generate a clean, spec-compliant robots.txt for Googlebot, OAI-SearchBot, GPTBot, ClaudeBot, and PerplexityBot. - [Does My Website Have a Valid /llms.txt File?](https://gocrawl.tech/q/does-my-website-have-llms-txt): Check if your domain serves a valid /llms.txt file for AI search agents, verify markdown section formatting, and test linked URLs for 200 OK status. - [How Do I Generate an /llms.txt File for My Website?](https://gocrawl.tech/q/how-to-generate-llms-txt): Automatically crawl your top pages and generate a 3-file AI grounding pack: /llms.txt, /llms-full.txt, and a zero-JS semantic entity HTML page. - [Does My Website Have Valid JSON-LD Schema for AI Search?](https://gocrawl.tech/q/does-my-website-have-json-ld-schema): Inspect your page for structured JSON-LD schema.org entities (Organization, WebSite, SoftwareApplication, FAQPage) and generate missing structured data. - [Can AI Crawlers Read My React, Next.js, or SPA Website Without JavaScript?](https://gocrawl.tech/q/can-ai-crawlers-read-my-javascript-site): Test what non-JS AI crawlers (GPTBot, ClaudeBot, PerplexityBot) actually see in your raw server HTML response versus client-side JavaScript. - [Will My Page Title or Meta Description Get Cut Off in Google Search?](https://gocrawl.tech/q/will-my-title-get-cut-off-in-google): Measure the exact pixel width and character length of your live and <meta name="description"> tags and preview your Google SERP snippet. - [Are My Page Keywords Aligned With My Title, H1, and Meta Description?](https://gocrawl.tech/q/are-my-keywords-aligned-with-title-and-h1): Analyze 1-word, 2-word, and 3-word N-gram keyword frequency on your page and verify semantic alignment across Title, H1, H2, and Meta Description. - [Why Is Google Ignoring or Rewriting My Page Title?](https://gocrawl.tech/q/why-is-google-rewriting-my-page-title): Check the 5 technical triggers that cause Google to rewrite your <title> tag in search results: pixel overflow, H1 mismatch, keyword stuffing, or boilerplate. - [Is My Page Indexable by Google and AI Search Engines?](https://gocrawl.tech/q/is-my-page-indexable): Verify HTTP status code, meta robots directives, X-Robots-Tag headers, and self-referencing canonical tags to confirm whether your URL can be indexed. - [Does My URL Have a Redirect Chain or Redirect Loop?](https://gocrawl.tech/q/does-my-url-have-a-redirect-chain): Trace every HTTP 301, 302, 307, and 308 redirect hop from source to destination, measure per-hop latency, and generate a flattened 1-hop server rule. - [Does My http:// and www Redirect in a Single Hop to https://?](https://gocrawl.tech/q/check-http-to-https-www-redirects): Test all 4 canonical domain variants (http://, http://www., https://, https://www.) to ensure 3 variants redirect cleanly in 1 hop to your canonical origin. - [Does My Website Have Broken Links or Crawl Errors?](https://gocrawl.tech/q/does-my-page-have-broken-links): Run a live multi-page crawl on your domain to detect 404 broken links, 5xx server errors, redirect chains, and orphan pages. - [Is My sitemap.xml Valid and Free of Redirecting or Noindex URLs?](https://gocrawl.tech/q/is-my-xml-sitemap-clean): Check your domain's indexability health and learn how to purge 3xx redirects, 404 errors, and noindex pages from your XML sitemap. - [How Does My Website's SEO and AI Readiness Compare to My Competitor?](https://gocrawl.tech/q/compare-website-seo-and-ai-readiness): Run a live head-to-head Technical SEO and AI Search Readiness comparison between your domain and any competitor domain. - [How Do I Map 301 Redirects When Redesigning or Migrating a Website?](https://gocrawl.tech/q/how-to-map-301-redirects-for-site-migration): Audit your domain before a site migration and generate ready-to-deploy 301 redirect rules for Nginx, Caddyfile, Apache .htaccess, and Cloudflare. - [What Is My Website's Technical SEO and AI Search Health Score?](https://gocrawl.tech/q/website-seo-and-ai-search-score): Calculate your website's 0-100 executive health score and letter grade across Crawlability, Indexability, On-Page SEO, Link Graph, Speed, and AI Search Readiness. - [How Do I Audit a Client or Prospect Website for an SEO Agency Pitch?](https://gocrawl.tech/q/how-to-audit-prospect-website-for-seo-pitch): Generate an executive Technical SEO & AI Search teardown report for any prospect domain to win agency retainers with real crawl data.