Desktop App, CLI & MCP Server Guide

CiteScope Desktop (Windows, macOS, Linux)

The desktop application runs crawls directly from your machine without cloud URL limits. Every crawl is saved locally in an embedded SQLite WAL database so you can compare crawls over time, query them from the CLI, or inspect them in Cursor and Claude Code.

Installation & Local Database Location

Configuring Crawl Scope & JavaScript Rendering

• Polite Rate Limiting: By default, CiteScope crawls at 10–15 requests/sec per host with automatic robots.txt compliance and Retry-After backoff so production servers and CDNs are not overloaded.
• Raw Server HTML Mode (Default): Parses the exact initial HTTP response that non-JavaScript crawlers (such as GPTBot, ClaudeBot, and PerplexityBot) receive.
• Smart Hybrid JS Rendering (Headless Chromium): Renders client-side React, Next.js, Vue, or Angular pages in headless Chromium and diffs the hydrated DOM against raw HTML to flag client-only links, missing SSR text, and canonical tag mutations.
• Staging Basic Auth: Supply optional HTTP Basic Auth credentials to audit password-protected staging environments before launch.

Exports & Deliverables

• 3-File AI Grounding Pack: Generates /llms.txt (concise Markdown directory), /llms-full.txt (complete page digest), and product-overview.html (plain HTML with JSON-LD schema) ranked by internal PageRank.
• Clean XML Sitemap: Exports a spec-compliant sitemap.xml containing only 200 OK, self-canonical, indexable pages.
• 301 Migration Redirect Mapper: Matches an old crawl against a staging/redesign crawl using path similarity, content hash, and 64-bit SimHash to generate ready-to-deploy nginx.conf, vercel.json, or Cloudflare _redirects rules.

Automated Audits with citescope ci

The same binary runs headless in your terminal or CI/CD pipeline. It crawls a target URL, prints a structured Markdown scorecard to standard output, and optionally exits with code 1 if critical SEO or AI search regressions are found.

Basic CLI Usage

./CiteScope-Linux-amd64 ci https://staging.example.com --max-urls 1000 --rps 15 --workers 10 --fail-on-critical

CLI Subcommands & Flags

GitHub Actions Workflow Example

Add .github/workflows/citescope-audit.yml to your repository to audit staging deployments on every pull request:

name: Technical SEO & AI Search Gate
on:
  pull_request:
    branches: [main]

jobs:
  citescope-audit:
    runs-on: ubuntu-latest
    steps:
      - name: Download CiteScope CLI
        run: |
          curl -fsSL https://citescope.app/downloads/CiteScope-Linux-amd64 -o citescope
          chmod +x citescope

      - name: Run Crawl & Fail on Critical Regressions
        run: |
          ./citescope ci https://staging.example.com --max-urls 500 --rps 15 --fail-on-critical > audit-report.md
          cat audit-report.md >> $GITHUB_STEP_SUMMARY

Connecting Cursor, Claude Code & Windsurf

CiteScope includes a native Model Context Protocol (MCP) server exposing 15 tools. Your AI coding assistant can start crawls, inspect page metadata, check AI bot firewall parity, generate /llms.txt files, and verify fixes directly from your editor.

Option A: Local Stdio MCP Server (citescope mcp)

Add the following block to .cursor/mcp.json in your project root or ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "citescope": {
      "command": "/usr/local/bin/citescope",
      "args": ["mcp"]
    }
  }
}

Option B: Zero-Install Cloud HTTP Endpoint (/api/mcp)

You can also query the hosted JSON-RPC 2.0 MCP endpoint at https://citescope.app/api/mcp (or inspect the tool manifest at https://citescope.app/api/mcp/tools):

curl -X POST https://citescope.app/api/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_llm_bot_parity","arguments":{"url":"https://vercel.com"}}}'

All 15 Available MCP Tools

Example Prompts in Cursor or Claude Code

• "Use CiteScope to crawl https://staging.example.com (up to 100 URLs), list all pages missing og:image or meta descriptions, and update the corresponding route metadata files in this repo."
• "Run check_llm_bot_parity and audit_llms_txt on our production domain, then generate a clean /public/llms.txt file using generate_ai_grounding_pack."

36 Extracted Per-Page Telemetry Fields

Every crawled page is parsed in a single streaming pass and stored in crawl_nodes (available in the Page Analytics Dashboard and 36-column CSV export):

Core Audit Rules & How to Fix Them