# How the AI visibility scan works

Methodology
Every score in the scan comes from real evidence: what the assistants behind ChatGPT, Claude, Perplexity, Gemini and Copilot can actually reach and read on your site. No model is involved in the judgement — it's a crawl, then a checklist.

## The scan in one paragraph

Your site is contacted about a dozen times. The scanner reads your homepage, robots.txt, llms.txt, sitemap.xml and a sample of internal pages the way an agent would, then runs each finding against a fixed checklist: can every AI crawler get in? Is there a real 404? Does the site answer an `Accept: text/markdown` request with markdown instead of HTML? Is your identity declared in structured data? Each check scores points, and the points add up to one number.

## The three layers

Checks are organised the way a fix list should be: what matters most at the top.

  - **Essential — up to 80 points.** Can an agent reach you, read you without JavaScript, follow your redirects, and find a recovery path when it hits a dead page. If these fail, nothing else matters.

  - **Recommended — up to 20 points.** The things that decide whether an agent _trusts_ you enough to name you: sitemap freshness, structured data for your entity, llms.txt quality, metadata found on the pages themselves.

  - **Bonus signals — up to 5 points, positive only.** Nice-to-haves like an existing llms.txt or a canonical URL. Bonus signals are never required: their absence can never lower your score.

## Every finding carries a fix you can copy

Each check shows three things: the **evidence** (the raw fact, like a status code or a character count), a **recommended fix** in plain English, and a **copyable fix prompt** you can paste straight into your developer workflow. The "Prompt to improve" button above the score bundles every failed and partial finding into one implementation brief, in priority order.

## The journey tree

Below the score you can see the scan's actual path: every fetch, its status, and the time it took. A half-empty journey with dead nodes is itself a finding — it shows exactly where an agent would have lost its way.

## The crawler check

robots.txt is evaluated with longest-match semantics for 12 user agents, including GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended, Bingbot and Applebot-Extended. A single blank line in the wrong place can open or shut off an entire assistant product, so the scan reports exactly which agents are allowed or blocked.

## Markdown negotiation

Agents increasingly ask for `text/markdown` when they fetch a page — it is cheaper to parse and closer to what a model consumes. The scanner checks whether your server serves a real markdown variant (with the right `Content-Type` and `Vary: Accept`), not just whether the page happens to be text.

## What we don't do

  - We do not execute JavaScript or take screenshots — the scan only sees what a raw HTTP client sees, which is what matters for agentic reachability.

  - We do not attempt secret or private areas of your site, and we never submit your URL to any AI model.

  - Free scans are rate-limited per IP so the scanner is never a nuisance to the site it is reading.

## What the number is not

The score is a snapshot of technical readiness for agentic discovery — not a rank, not a backlink or SEO score, and not a guarantee of citations. It tells you where a fix list should start.

HTML version: https://websitespa.net/methodology