Can AI search engines read your site?
Paste a URL. The check tests 13 AI crawlers from OpenAI, Anthropic, Perplexity, Google, Apple, Meta and others against your robots.txt and your server, then checks what they can actually read. No email, no signup.
AI bots
Site checks
Access is the precondition, not the result. Whether assistants actually mention and cite you is a content and authority question.
- 13
- AI bots tested
- ~15 s
- to a verdict
- €0
- free, no email gate
Access first. Then everything else.
AI search can only cite pages its crawlers can fetch and read. These checks cover the technical part, run server-side against the live URL.
- 01
robots.txt, per AI bot
Each bot is matched against its own group in robots.txt (or the * group), with longest-match rules, for this URL and for the homepage.
- 02
Server and CDN blocking
We request the page as a normal browser and as each AI crawler. If the browser gets 200 and a bot gets 403 or 429, a firewall or bot-protection rule is blocking it, whatever robots.txt says.
- 03
Search vs. training bots
Search and user-triggered bots decide whether AI assistants can read and cite you. Training bots only decide whether your content trains future models.
- 04
Content in raw HTML
Most AI crawlers don’t run JavaScript. If the text only appears after scripts run, they see an empty page.
- 05
Structured data
JSON-LD tells machines who you are, who wrote what and which entity a page is about.
- 06
llms.txt and directives
Whether /llms.txt exists, and whether the page sends noindex, noai or noimageai.
Questions about AI crawlers
What’s the difference between training bots and search bots?
Training bots (GPTBot, ClaudeBot, CCBot, Google-Extended and others) collect content to train future models. Search and user bots (OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User) fetch pages to answer questions right now, often with a link to the source. Blocking training bots is a legitimate business decision. Blocking search bots means AI assistants can’t read or cite you.
If I block GPTBot, do I disappear from ChatGPT search?
No. OpenAI uses separate crawlers: GPTBot for training and OAI-SearchBot for ChatGPT search. OpenAI’s documentation says sites that opt out of OAI-SearchBot won’t be shown in ChatGPT search answers, while GPTBot only controls training use (OpenAI crawler docs).
Does llms.txt matter?
It’s an optional proposal: a plain-text file that points AI tools to your most important content. Some tools read it; Google has said it doesn’t use it for Search. It’s worth a few minutes on documentation-heavy sites, but it won’t fix a site AI crawlers can’t reach.
How can a CDN block AI bots without me knowing?
Many CDNs and hosts ship bot protection that blocks AI crawlers by default or with one switch, often returning 403 to anything that identifies as an AI bot. That happens before robots.txt is ever read, which is why this check also requests the page with each bot’s user agent.
Everything is green. Will I show up in ChatGPT and Perplexity now?
Not automatically. Access is the precondition, not the outcome. Whether assistants actually mention and cite you depends on your content, your authority on the topic and how often you’re referenced elsewhere. Measuring that is what the AI Search Visibility Audit does.
They can read you. Do they recommend you?
The AI Search Visibility Audit tests the questions your buyers actually ask in ChatGPT, Perplexity, Gemini and Google's AI Overviews, measures how often you're mentioned and cited against your competitors, and shows what to change.
You talk to the person who runs the audit. Replies within 24h.