Marketing · 16 Oct 2026 · 19 min
How to get cited by ChatGPT, Perplexity and AI Overviews: what decides it, and what you can control
Every AI assistant that cites sources is a search engine with a language model on top. To be cited, a page has to be crawled by the right bot, sit in the right index, rank for one of the searches behind the question, and hold a passage worth quoting. What each company says, what the studies show, and what 229 sites' robots.txt files get right.
Every few weeks there's a new name for it. GEO, AEO, LLM SEO, AI search optimisation. Each comes with a list of tricks: special files, special markup, content cut into tiny chunks so a model can digest it.
Google's own guide to AI search, published in 2026, says to ignore most of that. Its advice is to prioritise effective SEO strategies over "AEO/GEO hacks", and it names chunking, llms.txt and inauthentic mentions as tactics you can ignore for Google Search.
That's the right place to start, because of what these products are. ChatGPT search, Perplexity, Google's AI Overviews and AI Mode, Copilot and Claude all work the same way underneath: a search engine finds pages, and a language model writes an answer from them and cites some. To be cited, a page has to get through four steps.
-
1
CrawledThe assistant's search crawler can fetch the page
Stopped by robots.txt blocking the search bot, a firewall that blocks bots, content that needs a login
-
2
IndexedThe page is in the index the assistant searches
Stopped by noindex, nosnippet, a page the index hasn't reached yet
-
3
RetrievedThe page ranks for one of the searches the assistant runs to answer the question
Stopped by a stronger page on the same point
-
4
CitedA passage on the page supports a claim in the answer
Stopped by an answer that's buried, vague, undated or unsourced
- of AI Overview citations rank in Google's top 10
- 38%
- of 229 sites block an AI search crawler
- 0
- of llms.txt files got zero requests in a month
- 97%
- of searches with an AI summary led to a click on it
- 1%
The short version
- Let the search crawlers in. Each AI company runs one crawler for search and another for training. Block training if you like. Block search and you can't be cited.
- Be in the index they search. Google's for AI Overviews and AI Mode, Bing's for Copilot, each company's own for ChatGPT, Perplexity and Claude.
- Rank for the questions behind the question. The assistant runs several searches to answer one question, and cites pages from all of them.
- Hold a passage worth quoting. A clear answer, a specific number, a date, a source.
- Be mentioned elsewhere. In one large study, brand mentions across the web tracked AI visibility more closely than links did.
- Build the page to be quoted. The answer in the first two lines, headings as questions, exact facts with sources, something only you have. There's a ten-point list below, in plain words.
- Skip the hacks. No AI company says it reads llms.txt, and Google says AI Overviews need no special markup.
How each assistant finds pages
| Assistant | Finds pages with | Searches | Training crawler, separate |
|---|---|---|---|
| ChatGPT search | OAI-SearchBot | Its own results, sometimes with other search providers | GPTBot |
| Perplexity | PerplexityBot | Its own index | None: PerplexityBot isn't used for training |
| Google AI Overviews and AI Mode | Googlebot | Google's index, the same as Search | Google-Extended, which doesn't affect Search |
| Microsoft Copilot | Bingbot | Bing's index | None named |
| Claude | Claude-SearchBot | Its own index | ClaudeBot |
There's also a third kind of visitor. When someone asks an assistant to read a particular page, it sends a user agent of its own: ChatGPT-User, Perplexity-User, Claude-User. OpenAI says robots.txt rules may not apply to ChatGPT-User, and Perplexity says Perplexity-User generally ignores them, because a person asked for the page. These fetches aren't how pages get into search answers. They're one person's request.
Step 1: let the search crawlers in
This is the step where sites most often rule themselves out, usually by accident. Blocking "AI bots" sounds like one decision. It's two.
Training crawlers collect pages that may be used to train future models. GPTBot, ClaudeBot, Google-Extended, CCBot and others. Blocking them keeps your writing out of training data. It doesn't affect whether you're cited.
Search crawlers fetch pages so the assistant can find and cite them. OAI-SearchBot, PerplexityBot, Claude-SearchBot, and for Google and Bing, the ordinary Googlebot and Bingbot. Block these and you're out.
OpenAI's documentation puts it plainly. The two are separate switches:
Each setting is independent of the others
Perplexity says the same of PerplexityBot: it's for search results, not for training models, and Perplexity recommends allowing it. Google says Google-Extended, its training control, doesn't affect a site's inclusion in Search at all:
Google-Extended does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search.
So blocking Google-Extended won't take you out of AI Overviews, and allowing it won't put you in. AI Overviews and AI Mode use the same Googlebot crawl as the rest of Search.
What 229 sites' robots.txt files do
To see whether sites get this right, I read the robots.txt files of the 245
sites in the catalogue behind the backlinks guide,
on 16 October 2026. 229 were readable: 215 publish a file, and 14 have none or
serve a page instead, which crawlers treat as no rules. For each crawler I
worked out whether it may fetch the site at all, the way crawlers do: a bot
follows its own group if it has one, and the * group otherwise.
- Not one site blocks an AI search crawler. Nor Googlebot or Bingbot.
- 13 block at least one training crawler.
- All 6 that block GPTBot still let OAI-SearchBot in. They opted out of training and stayed in search, which is exactly the split OpenAI describes.
- Only 18 files mention OAI-SearchBot by name, against 36 that name GPTBot. Most sites leave the search bot to the default rules, which here means allowed.
- 8 files carry Cloudflare's Content-Signal lines, which state preferences
such as
search=yes, ai-train=nofor whoever chooses to read them.
These are sites that want to be found, so it isn't surprising that none blocks search. The point is how they block training: crawler by crawler. A blanket rule against every bot with "AI" in its description would have taken them out of AI search too.
robots.txt isn't the only gate
A robots.txt that allows everything can still sit behind a firewall that doesn't. Content delivery networks such as Cloudflare offer settings that block AI crawlers at the network level, before robots.txt is ever read. If you've switched one on, check which bots it covers. OpenAI and Perplexity both publish the IP ranges their search crawlers use, so a firewall can let them through.
If you want to keep training crawlers out and search crawlers in, the robots.txt looks like this:
User-agent: GPTBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: *
Allow: /
OAI-SearchBot, PerplexityBot, Claude-SearchBot, Googlebot and Bingbot have no
group of their own there, so they follow * and are allowed. OpenAI says a
change can take about a day to reach its systems. The free
AI crawler checker reads a site's robots.txt the same
way and shows what each of these crawlers is allowed to do.
Step 2: be in the index they search
A crawler that can fetch a page doesn't mean the page is in an index yet.
Google: AI Overviews and AI Mode. Google's requirement is the same as for any search result: a page has to be indexed and eligible to be shown with a snippet. Its documentation is direct about the rest:
There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary.
The same controls that shape an ordinary snippet shape these too: nosnippet,
data-nosnippet, max-snippet and noindex. If a page has nosnippet, it
can't supply text to an AI answer.
Bing: Copilot, and possibly more. Microsoft says Copilot is powered by Bing's index. OpenAI says ChatGPT search sometimes partners with other search providers, without naming them. So a site that Bing hasn't indexed is missing from Copilot and may be missing from more. Two cheap things fix that:
- Add the site to Bing Webmaster Tools. It can import a site straight from Google Search Console, and since February 2026 it has an AI Performance report showing citations in Copilot.
- Use IndexNow, which tells Bing about new and changed pages as you publish them. Microsoft says it helps AI answers use the current version of a page.
Perplexity and Claude keep their own indexes. There's nothing to submit: allowing their search crawlers is the whole of it.
In India, this already applies in Hindi and more. Google brought AI Overviews to India in English and Hindi in August 2024. AI Mode followed in English in July 2025, in Hindi from September 2025, and in Bengali, Kannada, Malayalam, Marathi, Tamil, Telugu and Urdu from October 2025. A page in any of those languages can be cited.
Step 3: rank for the questions behind the question
Here's what changes most from ordinary search. An assistant doesn't run your question once. Google says both of its AI features may use what it calls a "query fan-out" technique: issuing multiple related searches across subtopics and data sources, then building the answer from what comes back.
Ask how to choose a salary hike percentage, and the assistant may also search what a typical hike is in your industry, how inflation affects it, and how to negotiate one. Pages that rank for any of those can be cited.
The data shows the effect. In July 2025, Ahrefs found that 76% of the pages cited in AI Overviews ranked in Google's top 10 for the search. In its March 2026 update, across 863,000 results pages, that had fallen to 38%: 31% ranked 11th to 100th, and 31% beyond that. Ahrefs changed its method between the two, and AI Overviews moved to a newer model, so the numbers aren't strictly comparable. But the direction fits the fan-out: more citations come from pages that rank for a neighbouring question rather than the one typed.
So:
- Find the sub-questions. People also ask, autocomplete and related searches show what people ask next. The free vs paid keyword tools post shows that method, and what it turned up for one topic.
- Give each one a heading of its own, phrased the way people ask it.
- Answer it under that heading, so the page ranks for the follow-up as well as the main question.
- Still rank. Ranking for the main search is still the surest way in. Half of the other 62% still ranked in the top 100 for it, and the rest most likely ranked for one of the searches fanned out from it.
Step 4: hold a passage worth quoting
Once a page is retrieved, the model picks the sentences that support what it wants to say. A page can rank and still not be cited, because nothing on it is specific enough to point to.
Neither OpenAI nor Perplexity publishes guidance on what it prefers to cite. OpenAI says only that results are ranked on several factors meant to surface relevant, reliable information, and that placement isn't guaranteed. The closest primary guidance comes from Microsoft and Google:
- Microsoft's Bing team advises short, self-contained answers of a sentence or two, lists, tables, question and answer blocks, clear headings, and content that's fresh, authoritative and well structured.
- Google advises unique, expert-led content that adds something beyond common knowledge, and warns against cutting pages into fragments:
There's no requirement to break your content into tiny pieces for AI to better understand it.
The two fit together. Write whole pages, and inside them, make every answer easy to lift out.
The second sentence can be cited on its own. It has a date, a number, a named source and a claim someone can check. The first says nothing a model could attribute to anyone.
What makes a passage like that, step by step, is in the plain-words list below.
Step 5: be mentioned where the assistants look
Links still matter, but in the AI results studied so far, mentions matter more.
Ahrefs looked at 75,000 brands in May 2025 and measured how closely each factor tracked how often the brand appeared in AI Overviews:
These are correlations, not causes: well-known brands get mentioned, searched and cited, and it's hard to say which comes first. But mentions came out ahead of every link measure, and 26% of the brands didn't appear in AI Overviews at all.
The assistants also lean on a small set of sites. In Ahrefs' studies, Wikipedia was the most-cited domain across ChatGPT, Perplexity and AI Overviews in 2025, and by its September 2026 count, YouTube and Reddit led the most-cited domains in AI Overviews. So:
- Be useful where your subject is discussed. Answer questions on Reddit and in communities, under your own name, without pitching.
- Put the best explanations on video if your audience watches YouTube.
- Get listed where people compare options: review sites, directories people actually use, "best of" roundups. The how to get backlinks guide covers which are worth it.
- Earn editorial mentions: quotes in articles, guest posts, podcasts.
What doesn't count is manufacturing them. Google's guide says seeking inauthentic mentions across the web isn't as helpful as it might seem. A hundred forum posts naming your brand is the pattern spam systems were built to find.
What to put on the page, in plain words
If you take one thing from this post, take this list. A page that gets cited usually has all ten of these. The example page after the list shows each one in place.
- A title that asks the question, or answers it. Write it the way people search. "Does blocking GPTBot remove my site from ChatGPT?" works. "Some thoughts on AI bots" doesn't.
- The answer in the first two lines. Before any background or story. Someone who reads only those two lines should have their answer.
- Who wrote it, why they know, and when. One line under the title: your name, what you do, and "Updated" with a date. Change the date only when you actually change the page.
- Headings written as the next questions. One heading for each thing people ask after the first question. "Which bot should I block?", not "Bot options".
- A one-line answer under every heading. Answer first, then explain.
- Exact facts, each with its source. Numbers, names and dates instead of "many", "some" or "recently". Put the link to where the fact came from right next to it.
- Something only you have. Your own numbers, a test you ran, a photo you took, an example from your own work. If an assistant wants to use it, it has to cite you.
- A table for any comparison. Two or more things side by side go in a table, not a paragraph.
- Short paragraphs, one point each. So the sentence that proves a point is easy to find, for a person and for a model.
- Questions and short answers at the end. The questions your customers actually ask, each answered in two or three sentences.
Does blocking GPTBot remove my site from ChatGPT?
No. GPTBot is OpenAI's training crawler. ChatGPT search uses a different one, OAI-SearchBot, and OpenAI treats them as separate settings.
Which bot should I block to stay out of training?
Block GPTBot, and leave OAI-SearchBot allowed.
OpenAI's crawler page says sites that block OAI-SearchBot won't be shown in ChatGPT search answers. Source: OpenAI, checked 16 October 2026
We checked 229 sites. All 6 that block GPTBot still allow OAI-SearchBot.
Each paragraph makes one point, so the sentence that proves it is easy to find.
Questions people ask
Does blocking GPTBot affect Google? No. Google uses its own crawlers, and its training control is a separate one, Google-Extended.
Then check the page isn't hiding its own words. Three things stop a good page from being read at all:
- Words must be text, not pictures of text. A screenshot of a table can't be quoted. Put the table itself on the page.
- The main text must load without JavaScript. Vercel studied AI crawlers on its network in December 2024 and found that none of the major ones, from OpenAI, Anthropic and Perplexity, ran JavaScript. Text that only appears after a script runs is invisible to them. Google's crawler does run it. To check, open your page's source (Ctrl+U in Chrome on Windows, Option+Cmd+U on a Mac) and search it for a sentence from the page. If it isn't there, those crawlers can't see it.
- Nothing should stand in front of it. No login, no paywall, nothing a crawler has to click through to reach the text.
And leave these out:
- Long introductions. Every line before the answer is a line the model has to skip.
- The same keyword over and over. It reads badly and doesn't help.
- Numbers you can't source. One unsupported figure makes the rest of the page harder to trust.
- Text hidden from readers and written for AI. It's against Google's spam policies, and the next section covers it.
What doesn't work
llms.txt. A proposal from September 2024 for a file that summarises a site for language models. Google says Search ignores it, and that creating one will neither help nor harm. Google's John Mueller wrote in June 2025 that no AI system currently used it, and no AI company has said it reads one for search or citations. Ahrefs tracked 137,000 domains and found 97% of their llms.txt files got zero requests in May 2026. Even so, 17 of the 215 robots.txt files in the catalogue point to one.
Special markup for AI. Google says there's no special schema.org structured data to add for AI Overviews or AI Mode. Structured data can still earn rich results, which is reason enough to use it, but it isn't a citation switch.
Chunking. Splitting a page into fragments for a model to digest. Google says there's no need, and it makes a page worse for the people reading it.
Hidden instructions. Text written for the model rather than the reader, such as instructions tucked out of sight telling an AI to recommend you. Hidden text is already against Google's spam policies.
Mass-produced pages. Hundreds of thin AI-written pages aimed at every variation of a question. Google's spam policies call that scaled content abuse, and pages it doesn't index can't be cited.
How to see whether it's working
Google. Search Console's Generative AI performance report shows impressions in AI Overviews and AI Mode, and since 31 August 2026 it's available to every site. That traffic also stays in the ordinary Performance report, under the Web search type.
Bing. The AI Performance report in Bing Webmaster Tools shows citations in Copilot and Bing's AI summaries.
ChatGPT. OpenAI says ChatGPT adds utm_source=chatgpt.com to the links it
sends, so those visits show up in any analytics tool by their source. The tag
briefly went missing in October 2026, which was reported as a bug and fixed.
Everything else. Ask the assistants yourself. Write down 20 questions your customers ask, put them to ChatGPT, Perplexity and Google once a month, and keep a sheet:
| Column | Why |
|---|---|
| Date | Answers change week to week, so compare months, not days |
| Question | Exactly as asked, every month |
| Assistant | ChatGPT, Perplexity, Google, Copilot |
| Cited you? | Yes, no, or mentioned without a link |
| Who was cited | The domains it used instead, which shows what to beat |
And keep the numbers in proportion. Ahrefs measured AI assistants at about 0.25% of referral traffic across nearly 82,000 sites in mid-2025, with ChatGPT sending more than 80% of it. Pew Research found people clicked a link inside an AI summary on just 1% of the visits where one appeared. A citation is mostly visibility, your name in front of someone at the moment they're deciding, and only sometimes a click.
The whole job, in order
- Check robots.txt allows OAI-SearchBot, PerplexityBot, Claude-SearchBot, Googlebot and Bingbot. Block training crawlers by name if you want to.
- Check your firewall or CDN isn't blocking those crawlers anyway.
- Make sure key pages are indexed in Google and Bing, and don't carry
nosnippetornoindexby mistake. - Add the site to Bing Webmaster Tools, and use IndexNow if your platform supports it.
- List the sub-questions for your main topics and give each a heading and a direct answer.
- Rewrite vague passages into specific ones: a number, a date, a source.
- Publish one thing nobody else has: data, a tool, a test.
- Be present where your subject is discussed, honestly and under your own name.
- Measure monthly: Search Console, Bing Webmaster Tools, ChatGPT's referrals, and your own 20 questions.
Where this leaves you
There's no separate discipline here, whatever it's called this month. The assistants find pages with search crawlers and search indexes, retrieve the ones that rank for the questions they run, and quote the passages that say something specific. Most of getting cited is getting found. The rest is writing sentences worth quoting.
So let the search crawlers in, rank for the follow-up questions as well as the main one, and make every answer on the page specific enough to point to. Then give people a reason to mention you. Assistants cite what the web already trusts, and that part has never been about a file.
