Skip to content

AI engines

How ChatGPT, Perplexity and Claude choose the sources they cite

The three engines document when they search and what the search returns, not a ranking formula; one study found ChatGPT cited 15% of the pages it retrieved.

A funnel filters many scattered squares down to three, with one highlighted in blue

SEO consultant, and more recently GEO, at MentionBox

None of the three publishes the formula. What OpenAI, Perplexity and Anthropic do document is the chain a question runs through before a link appears: the engine decides whether to search, a search returns candidate pages, the model reads some of them, and a subset of what it read becomes a citation. The volume lost at that last step is the figure most pages leave out. A report published on March 12, 2026 over 15,000 prompts measured that 15% of the 548,534 pages ChatGPT retrieved while answering were cited in the final response. Eligibility is settled earlier still, by a crawler: OpenAI's bots page states that sites opted out of OAI-SearchBot "will not be shown in ChatGPT search answers, though can still appear as navigational links".

This page is for whoever wants their pages picked up by an AI answer, and for the developer who has to explain to a client why a number moved. It goes engine by engine through what each one documents, then covers the gap between a page being read and a page being cited, why the three disagree on the same question, and what a site owner can act on.

The three steps behind a cited source

Every documented pipeline among the three has the same shape, and each step removes candidates.

  • Trigger. The engine decides whether to search. On the OpenAI API the model chooses under tool_choice: "auto", and the guide for the web search tool states that "like any other tool, the model can choose to search the web or not based on the content of the input prompt".
  • Retrieval. One or several searches run, each returning a ranked list. Our explainer on query fan-out covers that step, including the counts each vendor publishes.
  • Reading. The engine loads some of those pages into the model's context. OpenAI meters it with search_context_size, and Anthropic now lets Claude write code that filters the results before they arrive.
  • Citation. A subset of what was read is attached to sentences of the answer. OpenAI separates the two explicitly: the sources field "returns the complete list of URLs the model consulted", while inline citations "show only the most relevant references".

No vendor among the three documents how the last step picks its subset. Everything below is what they do document.

How ChatGPT chooses its sources

Start with eligibility, because it is the only hard switch a site owner holds. OpenAI documents three crawlers with three jobs: GPTBot, "used to crawl content that may be used in training our generative AI foundation models"; OAI-SearchBot, "used to surface websites in search results in ChatGPT's search features"; and ChatGPT-User, for "certain user actions in ChatGPT and Custom GPTs", which the same page says "is not used to determine whether content may appear in Search". Blocking GPTBot keeps a site out of training without touching its chances of being cited; blocking OAI-SearchBot removes it from ChatGPT search answers.

On the API side the mechanics are published parameter by parameter. A web_search_call item records what the tool did, and its action is one of three types: search, which "will usually (but not always) include the search queries which were searched"; open_page, supported on reasoning models to retrieve a full page; and find_in_page, to search inside one. So a page can be retrieved, opened, read in part, and still not cited.

Three settings move the candidate pool, and none of them ranks anything:

  • filters, which accepts up to 100 allowed_domains or up to 100 blocked_domains, written without a scheme.
  • search_context_size, at low, medium or high, which the guide describes as controlling "how much context from web search results is made available to the model before it generates a response".
  • include, where adding web_search_call.action.sources returns every URL retrieved rather than the citations alone.

The search is billed at $10 per 1,000 calls plus the tokens of the pages it feeds the model, per OpenAI's pricing page. What you observe through that route is the model and prompt you configured, which is a different object from the answer a buyer reads in the app; our comparison of the web search tool and ChatGPT answers sets out the five documented differences.

How Perplexity chooses its sources

Perplexity documents two crawlers rather than three, and splits them the same way. Its bots page presents PerplexityBot as "designed to surface and link websites in search results on Perplexity" and states that it "is not used to crawl content for AI foundation models", while Perplexity-User "might visit a web page to help provide an accurate answer and include a link to the page in its response".

The product is sold in two halves, which is the clearest published view of a retrieval step anywhere among the three. The Search API returns "ranked web results as JSON, not a generated answer": each result carries a title, a url, a snippet of page text, a date and a last_updated, with max_results defaulting to 10 and capped at 20 for a web search. There is no answer and no token billing, only $5.00 per 1,000 successful requests, or $1.00 per 1,000 on the lower-latency fast search type. The Agent API is the other half: a finished answer with citations, written by a model over those results.

The link between the two is spelled out, and it is the part to read if you count citations. Each citation number in the text refers to a result's id, and the documentation advises treating "the id and url fields of each search_results entry as the source of truth for citations" rather than parsing the numbers out of the prose.

The filters are published in full: search_domain_filter takes up to 20 domains or URLs per request, in allowlist mode or in denylist mode with a - prefix, but never both; search_recency_filter restricts results to the last hour, day, week, month or year; search_after_date_filter and search_before_date_filter bound the publication date in MM/DD/YYYY; last_updated_after_filter and last_updated_before_filter do the same for the update date; user_location localises results. Those last four have a direct consequence for a publisher: a page that exposes no publication or update date cannot satisfy a date filter, whatever its content.

How Claude chooses its sources

Anthropic is the only one of the three that publishes its trigger rule as two lists. Claude searches on recent events and announcements, current prices, rates, scores or statistics, information about specific organisations, people or products "that might have changed", and explicit requests to look something up. It answers without searching on established facts, maths and coding concepts, creative writing, analysis of content already provided in the conversation, and conversational turns. The guide then adds a caveat for anyone measuring: "triggering is steerable through your system prompt", so two teams can measure the same model on the same question and see different search behaviour.

Volume is published too. Simple factual queries "typically use 1–3 searches", comparative or multi-entity research "can use 10 or more", and max_uses is a hard cap that returns a max_uses_exceeded error rather than a bigger bill. Each search costs $10 per 1,000 on the Claude API, plus tokens for the content read.

The newest part of the pipeline is a filtering step inside the engine. With the web_search_20260209 tool version and later, Anthropic documents that Claude "writes and runs code that filters the results first, so only relevant content reaches the context window", instead of loading every result. A page can therefore be returned by the search, discarded by code the model wrote, and never seen by the model that writes the answer.

What comes back is precise. Each web_search_result carries a url, a title, a page_age described as "when the site was last updated", and an encrypted_content blob you must pass back on later turns. Citations "are always enabled for web search", and each web_search_result_location gives the url, the title and up to 150 characters of cited_text, which do not count toward token usage. The web fetch tool behaves in the opposite way on both counts: citations are off by default, and Claude "cannot fetch URLs that appear only in its own output", so a URL has to come from the user, a tool result or a previous search before it can be read at all.

What each engine documents, side by side

StepChatGPT, Responses APIPerplexityClaude API
Decision to searchthe model, under tool_choice: "auto"; "required" forces itthe Agent API searches for the answer; the Search API searches on requestthe model, on a published list of triggers, steerable by system prompt
Published search countnonemax_results from 1 to 20, default 101 to 3 simple, 10 or more comparative, capped by max_uses
Crawler that governs eligibilityOAI-SearchBot for search answers, GPTBot for trainingPerplexityBot for search results, Perplexity-User for answersClaude-SearchBot for search quality, ClaudeBot for training
What you can restrictup to 100 allowed or blocked domains, search_context_size, user_locationup to 20 domains or URLs, recency window, publication and update dates, user_locationallowed_domains or blocked_domains, max_uses, user_location
What comes back per sourceurl, title, citation character range; full URL list via includeid, url, title, snippet, date, last_updated, sourceurl, title, page_age, plus cited_text up to 150 characters
Filtering inside the enginesearch_context_size at low, medium or highnot documentedcode Claude writes, from tool version web_search_20260209
Price of the search$10 per 1,000 calls plus content tokens$5 per 1,000 requests, $1 on fast$10 per 1,000 searches plus content tokens
Ranking formulanot publishednot publishednot published

Parameters, fields and rates from each vendor's own documentation and pricing pages, read on November 1, 2026.

Retrieved, read and cited are three counts

The published fields exist because the three counts differ. The AirOps report of March 12, 2026 took 15,000 prompts, logged 548,534 retrieved pages and 82,108 citations, and found 15% of retrieved pages cited, with the rate moving by question type: 18.3% on product discovery, 16.9% on how-to questions, 13.1% on comparisons and 11.3% on validation searches. The report does not publish how it determined what ChatGPT retrieved, which is a reason to read the ratio as an order of magnitude rather than a constant.

Vendors treat the same gap as a schema problem. Peec AI's domains report returns a retrieval_count next to a citation_count and divides them into a citation_rate, because the two failures lead to different work: never retrieved is a reachability problem, retrieved and never cited is a content problem. Our guide on tracking AI citations of your pages covers the counting rules for both.

Why the three disagree on the same question

Three documented differences are enough to produce three different source lists. The trigger rules differ, so one engine searches where another answers from memory. The retrieval layers differ: Perplexity ranks results from its own crawl, OpenAI surfaces sites through OAI-SearchBot, Anthropic through Claude-SearchBot, and none of the three publishes the pool it draws from. And the reading budgets differ, from search_context_size to max_uses to a max_results of 10.

The measured consequence is larger than most readers expect. In an analysis we ran in October 2026, two engines answering the same question had less than a tenth of their combined list of cited domains in common. Google's own ranking is a weak predictor as well: in the AirOps data, pages ranking first in Google were cited 43.2% of the time against 12.3% for pages outside the top 20, and 32.9% of cited pages that did appear in a top-20 result page had been found only through a fan-out query rather than the original prompt.

Five things a page owner can act on

The documentation leaves little room for tricks, and the five items below all come from a published parameter or statement.

  1. Allow the search crawlers you want. OAI-SearchBot, Claude-SearchBot and PerplexityBot govern whether a page can appear as a source; the training crawlers do not. A robots.txt written once for GPTBot sometimes blocks all three by accident.
  2. Publish and maintain a visible date. Perplexity filters on publication date and on last-updated date, and Claude returns a page_age with every result. A page with no date cannot pass a date filter.
  3. Write for the sub-question, not the prompt. The retrieval step searches reformulated queries, and in the AirOps data a third of the cited pages that ranked in a top-20 result page were reached only through one of those reformulations.
  4. Put the answer in a quotable passage. Anthropic's citation block carries up to 150 characters of cited_text, and OpenAI's annotations mark a character range of the answer. A page that states its answer in one sentence gives the model something to attach a citation to.
  5. Keep the page readable without a browser. Anthropic states that its web fetch tool "does not support websites dynamically rendered with JavaScript". Content that only exists after client-side rendering is not reliably available at the reading step.

Four mistakes when reading a source list

  • Taking one list as the list. The same prompt sent twice can trigger a different number of searches, or none. A single run is a sample of one.
  • Reading a ranking into a citation order. None of the three documents citation order as a rank; citations attach to passages, so the order follows the answer's sentences.
  • Confusing a mention with a citation. An answer can name a brand and cite nobody, or cite a page without naming its owner. Our guide on tracking a brand in ChatGPT keeps the two counts apart.
  • Assuming Google's results carry over. The 43.2% against 12.3% split above says Google rank helps; the 32.9% found only through fan-out says it is far from sufficient.

Reading it for yourself

The shortest route to a real source list is the Claude API, because one call returns the searches, the results and the citations in the same response:

curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-opus-5-5",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "best AI visibility API for an agency"}],
    "tools": [{"type": "web_search_20250305", "name": "web_search", "max_uses": 5}]
  }' | jq '[.content[] | select(.type == "web_search_tool_result") | .content[].url]'

Swap the filter for select(.type == "text") | .citations[]?.url and you get the cited subset instead of the retrieved set. The difference between the two arrays, on your own prompts, is the measurement this page is about.

Across engines, a visibility API does the same job in one place. API for GEO returns a sources[] array per prompt with the url, the domain, a category and a mentionsBrand flag, at $13 per 1,000 answers down to $7 at its largest tier, with reads over stored runs not billed, checked on October 23, 2026. MentionBox, which publishes this site, has a commercial link with API for GEO. Our ranking of AI visibility APIs scores 55 of them on price, engines covered and data returned.

In our view the three engines publish their plumbing and withhold their judgement, which puts the work on the two ends of the chain. Being reachable by the sub-queries an engine runs, and being quotable once read, are the parts a publisher controls; the choice among the pages that clear both is the part nobody outside the three vendors can see.

Frequently asked questions

How does ChatGPT choose the sources it cites?

In four documented steps. The model decides whether to search at all, since OpenAI's guide for its web search tool states that "the model can choose to search the web or not based on the content of the input prompt"; a search returns candidate pages; a reasoning model can then open a page and search inside it, through the `open_page` and `find_in_page` actions; and the answer carries inline citations, which the guide describes as "only the most relevant references", next to a longer `sources` list of every URL consulted. OpenAI publishes no ranking formula for that last step, and bills the search at $10 per 1,000 calls plus the tokens of the pages it reads.

Which crawler decides whether my site can be cited?

The search crawler, not the training crawler. OpenAI's bots page presents OAI-SearchBot as the agent used "to surface websites in search results in ChatGPT's search features", and states that sites opted out of it "will not be shown in ChatGPT search answers, though can still appear as navigational links"; GPTBot is for training, and ChatGPT-User "is not used to determine whether content may appear in Search". Anthropic documents Claude-SearchBot as the crawler that "navigates the web to improve search result quality for users" and warns that blocking it "may reduce your site's visibility and accuracy in user search results". Perplexity documents PerplexityBot as "designed to surface and link websites in search results on Perplexity" and adds that it "is not used to crawl content for AI foundation models".

How does Perplexity pick the pages it links under an answer?

Its Search API returns what the documentation calls "ranked web results as JSON, not a generated answer", with a `title`, `url`, `snippet`, `date` and `last_updated` per result, 10 by default and up to 20 for a web search. Its Agent API then writes the answer, and the documentation tells developers to treat "the `id` and `url` fields of each `search_results` entry as the source of truth for citations", since each citation number in the text refers to a result's `id`. The ranking itself is not documented; the filters are, with up to 20 domains or URLs per request and recency windows of an hour, a day, a week, a month or a year.

Does Claude always search before it answers?

No, and Anthropic publishes the rule. Claude searches when a request depends on information that is "current, changing, or outside its training data", including recent events, current prices and facts about specific organisations or products, and answers directly on established facts, creative writing, analysis of content already in the conversation and conversational turns. The documentation adds that "triggering is steerable through your system prompt" and that `max_uses` is the hard cap, with searches billed at $10 per 1,000 on the Claude API.

Why do the three engines cite different pages for the same question?

Because they decide to search at different moments, search different indexes, and keep different amounts of what they read. Claude's own cap, `max_uses`, puts a comparative question at 10 or more searches; Perplexity's Search API returns 10 ranked results by default; OpenAI's `search_context_size` is a low, medium or high setting whose guide warns it "does not set an exact token count or guarantee a specific number of sources or citations". Each one also names its own search crawler, OAI-SearchBot, PerplexityBot and Claude-SearchBot, and none publishes the pool it draws from, so a page cited by one tells you nothing about the other two.

What is the difference between a page being retrieved and a page being cited?

A retrieval is a page the engine read; a citation is a page it showed as a source. OpenAI draws the line inside its own response, where the `sources` field "returns the complete list of URLs the model consulted" while inline citations "show only the most relevant references", and notes that "the number of sources is often greater than the number of citations". Peec AI ships the ratio as a field, with a `citation_rate` defined as `sum(citation_count) / sum(retrieval_count)`. A report of March 12, 2026 over 15,000 prompts put that ratio at 15% for ChatGPT.