Obility Editorial · · 4 min read
An answer can only cite pages its search step returned
When an assistant answers a buying question with sources, the model is not reading the whole web on the spot. It writes a short query, sends it to a search system, gets back a list of candidate pages, reads some of them and writes its answer from what it read. A page that never makes that list cannot be cited, however good it is.
So the first practical question is which search system each assistant asks. Most advice answers it with a single confident claim, usually that ChatGPT runs on Bing. The vendors’ own documentation is more varied than that, and the variation decides where you should check your pages.
Google and Microsoft name the index behind their answers
Google is direct about AI Overviews and AI Mode. Its Search Central documentation says that to be eligible as a supporting link, a page must be indexed and eligible to be shown in Google Search with a snippet, and that there are no additional requirements to appear. The gate for Google’s AI answers is the ordinary Google index.
Microsoft is just as explicit about Copilot. Its Microsoft Learn documentation says that when web search is on, Copilot generates a query of a few words informed by the user’s prompt and sends it to the Bing search service. The query is not the user’s prompt, which matters later when you try to work out why a page was or was not found. Copilot’s candidate list starts with Bing.
OpenAI does not name ChatGPT’s search providers
The Bing belief has a real origin. In May 2023 Microsoft announced that Bing was coming to ChatGPT as the default search experience. But when OpenAI launched ChatGPT search in October 2024, its announcement said only that the product uses third-party search providers, along with content supplied directly by its partners. It did not name the providers, and nothing in OpenAI’s documentation commits it to one.
What OpenAI does document is its own crawler. Its developer documentation says OAI-SearchBot is used to surface websites in ChatGPT’s search features, and that sites opted out of it will not be shown in ChatGPT search answers, though they can still appear as navigational links. So ChatGPT depends on at least two things you can influence: being present in whatever outside indexes it queries, and letting OpenAI’s own crawler in. SEO tests that claim to have caught ChatGPT using Bing or Google describe one moment in a commercial arrangement that can change without notice. Treat both as live until OpenAI says otherwise.
Claude and Perplexity point to other providers and their own crawlers
Anthropic does not describe Claude’s search setup in detail, but its public subprocessor list names Brave Search as a web search provider for all Claude products, alongside turbopuffer, a search database company. Brave operates its own search engine, separate from Google and Bing, so a page that is well indexed by both of those can still be missing from the pool Claude searches. Anthropic’s help centre also describes Claude-SearchBot, which it says indexes content to improve search results, and warns that blocking it may reduce a site’s visibility in Claude’s answers.
Perplexity’s documentation says PerplexityBot is designed to surface and link websites in Perplexity’s search results, and recommends allowing it in robots.txt and permitting requests from its published IP ranges. That last point catches teams out: a firewall rule that challenges unfamiliar bots can block the crawler even when robots.txt allows it.
Check your presence in each pool, starting from the answers
Begin with the buying questions you care about rather than with the indexes. In Obility’s Answer Engine Insights every observed answer keeps the engine, the prompt and the sources it cited, so you can compare the same question across engines side by side. A gap that shows up in one engine only is the useful signal. To take an illustrative case, if your comparison page is cited in Google’s AI Mode for a question but never by Copilot, check Bing before you rewrite anything.
The checks themselves are unglamorous. Confirm your key pages are indexed in Google Search Console and in Bing Webmaster Tools using each tool’s URL inspection. Search for the same pages on Brave Search. Read robots.txt and your firewall rules for OAI-SearchBot, Claude-SearchBot and PerplexityBot. Then look at your server logs, which Obility’s Agent Analytics separates from citations and human visits, to see whether those crawlers actually reach the pages that matter. A crawler request shows a page was fetched, not that it was used, so read it as access confirmed and nothing more.
Being indexed gets you onto the list, not into the answer
Do not read any of this as a secret lever. Presence in an index is a precondition. Once your page is among the candidates, the model still chooses what to read and cite based on how well the page answers the queries it wrote, and those queries are short and often different from what your buyer typed.
Provider arrangements also change. OpenAI already describes its providers only in general terms, and Anthropic’s subprocessor list can change whenever a vendor does. Recheck each vendor’s documentation every quarter, and when one engine suddenly stops citing you while the others hold steady, ask first whether its search pool changed before assuming your content got worse.