Citebase
Guides Pay $197

What you can check

Why an assistant recommends a competitor

There is no published rate that says roundups get a fixed share of citations. If someone sells you that number, ask for the prompts, the engines, and the date. The checks below are the ones you can do on your own site this week.

Start with the answer you already saved from the manual test. Open the link the assistant used. Most of this page is about that link, and about whether your pages can be used the same way.

Someone else already wrote the shortlist

ChatGPT search, Perplexity, and Google’s AI features in Search all point at web pages.

  • OpenAI says OAI-SearchBot is used to surface websites in ChatGPT’s search features. (platform.openai.com/docs/bots redirects to that page.)
  • Perplexity says PerplexityBot is designed to surface and link websites in Perplexity search results.
  • Google says AI Overviews and AI Mode surface relevant links, and that a supporting link has to be a page that is indexed and eligible to show a snippet in Google Search.

If a review, a comparison, a marketplace, or a “best tools” post names three products and you are not one of them, the assistant has a page it can cite that does not include you. Your homepage does not automatically become that page. Often the competitor’s homepage is not the cited page either. The cited page is the one that already compared the category.

Google’s guide to generative AI features says those features can show what is said about products across the web, including in blogs, videos, and forum discussions. The same page says seeking inauthentic mentions is less useful than it sounds, because Google’s ranking systems still focus on quality and its spam systems still apply. A planted roundup is not a plan this guide recommends. Reading the roundup the assistant already cited is.

Comparison pages are written as answers

A page a model can quote usually says the category, names the options, and says who each one is for, in sentences. A homepage line like “the operating system for modern teams” does not answer “membership software for a paid newsletter.”

Google’s AI features guide says important content should be available as text. It also says you do not need a new machine-readable file, an AI text file, or special schema to be eligible. The optimization guide is blunter about files named llms.txt: Google Search does not use them, and publishing one will neither harm nor help your visibility in Google Search. More on that in the llms.txt guide.

Google also says you do not have to rewrite pages in a special style for generative AI, and that its systems can match synonyms, so you do not need every long-tail variant of a query. You do need a page where a person can tell what you sell. A pitch that exists only in an image, or only after login, gives a crawler nothing to quote. Put the plain description in HTML: what it is, who it is for, what it costs, and what it refuses to be.

Say the category in words a buyer would recognize

Read the homepage, the pricing page, and the FAQ. Search the page for the noun in the buyer’s prompt: newsletter, podcast scheduling, form, cohort, membership. If the noun never appears, you are asking a retrieval system to guess from a metaphor.

That is not a secret ranking factor Google published. Google says an exact keyword match is not required. It is still worth describing the product in the language of the job, because “for a weekly solo newsletter” is information, and “for modern teams” is not. Link that page from the homepage. Google lists internal links among the ordinary practices that still apply to AI features.

The page has to be crawlable, and eligible to be quoted

For Google, the bar is specific. To be a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible to be shown in Google Search with a snippet. There are no additional technical requirements. Meeting them does not mean Google will crawl, index, or show the page. How Search Works is Google’s longer version of that limit. The technical requirements are the checklist for being eligible at all.

Blocking Googlebot in robots.txt, or marking the page noindex, takes the page out of Search. Google says robots.txt for Googlebot is how you manage crawling for Search, and AI features in Search sit on that crawl. nosnippet, data-nosnippet, max-snippet, and noindex limit what can be shown. A host, CDN, or firewall that challenges every unknown client has the same practical effect as a disallow, even when robots.txt says Allow. Google lists that case separately: crawling has to be allowed in robots.txt and by your CDN or host.

The bot you blocked may be the wrong one

This is the mix-up that sends people into a rewrite they did not need.

  • GPTBot is OpenAI’s training crawler. Disallowing it says the content should not be used to train foundation models. OpenAI says this setting is independent of search. You can allow OAI-SearchBot and disallow GPTBot.
  • OAI-SearchBot is ChatGPT search. OpenAI says a site that opts out will not be shown in ChatGPT search answers, though a navigational link can still appear. A robots.txt change can take about 24 hours to show up for search.
  • PerplexityBot is what Perplexity documents for search results. They say it is not used to crawl content for AI foundation models, and they recommend allowing it if you want the site in those results. Perplexity-User, the fetch that happens because a person asked a question, generally ignores robots.txt. Blocking the search crawler does not, on their own description, block every user-triggered fetch.
  • Google-Extended does not decide whether you are in Google Search. Google’s crawler docs say it is a standalone robots.txt token for training future Gemini models and for grounding in Gemini Apps and in Vertex AI’s Grounding with Google Search. It is not a ranking signal in Google Search. AI Overviews are a Search feature. The Search-wide crawl control is Googlebot. There is also a separate Search Console control for appearing in AI Overviews and AI Mode, covered in the AI Overviews guide.

If a competitor shows up and you disallowed OAI-SearchBot last year, or your firewall challenges every bot, look at that before you rewrite the homepage. If you only disallowed Google-Extended, that does not explain an AI Overview.

What not to conclude

  • One answer is not a verdict. Run the same prompt on four surfaces before you treat it as a pattern.
  • Google says no third-party tool has access to its internal ranking or AI systems. A screenshot of a public answer is evidence of that answer. A score that claims to be Google’s internal number is not.
  • Adding llms.txt is not one of Google’s requirements for AI Overviews. Google says Search ignores it.
  • There is no official percentage, from Google, OpenAI, or Perplexity, for how often listicles win. Do not buy one that cannot show its method.

Sources