AI discovery
How to test whether ChatGPT, Perplexity and Gemini know your company
Write 15 to 25 questions a real buyer would ask, in three groups: direct questions about your company, category questions, and comparison or recommendation questions.
Run each one several times in each assistant, on a clean session, and record the date, the exact wording, whether you are named, how you are described and which sources are cited. Repeat the same set every month or quarter and compare.
Why one check is not enough
AI answers are not stable. SparkToro ran 2,961 prompts across ChatGPT, Claude and Google's AI, each repeated many times, and found less than a 1% chance of getting the same list of brands twice. Parse found the top recommendation changed in 43.6% of 161,023 consecutive answers to the same prompt. A single screenshot tells you what happened once.
Step 1: write the question set
Use three groups. Write them the way a buyer would, not the way your marketing would.
| Group | Example | What it tests |
|---|---|---|
| Direct | "What does [Company] do?" / "Who founded [Company]?" | Whether the assistant knows you exist and describes you accurately |
| Category | "What are the main ways to monitor LLM output quality?" | Whether you appear when the buyer does not know you yet |
| Comparison | "Best tools for X for a 50-person company" / "[Company] vs [Competitor]" | Whether you make the shortlist and how you are positioned |
Fifteen to 25 questions is enough. Take them from sales calls and support tickets where you can.
Step 2: decide which assistants
Test the ones your buyers use. A common set is ChatGPT with search, Perplexity, Gemini, and Google AI Overviews or AI Mode. Add Claude or Copilot if your buyers work in environments where those are standard.
The engines behave differently. Perplexity shows its sources by default. ChatGPT cites sources when it searches. Google's AI features appear within search results and link to pages. Record which mode you used.
Step 3: run each question several times
Use a logged-out or clean session where possible, so personal history does not influence answers. Run each question 3 to 5 times per assistant. For a deeper study, run more and change the persona slightly ("I'm a CTO at a fintech company...") because answers can shift with the role described.
Step 4: record the same fields every time
- Date and time
- Assistant and mode (search on or off)
- Exact question wording
- Company named? Yes or no, and position if a list
- Founder named?
- How the company is described, copied verbatim
- Errors: wrong category, old pricing, wrong founder, confused with another company
- Sources cited, with URLs
A spreadsheet is enough.
Step 5: read the results
Look for patterns across runs, not single answers:
- Not known at all on direct questions: usually a crawl access or entity clarity problem.
- Known but described wrongly: inconsistent descriptions across your site and profiles, or old pages ranking.
- Known but absent from category answers: you lack pages that answer category questions, or third-party sources do not mention you.
- Present but positioned poorly: third-party sources describe you in terms you would not choose.
The cited sources are often the most useful part. They show which pages the assistant trusts for your category, which tells you where to be present.
Step 6: repeat on a schedule
Run the same set monthly or quarterly with the same method. Change the question set rarely, and log it when you do. Over time you get a distribution of outcomes, which is far more reliable than one run.
A note on tools
Several vendors sell AI visibility tracking. They automate the runs, which helps at scale. Ask how many runs they use per question, whether they show the raw answers, and how they handle variation. A report that gives you a single "rank" for an AI answer does not reflect how these systems behave.
Sources
- AI recommendations change with nearly every query: SparkToro, Search Engine Journal.
- Does AI recommend the same brand again?, Parse, 2026. 161,023 consecutive same-prompt answers, May to July 2026.