AI Brand Rank Is Meaningless. This Metric Actually Predicts When ChatGPT Recommends You.
New research proves AI brand rankings are essentially random noise. The metric that actually predicts whether ChatGPT recommends you is appearance rate — here's how to measure it and what moves it.
The AI visibility tracking market is worth more than $100 million a year, according to Search Engine Land. Most of that budget is going toward one thing: tracking where your brand "ranks" in ChatGPT or Google AI when a buyer searches your category.
New research from SparkToro suggests that investment has a real problem.
Rand Fishkin and researcher Patrick O'Donnell ran 2,961 AI prompt tests across ChatGPT, Claude, and Google AI. Six hundred volunteers each ran 12 different brand recommendation prompts, 60 to 100 times per prompt. The result: there is less than a 1 in 100 chance ChatGPT or Google AI returns the same brand list twice when asked the same question. When it comes to list order, the odds drop to roughly 1 in 1,000.
Your "rank #1 in ChatGPT" is not a position. It is one sample from a highly variable output.
Why this matters for your budget
Most AI visibility tools are selling rank tracking. They assign a position number, trend it over time, and report it to leadership as progress. Based on what SparkToro found, that number represents a single draw from a statistical distribution — not a stable ranking you can move.
This is the same trap that plagued early web analytics in 2004, when agencies sold "position 1 on Excite" as a KPI while search was rapidly becoming personalized, location-adjusted, and query-variant. The metric was real. The interpretation was wrong.
The Carnegie Mellon paper published at EMNLP 2025 that underpinned SparkToro's methodology found that LLM response consistency typically does not align with human expectations, and automated methods for estimating that consistency are "sufficiently imperfect to warrant broader use of evaluation with human input." The researchers studied 2,976 users to reach that conclusion.
If your team is optimizing for AI rank position, they are optimizing for noise.
The metric that actually tells you something
SparkToro's research didn't only surface the problem. It pointed to what matters.
When they ran each prompt dozens of times, a real signal emerged: some brands showed up in far more responses than others. One agency in the study appeared in 85 out of 95 Google AI responses to the same prompt. That's an 89% appearance rate. Not rank 1. Not rank 3. Just: this brand appears reliably when you ask this type of question.
The metric worth tracking is appearance rate — out of 50 (or 100) runs of a given prompt, how many times does your brand show up anywhere in the response?
Search Engine Land's framework for AI brand visibility formalizes this as the Brand Visibility Score: (answers mentioning your brand ÷ total answers for your space) × 100. If your brand appears in 22 of 100 ChatGPT runs for "best enterprise security platform," your score is 22%.
That number is meaningful. Rank position is not.
Running the measurement this week
Here is a setup you can hand to a team member today:
-
Choose 10 prompts that reflect how your buyers actually search in ChatGPT or Perplexity. Think "best [your category] tools for [your ICP]" or "top [your category] platforms for [specific use case]."
-
Run each prompt 20 times in fresh sessions. New incognito window or new conversation each time to prevent context carryover from influencing results.
-
Record only whether your brand appears — not where. Presence or absence. That's it.
-
Calculate appearance rate per prompt: appearances divided by 20 runs.
-
Run the same test on two or three competitors your buyers are actively comparing you to.
This gives you a real baseline. It takes an afternoon to set up, and about 90 minutes a week to maintain if you log results in a spreadsheet. It's more useful than anything a monthly AI rank report tells you.
What actually drives your appearance rate
AI systems like ChatGPT and Google AI Overviews don't hold a ranked directory of brands. They generate responses by drawing from their training corpus. The brands that appear most consistently are the ones most embedded in the documents the model treats as authoritative.
A Semrush study of 10 million queries found that commercial queries now trigger Google AI Overviews 18.57% of the time, up from 8.15% in October 2024. Transactional queries rose from 1.98% to 13.94% over the same period. The AI is showing up in buying moments now — not just informational research.
That shifts this from a brand awareness problem to a revenue problem.
Pew Research found that when a Google AI Overview is present, users click traditional organic results only 8% of the time, versus 15% when no AI Overview appears. The AI answer is where more and more decisions get framed. And documented buyer behavior data confirms that 47% of enterprise buyers now begin vendor research in AI tools, not Google.
If you are not appearing in those AI responses with any regularity, you are not in the room where the buying consideration starts.
Building a corpus footprint that moves your appearance rate comes down to one thing: the volume of trusted third-party coverage that exists about your brand. If TechCrunch has covered you, if Forbes has cited your research, if Harvard Business Review has featured your category — those documents are almost certainly in the AI's training data. If your only footprint is your own website and content you published under your own domain, your corpus presence is thin.
The relationship is direct: more trusted third-party coverage in publications AI engines index = higher document density for your category = higher probability your brand appears when the AI generates a response.
The publication audit Christian put together in February walks through how to identify specifically which publications are already driving citations in your category. That's where to start before you build or expand your earned media strategy.
The operating framework behind this
This is why Machine Relations is the right lens for what you're building. A placement in Forbes or TechCrunch is not just a PR milestone for human readers. It is a corpus contribution. Every time that piece exists in the AI's training data, it slightly increases the probability that your brand surfaces when the model answers a question in your category. The publication is the same one that shaped human buyer opinion for decades. The mechanism is the same. The reader changed.
Tracking AI visibility by rank position assumes AI works like Google did in 2015. Tracking appearance rate treats AI systems as what they actually are: probabilistic generators that pull from the documents they trust. One is measuring noise. The other is measuring the thing you can actually build toward and move over a quarter.
If your current AI visibility dashboard reports a position number and calls it a rank, you now know what question to ask your vendor. And if you want to see where your brand actually appears in AI answers for your category today, the visibility audit gives you a real baseline to work from.