Most AI Visibility Scores Are Statistical Noise. Here Is What Actually Holds.

The number on your AI visibility dashboard changed since the last time you checked it. Not because your marketing moved. Because AI engines are probabilistic by design, and a single measurement captures one possible outcome out of hundreds. New research from IQRush shows that rankings across SearchGPT, Gemini, and Perplexity require 33 to 94 repeated measurements before they stabilize. Your tracker probably ran one. Here is what that means and what you can actually trust.

AI Engines Are Built to Give Different Answers Every Time

This is not a bug in your measurement tool. It is a feature of the engines.

Every time ChatGPT, Perplexity, or Gemini answers a query, the model introduces randomness into its token selection. The documents it retrieves depend on index freshness. The query itself gets expanded into multiple sub-queries internally, and different expansions pull different source sets. Prior conversation history, user location, and prompt phrasing all shift the output.

SparkToro's study with Gumshoe.ai tested 12 prompts across ChatGPT, Claude, and Google AI with 600 volunteers running 2,961 total queries. The finding: less than a 1 in 100 chance that ChatGPT or Google's AI will return the same list of recommended brands in two responses to the same prompt. Factor in ordering, and the odds of an identical list in the same sequence drop to roughly 1 in 1,000.

That is not a different prompt. That is the same prompt, asked twice. Rand Fishkin put it plainly: any tool that gives a "ranking position in AI" is, his words, full of baloney.

But here is the nuance Fishkin added that most people miss: visibility percentage across dozens to hundreds of prompts run multiple times is a reasonable metric. When Bose showed up in 77% of headphone recommendation responses while competitors appeared in 38%, that gap represents something real. The problem is not measurement. The problem is measuring once and calling it truth.

How Much the Numbers Actually Move

TryRes ran 1,000 queries through Perplexity's Sonar API: 100 queries, each repeated 10 times. The results:

• 8.2 unique brands appeared per query on average
• Only 3.1 of those brands appeared in every run
• 62% of brand appearances were inconsistent across repeated measurements
• The Jaccard similarity between any two runs of the same query averaged 0.72, meaning roughly 28% of the response changed each time

One exception held: the number one recommendation was stable 75% of the time. Position one holds. Everything below it shuffles.

The IQRush paper by Ron Sielinski and Julius Schulte adds the operational question most trackers skip: how many measurements do you need before the ranking is trustworthy? Their finding across 30 platform-topic tests: between 33 and 94 answers with citations before rankings stabilize. Three of those 30 never stabilized at all, even after 125 questions, all on SearchGPT, because the top sites were too close to separate.

A 3.5-point difference in citation share between two brands on your dashboard might seem meaningful. In the IQRush data, a gap that size regularly fell within the margin of error. The tool showed Tom's Guide ahead of Runner's World. The statistics said the gap was noise.

Not All Engines Are Equally Noisy

This part trips up teams that assume their measurement budget covers every platform equally.

Gemini piles citations onto the same handful of sites within a single answer. Those citations look like strong signal, but many of them are telling you the same thing repeated. Each Gemini answer carries less independent information than the raw citation count suggests.

SearchGPT gives fewer citations per answer but spreads them across more sources. Each answer carries more independent information. The same number of answers on two engines does not give you the same confidence level, and a measurement budget that settles Gemini rankings can leave you guessing on SearchGPT.

The practical impact: your Perplexity visibility might be trustworthy after 40 measurements while your SearchGPT visibility needs 90 for the same confidence. A tracker that treats all engines identically is misleading you on at least one of them.

What a Model Update Does to Your Baseline

The variance is not only run-to-run. It is structural.

When Google switched from Gemini 2 to Gemini 3 as the default for AI Overviews on January 27, 2026, SE Ranking's 100,000-keyword study documented the damage: 42.4% of previously cited domains disappeared. That is 37,870 domains out of 89,262. They were replaced by 46,182 new domains that had never appeared before. Average sources per AI Overview grew 31.8%, from 11.55 to 15.22.

The churn was not evenly distributed. Among the 500 most frequently cited domains, only one disappeared. YouTube still led at 10.74%, followed by Reddit at 4.01%. The disruption happened almost exclusively in the long tail, which is exactly where most brands live.

No content strategy caused that change. No competitor outmaneuvered those 37,870 domains. A model update reshuffled the deck, and nearly half the players got new cards. Only 19% of AI Overview sources now overlap with the top 10 organic search results for the same query.

If your visibility tracker does not separate model-update effects from your performance, every trend line it shows you is contaminated. A drop after a model update is not a signal to change your strategy. It is an infrastructure event, and the correct response is to re-baseline, not react.

What You Can Actually Trust

Position one. The top recommendation in AI responses is stable roughly 75% of the time across engines and queries. If you are not in that slot, measuring your exact position below it is largely academic. The IQRush data shows that the typical margin of error on a top-10 site runs about five positions, and one in five is wider than ten. You might be sixth. You might be eleventh. The data cannot tell you.

Presence over time. Instead of tracking your rank, track whether your brand appears at all across repeated measurements for a given query. A brand that shows up in 70% of runs for a query owns that query in a way that a brand appearing in 15% of runs does not, regardless of what position either one is in on any single check.

Repeated measurement methodology. The IQRush paper provides a stopping rule: measure until both the ranking order stabilizes and the gap between top sites exceeds the margin of error. Neither condition alone is sufficient. Both must be true simultaneously. Any tracker that cannot perform this check is publishing rankings the data does not support.

The answer "not enough data yet." A tracker that tells you it cannot produce a reliable ranking for a given query is more valuable than one that prints a confident number every time you ask. Three of 30 IQRush tests could not separate their top sites within budget. The honest answer was to hold, not to publish a ranking.

Three Questions to Ask Your Visibility Tracker

Before you trust the number on your dashboard:

1. Does it measure once or repeatedly? If it checks your query once across each engine and prints a score, that score is a single draw from a distribution. It is not your visibility. It is one of hundreds of possible visibilities.

2. Does it show a range or a clean number? A clean number without a confidence interval is a design choice, not a measurement. Ask for the variance. If the vendor cannot show it, the vendor is not measuring. The vendor is sampling.

3. Can it tell you "not enough data"? The IQRush research shows that some queries never stabilize within practical budgets. A tracker that always produces a ranking, regardless of how much data it has, is filling the gap with confidence it has not earned.

If your current tool fails all three, you are paying for a random number generator with a professional interface.

How This Connects to Citation Architecture

This is why citation architecture matters more than point-in-time visibility scores. The structural pattern of earned media placements, owned content extractability, and entity signals across independent sources is what makes position one stable. It is what makes your brand appear consistently across 70% of runs instead of flickering in and out at 15%.

A visibility score tells you where you are in one snapshot. Machine Relations is the discipline that determines whether you stay there. The difference between a brand that holds position one at 75% stability and one that shuffles through the middle is not content volume. It is the density and consistency of third-party evidence that AI engines use to resolve which source to trust.

Share of citation measured correctly, across repeated observations with confidence intervals, is the diagnostic metric. Measured once, it is noise.

FAQ

Are all AI visibility tracking tools unreliable?

Not necessarily, but most are. The core problem is methodology, not capability. Searchless.ai's analysis argues that single-check tracking is "statistically invalid" because AI search is non-deterministic by design. Tools that run repeated measurements across multiple sessions and report ranges with confidence levels are closer to the truth. Ask your vendor whether they perform repeated sampling and whether they can report "not enough data" for thin queries.

How many times should you measure AI visibility for the same query?

The IQRush paper found stabilization requires between 33 and 94 answers with citations, depending on the platform and topic. Three of 30 tests never stabilized after 125 questions. There is no universal number. The right approach is to measure until both the ranking order settles and the gap between top positions exceeds the margin of error.

Why does position one hold when everything else shuffles?

AI engines weight top-of-answer placement more deterministically than mid-list positions. The sources with the strongest combination of entity clarity, earned media authority, and content extractability tend to lock position one because multiple independent signals converge on the same source. Below that, the signals are weaker and the model's randomness has more room to operate.

What should you track instead of AI visibility rankings?

Track presence rate (does your brand appear at all across repeated measurements), recommendation rate (are you mentioned versus recommended versus cited), and trend direction over time using consistent methodology. Share of citation measured across repeated observations with confidence intervals is the most diagnostic single metric. Avoid tracking exact positions below the top two or three, as the margin of error makes those positions statistically meaningless.

How do model updates affect AI visibility scores?

When Google switched to Gemini 3 for AI Overviews in January 2026, 42.4% of previously cited domains disappeared overnight and were replaced by 46,182 new domains. Model updates reshuffle citation patterns structurally. Visibility trackers that do not flag model updates separately will show performance drops that are infrastructure events, not strategy failures. The correct response is to re-baseline after a model update, not to change your content strategy in reaction to a structural shift.