How Citation Rate Is Actually Calculated — And Why Four Tools Give You Four Different Numbers

Citation rate is the number of eligible AI answer runs that cite your domain, divided by every eligible answer run in the panel, times 100.

Citation Rate = (eligible answer runs with ≥1 qualifying citation to your domain
                 ÷ all eligible answer runs) × 100

Count each run once in the numerator even if the answer links to you three times. Put every eligible run in the denominator, including the runs that cited nobody. Compute it per engine.

That is the formula. It is also the least interesting part of the answer, because the formula is not where citation rates go wrong. Three things decide what your number actually means, and none of them is the arithmetic: which denominator you chose, how many times you ran each prompt, and whether the engine retrieved anything at all.

Four denominators are in live use, and they are not interchangeable

The phrase "citation rate" is currently attached to at least four different divisions, each of them published as the formula by a working practitioner or vendor. Each is defensible. None converts into the others without the underlying log.

Unit of observation	Formula	Published as citation rate by	What breaks if you swap it in
Answer runs	runs citing your domain ÷ all eligible runs	Maxlytics: "valid runs citing your domain ÷ total valid runs"	Needs the run count declared. Five runs per prompt and one run per prompt give different numbers from the same prompt set.
Answers	answers citing your domain ÷ total answers	Kompozy: "answers that cite your domain ÷ total answers, per engine"	Equivalent to runs only if every run produced an answer. Diverges the moment refusals or errors enter the panel.
Prompts	prompts with ≥1 citing run ÷ all tracked prompts	Akshay Hooda: "prompts citing your domain ÷ prompts tested"	Strictly higher than the run-based rate. A domain cited in one run out of five scores 100% here and 20% above.
Responses containing any citation	responses citing you ÷ responses with ≥1 verified citation	FAII: "responses with ≥1 verified citation ÷ total responses"	Excludes uncited answers from the denominator, which inflates the rate against panels where retrieval often does not fire.

A fifth formula circulates under the same name but is a different metric entirely: citation share, your cited URLs divided by all cited URLs in the sample. Share falls when engines start citing more sources per answer, even if you were cited in every single one. It answers "how much of the cited authority do we own," not "how often are we cited."

The Machine Relations glossary definition uses the first row: eligible answer runs, with numerator and denominator reported alongside the percentage. That is the unit we recommend, because it is the only one that degrades gracefully — you can always aggregate runs up to prompts, but you can never recover runs from a prompt-level number.

The practical consequence is blunt. A citation rate quoted without its denominator is not a low-precision number. It is an unreadable one. You cannot tell whether 30% means "we appeared in 30% of 600 answers" or "we appeared somewhere in 30% of 40 prompts," and those describe different businesses.

Citation rate is a compound of two probabilities, not one

Here is the decomposition most dashboards do not expose, and the one that explains most unexplained drops:

Pr(cited) = Pr(the engine retrieved at all) × Pr(cited | it retrieved)

Answer engines do not search on every turn. When a model answers from parametric memory, there is no retrieval step, no source list, and no opportunity for any domain to be cited. That run still belongs in your denominator — it was an eligible answer — but nothing about your content could have changed its outcome.

Prominara's published methodology records whether retrieval fired on every run and splits the rate on exactly this identity, which is the correct treatment. Without that split, a citation rate that halves is ambiguous between two opposite diagnoses:

• Retrieval collapsed. The engine stopped searching for that query class — a model or product change. Your content is irrelevant to the fix.
• Selection collapsed. The engine searched as often as before and stopped choosing you. Now your content, freshness, and source role are the fix.

Those two failures call for entirely different work, and a single blended percentage cannot distinguish them. If your stack reports one number, you are guessing which one happened.

There is a further wrinkle inside the retrieval term. Google states in its Search Central documentation that both AI Overviews and AI Mode may use a "query fan-out" technique — issuing multiple related searches across subtopics and data sources — before composing a response. Most major answer engines do something equivalent. So a single "run" is not one retrieval; it is a bundle of synthetic searches you never see and cannot enumerate. Your page can win three of the hidden branches and still not appear, or win one and appear. The run remains the correct unit of observation — it is what the user experiences — but nobody should mistake it for the unit of competition.

Every citation rate is a sample estimate, and the sample is noisier than you think

AI answers are not stable readings. The same prompt, asked again, cites different sources — and the source list moves independently of the prose.

The clearest measurement of this is SE Ranking's June 2025 study, which ran the same 10,000 keywords through Google's AI Mode three times on the same day. Across all three runs, the average overlap was 9.2% of exact URLs and 14.7% of domains, and 21.2% of queries shared no URLs at all between runs. Roughly nine in ten cited URLs changed on a repeat search of an identical query, on the same day.

This is not an artifact of sloppy sampling. It is what the underlying systems do. Thinking Machines Lab sampled 1,000 completions of a single prompt at temperature 0 — greedy decoding, no deliberate randomness — and got 80 unique completions. The outputs were identical for the first 102 tokens, then split at token 103: 992 completions said "Queens, New York," and 8 said "New York City." The cause was not sampling temperature but batch-invariance in the inference kernels — server load changes the batch shape, the batch shape changes floating-point reduction order, and a flipped argmax cascades through everything after it.

The peer-reviewed picture agrees. Atil et al., in Non-Determinism of "Deterministic" LLM Settings — published at Eval4NLP 2025 — configured five API-based models to be deterministic across eight tasks and ten runs, and observed accuracy variation of up to 15% across runs, with a gap of up to 70% between best- and worst-possible performance. No model produced repeatable accuracy, let alone identical strings.

The providers say so themselves. OpenAI's guidance on the seed parameter states that the system will make a best effort to sample deterministically and that "determinism is not guaranteed". Microsoft's Azure OpenAI documentation is blunter: even with the seed and system_fingerprint matching across calls, it is "currently not uncommon to still observe a degree of variability in responses".

Add live retrieval on top of that and the source list moves for a second, independent reason: the index changed between your two runs.

A one-run citation rate is not a measurement. It is a draw.

How many runs you actually need

If a citation rate is a proportion estimated from a sample, its uncertainty follows the standard interval for a proportion:

SE = √( p × (1 − p) ÷ n )        95% CI ≈ p ± 1.96 × SE

Where p is the observed rate and n is the number of eligible runs. The arithmetic is unforgiving at small n:

Eligible runs (n)	Observed rate	95% confidence interval
30	25%	±15.5 points
100	25%	±8.5 points
300	25%	±4.9 points
1,000	25%	±2.7 points

Read the top row carefully. Thirty runs at 25% puts the true rate somewhere between roughly 10% and 40%. If your dashboard reports "citation rate 25%, up from 21% last month" off a sample that size, it has reported noise as a trend.

Three rules follow directly from the arithmetic:

1. Size the sample to the claim you want to make. Detecting a 2-point move needs a sample several times larger than detecting a 10-point move. Decide what change is worth acting on before you choose n.
2. Never pool engines into one rate. Retrieval behaviour and citation density differ per engine — Google says outright that AI Mode and AI Overviews "may use different models and techniques, so the set of responses and links they show will vary", and those are two surfaces from the same company. A pooled figure describes none of them, and it moves when your engine mix moves even if every engine held steady.
3. Freeze the prompt set inside a window. Adding prompts mid-period changes the denominator and manufactures a trend out of nothing. Batch prompt changes and date them.

Report the interval, or at minimum report n beside the percentage. "18 of 60 runs (30%, ±11.6)" is an honest sentence. "30%" is not.

The counting rules that silently change the number

Before any of the above matters, six decisions have to be written down, because each moves the result by more than most content work does:

• Domain or URL. Domain-level counting is more stable — SE Ranking's repeat runs kept 14.7% of domains against 9.2% of URLs. A URL-level rate is a stricter and more volatile instrument.
• Brand mention or linked citation. A mention with no link is presence, not citation. Mixing them produces a number that is neither.
• Deduplication within a run. If the same URL appears in the answer body and in the sidebar, that is one citation for that run. Counting both inflates share-based metrics and leaves run-based metrics unchanged — another reason the two must never be compared.
• Eligibility and exclusions. Define before collection what disqualifies a run: engine error, refusal, no answer generated. Excluding runs after seeing whether they cited you is how a measurement program starts lying to itself.
• Personalisation and locale. SE Ranking's local test found general local queries repeat only about 35% of domains and 19% of URLs within a single city, while adding an explicit city name to the query pushed domain repetition to roughly 55%. Locale is not a reporting dimension you add later; it is part of the instrument.
• Retrieval state. Whether the engine searched, per run, as covered above.

Write these down as a versioned protocol. If the protocol is not versioned, your time series is not a time series — it is a sequence of unrelated experiments sharing an axis label.

A worked example

A B2B software company runs 40 frozen prompts across three engines, five runs each, weekly. That is 600 eligible runs per week, 200 per engine.

Week one, the answers cite their domain in 46 of 200 ChatGPT runs, 71 of 200 Perplexity runs, and 12 of 200 AI Mode runs.

• ChatGPT citation rate: 46 ÷ 200 = 23%, 95% CI ±5.8
• Perplexity citation rate: 71 ÷ 200 = 35.5%, 95% CI ±6.6
• AI Mode citation rate: 12 ÷ 200 = 6%, 95% CI ±3.3
• Pooled: 129 ÷ 600 = 21.5% — a number that describes no engine they actually compete in

Week two, ChatGPT reads 39 of 200, or 19.5%. The intervals overlap heavily. Nothing happened. Reporting that as "citation rate fell 3.5 points" would send a content team to fix a problem that does not exist.

Now split AI Mode's 6% by retrieval state, and suppose 150 of the 200 runs never triggered retrieval. The rate among runs that actually searched is 12 ÷ 50 = 24% — competitive with ChatGPT. The company does not have a content problem on AI Mode. It has a query class that mostly does not trigger search, which is a prompt-set and demand question, not an authority one.

Same log. Same arithmetic. Opposite instructions to the team.

A live example of the distortion, caught while writing this

While researching this article, we found a secondary source attributing to SE Ranking the claim that "only 35% of cited domains repeat between runs" of the same prompt in Google AI Mode, presented as a general finding.

The 35% figure is real and it is SE Ranking's. It comes from their local search volatility test, and it describes general local queries — "restaurants near me" — repeated within a single city. SE Ranking's general AI Mode study reports 14.7% domain overlap across three same-day runs, not 35%.

So a correct number from a real study was transported onto a population it does not describe, and in the direction that makes AI answers look more than twice as stable as the general evidence says they are. Nobody fabricated anything. The population simply fell off during the journey.

This is the same failure pattern catalogued in the 2026 source register, and it is why the third provenance check — does the source contain this figure with this denominator? — catches things that link-checking never will.

The measurement contract

Before a citation rate is allowed to travel — into a board deck, a vendor comparison, or a budget argument — it should be expressible in one sentence:

In [engine], across [N eligible runs] from [frozen prompt set, version], during [window], [domain] was cited in [k] runs: [k/N]%, ±[interval], with retrieval firing in [r] of those runs.

If the sentence cannot be completed, the number is not ready to leave the room.

This is what separates a citation rate from a vanity metric. Not precision — the underlying system is genuinely noisy, and a well-run program still reports intervals several points wide. What separates them is that an honest citation rate carries its own instrument with it, so the next person can tell what changed: your content, the engine's retrieval behaviour, or the sample.

Method and limitations

The formulas here are arithmetic identities, not empirical claims. The competing-denominator table records formulas as published on the cited pages; it does not assert that any of them is wrong within its own stated protocol, only that the four are not interchangeable. The empirical inputs are: SE Ranking's June 2025 AI Mode study of 10,000 keywords run three times on the same day; SE Ranking's local AI Mode volatility test; Thinking Machines Lab's 1,000-completion temperature-0 experiment on Qwen3-235B-A22B-Instruct-2507; Atil et al.'s ten-run determinism study across five API-based models and eight tasks; and the OpenAI, Microsoft and Google Search Central documentation quoted above. All were checked against their primary pages on 10 September 2026.

Two limits are worth naming. First, the volatility figures come from Google's AI Mode and from general-purpose LLM inference; per-engine volatility differs, and a program should measure its own rather than inherit these. Second, the confidence-interval arithmetic assumes runs are independent draws, which repeated runs of the same prompt within a short window are not, strictly — correlated runs make the true interval wider than the table shows, not narrower. Treat those numbers as a floor on your uncertainty.

Related methods:

• What is citation rate — the definition and why executives track it
• Share of citation — the complementary share-based metric
• Citation Rate | MR Glossary — the canonical unit definition
• AI search visibility measurement framework — the full panel design
• Why AI citation studies disagree: the 2026 source register — provenance checks for any AI statistic