How Citation Rate Is Actually Calculated — And Why Denominators Give Different Numbers

Citation rate is the number of eligible AI answer runs that cite your domain, divided by every eligible answer run in the panel, times 100.

Citation Rate = (eligible answer runs with ≥1 qualifying citation to your domain
                 ÷ all eligible answer runs) × 100

Count each run once in the numerator even if the answer links to you three times. Put every eligible run in the denominator, including the runs that cited nobody. Compute it per engine.

That is the formula. It is also the least interesting part of the answer, because the formula is not where citation rates go wrong. Three things decide what your number actually means, and none of them is the arithmetic: which denominator you chose, how many times you ran each prompt, and whether the engine retrieved anything at all.

Four denominator choices are not interchangeable

These are four mathematical ways to summarise a panel, not a survey of current vendor implementations. Each needs its own eligibility and weighting rules. They cannot be interchanged without the underlying log.

Unit of observation	Formula	What breaks if you swap it in
Answer runs	runs citing your domain ÷ all eligible runs	Needs the run count and repeat weighting declared.
Answers	answers citing your domain ÷ total eligible answers	Equivalent to runs when each eligible run produces one answer. Declare how errors and refusals affect eligibility.
Prompts	prompts with ≥1 citing run ÷ all tracked prompts	With equal repeats per prompt, coverage is greater than or equal to run rate; with unequal repeats, it can be lower. In a hypothetical example, one citing run out of five for one prompt gives 100% coverage and 20% run rate.
Responses containing any citation	eligible responses with ≥1 qualifying citation to your domain ÷ eligible responses with ≥1 qualifying citation to any domain	A conditional-domain rate: it excludes citation-free answers from the denominator. It is undefined if no eligible response contains a qualifying citation.

Do not confuse the last row with any-citation frequency: eligible responses with ≥1 qualifying citation to any domain ÷ all eligible responses. That measures how often an answer cites anything, not how often it cites your domain.

Citation share is a different metric entirely: your cited URLs divided by all cited URLs in the sample. Share falls when engines start citing more sources per answer, even if you were cited in every single one. It answers "how much of the cited authority do we own," not "how often are we cited."

The Machine Relations glossary definition uses the first row: eligible answer runs, with numerator and denominator reported alongside the percentage. That is the unit we recommend, because it is the only one that degrades gracefully — you can always aggregate runs up to prompts, but you can never recover runs from a prompt-level number.

The practical consequence is blunt. A citation rate quoted without its denominator is not a low-precision number. It is an unreadable one. You cannot tell whether 30% means "we appeared in 30% of 600 answers" or "we appeared somewhere in 30% of 40 prompts," and those describe different businesses.

Citation rate can be a compound of two probabilities

When retrieval state is observable and a qualifying citation requires that retrieval event, the rate decomposes as:

Pr(cited) = Pr(the engine retrieved at all) × Pr(cited | it retrieved)

Answer engines do not search on every turn. An eligible answer with no qualifying citation still belongs in the denominator. But missing retrieval telemetry is not proof that no retrieval occurred: record the state as unknown rather than assuming a hidden backend event did not happen.

Prominara's published methodology describes recording retrieval state and splitting the rate. Where the assumptions above hold, a drop can reflect different changes:

• Retrieval fell. Fewer runs show search for that query class. Investigate engine, route and prompt-set changes before prescribing a content fix.
• Selection fell. Retrieval occurred as often as before, but your domain appeared less often within those runs. Investigate content, freshness and source selection rather than assuming a single cause.

A blended percentage alone cannot distinguish these changes or establish what caused them.

There is a further wrinkle inside the retrieval term. Google states in its Search Central documentation that both AI Overviews and AI Mode may use a "query fan-out" technique — issuing multiple related searches across subtopics and data sources — before composing a response. Most major answer engines do something equivalent. So a single "run" is not one retrieval; it is a bundle of synthetic searches you never see and cannot enumerate. Your page can win three of the hidden branches and still not appear, or win one and appear. The run remains the correct unit of observation — it is what the user experiences — but nobody should mistake it for the unit of competition.

Every citation rate is a sample estimate, and the sample is noisier than you think

AI answers are not stable readings. The same prompt, asked again, cites different sources — and the source list moves independently of the prose.

SE Ranking's June 2025 study tested the same 10,000 keywords in Google's AI Mode three times on the same day; 9,451 keywords received responses in all three rounds. For those common keywords, average overlap across the three datasets was 9.2% for exact URLs and 14.7% for domains, and 21.2% of queries shared no URLs at all between runs. These are AI Mode dataset-overlap statistics, not a cited URL's survival probability on one rerun or a probability for every engine.

This is not an artifact of sloppy sampling. It is what the underlying systems do. Thinking Machines Lab sampled 1,000 completions of a single prompt at temperature 0 — greedy decoding, no deliberate randomness — and got 80 unique completions. The outputs were identical for the first 102 tokens, then split at token 103: 992 completions said "Queens, New York," and 8 said "New York City." The cause was not sampling temperature but batch-invariance in the inference kernels — server load changes the batch shape, the batch shape changes floating-point reduction order, and a flipped argmax cascades through everything after it.

The peer-reviewed picture agrees. Atil et al., in Non-Determinism of "Deterministic" LLM Settings — published at Eval4NLP 2025 — configured five API-based models to be deterministic across eight tasks and ten runs, and observed accuracy variation of up to 15% across runs, with a gap of up to 70% between best- and worst-possible performance. No model produced repeatable accuracy, let alone identical strings.

The providers say so themselves. OpenAI's guidance on the seed parameter states that the system will make a best effort to sample deterministically and that "determinism is not guaranteed". Microsoft's Azure OpenAI documentation is blunter: even with the seed and system_fingerprint matching across calls, it is "currently not uncommon to still observe a degree of variability in responses".

Add live retrieval on top of that and the source list moves for a second, independent reason: the index changed between your two runs.

A one-run citation rate is not a measurement. It is a draw.

How many runs you actually need

For independent Bernoulli draws with a common citation probability, a normal-approximation interval for a proportion is:

SE = √( p × (1 − p) ÷ n )        95% CI ≈ p ± 1.96 × SE

Where p is the observed rate and n is the number of eligible runs. The table uses a hypothetical 25% rate to illustrate scale, not actual counts: 25% is not an obtainable k/n at n = 30. These approximations assume independent draws, do not account for repeated-prompt or run-date dependence, and are unsuitable with few successes or failures or rates near 0% or 100%. For dependent panels, use a design-aware interval; the MR run-date-clustering note explains why the design matters.

Eligible runs (n)	Hypothetical rate	Approximate 95% margin
30	25%	±15.5 points
100	25%	±8.5 points
300	25%	±4.9 points
1,000	25%	±2.7 points

Read the top row carefully. Under the stated assumptions, thirty runs at an assumed 25% rate give roughly ±15.5 points of uncertainty. A small observed difference needs a comparison fitted to the sampling design; overlapping separate intervals do not prove that nothing changed.

Three rules follow directly from the arithmetic:

1. Size the sample to the claim you want to make. Detecting a 2-point move needs a sample several times larger than detecting a 10-point move. Decide what change is worth acting on before you choose n.
2. Report per engine first. Retrieval behaviour and citation density differ per engine — Google says outright that AI Mode and AI Overviews "may use different models and techniques, so the set of responses and links they show will vary", and those are two surfaces from the same company. A predeclared pooled rate can describe the observed run mix, but it is neither any single engine's rate nor a consumer-app probability. It can move when the engine mix moves even if every engine holds steady.
3. Freeze the prompt set inside a window. Adding prompts mid-period changes the denominator and manufactures a trend out of nothing. Batch prompt changes and date them.

Report the interval and its assumptions, or at minimum report n beside the percentage. In a hypothetical independent-draw example, "18 of 60 runs (30%, approximate 95% margin ±11.6 points)" carries more information than "30%".

The counting rules that silently change the number

Before any of the above matters, six decisions have to be written down, because each moves the result by more than most content work does:

• Domain or URL. SE Ranking's AI Mode study reported average overlap across its three datasets of 14.7% for domains against 9.2% for exact URLs. Declare which unit you count; domain overlap and exact-URL overlap are different measurements.
• Brand mention or linked citation. A mention with no link is presence, not citation. Mixing them produces a number that is neither.
• Deduplication within a run. If the same URL appears in the answer body and in the sidebar, that is one citation for that run. Counting both inflates share-based metrics and leaves run-based metrics unchanged — another reason the two must never be compared.
• Eligibility and exclusions. Define before collection what disqualifies a run: engine error, refusal, no answer generated. Excluding runs after seeing whether they cited you is how a measurement program starts lying to itself.
• Personalisation and locale. SE Ranking's local test found general local queries repeat only about 35% of domains and 19% of URLs within a single city, while adding an explicit city name to the query pushed domain repetition to roughly 55%. Locale is not a reporting dimension you add later; it is part of the instrument.
• Retrieval state. Whether the engine searched, per run, as covered above; keep unavailable telemetry marked unknown.

Write these down as a versioned protocol. If the protocol is not versioned, your time series is not a time series — it is a sequence of unrelated experiments sharing an axis label.

A worked example

In this hypothetical example, a B2B software company runs 40 frozen prompts across three engines, five runs each, weekly. That is 600 eligible runs per week, 200 per engine. The approximate intervals below assume independent draws with a stable probability within each engine; actual repeated-prompt and run-date dependence needs a design-aware calculation.

Week one, the answers cite their domain in 46 of 200 ChatGPT runs, 71 of 200 Perplexity runs, and 12 of 200 AI Mode runs.

• ChatGPT citation rate: 46 ÷ 200 = 23%, 95% CI ±5.8
• Perplexity citation rate: 71 ÷ 200 = 35.5%, 95% CI ±6.6
• AI Mode citation rate: 12 ÷ 200 = 6%, 95% CI ±3.3
• Pooled: 129 ÷ 600 = 21.5% — a descriptive rate for this predeclared observed run mix, not any one engine's rate or a consumer-app probability

Week two, ChatGPT cites the domain in 39 of 200 runs, or 19.5%. The observed sample rate fell 3.5 percentage points. Under a simplifying independent-sample comparison, that is inconclusive evidence of a persistent underlying change, not proof that nothing happened. Separate interval overlap is not a change test; matched prompts and run-date dependence require a comparison that reflects the actual design before diagnosing a content problem.

Now suppose retrieval is fully observed: 150 of the 200 AI Mode runs did not trigger it, and all 12 citing runs are among the 50 that did. The retrieval-conditioned domain rate is 12 ÷ 50 = 24%, alongside the unconditional 12 ÷ 200 = 6%. The 24% cannot be compared with ChatGPT's unconditional 23%: compare like denominators, and obtain ChatGPT's retrieval-conditioned rate before making that comparison. The split helps investigate retrieval and source selection; it neither explains why retrieval occurred less often nor rules out a content problem. If retrieval state is unavailable, keep it unknown rather than treating those runs as no retrieval.

Same hypothetical log. Different denominators. Different questions for the team.

A live example of the distortion, caught while writing this

While researching this article, we found a secondary source attributing to SE Ranking the claim that "only 35% of cited domains repeat between runs" of the same prompt in Google AI Mode, presented as a general finding.

The 35% figure is real and it is SE Ranking's. It comes from their local search volatility test, and it describes general local queries — "restaurants near me" — repeated within a single city. SE Ranking's general AI Mode study reports 14.7% average domain overlap across three same-day datasets for the common keywords, not 35%.

So a correct number from a real study was transported onto a population it does not describe, and in the direction that makes AI answers look more than twice as stable as the general evidence says they are. Nobody fabricated anything. The population simply fell off during the journey.

This is the same failure pattern catalogued in the 2026 source register, and it is why the third provenance check — does the source contain this figure with this denominator? — catches things that link-checking never will.

The measurement contract

Before a citation rate is allowed to travel — into a board deck, a vendor comparison, or a budget argument — it should be expressible in one sentence:

In [engine], across [N eligible runs] from [frozen prompt set, version], during [window], [domain] was cited in [k] runs: [k/N]%, ±[interval], with retrieval observed in [r] runs and its state unknown in [u] runs.

If the sentence cannot be completed, the number is not ready to leave the room.

This is what separates a citation rate from a vanity metric. Not precision — the underlying system is genuinely noisy, and a well-run program still reports intervals several points wide. What separates them is that an honest citation rate carries its own instrument with it, so the next person can tell what changed: your content, the engine's retrieval behaviour, or the sample.

Method and limitations

The rate definitions are arithmetic ratios; the confidence intervals are approximations under the stated assumptions. The denominator table compares mathematical alternatives, not verified current vendor implementations; its four units are not interchangeable. The retrieval decomposition assumes a qualifying citation requires the observed retrieval event. The empirical inputs are: SE Ranking's June 2025 AI Mode study of 10,000 tested keywords, with 9,451 common to all three same-day response datasets; SE Ranking's local AI Mode volatility test; Thinking Machines Lab's 1,000-completion temperature-0 experiment on Qwen3-235B-A22B-Instruct-2507; Atil et al.'s ten-run determinism study across five API-based models and eight tasks; and the OpenAI, Microsoft and Google Search Central documentation quoted above. All were checked against their primary pages on 10 September 2026.

Two limits are worth naming. First, the volatility figures come from Google's AI Mode and from general-purpose LLM inference; per-engine volatility differs, and a program should measure its own rather than inherit these. Second, the confidence-interval arithmetic assumes independent draws with a common probability, not a guarantee for repeated runs of the same prompt within a short window. Positive within-prompt or run-date correlation can widen uncertainty; use an interval fitted to the actual design rather than treating the illustrative table as a guaranteed bound.

Related methods:

• What is citation rate — the definition and why executives track it
• Share of citation — the complementary share-based metric
• Citation Rate | MR Glossary — the canonical unit definition
• AI search visibility measurement framework — the full panel design
• Why AI citation studies disagree: the 2026 source register — provenance checks for any AI statistic