Google's AI Answers Contradict Their Own Sources 56% of the Time. Here's the Brand Audit.
A new NYT/Oumi benchmark found AI Overviews contradicts its own cited sources 56% of the time. Christian Lehman breaks down the 5-query brand audit that catches misrepresentation before your buyers see it.
Google's AI Overviews now contradicts its own cited sources 56% of the time, according to a new analysis by The New York Times and AI startup Oumi. That means even when your page is the source it links to, the summary searchers actually read may say something different. The system is 91% factually accurate on the SimpleQA benchmark, but at Google's scale of 5+ trillion annual searches, the 9% miss rate generates roughly 57 million wrong answers per hour. The real problem for brand operators isn't whether AI mentions you. It's whether AI accurately represents you when it does.
The source contradiction rate is worse than the accuracy rate
Most coverage of the NYT study focuses on the 91% accuracy figure. That number is misleading in isolation.
The Oumi analysis found that 56% of AI Overviews summaries contained information that did not match the sources they cited -- up from 37% under the previous Gemini 2.5 model (NYT/Oumi, April 2026). Google improved factual accuracy by upgrading to Gemini 3, but the trade-off was a system that sounds more confident while more frequently misrepresenting the pages it links to.
| Metric | Gemini 2.5 | Gemini 3 | Change |
|---|---|---|---|
| SimpleQA accuracy | 85% | 91% | +6 pts |
| Source contradiction rate | 37% | 56% | +19 pts |
| Estimated wrong answers/hour | ~42M | ~57M | +36% |
Christian Lehman's read on this: accuracy improvements mean nothing to your brand if the system is now more likely to cite your page and then say something different about it. The summary above the fold is what buyers read. 80% of search users rely on AI summaries for at least 40% of their searches, according to Bain. Click-through rates drop 60-70% when an AI summary is present, per Pew Research. Your buyers are reading the misrepresentation and not clicking through to verify it.
Why your brand queries get the least accurate model
Google doesn't run the same model on every query. As Ars Technica reported, the company uses faster, cheaper Gemini Flash models for most queries and reserves the full Gemini 3.1 Pro model for complex ones.
Most brand queries are classified as common. "Best project management tool for remote teams," "[your brand] vs [competitor]," "[category] solutions 2026" -- these all get the Flash treatment. The queries your buyers are actually running receive the less accurate, more contradiction-prone model.
The system also picks up new sources fast. The NYT analysis documented a case where a journalist published a blog containing fabricated information. AI Overviews cited it as a source within 24 hours. The same pipeline that makes Google responsive to fresh content makes it vulnerable to source pollution. Most brands have no monitoring in place to catch it.
The 5-query brand accuracy audit
Christian Lehman breaks down the audit in five queries. Run each in Google, read the AI Overview, then open every cited source and compare what the summary says against what the page actually says.
1. "[Your brand] vs [top competitor]" Check whether the AI summary accurately states your differentiators. Contradictions here typically flatten the comparison into generic phrasing that makes you interchangeable with competitors.
2. "Best [your category] for [specific use case]" Check whether claims about your product are current. AI Overviews frequently cites pages with outdated pricing, deprecated features, or old reviews.
3. "[Your brand] pricing" or "[your brand] reviews" Check pricing accuracy and review representativeness. Contradiction in commercial queries directly impacts purchase decisions downstream.
4. "[Your industry] solutions [current year]" Check whether you're positioned in the right category. Misclassification is common. AI might describe your analytics platform as a CRM because one source used that framing.
5. "[Your brand name]" -- naked brand search Read the full AI Overview summary. Does it describe what you actually do, or has it synthesized conflicting sources into something no one at your company would recognize?
For each query, document what the AI says, what the cited source says, and where they diverge. The contradictions you find are the AI visibility gaps actively shaping buyer perception without your input.
The fix is source consistency, not technical SEO
You cannot submit corrections to AI Overviews. There is no edit button, no dispute process, no feedback form that changes the output reliably. The fix is upstream: the quality and consistency of the sources AI pulls from.
When multiple authoritative sources say the same thing about your brand, AI summaries converge toward accuracy. When your only sources are your own website, a handful of listicles, and a Reddit thread from 2024, the system has nothing consistent to synthesize and fills the gap with inference.
The data is specific. 82% of all links cited by AI engines are earned media, not brand-owned content, according to Muck Rack's Generative Pulse analysis. AuthorityTech's research on AI search brand strategy found earned media generates 325% more AI citations than owned distribution. And the publications AI engines cite most in B2B -- TechCrunch, Forbes, Reuters -- are all earned media outlets.
The pattern: brands with consistent earned media presence in publications AI engines index get more accurate representation. Brands relying on owned content alone absorb the full 56% contradiction rate. As Jaxon Parrott outlined in his breakdown of how earned media drives AI search visibility, the mechanism is the same one that made PR valuable for decades -- third-party credibility in publications that both human and machine readers trust.
This is the infrastructure layer that Machine Relations addresses -- the discipline of building a citation architecture where AI engines have enough consistent, third-party source material to represent your brand accurately. Not because you gamed a prompt, but because your editorial presence gave the system something reliable to synthesize.
FAQ
How often should I audit what AI says about my brand? Weekly for your top 5 commercial queries. Monthly for the full 15-20 query set. AI Overviews sources refresh continuously. A correct summary today can contradict itself next week if a new, conflicting source enters the index.
Can I submit corrections to Google AI Overviews? No. There is no direct correction mechanism. The only reliable fix is improving source quality: consistent messaging across authoritative third-party publications that AI already indexes and trusts.
What type of content reduces AI misrepresentation? Earned media placements with specific, consistent claims about your brand in high-authority publications. AI engines cross-reference sources. When multiple trusted outlets make the same claim, the system converges. When sources conflict, you get the 56% contradiction rate.
Related Reading
- AI Visibility for HR Tech Companies: How People Platforms Get Cited in Enterprise AI Search
- AI Visibility for Fintech Companies: How to Get Cited by ChatGPT, Perplexity, and AI Search
Run the 5-query audit this week. If what AI says about your brand doesn't match reality, the gap is already costing you pipeline. Start with the AuthorityTech visibility audit to see exactly how AI engines represent you today.