Google AI Summaries: Assertively Incorrect, But More Prominent Than Before


Google, the search engine, is undergoing one of its least successful years since 2015, with its market share dipping just below 90% nearly every month in 2025, as reported by Statcounter. Nevertheless, Google continues to hold a leading position in the search domain and strives to reinforce its status with additional AI Overviews.

Introduced in May 2024, Google AI Overviews are AI-generated summaries of approximately 250 words that appear at the top of select search results. The precise number of search results featuring AI Overviews remains undisclosed, but SE Ranking identified them in as much as 18% of queries in 2024, dropping to 7% shortly after due to peculiar outcomes like the “glue on pizza” incident.

In March 2025, Pew Research also found that 18% of searches contained AI Overviews. By May 2025, SE Ranking observed a sudden rise to nearly 30% of Google searches featuring AI Overviews.

This surge could be an anomaly. Google has not revealed how many results are assigned AI Overviews but asserts the product is advantageous. “As users engage with AI Overviews, we notice they’re more satisfied with their outcomes,” a Google representative mentioned, adding that users are posing more inquiries on Google.

Google provided no substantiation for this assertion. However, the Pew study indicated that users perceive Google AI Overviews favorably, or at least, they are 50% less inclined to click on links when an AI Overview is available. This decline in clicks has spawned what numerous online publishers deem a “traffic apocalypse.”

But should AI Overviews be regarded as more trustworthy than expert-referenced articles? With OpenAI’s new models demonstrating inaccuracies 40% of the time, this question is crucial.

Mashable previously assessed AI Overviews for inaccuracies in May and December 2024, concentrating on authentic searches. Here’s what was discovered in 2025.

How precise are Google AI Overviews? Don’t ask Google.

A Google spokesperson stated, “The vast majority of AI Overviews are highly accurate, and we’ve continually enhanced the usefulness and quality of responses.”

Nevertheless, the AI Overview itself exaggerates its error rate. A Google AI Overview asserted that AI Overviews are wrong 60% of the time, which is a misrepresentation according to Google.

A Mashable article referenced by the AI Overview involved a journalism experiment where AI chatbots were provided an article excerpt and tasked with naming the article and its author, with a 60% error rate. This bears no connection to inaccuracies in Google AI Overviews, which Google claims to be more precise as they are “rooted in search results.”

Google asserts that AI Overview precision is akin to featured snippets, which have also been recognized for inaccuracies, including a conspiracy theory about President Obama.

In my evaluations, merely 1 in 5 AI Overviews yielded inaccurate or misleading replies. Don’t be overly critical of yourself, Google AI Overviews!

Google AI Overviews can err when AI Mode gets it right.

When AI Overviews make mistakes, consequences can be substantial. For instance, currently trending among Doctor Who aficionados: the 15th Doctor (Ncuti Gatwa) regenerated into a figure resembling Rose Tyler (Billie Piper), although the BBC hasn’t confirmed Piper as the 16th Doctor. Google AI Overview incorrectly asserted that Piper was not the 16th Doctor and Gatwa remained the Doctor, despite official communications.

However, Google AI Mode offers a nuanced response, akin to a Doctor Who enthusiast’s answer.

Google declined an interview with a search executive but indicated that AI Mode inquiries require more computational power than AI Overviews. A Google spokesperson mentioned that AI Mode relies on the latest Gemini models, and over time, more results will surface in AI Overviews.

This suggests Google is consciously delivering less accurate results while a more correct answer is merely a click away.

AI Overview aims high but misses.

Regarding scientific facts, Google AI Overview claimed that NASA’s Artemis II mission would launch in September 2025, referencing a January 2024 NASA press release. However, a December 2024 press release shifted the target date to September 2026.

AI Overview is unaware of DOGE.

Searching “Elon Musk DOGE controversy” produced an AI overview about a Dogecoin lawsuit resolved in November 2024, overlooking more recent happenings involving the U.S. government.

These outcomes may not be entirely reflective. In earlier assessments, most AI Overviews provided satisfactory summaries. As a new vegan, I valued its knowledge of vegan pesto and innovative car dent removal techniques.

Is it detrimental that AI Overview occasionally hallucinates? Perspectives may differ. When questioned, AI Overviews withdrew, and Google presented a featured snippet from Abeba Birhane, AI Accountability advisor to the Mozilla Foundation, stating, “Google’s AI overviews hallucinate too excessively to be dependable.”

Whether AI Overviews or snippets represent the future of Google search, it appears either option may contradict official Google declarations.