The extra outbound clicks that websites get when Google’s AI Overviews are removed seem to be of similar quality to the clicks with the AIOs still included. This finding comes from an updated, randomized field experiment, which showed a 39.8% decrease in organic clicks when the summaries are shown.
Researchers Saharsh Agarwal and Ananya Sen revised their working paper to include new analyses of click quality, treatment switches, and query types. The paper reported a 38% reduction when SEJ first covered it in April.
Revised figures show outbound clicks per search increasing and zero-click searches rising when AI Overviews are shown. Because AI Overviews were triggered on about 41% of queries, that amounts to a measurable reduction in outbound clicks across all searches.
What The Click Quality Data Shows
For clicks on queries where an AI Overview would have appeared, the authors compare how often people hit the back button to return to the results page, how often a visit ended within 10 seconds without interaction, and the total time spent on the destination site.
None of the three showed a statistically significant difference between groups. About 4 in 10 same-tab clicks led back to the results page in both conditions, roughly 18% of visits ended within 10 seconds, and time on site was statistically indistinguishable.
Google VP of Search, Liz Reid, has said that AI Overviews reduce “bounce clicks” but hasn’t released data to support the claim. The experiment gets at the same question from the other side. If the summaries absorbed mostly low-value visits, the additional clicks in the no-Overview group should look worse, but they don’t.
The authors write the result is “at odds with the view that AIOs primarily eliminate low-engagement website visits.”
Disclosure: the revised paper cites two Search Engine Journal articles in its discussion of Google’s claims.
Behavior Reversed When The Groups Switched
After the initial two-week period, the researchers rotated the assignments around.
Participants who were served AI Overviews saw their external clicks per search decrease, while those who no longer received them saw their external clicks per search go up. Interestingly, the zero-click rates mirrored each other.
Informational Queries Carry The Effect
When the authors examined the data by query type, they found that losses are concentrated in informational searches, while removing AI Overviews increased clicks per search. Navigational and transactional queries showed no measurable change, though those samples were smaller.
AI Overviews were triggered or intended to trigger on 53% of informational queries, compared with 15% of navigational and 6% of transactional ones. A position breakdown adds that the top three ranked results gained the most clicks when a top-of-page Overview was removed, with position one nearly doubling.
Why This Matters
Google’s answer to questions about lost traffic has leaned on increased click quality. This experiment tested the claim and found no measurable difference.
Looking Ahead
The authors write that aggregate traffic losses could grow if AI Overviews appear on more queries over time. The paper is a working draft on SSRN and hasn’t completed peer review.
Featured Image: takasu/Shutterstock
