Google A.I. Summaries and Search Traffic ⇥ pewresearch.org

Athena Chapekis and Anna Lieb, Pew Research Center:

Google users who encounter an AI summary are less likely to click on links to other websites than users who do not see one. Users who encountered an AI summary clicked on a traditional search result link in 8% of all visits. Those who did not encounter an AI summary clicked on a search result nearly twice as often (15% of visits).

Google users who encountered an AI summary also rarely clicked on a link in the summary itself. This occurred in just 1% of all visits to pages with such a summary.

I looked through this article and the methodology to see how this survey came together, since it seems to me the real question is if A.I. summaries are more or less damaging to search traffic than older features like snippets.

As far as I can figure out, the way Pew did this survey is that it looked for mentions of A.I. among users who consented to having their web browsing data tracked, and then categorized that traffic depending on whether it was a news article about A.I. or an A.I. feature being used. Any Google data without an A.I. summary was, as far as I can see, categorized as not containing an A.I. summary. But this latter category amounted to 82% of all Google searches, and there does not appear to be any differentiation in what features were shown for those. Some may have snippets; others may have some other “zero-click” feature. Some may have no such features at all. Lumping all those together makes it impossible to tell what impact A.I. summaries are having on search compared to Google’s previous attempts to keep users in its bubble.

This survey does a good job of showing how irrelevant the source links are in Google A.I. summaries to search traffic. Much like the citations at the end of a book, they serve as an indicator of something being referenced, but there is no expectation anyone will actually read it to confirm whether the information is accurate. There was such a citation to a Microsoft article ostensibly containing an Excel feature Google made up. Unlike citations in a book, Google’s A.I. summaries are entirely the product of a machine built by people who have only some idea of the output.