Website Owners Have Decreasing Control Over Their Presentation in Google’s Results ⇥ bloomberg.com

Julia Love and Davey Alba, Bloomberg:

Google now displays convenient artificial intelligence-based answers at the top of its search pages — meaning users may never click through to the websites whose data is being used to power those results. But many site owners say they can’t afford to block Google’s AI from summarizing their content.

[…]

Google uses a separate crawler for some AI products, such as its chatbot Gemini. But its main crawler, the Googlebot, serves both AI Overviews and Google search. A company spokesperson said Googlebot governs AI Overviews because AI and the company’s search engine are deeply entwined. The spokesperson added that its search results page shows information in a variety of formats, including images and graphics. Google also said publishers can block specific pages or parts of pages from appearing in AI Overviews in search results — but that would also likely bar those snippets from appearing across all of Google’s other search features, too, including web link listings.

I have quoted these two paragraphs in full because I think the difference between Google’s various A.I. products is worth clarifying. The effects of the Google-Extended control, which a publisher can treat as a separate user agent in robots.txt, is only relevant to training the Gemini and Vertex generative products. Gemini powers the A.I. overviews feature, but there is no way of opting out of overviews without entirely removing a site from Google’s indexing.

I can see why website owners would want to do this; I sympathize with the frustration of those profiled in this article. But Google has been distorting the presentation of results and reducing publishers’ control for years. In 2022, I was trying to find an article from my own site when I discovered Google had generated an incorrect pros and cons list from an iPad review I wrote. Google also generates its own titles and descriptions for results instead of relying on the page-defined title and meta description tags, and it has introduced features over the years like Featured Snippets, the spiritual predecessor of A.I. Overviews.

All of these things have reduced the amount of control website owners can have over how their site is presented on a Google results page. In some cases, they are often beneficial — rewritten titles and descriptions may reflect the actual subject of the page more accurately than one provided by some SEO expert. But in other cases, they end up making false claims cited to webpages. It happened with Featured Snippets, it happened with Google’s interpretation of my iPad review, and it happens with this artificially “intelligent” feature as well.