AI Overviews: Initial Tests and Studies

Google went public with AI Overviews in late May. I’ve been tracking queries and third-party studies since then.

AI Overviews occasionally produces incorrect answers. An account on X, Goog Enough, curates some of these errors. To its credit, Google has fixed many glitches, mainly by limiting the occurrence of Overviews and excluding user-generated content from the sources.

Here’s what we know thus far in AI Overview’s rollout.

15% of Queries

An article last week in Search Engine Land cited a study from BrightEdge, the search engine optimization firm, that found Overviews now shows for just 15% of queries. This aligns with my observations for queries I am following. Others in the industry have similar experiences.

Public AI Overviews only show for U.S.-based searchers signed in to Google accounts. Hence the visibility is likely even lower.

Labs vs. Public

Searchers who signed up for the Labs version of Search Generative Experience — the predecessor of AI Overviews — may see Labs-only results. To access public results, those searchers must opt out of Labs or create a Google profile in Chrome to test both.

I tested the query “how to choose a career.” When logged-in to my Labs profile, I see an Overview with the notation “Search Labs” above it.

Search Labs participants see a unique notation above the Overview. Click image to enlarge.

When using my non-Labs profile, I see the same AI Overviews with visible sources.

Public, non-Labs searchers see sources below the Overview. Click image to enlarge.

Optimizing for AI Overviews

In testing SGE results before the public rollout of Overviews, we knew getting a site listed in an AI answer wasn’t too difficult. From Google’s 2023 SGE patent:

• AI answers in Google summarize existing search results
• Google generates references (i.e., links in the answers) after creating the summaries.

The relative ease apparently remains. A simple test by search optimizer Cyrus Shepard immediately generated a link in AI Overviews. He first identified a query that produced an Overview for which his page ranked organically with a featured snippet. He then updated that page with a paragraph closely matching the text in the Overview. A link to his page appeared right away.

However, Shepard listed two caveats:

• His page lost the featured snippet.
• The link in the Overview disappeared quickly.

The link disappeared too soon for meaningful conclusions, such as the impact on featured snippets. However, the rapid change — in and then out of Overviews — suggests the fluidity of AI answers and caution in implementing new optimization tactics. Google is changing AI Overviews seemingly daily. While short-lived gains are possible, there’s too little data for long-term action.