The Maddening Adventure of Tracking AI Overviews in Google Search Console (GSC)

Google launched AI overviews in the SERPs in the United States officially on May 14th at I/O. This follows the launch of Google’s Search Generative Experience (SGE) in May of 2023. And with the US launch recently, Google explained AI overviews data would be available in Google Search Console, but that data would not be broken out separately. Therefore, you cannot apply a filter to see just AI overview data in Search Console. That’s extremely frustrating but does match what Google is doing with featured snippets.

Note, Google toyed with the idea of providing a featured snippet filter in GSC in 2016. I remember seeing that show up in my reporting since I was part of the beta, and I was excited to see that roll out. Unfortunately, it never did roll out… That said, featured snippets are a bit easier to track in several ways. First, they rank in the number one position and usually without other listings mixed in. Second, third-party tracking tools have gotten very good at tracking featured snippets. For example, Semrush has a featured snippet filter so you can view all featured snippets captured by their systems for a specific domain. Now, that doesn’t give you actual traffic numbers, CTR data, etc., but you can at least understand when your site ranks in a featured snippet.

Challenges to tracking AI Overviews:

There are several challenges site owners face when trying to track AI overviews. I’ll cover four below.

First, they only trigger in the United States for now. That will expand over time, but for now, it’s just US.

Second, they only trigger for logged in users. So they don’t appear when searching incognito.

Third, they are dynamic. I can often see differences when searching in accounts with SGE active in labs versus other accounts without SGE active. Both types of accounts can trigger AI overviews, but there are differences between the two. In other words, the account with the labs experiment for SGE active can show variations in AI overviews, or show an overview when typical US users don’t see overviews. I saw that on Friday several times actually.

Fourth, Google is heavily refining how AI overviews work, refining when AI overviews trigger, the sources being cited in link cards, and more. We saw that recently with all of the health and medical craziness that was being reported across the web. With the Ray update (named after the one and only Lily Ray), now the most authoritative health and medical sites often show up in link cards for many YMYL queries related to health. That wasn’t the case before the update…

For example, here’s a tweet of mine where I cover parked domains ranking without favicons before the Ray update for the YMYL query, “how to lose weight fast”:

I mentioned yesterday that Google should turn off AI overviews for YMYL queries until they can improve the situation, but I still see those overviews for MANY health/medical queries. Some are totally fine (even good), and some are not fine. And check these two link cards out for… https://t.co/PcmAiX88Cq pic.twitter.com/WQgDPDbWVI

— Glenn Gabe (@glenngabe) May 26, 2024

After the update, here are the link cards:

Ziptie + GSC: Tracking AI Overviews in the SERPs

I’ve been using Ziptie to track AI overviews for queries across a number of sites. It’s been very helpful in understanding when an AI overview triggers for a query, which websites are cited in link cards, tracking those changes over time, and more. It also provides rankings in the 10-blue links for sites you are tracking. This has helped me dig into specific examples where I should be able to track AI overviews in GSC. For example, queries where a site ranks lower on page one, but ranks in the AI overview (top of page one). I emphasized “should” earlier because I really can’t track AI overviews well. It’s incredibly frustrating and that’s the core point of this post.

Here’s a quick example. For one of the queries I’m tracking, the page ranks lower on page one in the 10-blue links, but ranks highly in the AI overview. Ziptie picks up the site ranking #7 in the SERPs and GSC shows an average position of about #9. I am filtering by just United States data since AI overviews have only rolled out in the US.

Note, the rank in GSC for link cards in an AI overview should technically be #1 since it’s part of the AI overview block that ranks first in the SERP. So, I should be able to see a higher rank in the United States once AI overviews rolled out. But when digging into the query in GSC (and especially since 5/14), it’s muddy. You cannot see a distinct difference there. It moves up a bit recently position-wise, but it’s still not extremely clear. Also, CTR hasn’t changed at all and clicks are pretty stable. And hey, maybe Google loves that I can’t see the difference in CTR and traffic when the site ranks in the AI overview.

Here’s another example where the site ranks #5 in the standard search results, but ranks well in the AI overview. First, position actually drops recently. What?? And then CTR and clicks are stable. Again, I cannot see a difference when analyzing that query in the United States.

Yep, this is incredibly frustrating. It’s also worth noting that even Google’s Elizabeth Tucker doesn’t have access to a specific AI overview filter or report in GSC. Barry Schwartz interviewed Elizabeth Tucker at SMX Advanced last week and she explained that during their conversation. That’s saying something given her role at Google, and it doesn’t bode well for site owners receiving an AI overview filter in GSC.

Moving forward: Google, GIVE US AI OVERVIEW DATA!

AI overviews are probably here to stay. Google is serious about providing those AI overviews and recently explained they should keep taking risks with new features as long as they carefully track and fix errors. That’s fine, but site owners really need to understand the impact from ranking in AI overviews. For example, are they driving a higher click through rate, more traffic, etc.? Heck, are they driving traffic at all? These are important questions that site owners want to know as AI overviews continue to roll out.

Also, based on the data, some site owners might even want to remove their content from AI overviews. I’m not saying to do that, but not every situation is the same. There may be some situations where site owners do want to remove their content or links from AI overviews. You can read my post about how to remove content from AI overviews if you are interested in doing that.

So, it would be amazing if Google provided a filter in GSC to view AI overview data or if they provided separate reporting like they do for Discover and Google News. Google said it’s serious about supporting publishers and maintaining a strong web ecosystem. Well, part of supporting publishers is providing data based on how their content is ranking across Search. If Google is serious about AI overviews, then let site owners be serious about tracking those overviews. Right now, that tracking is severely limited…

A simple filter like this would be incredible:

Or even separate reporting like Discover or Google News:

I’ll post more about tracking AI overviews as the situation evolves. But for now, I’ll continue the maddening process of trying to surface AI overview data in Search Console. And if you don’t hear from me in a few days, send help. 🙂