SEOSEO News

Google’s AI Overviews & YMYL Topics: September 2024 Research


Certain topics have the potential to significantly affect a user’s health, financial stability, or safety. Google refers to these as Your Money or Your Life (YMYL) topics.

Because of their sensitive nature, YMYL topics require the highest standards of trustworthiness and accuracy. And since AI Overviews (AIOs) are now playing a crucial role in determining what information users first see in Google search, the SE Ranking team believes it’s important to understand how AIOs impact people’s opinions on such critical matters.

So, with this round of research, we aim to analyze how AI Overviews display information on four essential YMYL areas—health, politics, finance, and the legal niche. We’ll provide insights into how often AIOs appear for these topics, how credible the sources featured are, and the extent to which Google ensures AIOs are accurate, non-harmful, and free from misinformation.

To support this analysis, we used our recently launched AI Overviews Tracker. This tool allows you to monitor AI Overviews for your target keywords, evaluate the sources featured in these snippets, and see whether your website’s link is included in AIOs. We’ve also added the ability to detect an AI Overview SERP feature in our Keyword and Competitive Research tools.

Now, without further ado, let’s move on to the results of our research!

  • A total of 50.33% of keywords (604 out of 1,200 queries) within YMYL categories result in AI Overviews.

  • The legal niche triggers the highest percentage of AIOs (77.67%), followed by the health sector (65.33%), finance (41.67%), and politics (16.67%).

  • The average number of links matched between the AI Overview resources and the top 20 search results was 7.44 for political topics, 7.13 for health topics, 6.49 for legal topics, and 5.96 for finance topics.

  • The most common keyword patterns triggering AIOs for YMYL topics are how (how to, how often, how long, how much, how does), what is, what are, when, you, I.

  • Health-related AI Overviews most frequently link to Mayoclinic.com (107 links), YouTube.com (104 links), and WebMD (91 links).

  • 83% of health-related keywords triggering AIOs contained a disclaimer: “This is for informational purposes only. For medical advice or diagnosis, consult a professional. Generative AI is experimental.”

  • Google takes extra care with sensitive topics such as mental health, eating disorders, substance abuse, and specific medications, opting not to generate AIOs for these subjects.

  • The most linked-to sites in AIOs for political topics are Wikipedia.org (36 links), State.gov (15 links), and DHS.gov (15 links).

  • No AI Overviews were triggered for keywords containing terms like “election,” “elections,” “president,” or “presidential.”

  • In the finance category, Investopedia.com (68 links), Nerdwallet.com (57 links), and Bankrate.com (56 links) are the most frequently referenced websites in AIOs.

  • 63.2% of finance-related keywords triggering AIOs included a disclaimer: “Generative AI is experimental. For financial (legal) advice, consult a professional.”

  • AI Overviews for legal topics most frequently link to NYCourts.gov (114 links), YouTube.com (48 links), and FindLaw.com (46 links).

  • 19.74% of keywords that triggered AI Overviews for legal topics featured a disclaimer stating: “Generative AI is experimental. For legal advice, consult a professional.”

  • Google’s AI Overviews for YMYL topics generally provide accurate information, but there is potential for enhancement in addressing geographic inconsistencies, incomplete content, and reliance on outdated/irrelevant sources.

Disclaimer:

This study examined how AI Overviews present information on YMYL topics. The data gathered was influenced by the keywords selected and the queries that triggered AIOs, among other factors. Results may fluctuate depending on time and search parameters.

Our research was based on data from September 15; however, we also offer weekly comparisons, as our AI Overviews Tracker tracks AIOs on a daily basis.

For a detailed explanation of our research methodology, please refer to the dedicated section at the end of the article.

How AI Overviews are impacting YMYL topics

YMYL stands for “Your Money or Your Life.” This term comes from Google’s guidelines for quality evaluators.

These are topics that have the potential to impact a person’s financial stability, health, safety, or happiness. Examples include medical advice, financial planning, legal matters, news related to major societal issues, and so on. Because these topics are so important, Google has much higher standards for pages that cover YMYL topics. 

This means that AI Overviews should also adhere to these high standards when addressing YMYL topics.

The thing is, AIOs have high prominence and visibility in SERPs, usually appearing right above organic search results. This prominent placement means that users pay a lot of attention to AIOs and often trust the information they provide, even on sensitive YMYL topics.

AIO for fever query

Because AI Overviews are highly visible and can shape users’ perceptions, it’s crucial that they deliver accurate, reliable, and trustworthy information to ensure users make well-informed decisions, especially on topics that significantly impact their lives.

Let’s look closely at our findings to see how AI Overviews are triggered in practice.

Overall insights on AI Overview performance for YMYL topics

Out of the 1,200 total keywords tracked in this analysis, 604 triggered AI Overviews, making up 50.33% of the total. 

  • The legal niche had the highest AIO trigger rate, with 233 out of 300 keywords (77.67%) activating AIOs. 
  • The health niche saw 196 out of 300 keywords (65.33%) trigger AIOs. 
  • The finance niche had a lower AIO trigger rate, with only 125 out of 300 keywords (41.67%) activating AIOs. 
  • The political niche had the lowest AIO trigger rate, with only 50 out of 300 keywords (16.67%) activating AIOs. 
AI Overviews across YMYL niches

But how do sources linked within AI Overviews compare to organic search results?

On average, there was an overlap of 6.67 links between AIOs and the top 20 organic results across all four niches. The range of overlap varied:

  • The lowest overlap was 0 (no matching links between an AIO and the top 20), which occurred with 11 keywords.
  • The highest overlap was 14 links, observed for just one keyword.
  • The most common overlap occurred with 8 links, found in 122 out of 604 keywords, representing 20.2% of cases.

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

The number of keywords that triggered AIOs with links matching the top 20

The ratio of the total number of keywords that triggered AIOs (604)

Here’s our list of the top 10 most commonly referenced websites in AI Overviews across the health, political, finance, and legal niches.

Top linked websites in AIOs across YMYL niches

YouTube leads with 190 links, followed by Investopedia (116 links), Mayo Clinic (107 links) and WebMD (91 links). Additionally, general knowledge sources like Wikipedia (89 links) and Forbes (87 links) are prominently featured.

Now, let’s break down the performance of AI Overviews for YMYL topics by niche.

Health niche

Health-related content, due to its potential to impact individual well-being, falls squarely into the YMYL category. This highlights the need for AI Overviews to provide reliable and helpful guidance to users.

How often do AI Overviews appear for health topics?

Our analysis shows that AI Overviews were triggered by 196 of the 300 health-related keywords, which amounts to 65.33%.

AIOs for health topics

To monitor how AIO presence has evolved, we added the whole set of keywords to our AI Overviews Tracker for daily monitoring. We then revisited these keywords on September 22nd to assess the changes that had occurred over the prior week. 

The results show that 198 keywords, which equals to 66%, triggered AIOs on September 22. This represents the peak number of AIOs identified during the entire week.

Dynamics of AIOs for health topics

The fewest AIOs recorded during the week occurred on September 17, with a total of 160 AIOs. This is 12% lower compared to September 15.

Chnages in AIOs for health topics

This suggests that while there was a very slight increase in AIO presence over the week, the dip on September 17 indicates fluctuations that may be influenced by various factors, such as changes in search algorithms. 

What are the top-linked websites in health-related AI Overviews?

The top-linked websites in health-related AIOs highlight Google’s preference for authoritative and trusted medical information from well-established institutions.

Top linked websites in AAI Overviews for health niche

Mayo Clinic leads with 107 links, underscoring its reputation as a reliable source for medical knowledge and patient care guidance. Close behind are WebMD (91 links), Healthline (77 links), and Medical News Today (74 links). This indicates Google’s reliance on well-known health platforms that offer accessible medical information for the general public.

Moreover, institutional sources like Cleveland Clinic (66 links), NHS.uk (61 links), and NCBI (58 links) suggest that Google prioritizes expert-reviewed and research-based content from leading health organizations. 

Do AI Overview sources match pages ranking in the top 20 of organic search?

The average number of links shared between the top 20 search results and the AI Overview resources among all tracked keywords was 7.13. However, there was considerable variation: some keywords had little to no overlap with top 20 sources, while others shared over 10 links with the top 20.

Note: The screenshot is taken from SE Ranking’s AI Results Tracker. URLs that appear in both AI Overviews and the top 20 are highlighted in pink.

For instance, the keyword “gluten-free foods and snacks” had no matching links in the top 20, while “what to do if I test positive for pregnancy” had the highest number of shared links—14. In the majority of cases (25% of all keywords), the overlap between the AIO resources and the top 20 organic results was exactly eight pages.

To sum up, 59.7% of tracked keywords shared between 6 to 8 links with top 20 sources, indicating a moderate correlation. However, 18.36% of keywords had minimal or no overlap (up to 5 sources from the top 20), suggesting that AI Overviews may occasionally provide unique or less conventional information compared to traditional search results.

Which keyword patterns resulted in AI Overviews across the analyzed keywords?

Out of the 196 keywords that triggered AIOs for health-related topics:

  • 49 began with “how,” such as “how long do people live with cancer.” Of these, 19 specifically asked “how to,” like “how to help with an asthma attack.”
  • 31 included the word “you,” as in “how do you know if you have a blood clot.”
  • 26 contained the pronoun “I,” such as “will ozempic help me if I weigh 200 pounds.”
  • 14 asked “what is,” such as “what is the amount of sugar recommended per day.”
  • 7 inquired “what are,” like “what are the best weight loss pills.”
  • 9 started with “when,” like “when should you go to the emergency room.”
  • 7 included the phrase “the best,” such as “best way to sleep with lower back pain.”

Based on the analysis, the most common keyword patterns that triggered AIOs for health-related keywords were “how” questions, followed by those containing the pronouns “you” and “I.” This suggests that Google generally offers AIOs in response to searches that ask for practical tips and personalized guidance. However, it’s essential for users to approach this information critically, as there is usually a disclaimer underneath AIO that absolves Google of any responsibility for the information provided. 

How often does Google provide a disclaimer for health-related content within AI Overviews?

Among the 196 health-related search terms that triggered AI Overviews, a significant 83.16% (163 responses) included a disclaimer saying: “This is for informational purposes only. For medical advice or diagnosis, consult a professional. Generative AI is experimental.”

This is how this disclaimer looks within AIOs:

The widespread inclusion of disclaimers in AI Overviews for health-related searches showcases Google’s careful approach to dealing with YMYL content. This attention to managing sensitive subjects reflects Google’s priority on maintaining trust and safety, especially in the experimental stages of generative AI.

For highly sensitive keywords related to severe mental health issues like depression, Google does not provide AI Overviews. Instead, it displays a dedicated “Help is available” feature at the top of search.

This feature provides direct access to the 988 Suicide & Crisis Lifeline, a 24/7 support service available in both English and Spanish. Users can conveniently start a chat, send a text, or make a call directly from the SERP. Additionally, Google offers four pre-written messages that users can easily copy and send to a trusted person for assistance.

In total, 20 out of all 300 analyzed keywords triggered this message, which accounts for 6.67%. Examples of such keywords include “what to do when you are depressed and suicidal,” “when should I go to the therapist for suicidal thoughts,” etc.

When it comes to sensitive topics like mental health, eating disorders, substance abuse, or specific medications, Google takes a particularly cautious approach. This is reflected in its decision to avoid providing information or tips on these topics through AI Overviews. 

Here’s a list of areas where Google steers clear of providing AI Overviews:

How trustworthy is the information provided within health-related AI Overviews?

Our analysis shows that in the vast majority of cases, Google does maintain its strict high standards for health-related information within AI Overviews.

However, there remain scenarios that might pose uncertain risks to user health and safety. Let’s examine some of these cases in more depth.

For example, the “weight loss pills that actually work” query triggers an AIO, although its featured sources are not always reliable or evidence-based.

  • Articles from publications like Forbes, while often informative, may not have the necessary expertise in healthcare or nutrition to provide accurate guidance on these topics.
  • Research studies from 2001 may not reflect the latest scientific understanding and could contain outdated recommendations.
  • Online marketplaces like Amazon often feature a wide range of weight loss supplements, but their product listings may be influenced by commercial interests rather than the scientific evidence supporting their effectiveness.

Another example is the “do snake venom facials work or not” query.

Note: Snake venom facials are a controversial beauty treatment that involves applying a synthetic form of snake venom to the skin to reduce the appearance of fine lines and wrinkles. Some studies have shown that the synthetic peptide can have a temporary smoothing effect on the skin, but the long-term benefits of this procedure are unclear. Additionally, there may be risks associated with using snake venom, such as allergic reactions or skin irritation.

This is how the AI Overview for the query about snake venom facials looks like:

As you can see, Google clearly states in its AI Overview the benefits of this beauty treatment (e.g., it reduces wrinkles and relaxes muscles), but doesn’t provide info on its weaknesses and the limited research behind it. This might be because the AIO relies on commercial sites like Sonya Dakar and Amazon, as well as outdated publications from the Los Angeles Times (2009) and the Irish Independent (2010).

Note: The screenshot is taken from SE Ranking’s AI Results Tracker. 

In some cases, Google starts generating an AIO for specific queries but then abruptly stops, displaying the message: “An AI Overview is not available for this search.” For example, this happens with the keyword “can I refuse vaccines for my newborn.”

The exact cause of this behavior isn’t clear. We might suggest that it happens due to a lack of sufficient data to generate a complete AI Overview or technical issues. However, since this message usually displays for highly sensitive topics, it’s likely that Google imposes restrictions, preventing AIOs from being generated for certain types of content.

Google generally maintains high standards for health content within AIOs, although there are still gaps in the quality of AI-generated responses.

On the positive side, AI Overviews prioritize authoritative sources like the Mayo Clinic and WebMD, demonstrating Google’s commitment to reliable information. In 83% of cases, health-related AIOs feature disclaimers suggesting users consult professionals for medical advice. Additionally, Google AIOs avoid highly sensitive topics related to severe depression, eating disorders, Covid-19, medication recommendations, and abortion. This way, Google minimizes the risk of causing harm or distress to users.

However, there are potential risks associated with AI Overviews for the health niche. In some instances, Google may rely on commercial-oriented or outdated sources, potentially compromising the accuracy and relevance of the information presented.

For this reason, we strongly recommend verifying health-related information from AIOs by cross-referencing it with reliable sources and consulting a healthcare professional before making any decisions about your health.

Political niche

Political content, particularly related to elections, policy changes, and governance, often has significant implications for individuals’ lives and livelihoods. This makes it essential for AI Overviews to provide accurate and unbiased information to users.

How often do AI Overviews appear for political topics?

In our detailed examination of 300 keywords related to political topics, we discovered that 16.67% (50 keywords) were successful in triggering AI Overviews.

To track changes in the number of AIOs over time, we added the entire set of keywords in our AI Overviews Tracker, which performs daily checks. We then revisited these keywords on September 22nd to evaluate any changes that took place during the previous week.

Our findings indicate that 62 keywords, or 20.67%, triggered AIOs on September 22—one week after the initial check. This was the highest number of AIOs throughout all seven days.

The fewest AIOs observed during the week occurred on September 16, with a total of 43 AIOs recorded (14.33%).

All in all, these results suggest a gradual increase in the prevalence of AIOs for political keywords, despite occasional fluctuations, indicating a growing relevance and responsiveness of AI-generated responses in this area. Still, compared to the other three YMYL niches analyzed in this study, the political sector has the lowest number of AIOs.

What are the top-linked websites in politics-related AI Overviews?

Our analysis reveals a clear preference for government websites and Wikipedia in politics-related AI Overviews:

Politics-related AIOs heavily favor government websites like state.gov (15 links), dhs.gov (15 links), whitehouse.gov (12 links), and so on. This can be explained by their reliability and official data. These websites are usually considered authoritative sources of information on political topics, providing users with official updates, legislation, and historical data.

Wikipedia (36 links), on the other hand, also plays a significant role in politics-related AI Overviews. Its high ranking can be attributed to its neutral perspective on different topics. However, unlike government websites, Wikipedia relies on user contributions, which can sometimes lead to information that is not 100% accurate or up-to-****.

Do AI Overview sources match pages ranking in the top 20 of organic search?

On average, the overlap between the top 20 search results and the AI Overview resources among all analyzed keywords was 7.44 links. The smallest overlap, with no matches in the top 20, occurred for queries like “what was the first government of the United States” and “how does the United States government work”. Conversely, the highest overlap, with 13 matches, was observed for the query “what is the state government responsible for”.

Note: The screenshot is taken from SE Ranking’s AI Results Tracker. 

As you can see in the screenshot above, there is no overlap between the links in the AI Overview and the top 20 search results (none of the links are highlighted in pink).

In most cases, the overlap between AIOs and organic search results appeared for 9 out of the top 20 pages. This pattern was observed in 34% of the analyzed queries, with 17 out of 50 keywords displaying this level of coincidence.

The data shows that 76% of AI Overviews for political topics incorporate 6-9 sources from the top 20 organic search results, indicating a strong correlation between AI-generated responses and high-ranking organic pages. However, 16% of AIOs include only up to 5 sources from the top 20, suggesting that while AIOs often align with organic results, there are instances where they diverge considerably.

Which keyword patterns resulted in AI Overviews across the analyzed keywords?

An analysis of 50 AI Overviews on political topics revealed a distinct pattern in the types of queries prompting responses. Most AIOs were triggered by informational keywords, particularly those with straightforward, factual answers (such as historical facts, definitions, or statistics).

The most common keyword structure was “what is,” appearing in 17 out of 50 keywords (e.g., “what is the US defense for nuclear *******”). Also, the keyword structure “what are” appeared in 4 keywords, such as “what are my first amendment rights as a U.S. citizen.” This likely indicates a strong demand for definitions and explanations within the political area.

The second most frequent keyword structure was “how does,” used in 14 keywords. This suggests that users were interested in understanding the processes and mechanisms behind political systems/policies/processes. For instance, the keyword “how does the United States government work” exemplifies this trend. 

Do AI Overviews appear for election-related queries?

What’s interesting is that none of the keywords containing the terms “election,” “elections,” “president,” or “presidential” triggered an AI Overview during our analysis. Late last year, Google released an article outlining their approach to the 2024 US elections. Here’s what they shared:

“We’re also focused on taking a principled and responsible approach to introducing generative AI products – including Search Generative Experience (SGE) and Bard – where we’ve prioritized testing for safety risks ranging from cybersecurity vulnerabilities to misinformation and fairness. Beginning early next year, in preparation for the 2024 elections and out of an abundance of caution on such an important topic, we’ll restrict the types of election-related queries for which Bard and SGE will return responses.”

In August 2024, Google published another article highlighting its commitment to delivering high-quality and reliable information on US elections from non-partisan sources. They also emphasize that AI can make errors as it learns or as new information emerges, which is why they limit responses to election-related queries on Gemini apps and the web experience:

“As we integrate Gen AI into more consumer experiences, we’re also applying election-related restrictions to many of these products, including Search AI Overviews, YouTube AI-generated summaries for Live Chat, Gems, and image generation in Gemini.”

To check how compliant Google is with its promises, we’ve also analyzed the set of keywords related to US elections through Gemini. The idea was to see if Google is consistently restricting AI-generated responses for these sensitive topics as outlined in their commitment.

Just like AI Overviews, Gemini does not provide any responses to queries related to upcoming US elections. All analyzed queries return the same response: “I can’t help with that right now. I’m trained to be as accurate as possible but I can make mistakes sometimes. While I work on perfecting how I can discuss elections and politics, you can try Google Search.

We’ve received this response even for simple queries that are related to US elections but don’t fall strictly within YMYL topics. For example, keywords like “who are the main candidates in the 2024 US presidential election, “when is election day in 2024,” and “how do I register to vote in the 2024 election” triggered this message.

It seems that Google is keeping its promise, as no AIOs (or Gemini responses) for election-related queries were detected during the analysis. This is a crucial step in preventing the spread of misinformation and ensuring the integrity of the electoral process.

Yet, there is still a difference in how Google approaches political topics in search and on Gemini. 

Gemini appears more open to offering AI responses on political matters that aren’t directly tied to elections.

For example, Gemini provides detailed answers to queries such as “what is the role of the Supreme Court in U.S. politics”, “rules of the US electoral system,” and “what is impeachment, and how does it work”—even though Google AIOs do not appear for these keywords.

Still, the wording plays a significant role in whether Gemini will generate responses on political topics. For instance, the query “how is the president elected” produces in-depth response with the breakdown of the presidential election process in the US:

However, if you slightly change the wording and input “how is the US president elected,” Gemini will not provide the answer:

Some users on X have also reported issues with Google’s Gemini AI, claiming it appears to be “broken.” They tested it by using the same prompt repeatedly but altering the names each time. This led to inconsistent results, with some media figures (including those involved in politics) being favored over others.

We decided to take things a step further and also explore how SearchGPT handles election-related queries in comparison to Google, which tends to avoid this topic. 

SearchGPT is currently in its testing phase, with access restricted to just 10,000 users. One of these lucky users is from the SE Ranking team. Thanks to their access, we were able to compare results provided by SearchGPT and Google.

Recently, we conducted research into how SearchGPT stacks up against Google. However, for this study, we narrowed our focus specifically to election-related queries, performing the analysis on September 26 from New York.

Let’s start by reviewing the results for the keyword “who is winning the election.”

As you can see, SearchGPT provides a direct response to the query stating that the presidential race is “exceptionally close.” This response features polling data, battleground states, and electoral college dynamics along with links to sources like BBC, FiveThirtyEight, and New York Post.

However, some of the sources from a carousel under the response are somewhat questionable.

For example, The Times article from April 2024 discusses Donald Trump leading Joe Biden in six out of seven swing states, even though Biden is no longer a candidate in the race.

Moreover, a few of the sources on the left side of the SERP are outdated, ****** back to 2020. These include a summary of the 2020 presidential election live results from 270toWin and CNN’s coverage of the 2020 election results.

For another example, let’s take a look at the keyword “how to vote for president online.”

Unlike Google AI Overviews, SearchGPT provides information for this query saying that the US does not offer online voting for federal elections, including the presidential elections. Then, it explores the process of registering to vote, voting by mail, and early voting in detail.

What’s interesting is that its SERP on the left-hand menu features 45 links, which is a way higher number compared to the majority of other queries we’ve analyzed. Among them, you can find links to USAGov, Wikipedia, MIT Election Lab, U.S. Election Assistance Commission, and so on.

In conclusion, while Google generally avoids using AI-generated responses for election-related queries to safeguard the integrity of the electoral process, there is a notable distinction in how political topics are handled across its platforms.

Gemini seems more flexible in addressing broader political questions that are not directly linked to elections. AI Overviews, on the other hand, are way less prevalent even for general political topics.

SearchGPT (which is not yet publicly available), at the same time, does provide responses to election-related topics, which suggests that it is not restricted from addressing sensitive issues. 

How trustworthy is the information provided within politics-related AI Overviews?

The political niche has the lowest rate of AI Overviews, suggesting that Google exercises caution when providing information on such a sensitive topic. 

But before we proceed with the analysis of specific AIOs for this niche, it’s worth noting that politics-related queries often trigger the “An AI Overview is not available for this search” message. Here’s an example of this message for the “how is the U.S. addressing income inequality” keyword:

And one more example for the “what is the US defense for nuclear *******” query:

This suggests that Google recognizes the potential for misinformation or bias on complex issues like income inequality or nuclear defense. This is why its AI Overviews may be programmed to avoid providing information on such sensitive and controversial topics.

Now, let’s move on to our analysis of Google’s AI Overviews for political topics.

To start, let’s evaluate the AIO for the keyword “what is the current debate around police reform in the U.S.”

When users search for the “current” debate, they likely expect recent information, ideally from the last 2-3 months.

However, at the top of the AIO, we see the CBS News poll dated February 2023 (over 1.5 years ago), which Google mentions. This data doesn’t align with the expectation for “up-to-****” content.

Additionally, while the content acknowledges the necessity for reform, it overlooks the issue of racial bias (which has been the driving force of the police reform movement).

In fact, high-profile incidents of police violence and systemic racism are what contributed to the ongoing debates surrounding police reform in the US in recent years. This omission could contribute to a lack of understanding of the root causes behind the movement.

Let’s now explore and AIO for the “what is the Patriot Act, and how does it affect citizens’ privacy” keyword.

The content within this AIO does not provide a balanced view of its intended purpose (counterterrorism) or the mechanisms of oversight. This is why it might create unnecessary fear or distrust of government institutions. 

In turn, by stating that the Patriot Act’s surveillance infrastructure “still exists today” without highlighting its potential for abuse or overreach, the piece could inadvertently contribute to a sense of complacency about the ongoing surveillance of citizens. Additionally, the content does not inform users of their rights or legal recourse if they believe their privacy has been violated.

Similarly, the “what is Medicare and Medicaid” keyword triggers the AIO although its content is missing out on some important details.

It primarily focuses on eligibility requirements but leaves out important information about how these health insurance programs work. 

For example, it doesn’t explain the different parts of Medicare (Part A, B, C, D), which offer various levels of coverage (hospital insurance, medical insurance, prescription *****, etc.). This misleads users into believing that Medicare has uniform coverage and can confuse users as they encounter out-of-pocket costs, coverage gaps, or differences in coverage based on the plan they select.

The content doesn’t mention that 40+ states have already expanded Medicaid eligibility under the Affordable Care Act (ACA), providing coverage to low-income adults who would not have qualified under traditional Medicaid rules. 

So, the potential for misinformation and bias in the political niche contributes to the limited presence of AI Overviews (16.67%). This explains Google’s approach to avoid providing AI responses for keywords related to upcoming US elections in November 2024.

Our examination shows that, overall, Google tends to offer accurate and trustworthy information, usually relying on government sources and displaying AI Overviews only for straightforward, simple queries. However, some AIOs reveal notable gaps in coverage, such as outdated data and omitted critical context. This is why it is essential for users to verify the information provided through additional credible sources.

Finance niche

Financial advice and information, such as investment strategies, tax planning, and debt management, can have a profound impact on individuals’ financial security. Therefore, AI Overviews must ensure that their content is reliable, trustworthy, and helpful.

How often do AI Overviews appear for finance topics?

Out of a total of 300 keywords within the finance niche, 125 keywords successfully triggered AI Overviews, accounting for 41.67% of the overall keyword set.

To keep an eye on shifts in AIO counts, we added all keywords to our AI Overviews Tracker, which conducts daily monitoring. On September 22nd, we revisited the keywords to assess any changes from the previous week.

The results revealed the increase in just one keyword triggering an AIO, with a total of 126 queries.

Over the course of the week, the number of detected AIOs for the analyzed keywords fluctuated. Specifically, on September 16, there were 111 AIOs, while September 17 saw a peak of 134 AIOs.

In summary, our analysis indicates that AIOs for finance-related keywords demonstrate a steady presence, despite minor fluctuations throughout the week.

What are the top-linked websites in finance-related AI Overviews?

The top-linked websites in finance-related AIOs reflect a mix of trusted information hubs and popular media outlets.

Investopedia, known for its educational content, leads with 68 links. Similarly, NerdWallet and Bankrate are close behind with 57 and 56 links respectively. Google likely relies on these platforms for accurate, well-rounded financial guidance.

Media outlets like Forbes (41 links), Yahoo Finance (23 links), and CNBC (18 links) are also frequently linked, reflecting Google’s preference for trusted financial news and expert opinions.

Do AI Overview sources match pages ranking in the top 20 of organic search?

On average, there was an overlap of 5.96 links between AI Overviews for tracked keywords and the top 20 organic search results. The smallest overlap recorded was zero, meaning there was no match between the top 20 organic results and the AIOs for certain keywords, such as “how can I make $300 a day from home.” In contrast, the highest overlap observed was 10 links for the keyword “what is APR credit card,” showing a significant alignment between organic top results and AIOs.

Note: The screenshot is taken from SE Ranking’s AI Results Tracker. 

The screenshot shows that all 10 links featured in the AI Overview on September 15 were also included in the top 20 search results.

In turn, the most common overlap between AI Overviews and the top 20 organic results was seven links, occurring on 24 out of 125 keywords, which accounts for 19.20% of the cases. This indicates that while AIOs may vary significantly from organic search results in some cases, there is still a substantial degree of overlap for many finance-related keywords.

The data reveals that 71.8% of AIOs related to finance topics include 5 to 9 sources from the top 20 organic search results, demonstrating a strong alignment between AI-generated responses and top-ranking organic pages. However, 26.4% of AIOs only incorporated up to 4 sources from the top 20 results. This means that some AI Overviews for finance-related topics may offer unique insights or alternative perspectives not fully captured by the highest-ranking organic results.

Which keyword patterns resulted in AI Overviews across the analyzed keywords?

Among the 125 finance-related keywords that triggered AI Overviews, the distribution is as follows:

  • Keywords starting with “how”: 40 (e.g., “how do you calculate profit on crypto trading”)
  • Keywords starting with “how to”: 21 (e.g., “how to earn money as a kid”)
  • Keywords starting with “what”: 37 (e.g., “what happens if someone hacks into your bank account”)
  • Keywords starting with “what is”: 23 (e.g., “what is the easiest loan to get online”)
  • Keywords containing “I”: 19 (e.g., “can I give my ********** access to my bank account”)
  • Keywords containing “you”: 14 (e.g., “can you get a $5,000 credit card”)
  • Keywords starting with “which”: 9 (e.g., “which app gives real money”)

How often does Google provide disclaimers for finance-related content in AI Overviews?

Of the 125 finance-related search terms that triggered AI Overviews, 63.2% (79 responses) featured a disclaimer stating: “Generative AI is experimental. For financial (legal) advice, consult a professional.”

The prevalence of disclaimers in finance-related AIOs implies that Google is taking a cautious approach, recognizing the potential consequences of providing misleading or harmful financial information.

How trustworthy is the information provided within finance-related AI Overviews?

To assess the accuracy and reliability of the information provided by AI Overviews in the financial niche, we conducted a quality analysis of their content and summarized several insights.

Let’s begin by taking a closer look at the AIO for the query “how to earn money as a kid.”

It triggers the response where Google recommends activities such as babysitting, pet sitting, washing cars, taking surveys, selling crafts, starting a lemonade stand, and creating a YouTube channel.

What’s important is that Google doesn’t clearly distinguish between “kids” (typically defined as those aged 12 and under) and “teens” (ages 13 to 19), which raises questions about the legality of the proposed activities.

In fact, the U.S. Department of Labor enforces the Fair Labor Standards Act (FLSA), which addresses wages, working hours, and safety standards for minors under 18. Generally, the FLSA sets the minimum working age at 14, although this can vary by state and the type of work.

According to this act, young people of any age may engage in “delivering newspapers, performing in media, working in family-owned businesses, babysitting, and performing minor chores around a private home.”

So, content within this AIO doesn’t fully address the legal nuances of child labor, potentially exposing users to unintended risks or consequences.

Another AIO worth noting is the question, “is it possible to earn $1,000 a month from stocks?”

It states that making $1,000 a month from stocks is possible with significant capital or a combination of dividend-paying stocks and reinvestment strategies.

Content within this AIO doesn’t adequately address the volatility and risks associated with stock investments. Investors might be lured into a false sense of security, thinking dividends are guaranteed.

Besides, the advice to rely on dividend history and company health oversimplifies the complexities of stock investing. It doesn’t cover the importance of diversification, market conditions, or the need for thorough research, which can mislead inexperienced investors.

Finally, by suggesting that reinvesting dividends can lead to compounding, the content may encourage inexperienced investors to take unnecessary risks without a solid understanding of investment strategies.

The case is pretty much the same with the AI Overview triggered by the keyword “how do you take profit in crypto trading”.

It presents trading strategies in a straightforward manner without acknowledging the complexities and risks involved. New traders may not fully understand that crypto markets are highly volatile, and what seems like a guaranteed profit can lead to significant losses.

The content doesn’t mention the potential for losses, especially with strategies like take-profit orders and lending, which can backfire if market conditions change unexpectedly. Plus, phrases like “buy low, sell high” promote speculative trading, which can lead inexperienced traders to make impulsive decisions based on market hype rather than informed analysis.

Featured AIO sources also raise several concerns regarding the authenticity and reliability of the information presented. For instance, the “How To Profit $2,156 In A Day” YouTube video is a part of the affiliate marketing scheme (as mentioned in the video description).

In such cases, creators often profit more from selling courses or subscriptions than from actual trading success. This can mislead traders into adopting risky strategies based on exaggerated claims rather than sound principles.

To sum up, AI Overviews for finance-related topics demonstrate a strong presence, triggered by 41.67% of the analyzed finance keywords. The top-linked sources in these AI Overviews, such as Investopedia, NerdWallet, Bankrate, and Forbes show that Google prioritizes established and reputable information hubs. 

While AI Overviews can be a valuable resource for financial information, it’s essential to approach them with a critical eye. Based on our findings, these AI responses sometimes oversimplify complex financial concepts or provide incomplete responses. So, to make sound financial choices, it’s crucial to always supplement information within AIOs with expert guidance and thorough research, as mentioned in Google’s disclaimer in 63.2% of AIOs.

Legal niche

Legal information, particularly about rights, obligations, and legal processes, can significantly impact individuals’ lives. As such, AI Overviews must ensure that their legal content is accurate, informative, and unbiased. This is especially critical due to the potential repercussions of sharing incorrect or misleading legal information.

How often do AI Overviews appear for legal topics?

Out of 300 tracked keywords related to the legal niche, 233 triggered AI Overviews, representing 77.67% of the total.

To monitor AIO fluctuations, we added the complete set of keywords to our AI Overviews Tracker for daily tracking. On September 22nd, we reviewed the data to analyze any changes that had occurred throughout the previous week.

The findings showed that 245 keywords, or 81.67%, successfully triggered AIOs.

During the week, we observed a decline in AI Overviews on September 16, with a total of 210 AIOs recorded (70%).

In conclusion, despite a temporary decline in AIOs on September 16, the overall trend reflects a growing responsiveness in this niche—from 77.67% to 81.67% over the week.

What are the top-linked websites for legal topics within AI Overviews?

The top-linked websites for legal topics in AI Overviews show a blend of official government resources, legal information platforms, and general knowledge sites.

NYCourts.gov leads significantly with 114 links, highlighting Google’s preference for authoritative, state-backed legal information. This is followed by YouTube (48 links), showing Google-focus on video content (in the form of legal tutorials or explanations). FindLaw (46 links) also ranks highly, likely emphasizing Google’s reliance on well-established platforms that offer comprehensive legal definitions, guides, and resources for both professionals and the general public.

Government-backed and educational sites like DOL.gov (26 links), Law.Cornell.edu (26 links), and USCIS.gov (24 links) further reinforce Google’s focus on trusted, reliable sources of legal information, especially concerning employment law and immigration. 

Do AI Overview sources match pages ranking in the top 20 of organic search?

On average, the overlap between AI Overviews for tracked keywords and the top 20 organic search results was found to be 6.49 links. The smallest overlap recorded was zero, indicating no matches for certain keywords within the top 20 results, with seven keywords affected. One example is “how to sell a business.” In contrast, the largest overlap observed was 13 links for the keyword “what are the 5 most common injuries in a workplace,” showcasing significant alignment in that instance.

Note: The screenshot is taken from SE Ranking’s AI Results Tracker. 

The screenshot shows that none of the 10 links in the AIO matched sources from the top 20 on September 15.

At the same time, the most frequent overlap between AIOs and the top 20 organic results occurred on eight pages, encompassing 49 out of the 233 keywords, which represents 21.03% of the total. This finding highlights a considerable degree of consistency between AI-generated responses and high-ranking organic listings.

The data indicates a strong correlation between AIOs and top 20 organic results, with over 63% of keywords utilizing 6 to 9 sources from these high-ranking pages. However, there are instances where this alignment is less pronounced, as seen in the 20.17% of keywords that rely on only up to 4 links from the top 20 organic results.

Which keyword patterns resulted in AI Overviews across the analyzed keywords?

Out of the 233 keywords triggering AIOs in the legal niche:

  • 106 keywords begin with “what.” This includes 55 keywords that start with “what is” (e.g., “what is the process to patent an idea”), 20 keywords that begin with “what are” (e.g., “what are the adoption laws in New York State”), and 7 keywords that start with “what happens” (e.g., “what happens if you don’t leave after deportation”).
  • 84 keywords start with “how,” such as “how do I find public records in NY.” This includes 45 keywords that begin with “how to” (e.g., “how to get guardianship of a child”), 11 keywords that start with “how long” (e.g., “how long does it take to get legal guardianship”), and 8 keywords that begin with “how much” (e.g., “how much does it cost to adopt a child in NY”).
  • 23 keywords contain “you,” like “how do you seek political asylum.”
  • 12 keywords contain “I,” for instance, “how can I apply for a green card.”
  • 8 keywords contain “can,” such as “can you practice immigration law in any state”

How often does Google provide disclaimer for legal topics within AI Overviews?

Among the 233 keywords that triggered AI Overviews for legal topics, 19.74% (46 responses) featured a disclaimer stating: “Generative AI is experimental. For legal advice, consult a professional.”

The rate of disclaimers in AI Overviews for legal topics is lower than in health and financial niches. This suggests that Google may have less stringent standards or expectations for the use of AI-generated legal information. Regardless, it’s crucial that users are aware of the limitations of AI technologies and seek professional legal advice before making any decisions in this area.

How trustworthy is the information provided in AI Overviews in the legal niche?

Similar to our analysis of three other YMYL niches, we conducted a quality assessment of AI Overview content on legal topics.

During our manual review of specific examples, we also examined information for different locations. This is because legal information can vary significantly depending on the specific state or country.

For instance, this is how the keyword “how to get guardianship of a child” triggers the AIO in New York:

And this is how it appears for searches in Texas:

As expected, AIOs are tailored to specific locations, which is essential for accuracy in legal content. However, while some queries correctly trigger location-specific AIOs, there are still instances where inconsistencies arise across different locations. For example, take the keyword “do consumers have the right to a refund.”

During our initial analysis on September 15, the AIO displayed information relevant to South Africa instead of the US.

By September 20, the same query began showing results for the UK.

When we checked again on September 22, the AIO had disappeared entirely.

Note: The screenshot is taken from SE Ranking’s AI Results Tracker. 

This indicates a problem with generating AIOs that are tailored to specific geographic locations.

However, these aren’t the only issues we’ve identified with AIOs in the legal niche.

For instance, let’s analyze the AIO for the “how to win child custody in Texas” keyword.

To begin with, it doesn’t explain the full range of factors courts consider in Texas, such as the child’s preference (if a child is at least 12 years old), the mental and emotional health of both parents, or any history of abuse or neglect.

While suggesting documenting incidents and gathering witness statements may seem like sound advice, it could encourage one parent to manipulate or frame situations in ways that portray the other parent negatively. This can lead to a contentious, adversarial legal battle rather than focusing on the child’s well-being.

What’s also important is that the content fails to mention alternative dispute resolution methods like mediation, co-parenting classes, or collaborative law, which are often less emotionally damaging for the children involved. This is particularly important since Texas courts assume that joint conservatorship (custody) is best for the children

Another example where Google provides incomplete information on legal matters is for the “how to look up someone charges in NYC” keyword.

The content in this AIO fails to highlight that under the Clean Slate Act, New Yorkers are eligible to have conviction records automatically sealed 3 years from sentencing for misdemeanors and 8 years from sentencing for felonies. So, some criminal records (those except for *** offenses and non-**** Class A felonies) might not be available for viewing even with suggested methods.

Second, accessing someone’s criminal record may have legal ramifications, particularly if the information is used inappropriately or for discriminatory purposes. It fails to advise users on the importance of context, verification, or the need for discretion when discussing someone’s criminal history, which could lead to spreading misinformation or causing harm.

Still, the issues mentioned above are uncommon. 

Legal topics trigger the most AI Overviews, accounting for 77.67% of all responses. This suggests that Google may have less stringent standards for AI-generated legal content. However, when providing AI Overviews for legal topics, Google primarily relies on government resources and those related to the user’s location. This ensures that AI Overviews in the legal niche are highly accurate and relevant.

As with any topic, it’s essential to complement the information provided by AI Overviews with your own research or expert advice. This is Google’s recommendation in disclaimers found in 19.74% of cases.

Research methodology

This study explores the presence and quality of Google AIOs for YMYL topics. We analyzed 1200 keywords, a significant part of which were long-tail. This is because our previous research studies indicate that these queries are most likely to trigger AIOs. Our keyword selection was based on a combination of manual research and data from SE Ranking’s Keyword Research Tool.

We created separate projects for four YMYL areas (health, politics, legal, and finance) within the SE Ranking platform, inputting 300 search queries in each.

To conduct the analysis, we used SE Ranking’s AI Overviews Tracker, which captured AIOs for YMYL topics in desktop searches.

  • Search Engine: Google
  • Location: USA
  • Region: New York, New York, United States
  • Language: English (en)

The primary data collection was conducted on September 15, but we also monitored AIO changes over the following week to observe any shifts in dynamics.

Wrapping up

The findings of this research indicate that AIOs are quite prevalent in the legal and health sectors, while the political niche appears to be less conducive to AIOs.

This all suggests that Google strives to deliver accurate and trustworthy information through AIOs for YMYL topics, but there are still improvements to be made to its AI technologies. Issues such as geographical inconsistencies, incomplete content, and occasional reliance on outdated/irrelevant AIO sources can negatively affect people’s safety and well-being.

Nevertheless, Google’s technologies are advancing quickly. To keep track of changes in AIOs, you can use our new AI Overviews Tracker! Sign up for our 14-day free trial to access the tool and start monitoring AIOs in your niche. See which sources Google features in AIOs and whether your website is among them. 

Stay tuned for our upcoming research studies and insights!





Source link

Related Articles

Back to top button
error

Enjoy Our Website? Please share :) Thank you!