Reimagining search in the AI era: Insights from ex-Bing Search director

I chuckled when I saw the words “Search was broken” on the LinkedIn profile of Stefan Weitz.

This assessment had nothing to do with Google Search in 2024. Rather, he was talking about Google in 2010. This was a time when Google was loved despite only answering one in four queries successfully, according to Weitz.

So what does the former director of Microsoft’s Bing search engine think of search in the generative AI era? Is Google still broken? Even more broken?

I caught up with Weitz yesterday to discuss the evolution of search in this AI era and how it can improve. Weitz is now co-founder and CEO of HumanX, an organization building the premier community for artificial intelligence (AI) and its deployment – but he still has plenty of thoughts about search.

Search is still broken. Search engines are great for many things. However, search engines still struggle to effectively help users accomplish their goals, Weitz said:

• “Today, search feels like far too much work for complex tasks. You, as a searcher, are doing a query, analyzing the results, then doing another query [or more] on top of that to dig deeper or take action.
• “At Bing, the idea was to connect the query and intent through action… The idea was to help you get to the endpoint, not just give you information. So that’s still broken, I think.”

Dig deeper. Survey: 54% of people look through more search results vs. 5 years ago

How search + LLMs are evolving. Voice agents were a big deal seven or eight years ago. Alexa, Siri, Cortana and others attempted to become a primitive version of the “Star Trek” computer.

While LLMs are magical, Weitz said they won’t get us to the “Star Trek” computer or AGI (artificial general intelligence) – though he thinks they will help lead us there – for two reasons:

1. LLMs are applied statistics – they don’t have true knowledge. LLMs can understand that an entity appears millions of times across the web, but have no understanding of the world.
2. It feels like we’re talking to a computer – there is still lots of work to be done in divining what people are actually asking for.

The future of AI Overviews and answer engines. Google will continue to push AI Overviews, while rivals (ChatGPT, Microsoft Copilot, Perplexity, Claude, etc.) will continue to develop new AI search experiences. The key to success will be ensuring the AI experiences are truly adding value for people, Weitz said:

• “You’re only as good as your worst failure. If people start using a technology and it fails, then they begin to churn out of the overall experience.
• “People may decide, I don’t really want this AI thing in my search results because, on balance, it isn’t accurate and useful.
• “Nobody notices when search engines work perfectly. They notice when they don’t.”

Changing search habits. As highlighted in U.S. vs. Google antitrust trial, it’s hard to change user behavior. Weitz’s own search behavior has changed in the past two years and he expects it has and will continue to change for more users. Maybe not for navigational queries (looking for a particular website or webpage) but for those who want answers to more complex questions.

• “The multimodal work we’re seeing – around text/image/video – are natural use cases. Say I’m getting bugs in my house… I can’t figure it out… I can describe what I need using poorly-formed thoughts. I can take a picture and it tells me ‘here’s what it is’ and spits out six different ways to solve it or help me accomplish whatever I want.
• “In traditional search, you’ll get results for ‘bugs in house’. It’s a pain.”

Needed AI search innovation. If Weitz were put in charge of Google Search tomorrow, what things would he address to improve today’s experience?

• Need for understanding. Generative AI tools are already better at understanding long, slang-riddled or even incoherent sentences and providing answers to them. Google needs to figure out new ways to better understand queries and provide answers.
• Task orientation. Google is getting there, but needs to improve on follow-on actions for searchers. After you identify an issue, tell users how to buy the product, how to install it, and so on.
• Never forget. Search needs to be less device-specific and become a true assistant (like Apple’s Rewind) that never lets searchers forget something they’ve read or seen in the past.

Flashback to 2010. Bing’s Stefan Weitz: Rethinking The Search Experience