Google Search AI Mode Shows Both Promise and Potential Peril
A personal case study trying to Google myself and analyzing the the new AI Overviews
“Google is no longer a search engine. It’s an answer engine.”
— Ben Thompson, tech analyst and founder of Stratechery as quoted in Intelligencer
At the latest Google I/O conference, Google announced that it turned on AI mode for all users with strong reactions on all sides. Google CEO Sundar Pichai called AI mode “the total reimagining of search.” At the same time, content creators have provided very negative feedback on AI mode. For a quick summary, The Verge published a quick article titled “News publishers call Google’s AI Mode ‘theft’.”
However, I understand that others who aren’t news publishers might want to understand how this might affect them as individuals. To explore this further, I decided to do a bit of experimentation using my own name as a search term as a potential guide for others forming their own opinions. Cutting to the chase, I am actually pretty ambivalent right now. I like many of the high-level summaries this technology can provide that spark further reading, as well as some of the very specific objective answers. However, I also see problems with slight hallucinations, shoveling B.S., and complete misattributions. For users, I believe there needs to be education about Google Search AI. For content providers, I believe Google should provide opt-outs to keep certain results from appearing in the AI Overviews produced by AI mode.
Good high-level summaries of people
I started this inquiry with “what is Stephen Pao doing right now?” Overall, the new AI mode didn’t produce a terrible result, responding that I’m a consultant and board advisor to early-stage technology companies (my “cop out” answer.) It would have been nice if Google Search AI mentioned that I’ve been doing this consulting as a “retired techie!” However, this might be more my fault than Google’s!
Good high-level summary of services offered
Professionally, I also appreciate that AI mode can be useful from a business perspective. For example, I think Google Search AI did a good job answering the question “What kinds of consulting does Stephen Pao provide?” Overall, I like that Google Search AI summarized this well — better than what I did on my own LinkedIn profile!
Good answers to specific questions
I also think AI mode is beneficial to help find the answers to specific questions. For example, I like that Google Search AI could answer a very specific question like “Where is Stephen Pao living right now?” I must admit that I have often struggled to figure out this type of information about old colleagues, trying to infer current locations from LinkedIn, social media, and other search engine results. AI mode makes this much easier!
Good high-level summaries for content creators
From the standpoint of my life as a content creator, Google Search AI seems to be pretty good at providing high-level summaries about authors that might entice people to read more. For example, I typed into Google “What kinds of things does Stephen Pao write about?”
I liked that the AI mode result helped to provide a higher level view of my writing about retirement, the tech industry, personal growth & aging, and travel. It also provided in the AI Overview a link to this Substack, as well as on the right hand side, links to both my Medium and Infoworld presence. Overall, I think this AI result benefitted me.
Even when drilling into subtopics (in this case, “investing”), Google Search AI did a good job merchandising individual articles, as well as providing links to my Substack and Medium spaces.
I was particularly entertained when I asked Google Search AI a very specific question —what I liked to do with friends. It kind of nailed it!
However, despite that I find AI mode extremely impressive technology, I have some issues with it.
Stealing content isn’t my problem
Most of the industry is complaining that Google is “stealing” content, providing results on the Google site itself rather than referring people to the sites hosting the underlying content. Some of this is indeed true. My own impact of this phenomenon is pretty limited given that my own Substack is primarily an email newsletter. Most of my traffic is from my email subscribers, not from search. I also don’t advertise on my site, so actual visits and click-throughs aren’t that important to me. However, I am writing this Substack to express myself, and I do feel that AI mode presents problems in this regard.
A problem: slight hallucinations
Once getting into more detailed content, I believe Google Search AI can fail at representing what I have been writing. For example, when I asked Google Search AI about the specific financial advice I give, I was disappointed. Nothing in the AI Overview was necessarily untrue in the context of my broader belief system, but I think that the AI mode slightly hallucinated, as I try to limit my writing and expression of opinion to my own experiences. For example, I haven’t really written about Social Security because I am only 57 years old (going on 58 this year), and I have no personal experience with it. However, both the AI Overview and the links on the right both refer to Social Security, likely because this topic is relevant to retirees, but not relevant to the specific advice I give.
A bigger problem: B.S.
Sometimes AI mode answers questions generically, reminiscent of the high school student writing a book report about a book they didn’t read! When I asked it about my thoughts on portfolio allocation (actual article linked here), it basically responded with how my advice “likely aligns” with other sources and what I would “likely advise” people to do. Did the AI mode read the post or not?
The biggest problem: mistaken attributions
When I queried Google Search AI about my writings on mental health, the biggest problem was revealed. The AI Overview mistakenly attributed to me work done at the National Institute of Mental Health (NIMH) by Dr. Maryland Pao (no relation). The AI Overview read like authoritative text but was erroneous and needed to be closely scrutinized. My head spins when I think about how this form of mistaken attribution opens a major can of worms! Of course, in this case, the work done by Dr. Pao contributes well to society. I am not sure how I’d feel if someone else did something sketchy or even illegal that Google Search AI attributed to me!
Conclusions
Even this personal case study demonstrates some real potential of this technology, as well as some very real hazards. I know that Google has provided a disclaimer exists “AI responses may include mistakes,” but I think Google needs to do more.
Google should provide more education about how to use this technology. Simply turning it on by default to all users accustomed to the accuracy of search feels borderline irresponsible without the proper expectation setting given the relative (lack of) maturity in this technology.
In addition, Google should provide content providers the ability to have their results “opted out” of these AI Overviews in AI mode without removing their pages from search results or removing non-AI snippets. (There are some controls to control snippets, but it seems this feature impacts both AI and non-AI snippets.)
Personally, I am a fan of this AI mode, but I have concerns without the right education and controls.
Do you share my concerns?
This was very interesting. How does Google compare with Chat GPT?