Not everything Google does or touches turns to gold.
Case in point: The new Gemini AI addition to Gmail is not all it's cracked up to be. While it works amazingly well when it comes to composing emails and summarizing a thread, the great irony is that most search-related prompts are not even remotely helpful.
(Note: I asked Google reps about my test results and they have not responded.)
I noticed the Gemini icon for the first time just about one week ago and started diving in right away. The AI bot is starting to roll out for many users with an update that includes new search functions, enhanced smart replies, and a few inbox clean up prompts.
Smart replies and summaries
Before I cover what didn’t work for me, let me just say: I can see where this is all heading and I’m mostly pleased with the basic functions, like smart replies and summaries.
I’m used to AI providing some basic help with my email replies, since I’ve used ChatGPT many times to help me compose and revise emails. Gemini does an exemplary job. When you want help, you can open a sidebar and enter prompts. On my phone or in the browser, I could also ask Gemini to “polish” my own email, adding more details and context in seconds.
I also really liked the summaries. At the top of the screen, there’s a button called “Summarize this email” and the little star icon for Gemini. You’ll see a summary with action steps, and in all of my testing, Gemini was accurate and helpful. I found I didn’t have to read back on a thread as much and used Gemini to catch me up on the conversation.
Where things went awry
I wasn’t here for the smart replies and summaries, though. I’ve been able to do that with other AI bots for the last three years.
I want an AI that goes much, much further than that with my email — e.g tools for helping me understand not just one email thread. I have around 650,000 emails in my Gmail and it’s a treasure trove that Gemini could easily explore.
I wanted to be able to find out who emailed me the most in one particular month, which topics I discussed most often this year, and create a mass email to let people I interact with the most know that I will be out a couple days in June.
Unfortunately, Gemini seems woefully inadequate and returns incorrect results. When I asked the bot to find the people I emailed the most this year and also in May, the results were not correct.
Gemini only listed two people and I had barely interacted with them. It’s possible Gemini just found the most recent interactions, but I had asked for results from 2025 and all of May.
When I asked Gemini about topics I had discussed most often, the AI was blissfully unaware of which emails were just spam sent to me. My prompt was “Which topics did I discuss and reply to the most in 2025” and Gemini listed a bunch of email newsletters.
That was an error, because Gemini was only looking at emails sent to me the most, not those where I interacted.
I also asked Gemini to compose an email to the people I interact with the most, explaining that I will be out June 5-6. Once again, Gemini only found the people that emailed me the most.
While the email the bot composed was helpful, what I wanted was the bot to do the heavy lifting — compose an email with each person in a blind copy. I just wanted to click send.
What else is broken
Gemini is also supposed to help with inbox cleanup duties, but this was mostly a miss. I asked Gemini for Gmail on my iPhone to look for old emails with large attachments and the bot showed me every email with an attachment, not the ones with the biggest attachments. And, they were not old emails -- they were all from the current month.
I also asked Gemini to show me the emails with the largest attachments. For some reason, that prompt didn’t work. “I can’t help with that” was the response.
This prompt did work, though: “Show me all emails with an attachment from May 2024.” I was able to then delete all of those messages quickly, which was helpful. The problem is that Gemini seemed to work about 25% of the time when I was trying to clean up my inbox. It is hit or miss.
I really wanted the bot to understand my goals. Inbox clean up is fine, although anyone who has used Gmail for a while knows we’ve been able to tame our inbox using searches for many years.
For example, I can type “larger:5M after:2024/05/24 before:2025/05/25” to find files with attachments over 5MB this last year. There’s also a filter to help guide you through that process.
Instead, I wanted Gemini to be more like a smart assistant. More than anything, Gemini seemed to only search recent emails. In one query, I asked which emails seem urgent, and the bot only mentioned two from the last week.
I asked which emails had a shipping label attached and the bot only found four, even though there are several dozen from the last two months.
What this all means
Gemini in Gmail is in more of a testing phase. Google is adding new features and enhancing the AI as time goes on, likely based on feedback or date they collect. For now, the AI is not really worth it for me, since the results are so unpredictable or outright incorrect.
I expect the technology will improve, but I’ll probably be leery of diving in again until it becomes obvious that Gemini will work as expected. I want the bot to make me more productive and to work reliably every time I type in a prompt. We’re obviously not there yet.
More from Tom's Guide
- I test AI for a living — and ChatGPT is still missing these 5 key features
- Apple has gotten in the way of its own AI rollout — here’s how it can get Apple Intelligence back on track
- The ‘3-word rule’ that makes ChatGPT give expert-level responses
You must confirm your public display name before commenting
Please logout and then login again, you will then be prompted to enter your display name.
