Google Gemini flaw exploited to turn AI-powered email summaries into the perfect phishing tool — everything you need to know

A flaw in Google Gemini for Workspace can be exploited by hackers to insert malicious instructions that could misdirect the AI tool and cause it to direct users to phishing sites.

As reported by Bleeping Computer, this vulnerability works by creating email summaries that look entirely normal, but include malicious instructions or warnings that are hidden and automatically obeyed by Gemini when it generates a message summary.

The process works by creating an email that holds an invisible directive for Gemini, by hiding instructions in the body text at the end of the message using HTML and CSS code then setting the font size to zero and the color to white. Since this additional text doesn’t include any attachments or links, it won’t be flagged or caught by the best antivirus software or email programs so it is likely to make it through to a potential victim's inbox.

When a target opens an email, then requests that Gemini summarizes the contents, the AI program will automatically obey the hidden instructions that it sees. Users often put their trust into Gemini’s ability to work with content as part of Workspace; the alert is considered a legitimate warning instead of a malicious injection.

Similar attacks have been reported over the last year, though safeguards have been implemented in order to block the misleading responses, the technique has remained successful overall which is why it is still in use.

Bleeping Computer says that when they asked Google about defenses to counter these types of attacks, a spokesperson referenced a blog post about prompt injection attacks and said that some of the mitigations are in the process of being implemented or are about to be deployed. Google also said it has no evidence that this attack has occurred in the wild.

Figueroa, the manager at Mozilla’s GenAI Bug Bounty Program who detected the flaw, offers a few ideas to prevent this threat: have security teams remove, naturalize or ignore content styled to be hidden in body text. Alternatively, implement filters that scan Gemini for urgent messages, URLs, phone numbers and flag those for additional review from users.

For now though, you just need to be careful when having Gemini summarize your emails as you never know what could be hiding inside them. Hopefully, Google rolls out a fix for this new type of attack sooner rather than later.

More from Tom's Guide

• Nearly 2 million people hit by malicious Chrome installations that can track you — what to do now
• This dangerous banking trojan now uses scheduled maintenance to hide its malicious activities — don’t fall for this
• New Android attack could trick you into compromising your own phone — everything you need to know