If you’re a Google Gemini user, it’s time to review the privacy settings for Google’s AI chatbot. It looks like Gemini might have access to documents in Google Drive that it shouldn’t be able to read. That’s what happened with one Gemini user who was stunned to discover that Gemini could summarize tax returns in a PDF file hosted in Google Drive.
This was a document the user never gave Gemini access to, prompting an investigation into Google Gemini’s behavior with Google Drive. The conclusions aren’t great, as it looks like Gemini might be ingesting data from Drive without the user’s permission or knowledge. Even if that data isn’t used to train the AI specifically, it’s still a potential violation of privacy.
This incident further proves that any AI product needs strong privacy features, especially if it’s made by Google.
UPDATE: A Google spokesperson provided BGR this comment about the Gemini behavior in Google Drive, stressing that a user has to start a Gemini interaction in Workspace for the chatbot to get access to the data. Even then, the data or documents Gemini can summarize isn’t stored or used to train the AI:
Tech. Entertainment. Science. Your inbox.
Sign up for the most interesting tech & entertainment news out there.
By signing up, I agree to the Terms of Use and have reviewed the Privacy Notice.
Our generative AI features are designed to give users choice and keep them in control of their data. Using Gemini in Google Workspace requires a user to proactively enable it, and when they do their content is used in a privacy-preserving manner to generate useful responses to their prompts, but is not otherwise stored without permission.
Furthermore, Google has a blog post from August 2023 that covers the AI privacy protections in Google Workspace.
The original post follows below.
A few days ago, I had this to say about the advantages of having a cloud-based Docs product that offers end-to-end encryption, as I was talking about Proton Docs at the time:
End-to-end encryption protects against data breaches and mass collection, and it ensures your data will not be used to train AI. These are all things to keep in mind when trusting any third-party cloud service provider with your data, especially documents you’d like to keep private.
It’s not that Google Docs and Microsoft 365 won’t secure your docs, because they do. Those companies also won’t train AI with your documents. But Proton’s extra end-to-end encryption protection is much better. And Google and Microsoft can’t match it yet.
Gemini is either hallucinating or this setting is so buried that I can’t find it in either my @Google account or Drive settings or Gemini Privacy Hub after searching for 15 minutes. (Pardon my french in the screenshot but felt like expressing anger to the machine…) pic.twitter.com/xyRi8pyTTb
— Kevin Bankston (@KevinBankston) July 10, 2024
Given what Kevin Bankston went through, we have reason to worry about how Google handles Drive documents. The Gemini user accidentally discovered that his tax returns in Google Docs were scanned by Google’s AI without his permission. He then tried to find out what had happened and how to change privacy settings to prevent Gemini from taking over data it shouldn’t.
The task is rather difficult, as even Gemini can’t help point users in the right direction for managing their privacy settings. The chatbot hallucinates information about its own privacy settings.
Bankston was able to confirm that Gemini doesn’t train on the data in Gemini Workspace, but that wasn’t good enough to explain why the chatbot accessed the PDF file in the first place. The privacy blunder is all the more shocking as Bankston is a paying Google One user, which means he’s getting the premium version of Gemini.
If you ever used Gemini with a particular PDF, it will re-invoke for every other PDF you load until you close it again. Same with GDocs–it wasn’t on in any of my Docs, then I turned it on in one, and now it auto-summarizes any I open. Quite a sneaky sticky setting @Google!
— Kevin Bankston (@KevinBankston) July 10, 2024
The user further discovered that Gemini had read all of his PDF files that are hosted in Drive once he had asked Gemini for help with another one of them.
I found the setting that’s supposed to keep Gemini out of my docs…and it was already turned off! Yet Gemini is absolutely giving me document summaries. A setting that doesn’t work is even worse than no setting. pic.twitter.com/U4i9dhsMCA
— Kevin Bankston (@KevinBankston) July 10, 2024
The startling discoveries continued after someone gave Bankston the correct privacy settings for Gemini AI. The setting that’s supposed to keep Gemini out of Google Drive docs was enabled. But Gemini accessed those documents nonetheless.
4) (bonus problem) Again I ask: how many people have unwittingly had their private documents summarized by Gemini–which means that those summaries are all hanging out in your Gemini chat logs unless and until you delete them, unwittingly creating yet more privacy attack surface.
— Kevin Bankston (@KevinBankston) July 11, 2024
A little more digging may have helped Bankston identify the real issue. He signed up for Workspace Labs last year to test Bard before he became a Google One Gemini user. This might be the reason why Google Gemni had access to Google Drive documents it shouldn’t have been able to read.
My working theory until Google can clarify is that using Workplace Labs overrode the explicit setting turning off Workplace integrations of Gemini (or, more likely chronology, I signed up for Workplace Labs and the later introduction of the controls didn’t override that).
— Kevin Bankston (@KevinBankston) July 11, 2024
Unfortunately, that’s just one theory for the moment. It’s up to Google to explain and clarify this glaring privacy issue. Even if only one user is affected, which I doubt is the case, it’s still a massive issue. AI like Gemini should never access any user data without explicit permission.
I’d rather have to deal with an avalanche of prompts to agree to Gemini getting data — which is what happens with ChatGPT access in iOS 18 — than somehow give the AI the power to access my data in bulk.
If you suspect Gemini has accessed any of your private files in Google Drive, just try to see whether it can summarize data from them.