If you use ChatGPT as a private sounding board, therapist, coding partner, or a place to work through personal thoughts, you may want to think twice before hitting send.
A report from 404 Media reveals that OpenAI employs hundreds of human contractors to read, summarize and critique real user conversations. While OpenAI emphasizes that accounts remain anonymous, internal documents show that reviewers see whole conversations, and that sensitive personal details routinely bypasses the company’s automated safeguards.
The good news is you can stop your future chats from being funneled to human contractors. Here is what is happening behind the scenes, the real privacy risks, and how to opt out.
Project Lily: How humans train ChatGPT
Internal documents reveal that the review initiative operates under the internal codename "Project Lily."
OpenAI pays outside contractors, often over $50 an hour via third-party staffing firms, to manually train the chatbot to sound less robotic. Contractors read a user’s prompt, summarize the user's intent and evaluate four potential AI responses on a 1-to-7 scale.
According to leaked training guides, these reviewers are specifically instructed to:
Eliminate 'AI-speak': Penalize unnecessary checkmark emojis, engagement-bait endings and cluttered formatting.
Stop anthropomorphizing: Ensure ChatGPT never claims human lived experiences (e.g., claiming "As a chef, I like to..." is strictly banned).
Cut down sycophancy: Flag answers where the bot excessively flatters users, validates irrational thoughts or amplifies personal frustration. This is a known issue that has previously brought legal scrutiny over user safety.
Why prompts aren't anonymous
OpenAI states that it does not provide usernames to reviewers and that all conversations pass through an automated "Privacy Filter" designed to redact identifying details. However, the system has major vulnerabilities.
OpenAI openly admits on its site that the filter can make mistakes, missing uncommon identifiers or ambiguous personal context. Even more concerning, 404 Media discovered that the dashboard shown to contractors often includes a "user memories summary." This block provides an overview of the user’s previous interactions, which can inadvertently reveal their general location, profession or personal life context.
Unlike public social media feeds, a clean, empty chatbot window creates a false sense of private intimacy. Some prompts seen by contractors even showed users asking ChatGPT to "keep this between us" — unaware that a prompt instruction does not override the platform's backend data collection.
OpenAI isn't the only tech giant doing this. Anthropic confirmed that it uses human reviewers for Claude when users leave training enabled, while Google’s Gemini clearly posts a disclaimer that human reviewers read select saved chats.
How to stop OpenAI from using your Chats
If you have a personal account (Free, Plus, or Pro), OpenAI enables model improvement by default. (Enterprise, Business, and Edu accounts have it turned off automatically.)
To stop your future conversations from being routed to human trainers:
Open ChatGPT in your browser or mobile app.
Click your profile icon in the bottom corner and select Settings.
Select Data Controls.
Find the toggle for "Improve the model for everyone" and switch it Off.
Final thoughts
Keep in mind, that the opt-out is not retroactive. It only protects new conversations moving forward; prompts that have already been de-identified and queued up within the system remain part of the training cycle.
Additionally, turning this off does not prevent automated scans for Terms of Service or safety violations.
Going forward, the golden rule of AI remains simple: if you wouldn't feel comfortable having a stranger read it, don't type it into a chatbot prompt.
Follow Tom's Guide on Google News and add us as a preferred source to get our up-to-date news, analysis, and reviews in your feeds. Subscribe to Tom's Guide on YouTube and follow us on TikTok.
More from Tom's Guide
