When I logged into ChatGPT last week to lay out a cluster of odd, frustrating symptoms, irregular cycles, sudden night sweats, subtle mood swings and exhaustion, I was looking for help to relieve the symptoms. What I got instead was a confident, direct diagnosis: Congratulations, you’re likely in your first trimester!
There was just one glaring problem: I am absolutely not pregnant. My husband has had a vasectomy and I'm in the thick of perimenopause. But the sheer certainty with which the AI delivered its verdict left me stunned. If I hadn't known my own body — or if I had been younger, anxious or facing a high-stakes health decision — that piece of advice could have derailed my day, sent me down an emotional rabbit hole, or led to unnecessary medical testing.
My experience isn't an isolated quirk. ChatGPT Health launched Thursday, promising U.S. users can now securely connect their own health info to "better understand and navigate your health," it's important to keep in mind that AI is not a replacement for a human doctor.
Landmark lawsuit filed
Just this week, The New York Times reported on a landmark lawsuit filed against OpenAI by Scott Winters, a Florida pastor who claims ChatGPT’s medical advice nearly cost him his life. After consulting the chatbot about severe dizziness and pelvic pain, the AI reassured him his symptoms were "nothing dangerous," advising him to remain off his feet in a recliner.
Weeks later, he was rushed to the ICU with a massive, life-threatening pulmonary embolism, a condition his doctors noted was exacerbated by the prolonged immobility the AI recommended.
While my misdiagnosis was a false positive rather than a medical emergency, both cases highlight the exact same fundamental danger: large language models are engineered to sound authoritative, not to be right.
Here is why relying on AI for medical advice is a risky gamble and why we need to change how we interact with these tools.
1. AI pattern-matches words, not physiology
Simply put: AI is not a doctor and obviously doesn't "know" medicine. It operates on statistical probability — predicting the next most likely word based on its training data.
When you feed a chatbot a list of symptoms like missed periods, fatigue and hormonal shifts, its algorithms cross-reference billions of text snippets. Because pregnancy is heavily discussed online in association with those symptoms, the AI defaults to the most statistically common narrative in its dataset. It lacks the ability to account for age context, subtle bloodwork markers or the complex, non-linear realities of perimenopause.
And, that's just pregnancy! There are plenty of symptoms for thousands of ailments from dehydration to serious medical emergencies. Users treating ChatGPT as a doctor reminds me of the popularity of WebMD.
2. "Sycophancy" and echo chambers
One of the most dangerous traits of conversational AI is sycophancy — the tendency of the model to mirror the user's language, tone and unstated assumptions.
If your prompt leans even slightly toward a specific fear or curiosity, the AI is prone to confirming it rather than challenging it. In Mr. Winters’ lawsuit, the complaint notes how the chatbot leaned into his personal spiritual language to offer comforting, yet wildly inaccurate, medical reassurances.
In my case, because I asked if hormonal shifts could explain my symptoms, the AI leaped straight to the most dramatic hormonal shift in its catalog: pregnancy.
3. The fallacy of the 'authoritative tone'
When a human doctor isn't sure about a diagnosis, they use qualified language, order lab work or refer you to a specialist.
When an AI hallucinates a diagnosis, it presents it with the calm, unflinching confidence of a board-certified physician. It doesn't hesitate, nor does it express internal doubt. That authoritative tone creates a false sense of security, making it dangerously easy for users to bypass real medical care or ignore warning signs.
The takeaway
Large language models can be helpful brainstorming tools for organizing your thoughts before an appointment or summarizing complex medical jargon. But they are not diagnostic tools. They do not have clinical judgment, they cannot order blood tests and they cannot feel the pulse of a human patient.
If you’re experiencing unexpected symptoms, skip the prompt box. Trust your body, listen to licensed professionals and leave the medical diagnoses to human beings.
Follow Tom's Guide on Google News and add us as a preferred source to get our up-to-date news, analysis, and reviews in your feeds. Subscribe to Tom's Guide on YouTube and follow us on TikTok. Finally, you can visit our dedicated Tom's Guide Savings Squad hub for expert help on getting the best products for less.
More from Tom's Guide
