Doctors and health tech experts agree that AI has a growing role in healthcare, but many are uneasy about using chatbots directly as medical advisors for patients. While tools like the newly announced ChatGPT Health aim to offer more privacy and personalization, concerns remain about misinformation, data privacy, hallucinations, and regulatory oversight. Many clinicians believe AI’s safest and most impactful use lies behind the scenes (helping doctors reduce administrative work and improve access to care) rather than replacing medical judgment at the patient level.
Key Points
A surgeon shared a real-world example of ChatGPT providing misleading medical risk statistics that didn’t apply to the patient’s condition.
Despite these risks, some doctors welcome ChatGPT Health, which promises privacy protections and safeguards around patient data.
ChatGPT Health allows users to upload medical records and sync health apps, raising concerns about data moving outside HIPAA-compliant systems.
AI hallucinations remain a major concern, especially in healthcare, with some models shown to be more prone to factual errors.
Experts argue that limited access to doctors (with wait times of three to six months) makes AI support appealing to patients.
Many clinicians see greater value in provider-side AI, such as automating administrative tasks that consume up to half of doctors’ time.
Stanford’s ChatEHR and Anthropic’s Claude for Healthcare focus on helping clinicians and insurers work more efficiently.
A fundamental tension exists between patient safety priorities and tech companies’ responsibility to shareholders.
Key Quotes
“I recently had a patient come in… with a dialogue printed out from ChatGPT that said this medication has a 45% chance of pulmonary embolism.” — Dr. Sina Bari
“Formalizing it so as to protect patient information and put some safeguards around it… is going to make it all the more powerful for patients to use.” — Dr. Sina Bari
“All of a sudden there’s medical data transferring from HIPAA-compliant organizations to non-HIPAA-compliant vendors.” — Itai Schwartz
“If your choice is to wait six months for a real doctor, or talk to something that is not a doctor… which would you pick?” — Dr. Nigam Shah
“Imagine cutting 20, 30 minutes out of each of them — it’s a dramatic time savings.” — Mike Krieger, Anthropic
“Patients rely on us to be cynical and conservative in order to protect them.” — Dr. Sina Bari
Implications
AI health chatbots may continue to grow in popularity, but unchecked use could expose patients to misinformation and privacy risks.
Regulators may face pressure to clarify how patient data can be shared with AI platforms.
The healthcare system may see faster, safer AI adoption through tools that support clinicians rather than directly advising patients.
Reducing administrative burdens with AI could improve access to care and reduce reliance on consumer chatbots for medical guidance.
The ethical balance between innovation, patient safety, and corporate incentives will remain a central challenge as AI becomes embedded in medicine.
AISQ | Squirrly created this web Customer App for all of you who own licenses for AISQ’s Next Level Marketing AI, AISQBusiness, Squirrly SEO, Hide My WP Ghost and more.
