In an emergency department at Beth Israel Deaconess Medical Center in Boston, a patient with a pulmonary embolism began to worsen after initial improvement. The medical team suspected the medication was failing. An AI reasoning model, scanning the same electronic health record, came to a different conclusion: a history of lupus, it proposed, could be driving heart inflammation that better explained the decline. It was right. That case, reported by NPR on April 30, 2026, describes a study in the journal Science in which an OpenAI reasoning model matched or outperformed experienced physicians on real emergency-department cases — and it captures both halves of the AI patient safety question now facing medicine: the technology is becoming genuinely capable, and its value still depends entirely on how clinicians verify, question, and act on what it says.
AI has moved from pilot projects into routine care at remarkable speed. The FDA’s public list of AI-enabled medical devices contained 1,466 unique device entries when reviewed on July 24, 2026, and the American Medical Association’s most recent physician survey found that more than 80% of physicians now use AI in their professional work — roughly double the 38% who said so in 2023. Yet the research record is blunt about the trade: algorithms that catch cancers in one setting miss sepsis in another, and physicians who lean on AI uncritically can inherit its errors. This article examines what the verified evidence says about when AI helps, when it harms, and why the physician’s oversight role — not the algorithm itself — is emerging as the deciding factor in whether everyday AI use makes patients safer.
How fast AI entered the exam room — and why that pace matters
The adoption curve is steep on every available measure. In February 2025, the AMA reported that 66% of surveyed physicians used health AI in 2024, a 78% jump from 38% in 2023, based on a survey of nearly 1,200 physicians across specialties and employment types. By the AMA’s 2026 survey wave, usage had passed 80%, and more than three-quarters of physicians said AI improves their ability to care for patients, up from 65% in 2023, according to the association’s updated research summary. The trade press tracked the same finding: Fierce Healthcare summarized the 2026 survey this way.
Regulatory clearances tell the same story from the supply side. A count of the FDA’s AI-enabled medical device list, performed on July 24, 2026, shows authorizations accelerating year over year — from 18 devices with decision dates in 2016 to 333 in 2025. Radiology dominates the list, which matters because radiology is where the strongest clinical trial evidence for AI also exists.
Speed creates its own risk. A tool adopted by four-fifths of a profession within three years will be used by clinicians with very different levels of training, skepticism, and institutional support. That gap — between what the tools can do and what users understand about them — is where physicians either add safety or lose it.
Where physicians and AI already work well together
The strongest controlled evidence for clinical AI comes from Sweden. The MASAI trial, published in The Lancet Oncology in August 2023, randomized 80,033 women in a national mammography screening program to either AI-supported screen reading or standard double reading by two radiologists. The AI group detected 244 screen-detected cancers against 203 in the control group — a cancer detection rate of 6.1 per 1,000 screened participants versus 5.1 per 1,000 — while recall rates (2.2% versus 2.0%) and false-positive rates (1.5% in both) stayed comparable, and screen-reading workload fell by 44.3%, according to the trial’s published safety analysis. Precision matters here: the detection-rate ratio of 1.2 reached a p-value of 0.052, just past the conventional significance threshold, so the trial’s headline conclusion was that AI-supported screening was safe and far less labor-intensive — not that it definitively catches more cancer. The authors’ own interpretation is the important part for everyday practice: AI did not replace the radiologist. It triaged examinations so that a single radiologist read low-risk cases and two radiologists, aided by AI risk scores and detection marks, read the highest-risk ones. The physician stayed inside every decision; the system simply spent human attention where it mattered most.
Documentation is the other proven win. On August 14, 2024, Kaiser Permanente announced that an ambient-listening documentation tool from Abridge had become available to clinicians across its 40 hospitals and more than 600 medical offices in eight states and Washington, D.C. — what the organization described as the largest implementation of its kind in the United States at the time. Two design choices in that rollout are worth underlining, because they are exactly the physician-oversight behaviors this article argues for: the tool requires patient consent, and clinicians review every note before it enters the medical record. “It allows doctors to have more meaningful interactions with patients by devoting their full attention to patient care without the distraction of typing,” said Dr. Linda Tolbert, executive medical director of Washington Permanente Medical Group, in the announcement. The pitch is not that AI writes the note; it is that a freed-up physician looks at the patient instead of the screen — and then checks the machine’s work.
Diagnostic AI has made real progress too, though with caveats that keep the physician central. Google’s AMIE system, described in Nature on April 9, 2025, was tested against primary care physicians in 159 simulated case scenarios with trained patient-actors, using a randomized, remote objective structured clinical examination design. AMIE showed superior diagnostic accuracy to the physicians on several measures — but the patients were simulated, the consultations text-based, and the authors framed the system as a research step toward conversational diagnostic support, not a deployable product. And in the Science study behind the Beth Israel opening scene, the researchers themselves drew the boundary: the AI worked from text alone, while clinicians integrate images, sounds, and nonverbal cues. “It works for making diagnoses in the real world,” said study author Dr. Adam Rodman of Beth Israel, as NPR reported — a conclusion about capability, not about removing clinicians from the loop.
The AMA video above is an industry-body production, so treat it as informed advocacy rather than independent evidence — but it usefully captures the framework physicians themselves are being taught: AI as an assistive layer under human control, what the AMA calls “augmented intelligence.”
The failure modes are documented, not hypothetical
Against those successes sits the most cited cautionary tale in clinical AI. In 2021, a University of Michigan team externally validated the Epic Sepsis Model — a proprietary prediction tool then implemented at hundreds of U.S. hospitals — against 38,455 hospitalizations at their own academic health system. Published in JAMA Internal Medicine, the study found the model identified sepsis with an area under the ROC curve of 0.63, far below the performance its developer reported. More consequentially for patients, the model missed 67% of patients who actually developed sepsis, while generating alerts for 18% of all hospitalized patients — “creating a large burden of alert fatigue,” the authors wrote. A tool that alarms constantly and still misses two-thirds of its target condition does not just fail to help; it trains clinicians to ignore alarms, including good ones. The physician’s role here is unglamorous and essential: demanding local validation before trusting a score, and knowing that a widely deployed product is not the same thing as a proven one.
The second failure mode is subtler: even excellent AI may not improve the physician who uses it. In a randomized clinical trial published in JAMA Network Open in October 2024, Ethan Goh and colleagues gave 50 physicians access to GPT-4 while they worked through diagnostic cases. The physicians with the LLM scored a median of 76% on diagnostic reasoning, statistically indistinguishable from the 74% scored by physicians using conventional resources. The surprise came from the control condition: GPT-4 working alone scored 16 percentage points higher than the conventional-resources physician group. The authors’ conclusion was not that physicians should step aside — it was that “the need for technology and workforce development to realize the potential of physician-artificial intelligence collaboration” is the real bottleneck. Doctors largely used the chatbot like a search engine, and the study built in no workflow or training to do otherwise. Buying the tool is not the same as building the skill.
That warning reflects a documented hazard: generative models sometimes produce fluent, confident, wrong answers, and patients are now bringing chatbot output into consultations. A physician who cannot explain why a plausible-sounding AI answer is wrong has lost control of the clinical conversation.
Oversight, liability, and trust: the unresolved rules
Physicians know the stakes. In the AMA’s 2024 survey, 47% of physicians ranked increased oversight as the single most important regulatory action that would build their trust in AI tools, ahead of every other option. The AMA’s policy principles push on transparency (patients and physicians should know when AI is being used), data privacy, payor use of automated decision-making, and — most painfully unresolved — physician liability when an AI-enabled tool contributes to harm. “There remain unresolved physician concerns with the design of health AI and the potential of flawed AI-enabled tools to put privacy at risk, integrate poorly with EHR systems, offer incorrect conclusions or recommendations and introduce new liability concerns,” said Dr. Jesse M. Ehrenfeld, then the AMA’s immediate past president, when the 2024 survey results were released. The liability question cuts both ways in daily practice: a physician who follows a bad algorithm may be exposed, and so may a physician who ignores a good one. Until case law and regulation settle that, the safest professional posture is the one the evidence already supports — use AI as a second opinion whose reasoning you can check, and document why you accepted or overrode it.
Globally, the World Health Organization’s 2021 guidance on ethics and governance of AI for health — the product of eighteen months of expert deliberation — set six consensus principles, including preserving human autonomy and ensuring human responsibility for medical decisions. The through-line from Geneva to the AMA to the MASAI trial design is consistent: keep a qualified human accountable for the decision, and make the machine’s reasoning visible enough to audit.
The PBS segment is journalism, not peer review, but its framing matches the research record: the same technology that surfaces a missed lupus diagnosis in Boston can flood a ward with false sepsis alarms in Ann Arbor. The difference is rarely the algorithm alone — it is the validation, workflow, and clinical supervision wrapped around it.
What physicians can do now to keep AI safe
The evidence points to a short, concrete checklist rather than a posture of blanket enthusiasm or refusal:
- Ask for local validation. The Epic sepsis study exists because one health system checked a vendor’s model against its own patients. Before trusting any score, ask whether it was validated on a population like yours.
- Review before signing. Kaiser Permanente’s ambient-documentation rollout builds in physician review of every note and explicit patient consent. Treat those as minimum standards, not optional features.
- Use AI as a second opinion, not a first answer. The Goh trial showed physicians underused the model’s strengths; forming your own differential first, then interrogating the AI’s, guards against automation bias.
- Tell patients when AI is involved. AMA policy and WHO guidance both treat transparency as foundational to trust — and consent is already standard in leading deployments.
- Push your institution on liability and training. Nearly half of physicians want more oversight; that pressure is how procurement standards and case law get written.
None of these require technical expertise beyond what medical training already provides. They require the same habits physicians apply to a new drug: check the evidence, watch for side effects, and stay responsible for the decision.
The next test
Return to that emergency department in Boston, where the machine saw lupus in the record and the humans did not. The study’s authors were careful about what the result means: an AI that reads text superbly still cannot examine a patient, notice what the record omits, or weigh what a family is too frightened to say. The AMA’s 2026 survey shows a profession that has, in three years, gone from mostly avoiding these tools to mostly using them — while still naming privacy, flawed outputs, and liability as its top concerns. Both things are true at once, and that is precisely the point. The FDA list will keep growing past its current 1,466 entries; new models will keep outperforming old benchmarks. Whether that translates into safer everyday care will be decided in the least technological place imaginable: in the judgment of the physician who reads the output, checks it against the patient in front of them, and signs the note.
References
- NPR — “In real-world test, an AI model did better than doctors at diagnosing patients” (April 30, 2026): https://www.npr.org/2026/04/30/nx-s1-5804474/ai-doctors-openai-patient-care-diagnosis
- American Medical Association — “2 in 3 physicians are using health AI—up 78% from 2023” (February 26, 2025): https://www.ama-assn.org/practice-management/digital-health/2-3-physicians-are-using-health-ai-78-2023
- American Medical Association — “Augmented intelligence in medicine” (accessed July 24, 2026): https://www.ama-assn.org/practice-management/digital-health/augmented-intelligence-medicine
- American Medical Association — “AMA principles for augmented intelligence development, deployment, and use” (PDF): https://www.ama-assn.org/system/files/ama-ai-principles.pdf
- U.S. Food and Drug Administration — “Artificial Intelligence-Enabled Medical Devices” (accessed July 24, 2026): https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-enabled-medical-devices
- Wong A, et al. — “External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients,” JAMA Internal Medicine (2021): https://doi.org/10.1001/jamainternmed.2021.2626
- Lång K, et al. — “Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI),” The Lancet Oncology (August 2023): https://doi.org/10.1016/S1470-2045(23)00298-X
- Goh E, et al. — “Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial,” JAMA Network Open (October 2024): https://doi.org/10.1001/jamanetworkopen.2024.40969
- Tu T, et al. — “Towards conversational diagnostic artificial intelligence,” Nature (published April 9, 2025): https://www.nature.com/articles/s41586-025-08866-7
- Kaiser Permanente — “Kaiser Permanente Improves Member Experience With AI-Enabled Clinical Technology” (August 14, 2024): https://about.kaiserpermanente.org/news/press-release-archive/kaiser-permanente-improves-member-experience-with-ai-enabled-clinical-technology
- World Health Organization — “Ethics and governance of artificial intelligence for health” (2021): https://www.who.int/publications/i/item/9789240029200
- American Medical Association (YouTube) — “Navigating AI in Health Care”: https://www.youtube.com/watch?v=9uVVY3kOkXg
- PBS NewsHour (YouTube) — “The promises and potential pitfalls of artificial intelligence in medicine”: https://www.youtube.com/watch?v=CWBTV0Qcxts
- Fierce Healthcare (X) — post summarizing AMA 2026 physician AI survey (March 13, 2026): https://x.com/FierceHealth/status/2032509495881117928
- AMA Ed Hub (X) — post on AI chatbot misinformation guidance (June 5, 2026): https://x.com/AMAEdHub/status/2062957627294851522
- U.S. Navy via Wikimedia Commons — “CP25 Leukemia Patient Consultation (9123625),” public domain (June 14, 2025): https://commons.wikimedia.org/wiki/File:CP25_Leukemia_Patient_Consultation_(9123625).jpg