This Nature Medicine published study has a strong warning for AI in healthcare.
Frontier AI in healthcare has a hidden failure mode: it can look medically brilliant while being clinically unready.
The authors tested frontier AI models on health benchmarks, then added stress tests to see whether the models were actually robust or just good at passing exams.
Found that the models were brittle.
i.e the models could give the right answer in a normal test, but fail when the question was slightly changed, when important information was removed, or when the image-text setup was altered.
One strange result was that some models could still guess the correct answer even when key inputs were removed, which suggests they may be using shortcuts rather than truly understanding the medical case.
the models sometimes gave convincing explanations that sounded medical and logical, but the reasoning was flawed.
The final conclusion is not “AI is useless in medicine” but that "benchmark success is not the same as clinical readiness.”
"GPT-5.5 Pro Outperforms 99.9% of Doctors and Predicts AI Superiority in Medicine by Next Year"
An optimistic AI viewpoint since there are no studies in real world medicine; all we have now are simulations, case vignettes, patient actors, etc. Beyond that, performance metrics need work as we've recently open-source published. Here's what the editors @NatureMedicine wrote: This study cuts through the optimism surrounding medical AI by showing how easily benchmark success can be mistaken for real readiness. In medical AI, impressive scores are clearly not the same as trustworthy capability.” [link below] Our results were independently confirmed by @yishan in current frontier models, such as GPT-5.5 Pro, text below https://www.nature.com/articles/s41591-026-04501-8 https://x.com/yishan/status/2070742742133780960"
