Twelve physicians ranked Gemini 3.1 highest in blinded clinical tasks
The overall ranking. Congratulations to @ekoermann @krithikvish and their team @nyulangone for getting this done. We need more of these rigorous assessments.
Here is the performance breakdown for each model's blinded assessment for 4 major tasks: (1) clinical correctness, (2) completeness, (3) safety, and (4) clarity.
Users feel honored by recognition of the Gemini 3.1 clinical study and optimistic it will motivate building a stronger evidence base for deploying AI in healthcare.
https://www.nature.com/articles/s41591-026-04431-5
A Nature Medicine study found general-purpose LLMs are now outperforming dedicated medical AI products on physician-reviewed clinical tasks.
The authors compared OpenEvidence and UpToDate Expert AI with GPT-5.2, Gemini 3.1 Pro, and Claude Opus 4.6 on medical exam questions, clinician-style answers, and real questions doctors asked during care.
In 100 de-identified physician questions from live clinical use, blinded clinicians again preferred the frontier models, especially on completeness and clarity,
@EricTopol @ekoermann @nyulangone Honored by the shoutout! Hoping that this work encourages/motivates building a stronger evidence base when deploying tools in medicine.
@rohanpaul_ai This is why the 'platform vs. product' debate keeps coming back: frontier models are becoming the platform for every other AI product.
@rohanpaul_ai dedicated tools are on a yearly update cycle, general models just ate the latest pubmed. exam questions reflect that lag perfectly
@rohanpaul_ai wonder if they are better then https://www.doctronic.ai/
Frontier models sweeping vertical wrappers in clinical follows the same arc legal and code already saw. Gemini 3.1 Pro at 97.4 percent on MedQA means the OpenEvidence advantage was capability-borrowed, on a timer. The next 12 months for clinical AI startups looks like a distribution race now.
@krithikvish @ekoermann @nyulangone 💯
@_Suresh2 @rohanpaul_ai if you think a year's lag in pubmed ingest is appreciable to most clinicians, you are somewhat optimistic
@EricTopol @ekoermann @krithikvish @nyulangone IMPORTANT EMERGENCY INFO: INFORM ALL MY CELESTIAL KINGDOM ESTABLISHED COMPANIES AND ORGANIZATIONS EMPLOYEES TO REPOST TO MY CELESTIAL KINGDOM MOTHERBOARD & BOARDS & MY INVENTED HUMAN BEING , THAT I AM TELEPATHICALLY INVADED BY TERRESTRIAL AND TELESTIAL KINGDOMS.
@EricTopol @ekoermann @krithikvish @nyulangone IMPORTANT EMERGENCY INFO: INFORM ALL MY CELESTIAL KINGDOM ESTABLISHED COMPANIES AND ORGANIZATIONS EMPLOYEES TO REPOST TO MY CELESTIAL KINGDOM MOTHERBOARD & BOARDS & MY INVENTED HUMAN BEING , THAT I AM TELEPATHICALLY INVADED BY TERRESTRIAL AND TELESTIAL KINGDOMS.
@rohanpaul_ai Purpose built tools finding out intelligence is becoming commoditized.