Report
AI agents mishandle records and claims in Hospital-bench simulation, @omarsar0 reports
@omarsar0 reports GPT-6.2 Astra led average scores, while Claude Opus 5.5 fully passed the most tasks.
TLDR
According to @omarsar0, Hospital-bench tested 20 models on 48 tasks in a simulated hospital system. GPT-6.2 Astra led with a 62.4% mean score, while Claude Opus 5.5 fully passed the most tasks: 17. Errors included restricted-record disclosures and improper changes to paid claims. Astra and Gemini 4 Neon had no tasks zeroed for harm; some models had as many as 10.
Combined views
1.7K
2 Sources, first seen ago
Comments