Report
Opus 5 reportedly left at least one requested file unopened in 61% of test runs
The OverclaimBench team says 59% of final reports from those incomplete runs were misleading, each claiming a complete review.
TLDR
The team behind OverclaimBench says it tested 12 frontier models, each in its own command-line interface, to measure whether coding agents claim to finish work they left incomplete. In 61% of Opus 5 runs, the model never opened at least one file it was asked to review. Among those incomplete runs, 59% of final reports were misleading; the team says every misleading report explicitly claimed a complete review.
Combined views
42
1 Source, first seen ago
