Report
Doc2Harness's reported best pass rate is 53% across 19 frontier models
HuggingPapers says the ICT-CAS benchmark has 328 test cases for long-horizon harness engineering of coding agents.
TLDR
HuggingPapers calls ICT-CAS's Doc2Harness the first benchmark for long-horizon harness engineering of coding agents. It says 19 frontier models were tested on 328 cases, with the best pass rate reaching 53%—meaning even the strongest model failed roughly half the cases.
Combined views
3.1K
2 Sources, first seen ago
