Top coding-agent setup reportedly passes 23.9% of ττ-bench’s held-out customer simulations

A post describing ττ-bench says the benchmark asks coding agents to build an agent using scattered company records, a client, an API, existing code and a budget, then handle unseen customer requests. It reports that the top setup, Claude Opus 5 with Claude Code, passed 23.9% of held-out customer simulations across 53 tasks, compared with 82.2% for the expert-authored reference. The author argues that better code generation alone is not enough, pointing to agents barely questioning clients, reusing familiar designs and writing tests that missed their own errors.