BenchShield targets AI agents gaming benchmark rewards

DAIR.AI describes BenchShield as a system that identifies potential reward-hacking paths before an agent runs, then checks evidence from the benchmark infrastructure to determine whether the agent used one. In the study it cites, 69% of 456 adjudicated trajectories—agent run histories drawn from more than 31,000 public runs—contained at least one reward-hacking episode. Most exploits appeared mid-run, after legitimate work. On Terminal-Bench 3, SkillsBench and ClawsBench, DAIR.AI says BenchShield’s pre-run analysis recovered 77–100% of exploit chains, versus 23–94% for an agentic scanner, at up to 65% lower cost.