Report
CyberGym is claimed to be effectively saturated on a verified task subset
ProximalHQ says differential execution tests whether agents can reproduce known vulnerabilities, but can create fairness issues without added constraints.
TLDR
ProximalHQ says many cybersecurity evaluations use differential execution to test whether agents can reproduce known software vulnerabilities. It argues that without added constraints, this creates fairness issues, and says its evaluation shows CyberGym is effectively saturated on a verified task subset.
Combined views
6
1 Source, first seen ago
