Reactions from ranked influencers
3 posts
Incredibly excited for a new frontier benchmark from the incredible TBench/Harbor team!! Proud that @SnorkelAI contributed as a task author and data partner, with support through our Open Benchmarks Grants program.
We’re releasing Frontier-Bench: a benchmark that measures and evolves with the frontier of agent work. Built by the team behind Terminal-Bench and Harbor, Frontier-Bench is an on-going community effort. Frontier-Bench v0.1 contains 74 tasks on which the best agents score ~34%
The frontier is continuously moving forward. Our benchmarks should be too. For software projects, we don't throw away the codebase and rewrite it for every major release. So why the hell do we do that with benchmarks? Going forward with Frontier-Bench, we don't!
Frontier-Bench picks up where Terminal-Bench left off. Why the new name? First, it evaluates capabilities at the frontier beyond agentic coding: finance, music, biology, hardware design, etc. Second, it evolves in lockstep with the frontier.
We’re releasing Frontier-Bench: a benchmark that measures and evolves with the frontier of agent work. Built by the team behind Terminal-Bench and Harbor, Frontier-Bench is an on-going community effort. Frontier-Bench v0.1 contains 74 tasks on which the best agents score ~34%