Report
Some ProgramBench model evaluations reportedly cost over $10,000
A user says evaluating some models on ProgramBench now costs more than $10,000 and shares a ranking based on tasks per dollar.
TLDR
A user says frontier-benchmark evaluations have never been more expensive, with some ProgramBench model evaluations now costing over $10,000. They share a ranking that looks at tasks per dollar instead.
Combined views
639
2 Sources, first seen 11h ago
11 likes
