Researcher Disputes 76% HLE Score Claim for Astra
Florian Brand replies on X that the HLE benchmark does not contain 76% solvable tasks.
TLDR
Florian Brand, a research engineer at Prime Intellect, replied to @teortaxesTex on X that he does not think the HLE benchmark has 76% solvable tasks. The exchange concerns an earlier post that reported high scores for Astra across several AI benchmarks, including FrontierMath and Riemann Bench. Brand's statement directly challenges the 76% figure cited in the discussion of the Astra evaluation.
Combined views
2.1K
1 Source, first seen 32d ago