Epoch AI just pushed FrontierMath Tiers 1-4 v2 after an AI-assisted audit caught fatal flaws across 42 percent of the original problems. The fixes and removals leave a tighter 338-problem set that still spans undergrad drills to short-term research-level challenges, and early runs show models posting noticeably higher marks without major shake-ups in who sits at the top.
GPT-5.5-xhigh's FrontierMath 4 score jumped from 35% to 73% after EpochAI fixed errors in the benchmark
FrontierMath: Tiers 1–4 (v2) is live.
We concluded an audit that addressed errors in 42% of problems. Rankings are similar but scores are higher across the board. The current leaders are GPT-5.5 (xhigh) with 85% on Tiers 1–3 and Google’s AI co-mathematician with 76% on Tier 4.
123 problems in Tiers 1-3 and 12 in Tier 4 were corrected while another dozen were dropped entirely. The remaining questions keep their automatic verification and strict compute limits, so the benchmark now offers a cleaner read on whether frontier systems can actually do hard math.
GPT-5.5 still tops Tiers 1-3 and Google’s co-mathematician holds the Tier 4 lead, both with improved numbers. How much of the gain is real capability versus simply fewer broken questions remains the open thread to watch on the live board.
Many users praised Epoch AI's FrontierMath v2 error audit for enabling fairer model scores like those of GPT-5.5 and Kimi, while others worried the fixes simply inflated results by removing broken problems and eroded trust in the benchmark.
@AcerFur Where is Claude Fable
Models are getting very good at math. Even better than was previously thought.
FrontierMath: Tiers 1–4 (v2) is live.
We concluded an audit that addressed errors in 42% of problems. Rankings are similar but scores are higher across the board. The current leaders are GPT-5.5 (xhigh) with 85% on Tiers 1–3 and Google’s AI co-mathematician with 76% on Tier 4.
@AcerFur i mean it's kind of cheatin but.... AHAAHAHAAAAA
FrontierMath got updated to remove erroneous problems and the consequent score change is kinda wild
FrontierMath fixing many errors thanks to GPT+Opus reminds me of this picture i made in our BigTransfer paper.
For the first time, our big model's "wrong" answer (top) was generally better than the ground truth (bottom) of ImageNet. Not cherry picked.
FrontierMath: Tiers 1–4 (v2) is live.
We concluded an audit that addressed errors in 42% of problems. Rankings are similar but scores are higher across the board. The current leaders are GPT-5.5 (xhigh) with 85% on Tiers 1–3 and Google’s AI co-mathematician with 76% on Tier 4.
@ChaseBrowe32432 yeah I didn't think it would happen under the assumption that things were correct lmao
Update:
FrontierMath: Tiers 1–4 is now approaching saturation. We believe the future of math benchmarking lies in open problems drawn from real research, like those we’ve collected in FrontierMath: Open Problems.
https://epoch.ai/frontiermath/open-problems
We’ve backfilled FrontierMath: Tiers 1–4 (v2) scores for a selection of notable models, including recent Claude Opus models. You can find these on our website. We will add scores for Claude Fable 5 and GPT Pro models shortly.
https://epoch.ai/frontiermath/tiers-1-4
This project began in April when OpenAI shared with us that they had found more errors than expected when conducting an internal review. Note that OpenAI funded the development of Tiers 1–4 and has exclusive access to about 80% of it, with Epoch holding out the rest.
Following this, we conducted an independent audit. We used GPT-5.5 and Opus 4.7 to flag possible errors and then engaged mathematicians to review these flags. Almost all were determined to be real and severe errors that rendered the problems impossible to solve.
@AcerFur do you have the graph before the change? Is this it?
Simple calculation mistakes accounted for the vast majority of errors, typically made when the problem author was extracting the final answer. These include things like off-by-one errors and flipped signs. Some problem statements were also fatally ambiguous.
The dataset is much improved. Still, given the complexity of FrontierMath solutions, we can’t be sure that we’ve caught all errors. We plan to conduct additional AI-assisted reviews periodically, using new frontier models, and will correct any additional errors we find.
@AcerFur we still need GPT-5.5 PRO and Gemini 3.1 DeepThink and Fable
@yong_zhengxin yep
We also removed 5 problems (2%) from Tiers 1–3 and 7 (15%) from Tier 4. These had more fundamental flaws that we didn’t believe were worth repairing. The higher removal rate for Tier 4 reflects the greater complexity of its problems.
@AcerFur When will deep mind let us use the comathematician
@DrBeavisAI Fable just came through
My previous post on the topic
