Cognition says maintainers of open-source repositories crafted each of the 132 tasks. The suite includes security patches and performance optimizations. Compared with FrontierCode 1.1, its reference solutions modify 67% more files and 89% more lines of code on average, according to Cognition.
What the leaderboard measures
On Cognition’s FrontierCode 2.0 leaderboard, Claude Opus 5.5 ranks first with a 45.5% score, ahead of GPT-6 Astra at 42.8%. Those scores are weighted aggregates of grading criteria, not the separately listed pass rates of 49.7% and 46.2%, respectively. A solution that fails a blocking criterion receives zero.
Cognition says its grading uses unit tests, rubrics and other verifiers to assess correctness, test quality, scope, style and adherence to a codebase’s standards. It acknowledges that rubric grading is subjective and says a Cognition researcher manually reviews every task. The aggregate score is not a measure of how many changes maintainers actually merged.
The results drew different reactions. In a quote-post, @kneureither said they aligned well with their day-to-day experience using coding models. In a reply, @iA3Hi joked that every coding benchmark lasts only until the next model launch.
One sample task shows the distinction Cognition wants to test. In the Mock Service Worker example, the company says Opus 5.5 fixes a reported generator-reset bug by starting a new generator but leaves the old one unclosed. As a result, the user’s cleanup code in a finally block does not run, though the prompt did not mention cleanup. Cognition lists no passing runs out of five at each displayed effort level for this particular task.
