Paweł Huryn Shares GPT-6 Astra Bug Hunt Scores
AI product manager shares test results for multiple models on 105 bugs across two repositories.
TLDR
In a post on X, Paweł Huryn described testing GPT-6 Astra along with Fable 5.1, GPT-5.6 Sol, and Gemini 3.8 Flash. The task involved finding and fixing bugs in two real repositories that contained a total of 105 bugs. Huryn reported these results: GPT-6 Astra found and fixed 48, Fable 5.1 found and fixed 43, GPT-5.6 Sol found and fixed 42, and Gemini 3.8 Flash found and fixed 20. He also noted the efficiency of the process in terms of time and cost, and included a screenshot of a BUG HUNT BENCH Leaderboard table dated 4 Sep 2026.
Combined views
362.2K
3 Sources, first seen 26d ago