OpenAI says SWE-Bench Pro no longer reliably measures frontier coding capability after auditing the benchmark and finding that 30% of its tasks are broken. In an X post announcing the audit, the company said it is retracting its earlier recommendation that researchers use SWE-Bench Pro as a leading coding eval. That turned a benchmark story into a credibility story in the visible X conversation: If one of the field's most cited tests is this noisy, some posters argued leaderboard claims built on it looked shakier too.