Mercor AI quietly revised its Fable 5 numbers on the APEX-SWE benchmark, dropping the model from a June high of 65.5% Pass@1 to 54.8% in the July re-release, a roughly ten-point slide that still leaves it ahead of Opus 4.8 at 45.3%.
Fable 5 is back and we’ve got results for the re-released version on APEX-SWE.
While it did not perform as well as its earlier version from June, the model still significantly outperforms Opus 4.8.
Fable 5 (June): 65.5% Pass@1 Fable 5 (July): 54.8% Pass@1 Opus 4.8: 45.3% Pass@1
This re-release scored about 10 points below the original Fable 5, however it still beat Opus 4.8 by more than 9 points.
No public detail has emerged on whether the change reflects a model update, prompt tweak, or harness revision, leaving observers to wonder how sensitive the benchmark's real-world tasks are to small shifts in evaluation setup.
Even after the regression the model remains competitive on the integration and observability tasks that make up most of the benchmark, suggesting the drop narrows but does not erase its lead over at least one major rival.
Negative users mocked the Fable 5 re-release as nerfed or Feeble after its performance dropped on APEX-SWE, viewing the regression as an unnecessary safety tradeoff.
No Digg Deeper questions have been answered for this story yet.
10% regression for Fable 5 is crazy
Fable 5 is back and we’ve got results for the re-released version on APEX-SWE.
While it did not perform as well as its earlier version from June, the model still significantly outperforms Opus 4.8.
Fable 5 (June): 65.5% Pass@1 Fable 5 (July): 54.8% Pass@1 Opus 4.8: 45.3% Pass@1
This re-release scored about 10 points below the original Fable 5, however it still beat Opus 4.8 by more than 9 points.
We're calling it Feeble now.
Fable 5 is back and we’ve got results for the re-released version on APEX-SWE.
While it did not perform as well as its earlier version from June, the model still significantly outperforms Opus 4.8.
Fable 5 (June): 65.5% Pass@1 Fable 5 (July): 54.8% Pass@1 Opus 4.8: 45.3% Pass@1
This re-release scored about 10 points below the original Fable 5, however it still beat Opus 4.8 by more than 9 points.
@BrendanFoody They lobotomized it :(
interesting
10% regression for Fable 5 is crazy
@BrendanFoody 10 points, 16% relative.
@beffjezos 7ZhZiwMFMymf1JmkjGxRyod8JMPu29c2aemU5yTxpump
$feeble
@BrendanFoody Feels understated, it's not the standout model it was before in my experience
@BrendanFoody incredibly meaningful drop in performance ye
@beffjezos Well it was anthropic begging for this really.
But I guess that's what they wanted, a state takeover of the industry. Of course that means no super intelligent models for regular people.
Just us dum dums out here.
@BrendanFoody Yep
@beffjezos easy to tell who’s not actually using the model.
@beffjezos Lol Rename to Feeble at this point
@BrendanFoody @davefontenot Opus 4.9
@hrkrshnn @BrendanFoody
@beffjezos yes but at least we'll be safer knowing that the terrorists will be done in by their potatoes
@BrendanFoody very misleading. when the request gets to fable 5, it performs exactly the same as before. it's just that more requests fallback to opus (with a clear notice)
@beffjezos Glad to have it, know it’s not what’s best
@beffjezos 🤣😂
@beffjezos In French, it’s “faible.” The “i” stands for inconsistency.
@beffjezos So we're going in reverse now?