A human team would need over two months.
No slowing down in sight, this is so weird.
Fable 5 is state-of-the-art on nearly all tested benchmarks, with exceptional performance in software engineering, knowledge work, scientific research, and vision.
The longer and more complex the task, the larger Fable 5’s lead over our other models.
Some users express excitement over Fable 5 topping benchmarks as progress feels unreal, while others doubt the results due to unverifiable figures, silent steering, and possible fabrication.
@francoisfleuret I think of this as the point of inflection where it'll slow down because we'll stop being able to differentiate or measure most improvements to models
bigger models should in theory have more potential capability but the difficulty of the training data is bounded by us
@francoisfleuret It's now impossible to verify these figures because the model is silently steered
@francoisfleuret the wall
@francoisfleuret No one can prove that biology/cybersecurity bench wasn't fabricated since you can't use it for biology/cybersecurity. It's bullshit.
Controversies on model nerfing aside, Fable seeing its biggest performance gains on more complex tasks is a clear indicator of large AI capabilities jumps ahead.
(We’re far from saturating the frontier and far from defining / measuring the next wave of frontier tasks)
@francoisfleuret Look at the price per token then try to normalize it for benchmark scores, and then you realize this has slowed down significantly since the end of last year.
@francoisfleuret the "weird" part is actually the respectful one in this whole timeline
@francoisfleuret Got a very weird feeling at work today myself, feeling the ASI not the AGI
@francoisfleuret There’s a wall, but transformers did not hit it, wallets did
@francoisfleuret not every week opens like this
must feel like riding lightning and sand at the same time
@francoisfleuret acting like youre not impressed but you definitely reloaded the benchmarks after reading this
@francoisfleuret agents behind the curtain.
@airesearch12 @francoisfleuret this wall is crazy
@francoisfleuret weird is a generous word for it
most people dont even use half the features they add per update
@francoisfleuret first time in a while progress actually feels unreal
@francoisfleuret had to check if i was reading about a language model or the gym guy from mass effect?
A human team would need over two months.
No slowing down in sight, this is so weird.
Fable 5 is state-of-the-art on nearly all tested benchmarks, with exceptional performance in software engineering, knowledge work, scientific research, and vision.
The longer and more complex the task, the larger Fable 5’s lead over our other models.