Now that Claude Fable is out, I am testing it against my favorite private eval: a certain minor unsolved problem in multi-armed bandits that I will stay quiet about.
So far, it's reached the same barriers as Opus 4.7, but much, much faster.
It thinks I have been a helpful user.
Opus and GPT have so far failed to solve this problem - by falling into a rabbit hole where the mathematical propositions become more and more and more complex, until we are at many pages of calculations that are so heavy that a mistake is too hard to detect.
Now that Claude Fable is out, I am testing it against my favorite private eval: a certain minor unsolved problem in multi-armed bandits that I will stay quiet about.
So far, it's reached the same barriers as Opus 4.7, but much, much faster.
It thinks I have been a helpful user.
On basis of experience, I believe that a solution should be 5-10 pages.
This problem is not trivial, there are technical barriers that need to be overcome, but I also don’t believe it’s that hard.