/Tech3h ago

Nikhil Chandak finds Anthropic's Claude Fable 5 fails to dynamically update forecasting predictions, tying GPT-5.5 on the FutureSim benchmark

Testing used a February-March subset to prevent data contamination

245456.6K

Original post

Nikhil Chandak@nikhilchandak29

🚨 Claude Fable 5 on FutureSim 🚨

While Anthropic folks have been using it internally for predicting evals, we actually put it to test on FutureSim.

We found it has very strong priors but fails to update predictions over time and ends up no better than GPT-5.5!

We report our results on Feb-March subset to minimize contamination with Fable's knowledge cutoff of Jan.

Sholto Douglas@_sholtodouglas

we don’t even run evals anymore we just ask Claude what the score will be

4:51 AM · Jun 10, 2026 · 6.1K Views

/Tech3h ago

Testing used a February-March subset to prevent data contamination

245456.6K

Original post

Nikhil Chandak@nikhilchandak29

🚨 Claude Fable 5 on FutureSim 🚨

While Anthropic folks have been using it internally for predicting evals, we actually put it to test on FutureSim.

We found it has very strong priors but fails to update predictions over time and ends up no better than GPT-5.5!

We report our results on Feb-March subset to minimize contamination with Fable's knowledge cutoff of Jan.

Sholto Douglas@_sholtodouglas

we don’t even run evals anymore we just ask Claude what the score will be

4:51 AM · Jun 10, 2026 · 6.1K Views

Sentiment

Sentiment building, check back later.

Cluster Engagement

Posts from X

Most Activity

No ranked X posts are available for this story yet.