Many users praised François Chollet's framing of intelligence as iterative failure and adaptation for valuing graceful recovery and learning velocity, while some dismissed it as ignoring benchmark biases that reward only final success.
Based on 18 visible X reactions from 40 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@fchollet This reframes evaluation nicely. We keep scoring models on how rarely they fail known tests—measuring yesterday. The real signal: how fast does a model absorb a novel failure and update? Graceful failure is table stakes; adaptation velocity is the actual bar.
@fchollet the problem is every benchmark rewards a confident one shot answer, not the failed attempt that led to it. so we keep training and shipping models that hide the retry loop instead of showing it. wrong thing to optimize for if adaptability is the actual goal
@fchollet the part that's hard to celebrate is that we don't score the recovery, we score the final answer. two runs that land on the same correct output look identical on a benchmark even if one took three wrong turns to get there and the other took one.
@fchollet graceful failure is underrated that iterative loop is exactly what makes systems resilient
@fchollet This reframes evaluation nicely. We keep scoring models on how rarely they fail known tests—measuring yesterday. The real signal: how fast does a model absorb a novel failure and update? Graceful failure is table stakes; adaptation velocity is the actual bar.
@fchollet the problem is every benchmark rewards a confident one shot answer, not the failed attempt that led to it. so we keep training and shipping models that hide the retry loop instead of showing it. wrong thing to optimize for if adaptability is the actual goal
@fchollet the part that's hard to celebrate is that we don't score the recovery, we score the final answer. two runs that land on the same correct output look identical on a benchmark even if one took three wrong turns to get there and the other took one.
@fchollet graceful failure is underrated that iterative loop is exactly what makes systems resilient
@fchollet Intelligence isn't about never failing. It's about learning faster than the failure.
@fchollet 💯
The process of trying, failing, updating a mental model, and trying again is the core of intelligence. We should celebrate models that fail gracefully and adapt instantly.
Gradually, more and more people began to get what intelligence is truly about… https://twitter.com/fchollet/status/2076462727422431312
Many users praised François Chollet's framing of intelligence as iterative failure and adaptation for valuing graceful recovery and learning velocity, while some dismissed it as ignoring benchmark biases that reward only final success.
Based on 18 visible X reactions from 40 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@fchollet 💯
The process of trying, failing, updating a mental model, and trying again is the core of intelligence. We should celebrate models that fail gracefully and adapt instantly.
Gradually, more and more people began to get what intelligence is truly about… https://twitter.com/fchollet/status/2076462727422431312