Yann LeCun Explains Why LLMs Lack Real-World Intelligence

"100 million words context window is already possible, which is roughly what a human hears in a lifetime. Inference support is the only bottleneck to achieve it.

And AI Models actually do learn during the context window, without changing the weights."

~ Anthropic CEO Dario Amodei

This is from 11 months back. --- From 'Alex Kantrowitz' YT Channel (Full Video link in comment)

"100 million words context window is already possible, which is roughly what a human hears in a lifetime. Inference support is the only bottleneck to achieve it.

And AI Models actually do learn during the context window, without changing the weights."

~ Anthropic CEO Dario Amodei

This is from 11 months back. --- From 'Alex Kantrowitz' YT Channel (Full Video link in comment)

During a Bloomberg interview, Yann LeCun (@ylecun ) explains why LLMs are limited in terms of real-world intelligence during a Bloomberg interview.

"Language is a very approximate, reduced, quantized, and simplified description of the world, and LLMs can only deal with discrete sequences of symbols. The world is much more complicated than language.

The biggest LLMs are pre-trained on the totality of all the publicly available text on the internet. That’s about 20 trillion words, or 30 trillion tokens.

A token is about 3 bytes. So total 10¹⁴ bytes of text.

This is the amount of data a four-year-old has seen through vision during four years. Now, the text, though, would take 400,000 years to read?

So, there is enormously more data from sensory input, like vision, touch, and everything else, than there could ever be through language."

A child does not need 400,000 years of reading to understand cups, doors, balance, faces, falls, or heat, because the body is already collecting dense feedback from vision, touch, motion, and consequence.

Text strips most of that away.

It turns a living scene into symbols, then asks the model to infer the missing world from traces left by people describing it.

That is why an LLM can sound fluent about physics and still have no native sense of how fragile glass feels in a hand.

Moravec’s paradox names this reversal: the things humans find intellectual can be easier for machines than the things toddlers do without applause.

The hard part is not producing an answer, but building a model of the world that survives contact with weight, friction, surprise, and failure.

----

Link to the full video on Bloomberg's site. Link in comment.

During a Bloomberg interview, Yann LeCun (@ylecun ) explains why LLMs are limited in terms of real-world intelligence during a Bloomberg interview.

"Language is a very approximate, reduced, quantized, and simplified description of the world, and LLMs can only deal with discrete sequences of symbols. The world is much more complicated than language.

The biggest LLMs are pre-trained on the totality of all the publicly available text on the internet. That’s about 20 trillion words, or 30 trillion tokens.

A token is about 3 bytes. So total 10¹⁴ bytes of text.

This is the amount of data a four-year-old has seen through vision during four years. Now, the text, though, would take 400,000 years to read?

So, there is enormously more data from sensory input, like vision, touch, and everything else, than there could ever be through language."

A child does not need 400,000 years of reading to understand cups, doors, balance, faces, falls, or heat, because the body is already collecting dense feedback from vision, touch, motion, and consequence.

Text strips most of that away.

It turns a living scene into symbols, then asks the model to infer the missing world from traces left by people describing it.

That is why an LLM can sound fluent about physics and still have no native sense of how fragile glass feels in a hand.

Moravec’s paradox names this reversal: the things humans find intellectual can be easier for machines than the things toddlers do without applause.

The hard part is not producing an answer, but building a model of the world that survives contact with weight, friction, surprise, and failure.

----

Link to the full video on Bloomberg's site. Link in comment.

During a Bloomberg interview, Yann LeCun (@ylecun ) explains why LLMs are limited in terms of real-world intelligence during a Bloomberg interview.

"Language is a very approximate, reduced, quantized, and simplified description of the world, and LLMs can only deal with discrete sequences of symbols. The world is much more complicated than language.

The biggest LLMs are pre-trained on the totality of all the publicly available text on the internet. That’s about 20 trillion words, or 30 trillion tokens.

A token is about 3 bytes. So total 10¹⁴ bytes of text.

This is the amount of data a four-year-old has seen through vision during four years. Now, the text, though, would take 400,000 years to read?

So, there is enormously more data from sensory input, like vision, touch, and everything else, than there could ever be through language."

A child does not need 400,000 years of reading to understand cups, doors, balance, faces, falls, or heat, because the body is already collecting dense feedback from vision, touch, motion, and consequence.

Text strips most of that away.

It turns a living scene into symbols, then asks the model to infer the missing world from traces left by people describing it.

That is why an LLM can sound fluent about physics and still have no native sense of how fragile glass feels in a hand.

Moravec’s paradox names this reversal: the things humans find intellectual can be easier for machines than the things toddlers do without applause.

The hard part is not producing an answer, but building a model of the world that survives contact with weight, friction, surprise, and failure.

----

Link to the full video on Bloomberg's site. Link in comment.

During a Bloomberg interview, Yann LeCun (@ylecun ) explains why LLMs are limited in terms of real-world intelligence during a Bloomberg interview.

"Language is a very approximate, reduced, quantized, and simplified description of the world, and LLMs can only deal with discrete sequences of symbols. The world is much more complicated than language.

The biggest LLMs are pre-trained on the totality of all the publicly available text on the internet. That’s about 20 trillion words, or 30 trillion tokens.

A token is about 3 bytes. So total 10¹⁴ bytes of text.

This is the amount of data a four-year-old has seen through vision during four years. Now, the text, though, would take 400,000 years to read?

So, there is enormously more data from sensory input, like vision, touch, and everything else, than there could ever be through language."

A child does not need 400,000 years of reading to understand cups, doors, balance, faces, falls, or heat, because the body is already collecting dense feedback from vision, touch, motion, and consequence.

Text strips most of that away.

It turns a living scene into symbols, then asks the model to infer the missing world from traces left by people describing it.

That is why an LLM can sound fluent about physics and still have no native sense of how fragile glass feels in a hand.

Moravec’s paradox names this reversal: the things humans find intellectual can be easier for machines than the things toddlers do without applause.

The hard part is not producing an answer, but building a model of the world that survives contact with weight, friction, surprise, and failure.

----

Link to the full video on Bloomberg's site. Link in comment.