The case against autoregressive LLMs alone as a path to human-level AI
Yann LeCun stands by his view that autoregressive LLMs alone won't lead to human-level AI. He argues that current reasoning and self-improvement methods remain limited, and that humans and animals learn new skills far more efficiently.
TLDR
In a September 20 reply, Yann LeCun reaffirms his argument that autoregressive LLMs alone will not lead to human-level AI. He says current systems' reasoning relies on non-autoregressive search and, as far as he can tell, operates in token spaceāan approach he calls limited and inefficient. He instead advocates searching in continuous representation space. LeCun also argues that current self-improvement methods work only where output quality can be scored without human intervention, such as math, code and accurately simulated scenarios. He says humans and animals learn new skills far more efficiently than current reinforcement learning methods.
Combined views
306.8K
10 Sources, first seen ago
