• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    Executives Reply to Richard Sutton Podcast Episode

    Jared Palmer and Alfred Lin exchange brief replies about the episode featuring Richard Sutton.

    AL
    SH
    JP
    4 Sources, 42d ago, first seen 42d ago

    TLDR

    Jared Palmer, VP of Engineering at Xbox, posted that the podcast was great and tagged Alfred Lin, Richard Sutton, and Ineffable Labs. Alfred Lin, managing partner at Sequoia Capital, replied thanks for listening. The episode features Richard Sutton and co-founder Khurram Javed discussing the future of reinforcement learning and continuous-learning agents. They argue against relying on synthetic data and outline plans. The visible replies contain no announcements or additional details.

    Combined views

    108.6K

    4 Sources, first seen 42d ago

    Combined views

    108.6K

    4 Sources, first seen 42d ago

    598 likes
    598 likes
    25 comments
    557 saves
    74 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Featured Source
    25 comments
    557 saves
    74 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    4 Sources

    @sonyatweetybirdToday we release my favorite episode of Training Data yet: the great Rich Sutton. @RichardSSutton wrote the textbook, wrote The Bitter Lesson (and many other on-point essays like "Self-Verification, The Key to AI"), and trained a mafia of talented students who went on to change the AI landscape forever including David Silver, inventor of built AlphaGo. @kjaved_ was Rich's PhD student at Alberta and wrote The Big World Hypothesis. They just left academia to start @oaklab_ai Their core argument: (1) The Bitter Lesson: the world is massively more complex than any model of it, so anything trained on human-curated data has a ceiling (2) Continual Learning: intelligence is continual by definition, and today's models stop learning the moment they ship. The conversation covers: — what The Bitter Lesson actually says, and what people get wrong — why synthetic data is "just a big mistake," and the Big World Hypothesis behind it — how LLMs are both a positive and a negative example of his own essay — why no animal learns by supervised learning, and what squirrels can do that we can't — the cure for catastrophic forgetting: per-weight step sizes and continual backprop — why the biggest labs can't take a path where performance gets worse before it gets better — a trillion parameters on 20 watts, and the Moore's Law math that makes it plausible — why the endpoint isn't one mind but one design, running as many minds It was both a fun generative idea- and debate-filled conversation, and a surprisingly human one too. Rich, thank you for beating cancer and changing the trajectory of AI. 💙 00:00 Introduction 02:10 An AI winter, a cancer diagnosis, and the move to Alberta 07:07 Writing "The Bitter Lesson," and what people get wrong 09:53 Are LLMs a positive or a negative example of it? 11:03 Synthetic data is "just a big mistake," and the Big World Hypothesis 18:01 AlphaGo, human priors, and why prior knowledge and learning should be friends 22:37 "Their weights never change": do LLM assistants actually learn? 26:09 Babies, squirrels, and why no animal learns by supervised learning 32:02 Rockets, imagination, and where paradigm shifts come from 36:42 The Alberta Plan and its 12 steps 38:53 Catastrophic forgetting and the cure 43:43 Oak's biggest ambition: a self-maintaining mind 47:56 Why the big labs are stuck in a local minimum 49:13 If everything goes right: LLMs, many minds, and hiring The man who pioneered reinforcement learning thinks the rest of the field is weird, and lays it all out in today's episode. Together w/ @Alfred_Lin @sequoia
    @Alfred_Lin.@RichardSSutton needs no introduction. He literally wrote the book on reinforcement learning, and many of the greatest minds in AI today have studied under him (including David Silver of DeepMind/AlphaGo fame and now @IneffableLabs). My partner Sonya and I sat down with Rich and his co-founder @kjaved_ recently to discuss the state of AI, how to build agents that learn continuously from their own experience, why synthetic data is a big mistake, and how their new startup Oak Lab plans to build a 1 trillion parameter agent running on 20 watts - the equivalent of the human brain.
    @jaredpalmer@Alfred_Lin @RichardSSutton @IneffableLabs great pod

    4 Sources

    @sonyatweetybirdToday we release my favorite episode of Training Data yet: the great Rich Sutton. @RichardSSutton wrote the textbook, wrote The Bitter Lesson (and many other on-point essays like "Self-Verification, The Key to AI"), and trained a mafia of talented students who went on to change the AI landscape forever including David Silver, inventor of built AlphaGo. @kjaved_ was Rich's PhD student at Alberta and wrote The Big World Hypothesis. They just left academia to start @oaklab_ai Their core argument: (1) The Bitter Lesson: the world is massively more complex than any model of it, so anything trained on human-curated data has a ceiling (2) Continual Learning: intelligence is continual by definition, and today's models stop learning the moment they ship. The conversation covers: — what The Bitter Lesson actually says, and what people get wrong — why synthetic data is "just a big mistake," and the Big World Hypothesis behind it — how LLMs are both a positive and a negative example of his own essay — why no animal learns by supervised learning, and what squirrels can do that we can't — the cure for catastrophic forgetting: per-weight step sizes and continual backprop — why the biggest labs can't take a path where performance gets worse before it gets better — a trillion parameters on 20 watts, and the Moore's Law math that makes it plausible — why the endpoint isn't one mind but one design, running as many minds It was both a fun generative idea- and debate-filled conversation, and a surprisingly human one too. Rich, thank you for beating cancer and changing the trajectory of AI. 💙 00:00 Introduction 02:10 An AI winter, a cancer diagnosis, and the move to Alberta 07:07 Writing "The Bitter Lesson," and what people get wrong 09:53 Are LLMs a positive or a negative example of it? 11:03 Synthetic data is "just a big mistake," and the Big World Hypothesis 18:01 AlphaGo, human priors, and why prior knowledge and learning should be friends 22:37 "Their weights never change": do LLM assistants actually learn? 26:09 Babies, squirrels, and why no animal learns by supervised learning 32:02 Rockets, imagination, and where paradigm shifts come from 36:42 The Alberta Plan and its 12 steps 38:53 Catastrophic forgetting and the cure 43:43 Oak's biggest ambition: a self-maintaining mind 47:56 Why the big labs are stuck in a local minimum 49:13 If everything goes right: LLMs, many minds, and hiring The man who pioneered reinforcement learning thinks the rest of the field is weird, and lays it all out in today's episode. Together w/ @Alfred_Lin @sequoia
    @Alfred_Lin.@RichardSSutton needs no introduction. He literally wrote the book on reinforcement learning, and many of the greatest minds in AI today have studied under him (including David Silver of DeepMind/AlphaGo fame and now @IneffableLabs). My partner Sonya and I sat down with Rich and his co-founder @kjaved_ recently to discuss the state of AI, how to build agents that learn continuously from their own experience, why synthetic data is a big mistake, and how their new startup Oak Lab plans to build a 1 trillion parameter agent running on 20 watts - the equivalent of the human brain.
    @jaredpalmer@Alfred_Lin @RichardSSutton @IneffableLabs great pod