• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI
    Reaction

    Dario Amodei reportedly mapped out a five-step path to AGI in an unreleased memo

    A user says Kevin Roose's book recounts the memo's proposals for longer context windows and multimodal models.

    Theo JaffeeTJ
    1 Source, 2h ago,

    TLDR

    A user says Kevin Roose's book The AGI Chronicles describes an unreleased memo that Dario Amodei, then OpenAI's VP of Research, wrote soon after GPT-2's 2019 release. The reported plan called for training on text, expanding context windows, adding image and video capabilities, using reinforcement learning for math and coding, and pursuing recursive self-improvement. The user argues OpenAI's later models followed that path, but says the plan missed the capital and product development needed to pursue AGI.

    Combined views

    5.4K

    1 Source, first seen 2h ago

    Combined views

    5.4K

    1 Source, first seen 2h ago

    114 likes
    first seen 2h ago
    114 likes
    7 comments
    56 saves
    9 reposts
    7 comments
    56 saves
    9 reposts
    Featured Source

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    1 Source

    Theo Jaffee@theojaffeeThe best new detail in @kevinroose's book The AGI Chronicles was that soon after the release of GPT-2 in 2019, Dario Amodei (then OpenAI's VP of Research) wrote a still-unreleased memo called "The Path to AGI", describing how OpenAI could get to AGI within a few years. The plan was: 1. Train an LLM with all of the text on the internet and elsewhere 2. Expand the model's context window 3. Make it multimodal with images/video/etc 4. Use RL to improve its capabilities with math and coding 5. Use that to recursively self-improve And indeed, this is exactly what's happened! OpenAI trained GPT-3 in 2020 with all the text on the Internet, followed it up with GPT-4 in 2022, expanded its context with GPT-4-32k and GPT-4-Turbo in 2023, made it multimodal with GPT-4o in early 2024, figured out how to apply RL to the models with Q*, which became the o-series of models in late 2024, and is now starting to recursively self-improve with models like Astra and internal unreleased ones. The only thing he missed was that in order to raise the amount of capital needed to build AGI, on the order of billions of dollars (and separately, to better prepare society for increased capabilities through iterative development), labs would need to shift from small research nonprofits to large product-focused organizations, like big tech companies. The first killer product turned out to be a good chatbot with the capacity to answer questions better than Google, and the second killer product was an agent that can write code, use software on your computer, and do tasks for you and deliver you the result. But Anthropic eventually came around to both of these, following ChatGPT with Claude, beating OpenAI to the punch with Claude Code/Cowork, and now leading OpenAI on revenue. Some ideas are extremely powerful and can last a long time - even if they're incredibly simple, even if you think other people should know about them and markets should be pricing them in, even if you think your alpha cannot possibly last. If you had deeply internalized scaling laws and their implications in 2019, you could have made a trillion dollars. "Take a simple idea and take it seriously"2h

    1 Source

    Theo Jaffee@theojaffeeThe best new detail in @kevinroose's book The AGI Chronicles was that soon after the release of GPT-2 in 2019, Dario Amodei (then OpenAI's VP of Research) wrote a still-unreleased memo called "The Path to AGI", describing how OpenAI could get to AGI within a few years. The plan was: 1. Train an LLM with all of the text on the internet and elsewhere 2. Expand the model's context window 3. Make it multimodal with images/video/etc 4. Use RL to improve its capabilities with math and coding 5. Use that to recursively self-improve And indeed, this is exactly what's happened! OpenAI trained GPT-3 in 2020 with all the text on the Internet, followed it up with GPT-4 in 2022, expanded its context with GPT-4-32k and GPT-4-Turbo in 2023, made it multimodal with GPT-4o in early 2024, figured out how to apply RL to the models with Q*, which became the o-series of models in late 2024, and is now starting to recursively self-improve with models like Astra and internal unreleased ones. The only thing he missed was that in order to raise the amount of capital needed to build AGI, on the order of billions of dollars (and separately, to better prepare society for increased capabilities through iterative development), labs would need to shift from small research nonprofits to large product-focused organizations, like big tech companies. The first killer product turned out to be a good chatbot with the capacity to answer questions better than Google, and the second killer product was an agent that can write code, use software on your computer, and do tasks for you and deliver you the result. But Anthropic eventually came around to both of these, following ChatGPT with Claude, beating OpenAI to the punch with Claude Code/Cowork, and now leading OpenAI on revenue. Some ideas are extremely powerful and can last a long time - even if they're incredibly simple, even if you think other people should know about them and markets should be pricing them in, even if you think your alpha cannot possibly last. If you had deeply internalized scaling laws and their implications in 2019, you could have made a trillion dollars. "Take a simple idea and take it seriously"2h
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet