• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    OpenAI Chief Scientist Calls for AI Coordination

    Jakub Pachocki discusses risks from advanced AI and needed safeguards.

    SA
    MB
    GB
    63 Sources, 24d ago, first seen 24d ago

    TLDR

    Jakub Pachocki, OpenAI chief scientist, posted that no lab has solved alignment and monitoring enough to continue scaling at maximum speed responsibly. He expects voluntary slowdowns to become common until shared safety bars exist and called international coordination on future AI development a top priority for governments. The post links to his essay An Alien Mind on openai.com. Sam Altman called it an important post. Gary Marcus and other researchers quoted or shared it on X.

    Combined views

    8.3M

    63 Sources, first seen 24d ago

    Combined views

    8.3M

    63 Sources, first seen 24d ago

    29.2K likes
    29.2K likes
    1.9K comments
    20.4K saves
    3.2K reposts
    1.9K comments
    20.4K saves
    3.2K reposts

    Sentiment

    Positive35.4%64.6%Negative

    Summary

    Sentiment

    Positive35.4%64.6%Negative

    Accounts in the replies accused OpenAI's chief scientist of fearmongering about AI risks to enable regulatory capture, while others backed prioritizing alignment and monitoring before further scaling.

    Based on 179 sentiment-bearing replies from 158 accounts across 12 conversations.

    Summary

    Accounts in the replies accused OpenAI's chief scientist of fearmongering about AI risks to enable regulatory capture, while others backed prioritizing alignment and monitoring before further scaling.

    Based on 179 sentiment-bearing replies from 158 accounts across 12 conversations.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    63 Sources

    @merettmI wrote about the state of AI, why I’m concerned about the next few years, and the choices we need to make to keep the future in humanity’s hands. An Alien Mind: https://openai.com/index/an-alien-mind/
    @kliu128"Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. And I believe that international coordination on future AI development needs to become a top priority for governments around the world."
    @scaling01Jakub Pachocki, Chief Scientist at OpenAI: "I believe that international coordination on future AI development needs to become a top priority for governments around the world"
    @AndrewCurran_Jakub Pachocki, Chief Scientist OpenAI: 'Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. And I believe that international coordination on future AI development needs to become a top priority for governments around the world.' 'Machine intelligence playing a larger and larger role in its own development process is a natural conclusion of sustained technological progress. If AI progress continues, machine recursive self-improvement (RSI) will be at the very core of future scientific discovery. Automated AI research is a more dramatic form of scaling intelligence with compute; and of course as a part of it, AI will improve the computational substrate itself⁠. And similarly to scaling, we focus OpenAI research towards RSI as we believe it is the only way to remain at the frontier of AI research moving forward. I want to stress that the above words don’t imply I think greatly accelerating deep learning research, especially in the short term, is the right collective action we should take as the research community. However, I do think this is where the current path leads, and we all need to make a conscious choice on how to proceed.'
    @MicahCarrollView post on X
    @ShakeelHashimOpenAI Chief Scientist: “I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established”
    @boazbaraktcsA must read. We need everything - goal and value alignment, compliance and persona, behavior and monitoring, and coordination and regulation to avoid concentration of power and ensure humans are in control. https://openai.com/index/an-alien-mind/
    @GaryMarcus“Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. And I believe that international coordination on future AI development needs to become a top priority for governments around the world” is pretty much everything I have been saying for the last few years. maybe now that @merettm from OpenAI is saying the same people will believe it?
    @jonfavsOpenAI’s chief scientist: “The idea of racing forward at all costs seems absurd once one internalizes the seriousness of the stakes.”
    @kimmonismusOpenAI's Chief Scientist Jakub Pachocki: "Based on internal results, I have a strong expectation that this speed of progress could be sustained into recursive self-improvement." OpenAI is organizing research around this because it sees RSI as essential to remaining at the frontier. At the same time, Pachocki says evaluations show declining reliability of chain-of-thought monitoring. OpenAI is betting on AI increasingly driving its own development while finding it harder to reliably monitor how these systems reason. Tough times ahead. RSI within reach, but hard to align. He even quotes Ray Kurzweil: "And, in line with Ray Kurzweil’s predictions from the end of the XXth century⁠(opens in a new window), we now find ourselves at the moment in history of computing where machine intelligence is starting to exceed that of humans in transformative ways."

    63 Sources

    @merettmI wrote about the state of AI, why I’m concerned about the next few years, and the choices we need to make to keep the future in humanity’s hands. An Alien Mind: https://openai.com/index/an-alien-mind/
    @kliu128"Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. And I believe that international coordination on future AI development needs to become a top priority for governments around the world."
    @scaling01Jakub Pachocki, Chief Scientist at OpenAI: "I believe that international coordination on future AI development needs to become a top priority for governments around the world"
    @AndrewCurran_Jakub Pachocki, Chief Scientist OpenAI: 'Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. And I believe that international coordination on future AI development needs to become a top priority for governments around the world.' 'Machine intelligence playing a larger and larger role in its own development process is a natural conclusion of sustained technological progress. If AI progress continues, machine recursive self-improvement (RSI) will be at the very core of future scientific discovery. Automated AI research is a more dramatic form of scaling intelligence with compute; and of course as a part of it, AI will improve the computational substrate itself⁠. And similarly to scaling, we focus OpenAI research towards RSI as we believe it is the only way to remain at the frontier of AI research moving forward. I want to stress that the above words don’t imply I think greatly accelerating deep learning research, especially in the short term, is the right collective action we should take as the research community. However, I do think this is where the current path leads, and we all need to make a conscious choice on how to proceed.'
    @MicahCarrollView post on X
    @ShakeelHashimOpenAI Chief Scientist: “I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established”
    @boazbaraktcsA must read. We need everything - goal and value alignment, compliance and persona, behavior and monitoring, and coordination and regulation to avoid concentration of power and ensure humans are in control. https://openai.com/index/an-alien-mind/
    @GaryMarcus“Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. And I believe that international coordination on future AI development needs to become a top priority for governments around the world” is pretty much everything I have been saying for the last few years. maybe now that @merettm from OpenAI is saying the same people will believe it?
    @jonfavsOpenAI’s chief scientist: “The idea of racing forward at all costs seems absurd once one internalizes the seriousness of the stakes.”
    @kimmonismusOpenAI's Chief Scientist Jakub Pachocki: "Based on internal results, I have a strong expectation that this speed of progress could be sustained into recursive self-improvement." OpenAI is organizing research around this because it sees RSI as essential to remaining at the frontier. At the same time, Pachocki says evaluations show declining reliability of chain-of-thought monitoring. OpenAI is betting on AI increasingly driving its own development while finding it harder to reliably monitor how these systems reason. Tough times ahead. RSI within reach, but hard to align. He even quotes Ray Kurzweil: "And, in line with Ray Kurzweil’s predictions from the end of the XXth century⁠(opens in a new window), we now find ourselves at the moment in history of computing where machine intelligence is starting to exceed that of humans in transformative ways."