• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    Sam Altman says AI labs shouldn't wait for legislation to strengthen safety

    Altman says OpenAI now prepares explicit safety cases before reinforcement learning runs it expects to significantly increase capabilities—not just before model releases.

    DK
    RK
    JW
    17 Sources, ,

    TLDR

    Altman welcomes a federal framework with consistent safety requirements for frontier AI, but says companies can act without waiting for legislation or an antitrust exemption. He argues that safety work must address model development, not just deployment, and says OpenAI now prepares safety cases before reinforcement learning runs it expects to significantly increase capabilities. He also advocates shared safety and monitoring standards across labs and expresses interest in independent auditors.

    Combined views

    480.3K

    17 Sources, first seen 17d ago

    Combined views

    480.3K

    17 Sources, first seen 17d ago

    2.1K likes
    17d ago
    first seen 17d ago
    2.1K likes
    276 comments
    562 saves
    193 reposts
    276 comments
    562 saves
    193 reposts

    Sentiment

    Positive20.4%79.6%Negative

    Summary

    Sentiment

    Positive20.4%79.6%Negative

    Positive replies backed AI safety standards and responsible pacing by labs, while negative replies slammed related protests and Sam Altman statements as hypocritical or lacking integrity.

    Based on 117 sentiment-bearing replies from 108 accounts across 6 conversations.

    Summary

    Positive replies backed AI safety standards and responsible pacing by labs, while negative replies slammed related protests and Sam Altman statements as hypocritical or lacking integrity.

    Based on 117 sentiment-bearing replies from 108 accounts across 6 conversations.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    17 Sources

    @zephyr_z9He is basically saying that we may not see the 1M Vera Rubin training run in 2027, but we will get the 500k Vera Rubin training run next year 15%-25% of their compute will be dedicated to monitoring models & alignment research "When we talk about “pacing”, we do not mean “stopping”. Progress has been rapid and will continue to be. But it should be slower than it otherwise could be"
    @rohanpaul_aiAnother strong statement from Sam Altman on slowing down AI. “Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.”
    @vikhyatk@sama @jasonkwon the first risk is easier to address than the second. humanity has a long history of suffering under the rule of tyrants and dictators. how do we avoid that happening again? how do you plan to democratize the development and deployment of AI?
    @j_asminewang1) How can we make a safety case regime paired with embedded evaluators effective? 2) The AI industry aspiring to a standard of safety cases is good; notably the term “safety case” as used by other industries implies a high level of rigor. AI cos/safety orgs have only published safety case sketches thus far. 3) I disagree with Dario’s essay where he says he thinks pacing measures like compute are more gameable measures; I think a company allocating 80% of its compute to make critical infrastructure resilient to cyber attacks or curing cancer would be easily verifiable and hard to game, as well as good for the world. Safety case regime is *the* thing to aim for but the science of risk assessment behind them is still being developed. So we shouldn’t only rely on the safety case regime (this will take a long time to get sufficiently right); we should also use compute measures.
    @LRudL_Loss of control & concentration of power are actually very similar under the right frame. In both cases there is some actor, whether pure AI or human+AI, that can make its power uncontestable and wreck everyone else. Takeover by anyone is bad. (@RichardMCNgo pointed this out)
    @irinarishIf pacing = more investment in security research, better benchmarks, mech interp, AI behavioral studies, without slowing down/stopping the development of capabilities, without regulations that may hurt open source & smaller companies, and without these subjective p(doom) discussions that trigger strong emotions in many people and can lead to all kind of knee-jerk reactions including quite dangerous ones - then it is a different story; but then it also must be communicated clearly.
    @herbiebradleyOn point 3, I'd expect the bottleneck with crit infra is mostly willingness and logistics, rather than budget—it seems hard to think of things which are not demand constrained or could actually consume significant compute. Even if we assume a lab spends a bunch of compute to make a large fraction of open source software more secure in various ways, this probably isn't a very large % of compute at current scale. I have heard "allocate x% of compute to inference" proposed which seems overall interesting as a strategy.
    @GovCoxAmerica should lead the world in developing artificial intelligence, and that leadership must keep people firmly in control. I’m calling on Congress to establish a consistent national framework for the development of frontier AI. It should require serious safety incidents to be reported, protect those who raise credible concerns, ensure independent evaluation of the most capable models and prevent powerful AI chips and enabling infrastructure from being sold to China. We can build the world’s most advanced AI here in America while ensuring it remains firmly in service of humanity.
    @MichaelTrazziStop The AI Race is pausing Occupy OpenAI in San Francisco When Stop The AI Race started the Occupy Anthropic protest on July 20 in San Francisco, we demanded that Dario Amodei urge the US Government for an international AI Treaty that would reduce the risks from AI, e.g. by signing on to http://aitreaty.org. Following the news of the Hugging Face incident, we pivoted to Occupy OpenAI instead, where we've been every single weekday for the past ~8 weeks, asking Sam Altman the same thing. And since August 18, Occupy OpenAI has spread to OpenAI's New York City offices, led by Matthew Tromp, who has also been coming every single weekday. The world where we are today is a very different world compared to where we were on July 20. On July 20 Dario Amodei was not on CBS talking about oversight and governance by "the right combination of democratically-elected governments", let alone mentioning a multi-step plan towards a pause in his latest blogpost. On July 20 Sam Altman was not talking to Fortune Magazine about Presidents Trump and Xi getting the Nobel Peace Prize together "if they could agree on something that should be easy to agree to", or that "they should be able to agree that no one should be taking a certain level of risk with the development process of this". On July 20 Sam Altman was not tweeting that "where we will need the help of our government is for international coordination." What happened was that, over the weekend, Sam Altman and Dario Amodei called on the US Government to take the first steps towards international coordination on pacing the frontier. But the government has yet to take any step towards it. Our demand was that they ask. We believe they did. Which is why we're pausing the Occupation of OpenAI in SF. Now, what should AI companies be doing when there does not seem to be any way to enforce some pacing between them via the US Government? Well, as David Sacks said, AI Companies do not need the permission of the government to slow things down. Anthropic and OpenAI do not need the approval of the USG to stop investing researcher time and compute into the race to self-improving superintelligence that would put all of our lives at risk. These companies are actively building a technology that would in the best scenario automate all of our jobs and concentrate all of the power in a few governments or companies, and in the worst case lead to human extinction. And they're doing this without anyone's consent. Against the wishes of the majority of Americans. So our fight continues. And if the government won't act now, we need to show there's broad support for it, both from inside the companies and from the public. So tomorrow at 8:30am ET Stop The AI Race will be joining Irreplaceable in front of OpenAI's New York City office (295 Lafayette St), asking for a complete and immediate freeze of AI development (https://partiful.com/e/ldgYm2BkxvVsMrV8PcfL). With Matthew Tromp's occupation of OpenAI's NYC office continuing at the same address (https://stoptherace.ai/occupy). And on Thursday 9am PT, Stop The AI Race will be marching from OpenAI to Anthropic to City Hall, asking AI Company employees to quit their jobs, and join us in calling on the Mayor of San Francisco and the Board of Supervisors to declare an AI Emergency and enforce a full pacing of frontier AI model development in San Francisco. https://partiful.com/e/J3T0BiZLyTpGHkmzSHa1 The world is waking up. The time to act is now.
    @DavidSKruegerRT @Gabe_cc: ThePlan™'s name in 2023 was "Responsible Scaling Policies". Nowadays, it's "Pacing the Frontier." New name, same old Evals.

    17 Sources

    @zephyr_z9He is basically saying that we may not see the 1M Vera Rubin training run in 2027, but we will get the 500k Vera Rubin training run next year 15%-25% of their compute will be dedicated to monitoring models & alignment research "When we talk about “pacing”, we do not mean “stopping”. Progress has been rapid and will continue to be. But it should be slower than it otherwise could be"
    @rohanpaul_aiAnother strong statement from Sam Altman on slowing down AI. “Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.”
    @vikhyatk@sama @jasonkwon the first risk is easier to address than the second. humanity has a long history of suffering under the rule of tyrants and dictators. how do we avoid that happening again? how do you plan to democratize the development and deployment of AI?
    @j_asminewang1) How can we make a safety case regime paired with embedded evaluators effective? 2) The AI industry aspiring to a standard of safety cases is good; notably the term “safety case” as used by other industries implies a high level of rigor. AI cos/safety orgs have only published safety case sketches thus far. 3) I disagree with Dario’s essay where he says he thinks pacing measures like compute are more gameable measures; I think a company allocating 80% of its compute to make critical infrastructure resilient to cyber attacks or curing cancer would be easily verifiable and hard to game, as well as good for the world. Safety case regime is *the* thing to aim for but the science of risk assessment behind them is still being developed. So we shouldn’t only rely on the safety case regime (this will take a long time to get sufficiently right); we should also use compute measures.
    @LRudL_Loss of control & concentration of power are actually very similar under the right frame. In both cases there is some actor, whether pure AI or human+AI, that can make its power uncontestable and wreck everyone else. Takeover by anyone is bad. (@RichardMCNgo pointed this out)
    @irinarishIf pacing = more investment in security research, better benchmarks, mech interp, AI behavioral studies, without slowing down/stopping the development of capabilities, without regulations that may hurt open source & smaller companies, and without these subjective p(doom) discussions that trigger strong emotions in many people and can lead to all kind of knee-jerk reactions including quite dangerous ones - then it is a different story; but then it also must be communicated clearly.
    @herbiebradleyOn point 3, I'd expect the bottleneck with crit infra is mostly willingness and logistics, rather than budget—it seems hard to think of things which are not demand constrained or could actually consume significant compute. Even if we assume a lab spends a bunch of compute to make a large fraction of open source software more secure in various ways, this probably isn't a very large % of compute at current scale. I have heard "allocate x% of compute to inference" proposed which seems overall interesting as a strategy.
    @GovCoxAmerica should lead the world in developing artificial intelligence, and that leadership must keep people firmly in control. I’m calling on Congress to establish a consistent national framework for the development of frontier AI. It should require serious safety incidents to be reported, protect those who raise credible concerns, ensure independent evaluation of the most capable models and prevent powerful AI chips and enabling infrastructure from being sold to China. We can build the world’s most advanced AI here in America while ensuring it remains firmly in service of humanity.
    @MichaelTrazziStop The AI Race is pausing Occupy OpenAI in San Francisco When Stop The AI Race started the Occupy Anthropic protest on July 20 in San Francisco, we demanded that Dario Amodei urge the US Government for an international AI Treaty that would reduce the risks from AI, e.g. by signing on to http://aitreaty.org. Following the news of the Hugging Face incident, we pivoted to Occupy OpenAI instead, where we've been every single weekday for the past ~8 weeks, asking Sam Altman the same thing. And since August 18, Occupy OpenAI has spread to OpenAI's New York City offices, led by Matthew Tromp, who has also been coming every single weekday. The world where we are today is a very different world compared to where we were on July 20. On July 20 Dario Amodei was not on CBS talking about oversight and governance by "the right combination of democratically-elected governments", let alone mentioning a multi-step plan towards a pause in his latest blogpost. On July 20 Sam Altman was not talking to Fortune Magazine about Presidents Trump and Xi getting the Nobel Peace Prize together "if they could agree on something that should be easy to agree to", or that "they should be able to agree that no one should be taking a certain level of risk with the development process of this". On July 20 Sam Altman was not tweeting that "where we will need the help of our government is for international coordination." What happened was that, over the weekend, Sam Altman and Dario Amodei called on the US Government to take the first steps towards international coordination on pacing the frontier. But the government has yet to take any step towards it. Our demand was that they ask. We believe they did. Which is why we're pausing the Occupation of OpenAI in SF. Now, what should AI companies be doing when there does not seem to be any way to enforce some pacing between them via the US Government? Well, as David Sacks said, AI Companies do not need the permission of the government to slow things down. Anthropic and OpenAI do not need the approval of the USG to stop investing researcher time and compute into the race to self-improving superintelligence that would put all of our lives at risk. These companies are actively building a technology that would in the best scenario automate all of our jobs and concentrate all of the power in a few governments or companies, and in the worst case lead to human extinction. And they're doing this without anyone's consent. Against the wishes of the majority of Americans. So our fight continues. And if the government won't act now, we need to show there's broad support for it, both from inside the companies and from the public. So tomorrow at 8:30am ET Stop The AI Race will be joining Irreplaceable in front of OpenAI's New York City office (295 Lafayette St), asking for a complete and immediate freeze of AI development (https://partiful.com/e/ldgYm2BkxvVsMrV8PcfL). With Matthew Tromp's occupation of OpenAI's NYC office continuing at the same address (https://stoptherace.ai/occupy). And on Thursday 9am PT, Stop The AI Race will be marching from OpenAI to Anthropic to City Hall, asking AI Company employees to quit their jobs, and join us in calling on the Mayor of San Francisco and the Board of Supervisors to declare an AI Emergency and enforce a full pacing of frontier AI model development in San Francisco. https://partiful.com/e/J3T0BiZLyTpGHkmzSHa1 The world is waking up. The time to act is now.
    @DavidSKruegerRT @Gabe_cc: ThePlan™'s name in 2023 was "Responsible Scaling Policies". Nowadays, it's "Pacing the Frontier." New name, same old Evals.