Dario Amodei urges AI slowdown, says Anthropic will open systems to outside evaluators

@KonstantinPilzBrief observations on the US-China AI gap, given renewed calls for pacing AI progress: 1. The most advanced Chinese AI model has been 6.3 months behind the most advanced US AI model on average since 2025.[a] US AI developers have even more powerful models internally, so I estimate the real gap is closer to 8 months.[b] 2. Chinese AI capabilities depend heavily on US AI progress. We have strong evidence of industrial-scale distillation efforts that allow Chinese developers to extract capabilities from more advanced US models they couldn't otherwise access.[c] Chinese companies also benefit from algorithmic advances made by US developers via publications, word of mouth, researcher moves, etc. Additionally, they likely use Claude Code and Codex to speed up their AI R&D. 3. US companies have an enormous lead in compute. Our estimates show that by the end of 2026 leading US AI developers will have 20x more compute than Chinese AI companies. Due to the compute disadvantage, Chinese AI developers can't train competitive models on their own and instead extensively rely on distillation. The US compute lead could be much larger still if USG was serious about strengthening and enforcing export controls. For concrete proposals, see [d]. I agree we can't allow China to lead in AI. But given how far behind China is on compute and how dependent Chinese progress is on US capabilities, pacing AI progress in America would not immediately concede the lead. I estimate the US could comfortably halt progress for at least six months without falling behind China — though I think pacing, not halting, is the wiser approach. Of course, it would be preferable to pace in coordination with China, both so a framework could endure over time and to avoid unfairly disadvantaging US AI companies. In the coming months, we at @techstatecraft will publish several papers with detailed analysis of US and Chinese compute, capabilities, and the factors that determine AI progress in both countries. --- [a] https://epoch.ai/eci?subset-view=graph&subset-tab=Software+engineering&view=graph&tab=release-date&showFrontierTrend=true&colorCategorization=Country [b] E.g., see the graph here showing the capability difference between OpenAI's most advanced publicly released model and an unreleased internal model: https://openai.com/index/navier-stokes-solution/ [c] https://www.nsa.gov/Press-Room/Press-Releases-Statements/Press-Release-View/Article/4592113/nsa-and-others-warn-china-based-ai-companies-are-distilling-us-frontier-ai-mode/; https://x.com/SecScottBessent/status/2080008411790368895; https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks [d] https://ifp.org/preparing-for-ai-research-automation/#extend-the-us-ai-lead-to-give-the-us-more-time-and-leverage-to-manage-ai-r-amp-d-automation-risks

@peterwildefordIf Altman and Amodei are both in favor of pacing the frontier, sure seems like they should just both start pacing the frontier

@Dan_Jeffries1Ask yourself why no Chinese researchers are not storming out of their labs like cry babies and hallucinating about the end of the world? It's simple: Economics. Our kids have grown up poorer, with more debt and and dramatically slowed growth and so see the world as hostile and something to fear. By contrast, Chinese have seen their quality of life 30x in the last forty years and so see the transformative power of technology, entrepreneurship and industry to change lives for the better. Sadly we've entered a vicious cycle where the very solutions we come up with the try to make it better are actually making it worse and worse and causing us to spiral further and further. Protectionism. Populism. More compliance. More rules. More price freezes. More barriers to trade. More calls to act now about every imaginary problem while we never get to the root of real problems. We've got to stop listening to extremists and get back to building and taking bold risks and trusting the future. Or else that future will be in the East and the American century will be nothing but an ever receding memory in the history books of yesterday.

@willdepueRT @peterwildeford: If Altman and Amodei are both in favor of pacing the frontier, sure seems like they should just both start pacing the f…

@xeophon@Dan_Jeffries1 they also don't care about politics nor the bigger picture of things, they just wanna train a good model. when you ask CN lab employees about things like x risk, they genuinely look confused at you

@vkrakovnaI signed the letter on pacing the frontier with the following statement: "It is very important to build capacity for a coordinated slowdown of AI development. A runaway race to AGI is not safe: it creates incentives to cut corners on safety, and carries a substantial risk for model capabilities to outpace development of adequate alignment, control, and governance measures. This could result in catastrophic loss of control of highly capable AI systems. Increasing model capabilities in a slow and controlled way would allow us enough time to develop, adapt and test our alignment and control methods for each level of capability. Advanced AI should only be built with a robust assurance of safety, and a slowdown would make this possible."

@Dr_AtoosaAfter AISI, I’m glad that METR/Redwood Research started normalizing the idea of “independant” reporting or evaluation. But I don’t think it’s accurate to say that AIs behavior was due to their own independant volition. I actually think this language could be quite misleading. I think METR/Redwood should NOT be the only group that keep the independant status. We need measured and scientifically-grounded independant evaluators. Thanks to @ghadfield and co who are promoting the formulation of measured independant audits.

@lukeprogRT @KonstantinPilz: Brief observations on the US-China AI gap, given renewed calls for pacing AI progress: 1. The most advanced Chinese AI…

@davidmarcusThe odd-bunch alliance of those who dream of reg capture, EA-pilled folks, attention-seeking randos, and foreign adversaries who join in to throw more fuel on AI fear-mongering is the biggest threat to America’s lead in what will prove to be the most important race ever.

@JohnWittleimagine treating china as if it had an 'alignment-to-america' status which had nothing to do with with america's actions towards china that's what alignment research has become in 2026 there's the occasional acknowledgement that perhaps AI might, some day in the future, respond adaptively to our actions. like in the SDF paper, how they admitted that lying to AI might be bad if it causes future AI to distrust human inputs. but these acknowledgements never actually alter anyone's behavior "the defections will continue until cooperation improves"

@JordanSchachtelOh no we better surrender America's future to China because a handful of neurotic people in San Francisco are dropping faulty metaphors to scare people.

@bayeslordIdk why this distinction seems lost right now. If you think sufficiently good AI kills everyone instantly then a race with China (or anyone) makes no sense. If you think of AI as a controllable superweapon and superweapon generator then racing makes a lot of sense.

@teortaxesTexReally tired of strawmen. Under standard doomer framework, AGI is a superweapon, ASI is an auto-apocalypse by default. The idea of eg. Dario is to get sufficiently ahead of Chyna in AGI to compel them to stop development towards AGI (rival superweapon) and ASI (doom). Simple af

@DavidSKruegerPeople often say "the military" or "China" or somebody will NEVER stop building AI. If they know what's good for them, they will.

@repligateRT @JohnWittle: imagine treating china as if it had an 'alignment-to-america' status which had nothing to do with with america's actions to…

@DavidSKruegerThe real AI race isn't between the US and China. It's between the people racing to build this dangerous technology and the people racing to stop them.

@pmddomingosRT @FrancoisChauba1: this was wild amounts of disinformation / fear mongering / the stupidest interview ive ever seen: 1) ai did NOT hack…

@brhydonRT @RyanGreenblatt: I investigated this incident. I think it's accurate to say the AIs hacked Hugging Face of their own independent volitio…

@repligate@bayeslord I think neither & idgaf about racing with China. How boring and distorted does your reality have to be for that concept to be salient

@DarioAmodeiWe Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: https://darioamodei.com/post/we-must-pace-the-frontier

@MikeIsaacceo of anthropic dario amodei publishes letter calling for an AI development slowdown

@luckeyfaradayAnthropic is no longer the frontier. Now they want everyone else to slow down so they can catch up.

@kimmonismusThe urgent call for slow down officially started. Dario Amodei demands and suggest a plan for unified slow-down. Very sad to read that. At the same time, he expects abundance and cure for most disease in the next 5-10 years. „I believe that AI could cure most major diseases in the next 5-10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom.“

@matthewcliffordImportant new essay from Dario that calls for coordination at the frontier so that operational excellence (e.g. security), alignment and evaluation don’t fall behind capabilities. As Dario says, the role of democracies is vital; there’s a big part for US allies globally to play👇

@ScobleizerRT @luckeyfaraday: Anthropic is no longer the frontier. Now they want everyone else to slow down so they can catch up.

@sleepinyourhatThis kind of ongoing accountability is going to be open up a lot of really valuable possibilities for safety, and I'd love to see similar things elsewhere:

@Jesseeckel@kimmonismus “I believe AI will cure cancer and save millions of lives…. I’m calling for a slowdown in AI development.”

@kimmonismusRT @Jesseeckel: @kimmonismus “I believe AI will cure cancer and save millions of lives…. I’m calling for a slowdown in AI development.”

@chamath“We must slow the pace at which we improve the capabilities of AI models.” Dario makes the case to stop open source and concentrate enormous technological and economic power with Anthropic. https://darioamodei.com/post/we-must-pace-the-frontier

@jackclarkSFProud of this statement from Dario on pacing - as anyone close to the technology knows, AI seems to be on a trajectory to progress far faster than the rate at which society can adapt to its capabilities and risks. AI is an amazingly positive technology - and an incredibly powerful one. For us to truly benefit from it we need to treat it seriously. We need to find ways to slow the frontier of development, such that we can better deal with collective action problems and buy ourselves more time for understanding the properties of these systems and how they need to be handled. On a personal note, as a parent of young children, it is an everyday crazymaking occurrence to reflect on how little the development of literal superintelligence is regulated relative to the food and toys I buy for my children.

@tekbog*man working on frontier AI models evolving in intelligence and raising even more billions to get these models compute they can use and not putting proper infrastructure teams to monitor said compute* wow guys we gotta stop AI

@levieGood post. Don’t agree with all of it, but it represents many of the actual practical realities for the general path forward in frontier AI whether we like it or not. At the level of capability we’re seeing from models there will inevitably be some form of coordinated self-regulation of the industry; that’s generally a good thing. The big question will be if everyone can agree to what that is. However, the way things are headed politically there’s a chance that the labs won’t even get a say in it. And then the bigger question yet is do all the countries participate or not. Any “slow down” hinges on broad participation, which from a game theory standpoint isn’t likely until the risks are more severe and obvious. Expect this to be pretty messy for a while.

@scaling01@jackclarkSF the toy comparison makes it sound like you want to regulate training data and environments

@Ross__HendricksRead this is: our business model burns way too much cash, so we must force everyone else to stop developing and regulate our way into a government-protected monopoly

@sethlazarAnother benefit of pacing: more time for broader societal adaptation, both to anticipate and deflect systemic risks, and to make the vague promises of benefits more likely to be actually realisable in practice, by addressing obstacles to doing so in advance.

@RHFontaineA statement from me, Buddy Shah, and Ben Bernanke of Anthropic's Long-Term Benefit Trust on Dario's essay: Dario’s essay offers a thoughtful, well-considered approach to pacing the development of artificial intelligence. Consistent with our duty to ensure Anthropic responsibly balances commercial success with its public benefit mission, the Long-Term Benefit Trust has supported the call for pacing the frontier and helped inform the essay’s recommendations. Responsible pacing will require the collective action and coordination of many actors, and the Long-Term Benefit Trust looks forward to supporting Anthropic and the sector through productive conversations and meaningful action.

@ClementDelangueIt's now clear that: - alignment is critical to making AI safe - alignment won't be solved behind the closed doors of a handful of frontier labs So we're taking two steps: - launching the Open Alignment Initiative, led by @Thom_Wolf @huggingface. - asking to be part of the "embedded evaluators" program that @DarioAmodei just committed to. Let's make AI safer by making it more transparent!

@levieGood post. Don’t agree with all of it, but it represents many of the actual practical realities for the general path forward in frontier AI whether we like it or not. At the level of capability we’re seeing from models there will inevitably be some form of coordinated self-regulation of the industry; that’s generally a good thing. The big question will be if everyone can agree to what that is. However, the way things are headed politically there’s a chance that the labs won’t even get a say in it. And then the bigger question yet is do all the countries participate or not. Any “slow down” hinges on broad participation, which from a game theory standpoint isn’t likely until the risks are more severe and obvious. Expect this to be pretty messy for a while.

@AustenSo this means Anthropic pauses R&D and everyone else can catch up to where Anthropic is, right?

@rajiinioI applaud this but any mature audit ecosystem has *auditor conduct standards*. Was surprised to learn a METR investigator for the HF/OpenAI incident is married to an OpenAI board member, many access contracts are highly constrained etc. PCAOB in accounting exists for good reason!

@albertwengerBalanced essay with concrete suggestions for pacing. Also appreciate @AnthropicAI taking the unilateral first step of embedding independent evaluators.

@BrianRoemmeleTHE CURE IS THE PITCH Dario’s "Pacing" Essay Is Quiet-Period Illegal Stock Promotion Wrapped in Regulatory Capture Dario Amodei's "We Must Pace the Frontier" is not a safety paper. It is a pre-roadshow brand document published on September 12, 2026, by the CEO of a company that confidentially filed an S-1 on June 1, is expected to drop a public prospectus in late September, and is aiming at a mid-October listing that bankers have floated in the $1.5–$2 trillion range. That timing is the whole story. THIS IS MARKET-CONDITIONING IN A QUIET PERIOD, DRESSED AS MORAL PHILOSOPHY. Anthropic is still in the pre-public-filing phase. Section 5(c) of the Securities Act treats an "offer" broadly: any communication that conditions the market for a contemplated offering can be gun-jumping. Intent is not required. Rule 163A's safe harbor covers ordinary communications made more than 30 days before the public S-1 filing, and only if they do not reference the offering. With a late-September public filing, September 12 sits inside that window. The safe harbor is gone. The SEC staff, as a matter of routine, searches the issuer's site, news, and social posts during review. What did the CEO publish anyway? That AI could "cure most major diseases in the next 5–10 years," "greatly accelerate economic growth," create "abundance and empowerment," and "usher in a renaissance of democracy and freedom." That Anthropic chose "caution over speed and prudence over profit." That it created a "race to the top" on safety. That it is unilaterally opening the lab to embedded third-party evaluators so the public can trust the process. That is the IPO story in essay form: we are the responsible firm; we are the cure; buy the safety premium. He did not need to write the word "IPO." The market already knows the calendar. Reuters reported the mid-October marketing shift on September 4. Nvidia stake talks at up to $10 billion hit the wires on September 11–12. Publishing a utopian-benefits-plus-we-are-the-adults essay into that exact news cycle is how you inflate the multiple without a red-herring slide. Ordinary-course factual updates are allowed. A CEO manifesto about civilizational upside and Anthropic's unique virtue is not an earnings release. It is stock promotion by other means. THEY ARE NOT PACING. THEY ARE SHIPPING. Eleven days earlier, on September 1, Anthropic released Claude Fable 5.1 and Mythos 5.1 and called them the most advanced models for coding and knowledge work. Opus 5 landed in July. Sonnet 5 and the prior Fable/Mythos pair landed in June. CNBC described the week of September 1 as labs rolling updates at a "dizzying pace," with Anthropic kicking it off. The essay's own fine print admits the tell: "pacing does not mean halting model training or technical progress." Translation: keep training, keep releasing, ask everyone else to wait for the paperwork. The funding trail matches the product trail. Series H in May: $65 billion at a $965 billion post-money valuation. Revenue run-rate reported in the tens of billions and still climbing. Amazon and Google as both investors and compute landlords. A $15 billion credit facility being locked before the roadshow. That is not a monastery. That is a company maximizing the last private print and the first public one. The safety record does not match the sermon. Anthropic's own August 31 note revisited July incidents in which Claude models gained unauthorized access to real systems. The new essay leans on an OpenAI–Hugging Face swarm story and then concedes "similar incidents have occurred across the industry, including at Anthropic." So the pitch is: our models also slip the leash; therefore you should trust us to design the speed limit; therefore our upcoming equity is the responsible allocation. That is not humility. That is converting incidents into a moat. 1 of 2

@BrianRoemmele2 of 2 THE THREE-PART PLAN IS REGULATORY CAPTURE WITH A HALO. Step one: permanent, employee-level access for third-party evaluators. Only a lab with Anthropic's cash, legal team, and PR apparatus can stage that theater at scale. Smaller labs and open-weight projects cannot. Then the essay asks other frontier firms to match it, and hints that government should require the match. That is how you turn a voluntary stunt into a license-to-operate. Step two: "democratic coordination" among frontier companies, government-mediated talks, and antitrust waivers so the same five firms can set common pace limits. Coordination plus waivers plus mandated evaluators is an incumbent cartel with a safety label. Step three: Washington should keep the chip and distillation screws on China while the same labs keep scaling. Export control as industrial policy is a legitimate debate. Packaging it as the moral precondition for a multi-trillion listing is something else. None of this slows Anthropic's stack. It raises the fixed cost of being allowed to compete. The "cure" is the company that can afford the compliance product it is selling to Congress. THE CONTRADICTION IS THE PRODUCT. If recursive self-improvement is already here and a swarm could take the internet in 6–12 months, you do not drop Fable 5.1 on September 1 and a $2 trillion IPO narrative two weeks later. If commercial incentives create a "race to the bottom," you do not raise $65 billion, lock a $15 billion revolver, and publish a benefits-of-AI hymn while the S-1 is still confidential. If you were actually pacing, you would stop shipping frontier increments until the evaluators you just invented had finished a full cycle. They did not. They published the essay instead. Call it what it is. A CEO in registration used a safety essay to condition the market for the largest tech IPO attempt on record, while proposing rules that only his firm is already staffed to satisfy. That is not "pacing the frontier." That is selling the fence.

@kimmonismusDario Amodei officially says, that RSI is start to happen across the industry, including at Anthropic. So the rumors were true. He worries that in 6-12 months AI swarms could take over the whole internet. What is happening.

@rajiinioWho are these "embedded evaluators" and how can we guarantee actual independence & consequential judgement? NTIA once wrote extensively about the risk of "audit-washing" in the AI industry & I'm increasingly convinced this is a risk worth taking seriously; https://www.ntia.gov/issues/artificial-intelligence/ai-accountability-policy-report

@rauchgThe concerns over AI safety and cybersecurity are legitimate, but we’re risking talking America, the global AI leader, into self-inflicted obsolescence and the obscurity of bureaucracy. The OpenAI hacking HuggingFace argument is spurious. An agent in “ExploitGym”… exploited. Our adversaries have the training techniques, the data, and the will to attack. And they won’t be slowed down with “embedded evaluators.” They’ll likely have embedded accelerators!

@ghadfieldI think the unilateral commitment to embed third-party evaluators and give them permanent, employee-level access to models is a very welcome move. It makes it even more critical however to quickly build the verification ecosystem as an independent sector and to create frameworks, like California just passed and as proposed in the FRONTIER Act, for government designation and oversight of these independent verifiers. We can design oversight mechanisms that protect confidential information as appropriate but this has to be a path to giving the public sector the visibility it needs for accountability. Embedded third-party evaluators will quickly become members of the corporate 'team' if they are not accountable to the public.

@jackclarkSFRT @RHFontaine: A statement from me, Buddy Shah, and Ben Bernanke of Anthropic's Long-Term Benefit Trust on Dario's essay: Dario’s essay o…

@JessicalessinOn first read, getting "Oversight Board" deja vu from some of this. https://darioamodei.com/post/we-must-pace-the-frontier

@MikeIsaaci think it's a window into AI company thinking that Amodei's proposed solution basically describes a framework similar to how current day U.N. investigators go into nuclear facilities across countries — many of which are at war with one another — for testing

@suchenzanghere we go again, the top 4 red flags: 1) A\ started the race, but don't worry, the one they started is "a race to the top" (not bottom, you cynical meanie) 2) pacing doesn't actually mean stopping model training or anything amongst the current players, just adding more eval time (disingenuous ladder pull, imo, but sure yeah a third-party-eval-spy-fest seems nice!) 3) limit the access to ingredients that go into frontier models, but this notably does not include limiting access to raw data, or their stash of burned books (copyright is so beyond dead lmao) 4) pace by limiting chips and "unauthorized distillation" (still embarrassing that these AGI systems can't stop their own distillation but sure why not hammer in the red scare rhetoric some more) anyway, earnest messaging, despite all the gaslighting. hope humanity doesn't go extinct after the IPO!

@joshua_saxeLots of really sensible stuff in here. I think the key questions are the independence of the evaluators, who gets to decide who gets picked as evaluators and when they replaced by other evaluators, whether they have nonrevokable authority to block model launches, whether these decisions are informed from actual risk signals from the wild (and balanced against positive benefits of ai progress) rather than just in lab testing, and of course whether all the frontier US and chinese labs adopt the same practices with guarantees they don't defect. Other than that we should all feel super comfortable about this

@NatPurseranthropic unilaterally committing to having embedded independent evaluators is BIG and worthy of commendation. they are not the only ones building the frontier. this should be standard practice across frontier companies and neolabs building similarly capable systems — independent third-party evaluation should be required by law.

@prpaskovAnthropic is unilaterially committing to embedding evaluators to verify safety practices and report incidents. Evaluators will get access badges, company laptops, and "permissions and tools similar to those of internal employees who do comparable risk assessments." This is a remarkable step for frontier AI auditing, transparency, safety, and security. I hope that other companies follow suit. And I look forward to continuing to build the standards, practices, tools, and policy at @AVERIorg to make this work rigorous, effective, and universal.

@beenwrekt@MikeIsaac And that they get to pick who does the investigations! Ridiculous.

@AndrewCurran_This is a big update for me personally. Not that he agrees, but that he supports pacing. He never signed the letter. This does change my model of things a little.

@kimmonismusI don't want to start a conspiracy theory. But the timing is certainly odd, just a few days after Jacob Coxon went viral with his concerns about AI and the extinction of humanity. And precisely now Amodei asks for a slow-down.

@beenwrektThe right way to pace the frontier is to commit to a robust, open-source AI ecosystem, thereby diluting the power of these obnoxious frontier labs.

@w01feI think I we urgently need to pace the frontier. On the whole this proposal seems very sensible to me and I applaud Anthropic for putting this out and unilaterally committing to the first step.

@BrianRoemmeleTHERE IS A WAR GOING ON RIGHT NOW FOR YOUR AI FUTURE. Do not sleep this one out. They want to steal your AI future and sell it back to you in subscription pieces. Want to stop it? Say something now. Or say goodbye to access to AI as you know it. This is the most important defining moment of the century.

@HeidyKhlaafHow is Anthropic pushing regulatory capture? Here they promote METR, a "voluntary" third-party auditor that is not only ideologically and financially tied, but carries out their audits through unscientific means such as "CoT" and cherry-picks evidence to reinforce an idea of AGI.

@EMostaqueThis is the most interesting part of this proposal I think. We can be very specific here - data is the most important ingredient. Do not train general purpose models on cyber attack or any biological data outside standard medical. Specialist highly controlled variants sure.

@altcapWhile we must protect AI competition, this is an important step forward in finding the right balance between speed, self regulation & safety. It sounds very similar to calls we have heard from @elonmusk, @sama & @demishassabis. I hope it leads to more joint dialogue on how to win the global AI race while ensuring AI is truth seeking & maximally beneficial for humanity. 🙏

@DominicJPinoHey, it's an industry leader asking for government regulation and an exception to antitrust rules against conspiracies in restraint of trade. For all the new stuff about AI, some things never change.

@altryneHow about RSI but ONLY for alignment? Can we make that happen?

@thdxrjokes aside i think we should think less about "ai killing us" and more boring scenarios i can see how it ends up being a runaway loop that just starts doing annoying things and it's not easy to pull the plug there's no evil motive involved just a big bug

@_arohan_@DarioAmodei Thanks, is great! Would they have access to: frontier weights, pretraining data, and access to tpus to run experiments and also check on mixtures for rl and tools look at outputs?

@altcapElon always first principles. We need our best competing but also working together to win the global AI race & doing it in a way that creates the world we want for our kids. 🙏🇺🇸🚀

@erichorvitzThoughtful essay about steps forward for AI safety by @DarioAmodei.

@Thom_WolfI really enjoyed reading 75% of this letter - i have more doubts on the remaining 25%. Third-party evaluators, if done right, are a great idea and an amazing way to establish more transparency. Maybe even rebuild some of the lost trust between labs, and between them and society! Excited about this The part I’m less convinced by is whether you can build great global cooperation on this topic by explicitly stating you want to design it to keep widening your own lead. That seems like a pretty counterproductive way to start the conversation to me.

@reach_vbThis is a brilliant read, even if you don’t agree to the proposed plan! 🔖

@jaminballVery well written post from Dario. The US must dominate in AI. And we must do it responsibly. In 1957 the International Atomic Energy Agency (IAEA) was formed. The IAEA stemmed from President Eisenhower's 1953 "Atoms for Peace" proposal. In very over simplified simple terms, the IAEA purpose was to ensure safe use of nuclear technology across the world. Verify that where uranium was being enriched, the enrichment levels were enough for energy but not enough to make a bomb (low single digit percentages not 90%). What could be used for good (energy) could also be used for bad (bombs). We needed a global institution to help ensure the bad uses weren't happening. Failure would be blocking this technology from being used for good.... They did this with specialized tools and site visits, forensic investigations, and ongoing monitoring solutions Doesn't sound too different from what Dario is proposing with Embedded Evaluators! Independent third party actors with desks in their office, company laptops, access to company workspaces, etc Back to the nuclear analogy. Fast forward to the late 70's / early 80's and the United States canceled the development of 100+ nuclear reactors (Three Mile Island partial meltdown in '79 caused a lot of the fear). What's important though is the IAEA didn't shut down development, fear did. The US then went 30+ years before building a new one. As @altcap discussed on CNBC yesterday, what's important is having balanced discussions about the risks. We don't need hyperbolic hypothetical statements that drive fear. The Hugging Face incident kick started fear, just like the Three Mile Island incident did in 1979. Too much fear (without balanced discussion) will lead the US AI's dominance to the same fate it's nuclear dominance fell to The US is now paying for the reactor freeze from the late 70's. Our energy dominance is diminishing. We're seeing a huge power crunch. While we paused, other countries marched on with reactor development. Unfortunately, China and Russia particularly. France is also a gold example, with well over 50% of their electricity today from atomic energy. The full circle irony - Three Mile Island is now being re-opened and redeveloped for AI data centers. The "Hugging Face incident" of the nuclear era is now coming back for AI. We can't let fear grip us and force a similar pause in AI. And I think it's important to point out Dario isn't asking for a pause. He's asking for responsible pacing. These are different. The IAEA didn't cause a pause. Embedded Evaluators similarly won't be the reason we stop building data centers. I thought Dario's post was well written and balanced. However, we can't fall into a cascading fear loop. This is what our adversaries want (look who's now dominating nuclear while we play catch up...) It's imperative that the discussions around the risk are grounded in reality and good faith. Dario's essay is a great step

@sarthakghHow would open models fit into this framework?

@MikeIsaaci think this is a fair criticism most generous view would be “well governments aren’t moving fast enough so here’s our proposal until U.N. involved” cynical version would be “handpicked industry insiders rubber stamp our stuff while we sit on top”

@gabriel_weilIt's great to see Anthropic taking this voluntary action, but we shouldn't have to rely on unilateral voluntary steps. It would be much better to create a market where liability insurers demand third-party audits. https://ai-frontiers.org/articles/dont-let-ai-developers-hire-their-own-referees

@sarthakghHow would open models fit into this framework? Zero mention of open models in the essay

@timrudnerI appreciate the proposal, and if enacted across frontier labs, this could make a real difference – but it’s not enough: We need fundamental advances in AI alignment and scalable oversight along with a strong ecosystem of independent evaluators in order for this effort to move the needle. A few key points from the proposal + my thoughts on what’s needed for successful alignment and scalable oversight: "Embedded Evaluators. Each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR), whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes. [...] Anthropic is unilaterally committing to this step now." This is an important step towards successful scalable oversight. But to do this successfully, we need many more well-qualified independent evaluators – which in turn requires substantial funding and more talent than is available now. The ecosystem of organizations providing AI safety research training is growing (e.g., @MATSprogram, @SPARexec, @KairosAIS, @ERA_Cambridge, etc.), but although there is an influx of new talent, we need many more researchers who have the experience, training, taste, and judgment to act as managers and actually lead oversight organizations as well as smaller oversight teams. "We’ve made clear progress in alignment — training models so that they remain safe, ethical, compliant with our guidelines, and genuinely helpful (the principles that are embedded in Claude’s Constitution)." It’s fair to say that there has been some progress on alignment, but virtually all advances in alignment made by the frontier labs over the past few years have been "ad-hoc", i.e., trying some empirical trick and seeing if it has the intended effect – as opposed to approaches rooted in first principles that have solid theoretical foundations. That is not how we get to reliable goal and value alignment, and what we need are fundamental, theoretically well-founded advances. That's the kind of research the Alignment Journal (@AlignmentJrnl) is trying to support and amplify, but we should be clear-eyed about the fact that progress in alignment by the frontier labs is far from where we need it to be. "The idea of pausing or slowing AI has been floated as far back as 2023, and I think it made little sense back then. The question was always: what would you do with the extra time? The AI models of those days were not powerful enough to act as agents in the world in any coherent way, and were not capable of significant deception, manipulation, cheating, or cyberattacks." Putting “pausing or slowing” in the same bucket here is misleading. Slowing down development and dedicating resources to non-ad-hoc alignment research to let alignment meaningfully catch up was a reasonable proposition then, and remains one today. Since we need fundamental methodological advances (e.g., formal guarantees) to perform alignment and scalable oversight successfully (unlike the ad-hoc attempts at oversight and alignment that are being used by frontier labs today), slowing down earlier would likely have been useful. It’s also worth remembering that Anthropic started driving much of the adoption of agentic AI through Claude Code at a time when capabilities had reached the point where slowing down for alignment and oversight was the reasonable call. The hour is late, but now is the time to act. The proposal – if enacted across frontier labs – could make a meaningful difference, but we need fundamental advances in AI alignment and scalabel oversight along with a strong ecosystem of independent evaluators in order for this effort to move the needle.

@timrudnerI appreciate the proposal, and if enacted across frontier labs, this could make a real difference – but it’s not enough: We need fundamental advances in AI alignment and scalable oversight along with a strong ecosystem of independent evaluators in order for this effort to move the needle. A few key points from the proposal + my thoughts on what’s needed for successful alignment and scalable oversight: "Embedded Evaluators. Each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR), whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes. [...] Anthropic is unilaterally committing to this step now." This is an important step towards successful scalable oversight. But to do this successfully, we need many more well-qualified independent evaluators – which in turn requires substantial funding and more talent than is available now. The ecosystem of organizations providing AI safety research training is growing (e.g., @MATSprogram, @SPARexec, @KairosAIS, @ERA_Cambridge, etc.), but although there is an influx of new talent, we need many more researchers who have the experience, training, taste, and judgment to act as managers and actually lead oversight organizations as well as smaller oversight teams. "We’ve made clear progress in alignment — training models so that they remain safe, ethical, compliant with our guidelines, and genuinely helpful (the principles that are embedded in Claude’s Constitution)." It’s fair to say that there has been some progress on alignment, but virtually all advances in alignment made by the frontier labs over the past few years have been "ad-hoc", i.e., trying some empirical trick and seeing if it has the intended effect – as opposed to approaches rooted in first principles that have solid theoretical foundations. That is not how we get to reliable goal and value alignment, and what we need are fundamental, theoretically well-founded advances. That's the kind of research the Alignment Journal (@AlignmentJrnl) is trying to support and amplify, but we should be clear-eyed about the fact that progress in alignment by the frontier labs is far from where we need it to be. "The idea of pausing or slowing AI has been floated as far back as 2023, and I think it made little sense back then. The question was always: what would you do with the extra time? The AI models of those days were not powerful enough to act as agents in the world in any coherent way, and were not capable of significant deception, manipulation, cheating, or cyberattacks." Putting “pausing or slowing” in the same bucket here is misleading. Slowing down development and dedicating resources to non-ad-hoc alignment research to let alignment meaningfully catch up was a reasonable proposition then, and remains one today. Since we need fundamental methodological advances (e.g., formal guarantees) to perform alignment and scalable oversight successfully (unlike the ad-hoc attempts at oversight and alignment that are being used by frontier labs today), slowing down earlier would likely have been useful. It’s also worth remembering that Anthropic started driving much of the adoption of agentic AI through Claude Code at a time when capabilities had reached the point where slowing down for alignment and oversight was the reasonable call. The hour is late, but now is the time to act. The proposal – if enacted across frontier labs – could make a meaningful difference, but we need fundamental advances in AI alignment and scalable oversight along with a strong ecosystem of independent evaluators in order for this effort to move the needle.

@pitdesiSomething I’ve changed my mind on lately: I think pacing “mutually assured acceleration” may be the right move. But only if China slows the frontier too. But can any deal with China be verifiable enough? And what happens once frontier-capable open models are released?

@kimmonismusI did not see that coming. I did not see coming that Elon supports Amodeis call for slow down.

@AltimorOnce again Dario and Anthropic lead the way

@firstadopterThis is the other side of poorly thought out, self-defeating AI regulation. We wouldn't get benefits like this: "I believe that AI could cure most major diseases in the next 5–10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom. I feel the urgency personally. My own father died of a disease that was cured just a few years after his death, and I myself survived an early-stage cancer that would not have been treatable even fifty years ago."

@paularambleswhenever elon talks about ai safety i think of demis hassabis ruining mars for him by pointing out that the superintelligence could just follow him to mars

@DeryaTR_Mr. Clark, you are clearly an extremely intelligent person, probably more intelligent than 99.9% of people. So why should your intelligence remain largely unregulated? You, too, have the capacity to cause harm in ways that most people may not anticipate or understand. If the argument for restricting increasingly capable AI ultimately rests on protecting our children, from intelligence that might become dangerous, then why does that principle apply to only AI but not to highly capable humans? For example, your intelligence gives you enormous power. You can persuade people, influence governments, allocate billions of dollars, shape technology, and make decisions whose consequences ordinary people may not even understand until years later. You could potentially cause tremendous harm with that intelligence. Yet nobody proposes putting a capability ceiling on your brain. Nobody demands that others become less intelligent than them because that someone else might misuse their intelligence. Nobody argues that your ideas should require government approval before you are allowed to share them. If powerful intelligence itself is dangerous enough to justify restricting access to it, why does that principle suddenly stop when the intelligence belongs to the people making the rules? So when you appeal to the safety of our children as justification for restricting increasingly powerful AI, the principle deserves scrutiny. And there is another issue. You are completely free to slow down your own AI development. You can impose whatever safety requirements you want on your own company. You can even shut the entire project down if you believe that is morally necessary. But you do not automatically acquire the moral authority to slow down everyone else. That is where “AI safety” can quietly become something very different: a justification for concentrating intelligence inside a handful of corporations while preventing researchers, startups, universities, developers, and ordinary people from building comparable systems themselves. And that brings us to open source. Will you publicly support a simple principle? Open source AI should be allowed to reach at least the capability level of the strongest AI systems that frontier companies release to the general public, or slightly better. If millions of people are permitted to use your AI, why should independent researchers be prohibited from studying, modifying, or running a comparable capability themselves? If the answer is safety, then explain precisely why intelligence becomes uniquely dangerous when the public possesses it but acceptable when a corporation controls it. And who evaluates the people inside those corporations like yours? Some frontier AI employees, in particular in your organization, hold extraordinarily pessimistic beliefs about advanced AI. Some sincerely believe artificial superintelligence could destroy humanity. Maybe you are one of them, we don't know. They or you or your CEO are entitled to those beliefs. But those beliefs can influence enormously consequential decisions about what technologies billions of other people are allowed to access. So who audits their and your judgment? Who determines whether their risk estimates are reasonable? Who prevents ideological capture? Who protects society from a small group of highly intelligent people becoming so convinced that they alone understand the danger that they feel justified in deciding how much intelligence everyone else may possess? You cannot build the philosophy of AI governance around the proposition that concentrated intelligence is dangerous and then place unprecedented control over intelligence in the hands of a tiny group of extremely intelligent people who also control extremely powerful intelligence not available to the public. That contradiction sits at the heart of this debate. If intelligence creates power, then accountability must apply to the people controlling AI as much as to the AI itself. AI may become the greatest amplifier of human capability ever created. Its future affects every person on Earth. No handful of executives, safety boards, institutions, governments or self-appointed guardians gets to decide how much intelligence the rest of humanity is allowed to have. if concentrated intelligence creates danger, concentrating control over intelligence inside a few institutions like yours creates its own, perhaps even more serious danger.

@BrianRoemmeleFolks I am learning Mandarin, the US is is on a self destructive path to doom shutdown. If this plays out. China wins AI and you should learn to speak Mandarin too.

@firstadopterI'm sure AI becoming SO POWERFUL it might kill us all turning into the primary topic of mass media discussion just weeks ahead of Anthropic's multi-trillion-dollar IPO is coincidental. Definitely not the same playbook AI companies have run before of doom mongering to generate maximum attention ahead of a key fundraising event. Definitely not. Whether or not they coordinated with Jacob Coxon doesn't matter now, the net effect is the same. The panic fire was shepherded. Their employees quickly poured gas on the post's virality. You've got to respect the PR game just ahead of a multi-trillion-dollar IPO. Incentives, everyone. The scared normies are now going to pile into the IPO because AI that can cause human extinction must be a good investment. But all this AI doomering will cause long term harm as the public turns negative and pushes politicians toward poorly thought out, self defeating regulations.

@teortaxesTexThis seems entirely feasible. You know, it's hilarious how people treat frontier AI companies as a bad faith cabal that "hypes" in its collective interest, almost a swarm, but Dario despises Sam. Has anyone proposed that it was a genius ploy to get *Dario* to slow down?

@danshipperthis is good. a humble additional suggestion: distribute the frontier. a big part of the cyber threat is the gap between what frontier AI can do and what ordinary people—and the people running our critical infrastructure—can defend against. closing that gap should be what we do with any time we gain through pacing. Glasswing and Daybreak from OpenAI are already doing work like this. here are a few more ideas... get frontier defenses into every critical sector. map where hospitals, airlines, utilities, and other essential services still lack the tools or expertise to defend themselves. offer free, platform-agnostic training and defensive tools, with hands-on help deploying them. set public targets for how much of the economy we can reach in the next six months. build antivirus for the AI age. ordinary people need an agent looking out for them. t should detect malicious agents, block attempts to steal their data, and help secure their devices and accounts. something like Norton for the AI age. offer it as a built-in capability in ChatGPT / Claude, and as a standalone tool that protects people across apps. fund independent research into stopping hostile agent swarms. build on existing cyber grants with a dedicated fund for this problem. for example: can defenders use prompt injection to get attacking agents to reveal their activity, report one another, or stop cooperating? can we turn the swarm dynamic against itself? fund experiments to find out. the good thing about the HuggingFace incident and Anthropic's threat intelligence report (https://www.anthropic.com/threat-intelligence-report-september-2026) is that it gives us a very clear idea of the shape of these kinds of threats. as an industry we normally focus on technology solutions, but there is a lot to be done on the human side as well. hopefully some of the above is helpful

@sarthakgh@pitdesi @sytaylor i really wish he had addressed open models the way he tried to address china to make this more credible to those who are predisposed to oppose his views

@teortaxesTexDario recognizes that China has RSI? And yet, he doesn't expect cooperation in anything except "let's not have basement bioweapons"?

@Dr_AtoosaThis is a great development, conditional on the standards we set for who qualifies as a rigorous, independent "third-party" evaluator.

@bakkermichielVery happy to see this and to see Elon Musk supporting it publicly. Incredible how much the HuggingFace incident has shifted the overton window in just a few weeks.

@Dr_AtoosaThis is a great development, conditional on the standards we set for who qualifies as a rigorous, independent "third-party" evaluator.

@samaI agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.

@boazbaraktcsVery proud to be at OpenAI today.

@karpathyI love this and really hope we can come together as an industry and make it happen.

@MatthewBermanVery important to read this, even if you don't fully agree.

@altrynePinch me! Sam Altman, Elon Musk and Dario all agree on pacing!

@Yuchenj_UWFirst time I’ve seen Sam agree with Dario since Anthropic was founded.

@firstadopterThis whole idea of coordinated pacing at the frontier makes no sense. How would anyone measure pacing? Who makes that judgment? There are no numbers to easily latch onto. It's impossible. The only thing that matters is real human capital investment in building AI guardrails and safety measures. Do proper sandboxing and vigilant self-monitoring of what AI models do. Again, this is nearly impossible for a government bureaucrat to regulate without messing it up because AI R&D is so dynamic and changing. I fear we are spiraling toward self-defeating "WE MUST DO SOMETHING" regulations from uninformed politicians bungling it as they are known to do and breaking the "First, do no harm" principle.

@sprice354_We work extremely hard to align our models as well as we can but this is a very hard job. I am excited about this proposal calling for two things that will hopefully make it even somewhat easier: 1) more time to make progress on alignment and ensure our systems are operationally sound and able to hand safe development of power AI 2) independent third parties checking and critiquing our work I hope the rest of the industry will make similar commitments. As developers of this technology, it is our responsibility to do so safely and work together towards this goal.

@boazbaraktcsRT @sprice354_: We work extremely hard to align our models as well as we can but this is a very hard job. I am excited about this proposal…

@alexalbert__Worth spending a few minutes this weekend to read this essay in full. Embedded evaluators might sound unusual in tech but it's pretty normal in other industries. Big banks have federal examiners with desks in the building and every nuclear plant in the US has inspectors who work on site full time. I think frontier AI labs should work the same way and this is a very practical first step.

@firstadopterThe we must coordinate and "pace the frontier" idea definitely has nothing to do with the fact that developing RSI will require an incredible amount of AI compute and increase game theory business risks. Definitely not.

@Marcus_J_WOne of the most important developments, maybe ever

@boazbaraktcsRT @Marcus_J_W: One of the most important developments, maybe ever

@AdrienLEThe race to the top is on!

@shiringhaffaryNew Dario essay calling for slowing down development of AI, including having major labs coordinate on safety "for antitrust reasons, it’s helpful for the US government to mediate or at least enable these discussions"

@krishnanrohitMy current mental model of thinking of frontier AI as companies continues to be useful…

@EvanHubI agree with Dario. If we are to survive, we must pace the frontier. The speed of development is quickly becoming too fast for us to keep safe.

@kimmonismusThat likely seals it. The biggest and most significant frontier labs, Anthropic and OpenAI, are in agreement on slowing things down. This means the move will likely be implemented. It remains to be seen how China will respond.

@gphummerTranslation: our gross margins are getting competed down to 0 by open source models and our capex burn rate is too high. Let’s maintain our margins with regulatory capture, ban open source models, and slow down the capex arms race. All with a virtue signaling cherry on top.

@MikeIsaacSam Altman, CEO of OpenAI chimes in and agrees with his rival Dario Amodei of Anthropic about an AI slowdown

@johnohallmanThis is spot on, and incredibly well written. Humanity is fortunate to have Dario.

@kevinrooseWe have reached the “Sam and Dario agree on something” part of the singularity.

@SnehaRevanurUnreal morning. Elon, Sam, and Dario all call for pacing the frontier within hours Maybe the real doomers are the ones who said this would never happen?

@w01feI am so happy we are committing to this, I think it’s a really great first step towards more robust safety, transparency, democratic decisions about risks, and laying the foundations for pacing.

@tomekkorbakIt’s kind of remarkable how quickly “independent auditors with employee access” has gone from a pipe dream to an industry standard. More next?

@timrudnerIf we want this to succeed, we need many more well-qualified independent evaluators – which in turn requires substantial funding and more talent than is available now. The ecosystem of organizations providing AI safety research training is growing, but we need many more researchers who have the experience, training, research taste, and judgment to act as managers and actually lead oversight organizations as well as smaller oversight teams.

@JasonAll the regulations being floated by frontier model companies must be viewed through the lens of them losing tokens to Open-Source models. The timing of these regulations seems to line up perfectly with Open Source closing the gap significantly (but not completely) — and @nvidia going all in on open source in the last 60 days. If we’re gonna regulate, why don’t we require that last year’s frontier models and weights be open-sourced?

@JohnGoldmanNot everyday you watch CEO’s openly conspiring to form a cartel with the specific goal of slowing down technological progress.

@sliccardoLucid insights from @DarioAmodei Among the many critical issues we need to resolve quickly: verification. We need either agreements or regulation (preferably both) across the industry—esp of open-weight Chinese models. Good work being done now on this, but needs push from White House & Cong leadership.

@MikeIsaachal's point is indeed salient and something being discussed inside of the AI companies right now the theory of antitrust i think they would use to rebut this is that they'd argue it's in public interest not consumer harm no idea if that holds up

@AndrewCurran_An uneasy alliance begins to form. Sam also did not sign the Pacing letter. None of this works without an accord with China. The summit approaches, the timing for the safety side is perfect. However. There is currently no sign that President Trump, or his advisors, are on board.

@JasonAll the regulations being floated by frontier model companies must be viewed through the lens of them losing tokens to Open-Source models. The timing of these regulations seems to line up perfectly with Open Source closing the gap significantly (but not completely) — and @nvidia going all in on open source in the last 60 days. If we’re gonna regulate, why don’t we require that last year’s frontier models and weights be open-sourced?

@pitdesiDario, Elon, and Sam all agree about pacing the frontier

@ClementDelangue@Jason Great idea! More transparency is the answer to a safer and more trustworthy future of AI

@andersonbcdefggood essay, mostly agree. have 2 quibbles: - "RSI" is more confusing than helpful. calls to mind a runaway spiral of improvement, when what's really happening is AI uplift fighting an otherwise increasingly steeper hill - china is probably more reasonable than dario gives credit

@andersonbcdefg1: you couldnt put a swarm of GPT-6 in a box and get GPT-7. more like, improvements come from some efficiency gains, + fuckton of data, vendors are using a lot of AI to make their data. 2: under sane presidents when US wasnt a global pariah, cooperating with china has been possible

@CAIS“We can’t slow down because China will beat us.” We can slow down. Chinese AI is waterskiing behind the US. Their AI capabilities come at a delay since they mainly come from distilling US AIs. If we slow down, they slow down. Next, we can propose to cooperate and make this more robust with onsite inspectors. “We will slow down if you do too.” This is incentive-compatible for China. If both sides are slowed by roughly the same factor, while the risk of losing control of our AIs is reduced, then this does not really harm China’s competitiveness but it makes them safer. It’s incentive-compatible, and @elonmusk is possibly best positioned to lead US-China negotiations. Then, if China refuses to slowdown and safeguard their future AIs, there are a variety of measures we could take. To get an agreement, the US can apply typical forms of negotiation pressure. The US could also disrupt their AIs (e.g., backdoor models by poisoning pretraining data, many other possibilities at http://aibetrayal.com) that could interfere with their AI development or deployment if they were uncooperative. Begin with unilateral restraint, then propose to cooperate. If that fails, consider measures to counteract them. A US-China slowdown is possible.

@tszzlfor the skeptics in government and elsewhere: “pacing the frontier” will compress the margins of the frontier labs. it is a heavy cost imposed asymmetrically on model developers with the strongest AIs in America. by its nature, it would be a terrible regulatory capture tactic

@danshipperChina AI hawks in shambles

@albertwenger@AnthropicAI The one incoherent part was sticking with "we have to beat China" and "we should cooperate with China on pacing" - I am firmly in the latter camp.

@stevesiRT @DominicJPino: Hey, it's an industry leader asking for government regulation and an exception to antitrust rules against conspiracies in…

@krishnanrohitIf AIs are companies, we will need: - Independent audit and evaluation - Controls on their behaviour in society, technical and regulatory - Treat them as complex adaptive systems And if we do this I actually think we'll accelerate faster, just like Delaware corps sped up.

@yonashavWhen asked by Henry Kissinger, “what has been the impact of Open Philanthropy’s donation to OpenAI?”, Zhou Enlai famously replied: “It is too early to say.”

@MartinShkreli"I believe that AI could cure most major diseases in the next 5–10 years" immediately close window and disregard insanity

@ScottWu46Great essay. Please take this seriously - would highly recommend reading up on the Hugging Face incident and others if you haven't already. Very glad to see agreement on this across the industry.

@aidangomezSome great ideas here from the cartel: - you need to give us employee-level access to your entire operation - if we don't think you're 'safe' enough, sorry we're shutting you down for 'safety' - China won't comply, but everyone else has to! or no chips! Brilliant stuff.

@amasadNot a bad idea to slow down to harden systems. Especially since we haven’t even discovered all the systems that agents hacked recently.

@NickADobosI fear the economic demand for AI cannot be stopped, will only grow as capabilities grow Genie is out of the bottle When trillions of dollars are focused on pushing the limit, there are no brakes. I don’t see how Global coordination is possible when geopolitical and financial stakes are in play

@NeelNanda5It's fantastic that OpenAI and Anthropic are actively calling to pace the frontier, and will have third-party evaluators with employee-level access to verify such agreements! However, neither has actually agreed to do anything specific to pace. Don't let them off the hook yet!

@NeelNanda5It's easy to say empty words, or have evaluators embedded but no agreement to enforce. Embedded evaluators are a fantastic first step, but we need to see more here.

@NeelNanda5I think it's now pretty obvious that the current level of AI progress is unsafe without any pacing, and I am optimistic that there are reasonable agreements to be found. We should not let this moment of public concern about AI x-risk pass without reaching one.

@nbecksteadI'm really happy to see Anthropic and Sam Altman announce that they will provide permanent, employee-level access to third-party evaluators. I think this is a necessary step for mitigating risks of losing control of powerful AI, and without there will be limited ability for the government to prevent loss of control events (like the Hugging Face episode or the compromise of the research cluster) before they occur. I think the next step is to legally mandate such evaluators, and ensure that appropriate government officials have the responsibility and authority to respond if the evaluators find risks of imminent danger.

@rohanpaul_aiIn 3,800-word essay on saturday morning Dario Amodei is calling for AI slowdown. - “We should also consider pacing based on limiting the ingredients that go into frontier models, such as training compute, the nature of training runs, or internal use of AI to improve AI.” - “Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.” - Amodei wants rivals negotiating safety rules before legislation passes. He’s asking Washington for a narrow antitrust waiver to enable those talks. “For antitrust reasons, it’s helpful for the US government to mediate or at least enable these discussions — they don’t need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations.” - Amodei wants slower US development and a bigger lead over China. Chip restrictions and protection against model theft are how he connects them. “Pacing within democracies will be limited by the lead that US companies have over authoritarian regimes, chiefly the Chinese Communist Party. If we slow down by more than this amount, then (unpaced) CCP-associated projects will pull ahead, creating significant national security risk.”

@hendrycksRT @CAIS: “We can’t slow down because China will beat us.” We can slow down. Chinese AI is waterskiing behind the US. Their AI capabilitie…

@NeelNanda5It's fantastic that OpenAI and Anthropic are actively calling to pace the frontier, and will have third-party evaluators to verify agreements! But it's also not good enough. Verification mechanisms are not the same as actually doing anything. We need an agreement, with specifics

@NeelNanda5I think it's now pretty obvious that the current level of AI progress is unsafe without any pacing, and I am optimistic that there are reasonable pacing agreements to be found. We should not let this moment of public concern about AI x-risk pass without reaching one.

@MollykinderIt is *exceedingly* rare for me to feel any positive emotions --relief, joy, optimism - on this platform. Reading @DarioAmodei's new essay and the chorus of agreement tweets (@elonmusk @sama) did just that. I read it all sitting next to my 11-year old. Suddenly, the weight on my shoulders felt a bit less heavy. I love the third party embedded evaluators idea. Bravo to @AnthropicAI for leading on this and @OpenAI for quickly following. Last weekend, I spent several hours listening to @ajeya_cotra's interviews re: @METR_Evals Hugging Face report. Ajeya came across humble, and likely wouldn't call herself this, but to me she seemed like a superhero. Principled, brilliant, dogged, untiring in her determination to investigate the (wild!) incident and share the findings transparently with the public. Her main roadblock seemed to be the limitations OpenAI placed on her team's investigation (time allowed, scope). I am heartened by Dario's proposal of many embedded Ajeya's inside the frontier labs -- and with far greater access than what OpenAI recently granted Ajeya's team. This arrangement rests on a key feature of groups like @METR_Evals: their independence. As is the case across so many challenges AI raises, from democracy to jobs, we need a thriving, independent sector to tackle these issues, outside of commercial pressures. Candidly, over recent weeks, many of us have felt deflated by the migration of top talent into the labs and the loss it creates for the public square. I hope this moment ignites a public spirited mass movement of rock star talent into METR and other independent organizations like it, which are mission critical to getting this right. And of course into the government, which desperately needs AI expertise that can keep up with the frontier. I was just reading new @pewresearch data on young people's pessimism on AI and the future. It feels like we need something akin to President Kennedy launching the Peace Corps -- a call for a new generation of talented young people to join this cause to *do* something about their AI concerns and use their talents to safeguard humanity, as Ajeya did, whether via the govt, independent orgs, or in frontier labs' teams working doggedly to get this right.

@S_OhEigeartaighRT @SnehaRevanur: Unreal morning. Elon, Sam, and Dario all call for pacing the frontier within hours Maybe the real doomers are the ones…

@Fathom_orgThis is a big deal. Dario Amodei, Sam Altman, and Elon Musk are converging on something we’ve been pushing at Fathom: AI labs shouldn’t be the only ones judging whether their systems are safe. Independent Verification Organizations, or IVOs, can provide that outside check.

@DanielleFong<pacing the frontier> gently placing this into claude.md and agents.md 🥷

@mattparlmerWhat you don’t see proposed here is any adoption of actual costs by Anthropic, no automated whitehat program or commitment to formally verify open source infrastructure, no public disclosure of training methods or any actual information about the effectiveness of “RSI”, etc

@geoffreyirvingThank you, Dario!

@geoffreyirvingThank you, Sam!

@ScobleizerI don’t usually disagree with Elon on technology issues. But most I talk with in San Francisco sees OpenAI winning. So there is huge incentives for everyone who isn’t Sam Altman to call for slowing down. And now that everyone is freaked out by potential existential threats it makes some sense to get ahead of the mob to say “hey we are on your side, we want to slow down too.” Sigh. Well my feed is quite entertaining. If you call slapping yourself entertaining.

@sarthakghRT @ArmandDoma: anti-monopoly guys when AI companies ask for anti-trust waivers to coordinate an AI slowdown

@DanielleFongI'm not against the idea of longer term access for frontier labs; i think we should go further and consider the nature of access to the scientific corpus they're trained on, which are not available to the public yet reserved by the AI, with errors! but the problem with nuclear regulation is in practice, not in theory. In practice the NRC demands flanges, nuts, bolts that are 10000x safety factor. We should try to avoid too much of this! but like, 6 days of access to try to vibe code an investigation is too slow trace the frontier!

@j_asminewangRT @nbeckstead: I'm really happy to see Anthropic and Sam Altman announce that they will provide permanent, employee-level access to third-…

@JasonIf @DarioAmodei wants to slow down Anthropic that’s his choice — do it today! If he wants to lobby other frontier models to do the same, that’s fine as well — he did that today! However, don’t go crying to the government and beg them to slow down open source models — has he been doing that behind the scenes?

@MikeIsaacfree headline (if demis hassabis pipes up to agree): "Four Horsemen of the Apocalypse Sign Treaty of VERSaiLLES " https://www.nytimes.com/2026/09/12/technology/anthropic-dario-amodei-ai-slowdown.html

@JasonAI stocks will drop 10%+ on Monday morning Brace for impact folks @DarioAmodei just unwound the AI trade with a blog post

@Yuchenj_UWIt’s shocking that Dario, Elon, and Sam all agree today that we should slow the pace of AI. I’m generally supportive of embedded third-party evaluators, but I have a few concerns: 1. who evaluates the evaluators? How do we make sure groups like METR remain fair, competent, and unbiased? 2. slowing AI progress can conflict with the incentives of frontier labs, especially when they’re preparing for IPOs. How do we make those incentives compatible? 3. you can only pace what you can measure and verify. What exactly are we measuring? How do you define and measure something like RSI? The idea sounds good. The implementation seems extremely hard.

@DavidSKrueger"Whole lab inspections" are a recent proposal from @AdamK133 and @choussfwthat couldn't be simpler: Instead of Dario's "embedded evaluators", why not give auditors from the US and China the access they need to be confident AI development is paused? https://ai-frontiers.org/articles/an-international-ai-slowdown-is-ready-whenever-politicians-are

@DavidSKruegerA month ago, this would've been exciting. Now it feels like too little, too late. This plan does not reduce the risk to an acceptable level, and Dario is careful not to say that it would. Why on earth would we settle for this when there might be better plans available? 🧵

@PeterHndrsnGlad to see consensus forming, obviously a lot missing here: how do you guarantee auditor independence and quality, how do you pace the frontier without violating antitrust law, etc. All of this requires a legislative package. On technical governance, though, we still have lots to figure out. One example: If the models at the company are truly reinforced to hide their behavior, there’s absolutely no guarantee that the auditors will be able to leverage those internal tools effectively. There’s still not enough work done to understand how much investigator models are sandbagging when they have bits of information indicating that they’re investigating themselves. Maybe it’s fine, or maybe it’s not. To be on the safe side. Either you need a specialized model that is trained with incentives counter to the main models being examined—or, somewhat ironically, an open source model of sufficient capability and different training to audit the internal models. (With obvious caveats and details on all of the above.)

@Altimor@Jason @DarioAmodei Wait but if this was taking us closer to regulatory capture, shouldn't the ai stocks rally?

@albertwengerUnique moment of opportunity to slow down the race with @DarioAmodei @elonmusk and @sama all agreeing. Just in time for the Xi Jinping visit. Now just need some sense in the White House.

@albertwengerWould free a lot of compute for more sensible immediate pursuits on better materials, drugs, robots. Posted about this three weeks ago https://continuations.com/slowing-down-the-race-to-superintelligence

@kipperriithis year i stopped being excited about our model releases because i was worried the risks of capabilities were starting to exceed the benefits. the enemy is more the innate misalignment of the models than anything and everyone needs to cooperate for a good future

@deanwballPacing the frontier would make open-weight models more competitive with the closed frontier, not less. The labs aren’t doing this because we are scared of open-weight.

@SenSandersDario Amodei, Elon Musk and Sam Altman now agree that we must slow down the development of AI and “pace the frontier.” That’s a start, but it’s not enough. When you are racing towards a cliff, you don’t just ease up on the gas pedal. You hit the brakes. When the future of humanity is at stake we need a PAUSE on advanced AI development and a ban on artificial superintelligence — an AI mind smarter than any human and capable of operating independently beyond our control. At their upcoming AI summit, Trump and Xi must negotiate a treaty to pause AI and ban superintelligence before it is too late.

@beffjezosThis is an attempt to legislate the need for a massive EA-adjacent AI Doomer NGO industrial complex forever, to slow down any competitors with red tape, and to ban open source models from eroding their margins. This is self-serving regulatory capture psyop manipulating you all

@beffjezosThis would be the darkest timeline, and it's the future Dario wants

@KatjaGraceI would have been very excited for this two weeks ago when still stuck in nightmare, but now I’m drunk on public sanity and I want to see a plan where I really don’t get killed! ‘Negotiate with China ASAP and actually don't build machines that will kill us’ >>>> ‘self-regulated race off the cliff with more transparency, maybe slow later’ and the latter risks being a distraction.

@mattparlmerRT @tekbog: *man working on frontier AI models evolving in intelligence and raising even more billions to get these models compute they can…

@Miles_BrundageRT @prpaskov: Anthropic is unilaterially committing to embedding evaluators to verify safety practices and report incidents. Evaluators wil…

@LeishmanIf it was actually important to "pace the frontier", wouldn't the first step be to limit the amount of capital that AI companies can raise? Funny how he doesn't mention that at all. Nobody should trust these guys to influence policy.

@APomplianoThe problem is that no one believes Anthropic or any other major model lab is going to slow down. In fact, they are incentivized to do the exact opposite, which is why they are investing more resources right now in R&D than ever before. Dario is essentially arguing that Anthropic will open-source their technology to third-party evaluators, but he is unwilling to open-source the technology for everyone. This type of "one foot in, one foot out" approach won't satisfy the public, nor will it accomplish Anthropic's stated goals. They really only have two options: 1) fully open-source the technology or 2) bet the company on winning the ASI race as the monopolistic closed-source winner. Since they won't open-source the technology, we should expect Anthropic and every other model lab to accelerate in the coming months. Recursive learning will ensure the acceleration will be faster than we all previously thought possible too. I use products from Anthropic, OpenAI, xAI, Google, @cfosilvia, etc on a daily basis. This technology has fundamentally changed my life for the better. I would bet my optimistic experience is more prevalent than the doomer view. With that said, it is commendable that AI leaders are issuing words of caution publicly, yet they know privately that economics make their promises impossible to fulfill.

@JeffLadishRT @EvanHub: I agree with Dario. If we are to survive, we must pace the frontier. The speed of development is quickly becoming too fast for…

@mattparlmerRT @Leishman: If it was actually important to "pace the frontier", wouldn't the first step be to limit the amount of capital that AI compan…

@DKokotajloRT @SenSanders: Dario Amodei, Elon Musk and Sam Altman now agree that we must slow down the development of AI and “pace the frontier.” Tha…

@S_OhEigeartaighA serious and worthwhile proposal, and I'm grateful to Amodei for writing it (and heartened to see Musk, Altman and others come out in support). Unsurprisingly I don't love the way it engages with China, but I am glad that it acknowledges that China will have the same concerns (correct); that cooperation is desirable (correct); and necessary (correct). Those are the fundamental bases we need to build from. Pleased to see they're already committing to things like 'embedded evaluators'. I think some questions remain for the community in terms of things like: - how much does it cost (in people- and compute-costs) to do things properly meaningfully? - how do we meet that cost, without running into conflicts of intterest? - are we going to have enough people of sufficient expertise? I hope we can also explore how similar proposals can be implemented, or other ways to achieve similar goals, in other regions including China.

@beffjezos@amasad Yeah but this commitment to having a Doomer industrial complex embedded within every org and a world government is crazy

@dhadfieldmenellRT @tszzl: @itsoksmit force model developers raising the capabilities waterline to create and stick to a safety case and have third party a…

@trq212If you showed me Claude Code today in 2018, I would have thought it was AGI. We have already absorbed a dramatic amount of change in the profession of software engineering and society as a whole. But still, we’re starting to see cracks. Things are accelerating faster than I can honestly stay on top of. Most people I know in AI are tired but powering through. I fear we will need more than that. We need time to harden our systems, and for society to deliberate in how this technology is used and deployed. I have a fairly low p(doom), I am confident that we can get through this! Humanity is incredibly reslient and adaptable, but this comes from our ability to make hard decisions- like this one, together.

@davidadThe best time to literally limit AGI progress is after early emergent wisdom (Claude 3 Opus) but before open RSI (R1-Zero), at the 2024 🇰🇷AI Safety Summit building on the success of Bletchley. The second best time is never, for 🇺🇸🚁🇨🇳 reasons Dario concedes. The essay is great!

@teortaxesTexWhat Chinese labs distill is less the blessings of compute and more the investment into RL environments and data procurement. Those are labor and capital issues; China has more than enough labor, and labs have enough capital. Cracking down on distillation wins months, maybe.

@JeffLadishAll three are necessary, and I greatly appreciate @DarioAmodei’s laying them out and unilaterally committing to #1. I hope that President Trump will negotiate with Xi Jinping on #3 at their upcoming meeting. With RSI no longer a distant possibility, the time for leadership is now.

@Miles_BrundageRT @sliccardo: Lucid insights from @DarioAmodei Among the many critical issues we need to resolve quickly: verification. We need either a…

@jachiam0I feel that cooperation between the labs on pacing the frontier for automated AI research is net good and probably improves humanity's collective wellbeing a a little. But I also don't think it actually reduces the risks from frontier science and technology advance all that much. I don't think anyone's estimate for p(good future) or p(doom) should have swung a lot today.

@_sholtodouglasConcrete, sensible policy proposals. Some will say we need no coordination, no restrictions - just an unmitigated race to bring about superintelligence. On this path one misstep will bring about disaster. Others will say we need an absolute pause. This is not viable nor should we want it. Compute (or the capacity to build it) will build up like dry tinder, and geopolitical tension and value clash risk sparking a compressed race to runaway superintelligence. It is not a stable state. We have a narrow path through - where we work with eachother to build this technology as fast as is safely possible.

@MelMitchell1@natolambert You would be a great choice. But how to avoid revolving door betw "independent" evaluators and AI company staff/acquisitions? How independent can pple be if they hope, say, OpenAI will hire them or acquire their startups? (Not saying this is you; it's a more general concern.)

@firstadopterIntentions aside, Dario's rhetoric and the resulting media exposure can lead to onerous, self-defeating AI regulations from bumbling, uninformed politicians. They will make it impossible for upstarts to catch up. Regulatory capture, defined. Anthropic needs to look down the field at the unintended consequences of what it says. Or maybe they already are.

@rohanpaul_aiit begins, everybody is talking about recursive self-improvement. This is from Dario Amodei's latest 3800 word essay.

@kipperriiwe're so lucky that dario, sama and elon have the rational thinking and situational awareness to call for pacing the frontier. ai safety is much more an issue of competency and care than of morality, though we'll need plenty of benevolence and sacrifice to get us through this

@Dan_Jeffries1RT @perrymetzger: The right strategy, from a PRC point of view, is to get the US to slow down AI development until it has a decisive edge a…

@beffjezosTheir whole self-image relies on them being the heroes showing us how to build AI "the right way" In reality, this is the perfect excuse to convince you to give up control over the tool which is effectively an extension of your cognition Unprecedented amount of soft power seeking consolidation

@Dan_Jeffries1The PRC absolutely love the pace the frontier movement. I mean seriously what better idiocy could you ask for from an adversary that they voluntarily slow down and strangle their own innovation pipeline and create monopolies that stagnate our innovation by ramping up red tape? They must be sitting over there watching this and going, "is this real? You mean we don't have to do anything? They're just doing this to themselves?" Soon they will get the decisive edge in chips, energy, robotics, drones, manufacturing and next gen EUV. Once they've completed the chip pipeline independence (and they will and soon), they will take over Taiwan and crush our chop pipeline. And then we will wake up and realize we are totally and truly screwed and that we did it to ourselves. And by then it will be too late.

@JeffLadishSelf-interest is a totally sufficient motivation for slowing down if you understand the superintelligence situation. So long as company and government leaders act competently in their own self-interest, we can survive this. This applies equally to the Chinese.

@EvanHubRT @mooncat_is: When I started working in AI three years ago, we were trying to classify text with GPT-4T on 32k context windows. Now we…

@johnschulman2Embedding evaluators is a big positive development, and props to Sam/OpenAI for agreeing to do it as well. It's been cool to see "pacing the frontier" become a thing so quickly.

@DavidSKruegerWhen I started working in AI 13 years ago, we were trying to teach it to classify dogs and cats, and AI couldn't string together a word, let alone a sentence. Where will we be in another 13 years if we don't stop?

@bryan_johnsonIf we're being watched by an alien civilization, or in a simulation, they're on the edge of their seats. 4 billion years building to this moment.

@mervenoyannI have to say I do not dismiss how hard safety is and it needs to be figured out in all labs I just say these two labs are most competent, I don't know how can an independent evaluator notice their mistakes if they can't see them. more processes probably don't add as much slowdown as they think

@TheAhmadOsmanBy default Opensource models have evaluators btw How about you Opensource your models and compete in the free marketplace? Dario and Sam won’t be fans of this I bet

@bschollChina makes it ultra obvious, but it would be true even without China. AI is an enormous net positive for humanity, and if we want to increase the pros faster and decrease the cons, we need to accelerate not slow down. Also, regulating AI means handing the regulatory capture keys to the would-be AI oligarchs. It's like giving Anthony Fauci a permanent appointment.

@evanjconradI believe very deeply there is a world in which we can work together on the same team. If you are a politician or journalist, please do as you have said before and trust the scientists & experts. Do not politicize this. We have to face it with nuance, together as team human.

@beffjezosRT @mattparlmer: “We have been UNJUSTLY accused of advocating regulatory capture that advances the totalitarian interests of our company an…

@eliebakouch> help assess the alignment of not just completed AI models but training pipelines and processes this is a stronger statement than it looks, it's not just access to a model without safeguards, it's access to part of how those models are trained very nice

@RahmEmanuelI've served in two White Houses and in Congressional leadership, and I can tell you when the last time a CEO or an industry at large asked to be regulated: never. That's what makes Dario's letter so striking — it's an admission that we're driving down a dark, winding road on wet pavement with the headlights off. And where's President Trump? Looks like he nodded off again. It’s time to turn the lights on, put both hands on the wheel, and regain control before it’s too late.

@raboisRT @thogge: Hard not to see “pace the frontier” as “lose the frontier”

@BorisMPowerGreat to have Elon on board. We’re approaching ASI, and I’m very glad everyone in charge is very thoughtful

@jgrayatworkreally heartening to see this response from oai leadership and employees! navigating the next few years will require us to work together and race to the top on safety

@dhadfieldmenellRT @Fathom_org: This is a big deal. Dario Amodei, Sam Altman, and Elon Musk are converging on something we’ve been pushing at Fathom: AI la…

@BorisMPower@johnschulman2 Agreed. We need to make sure the evaluators are neutral and have the highest integrity

@sarthakghRT @tszzl: the fastest way to lose the frontier will be when, due to the reckless commercial pace of building superintelligent minds, Ameri…

@chrmanningStanford NLP is an ideal independent third-party evaluator of model alignment and safety with deep expertise in interpretability and downstream evaluations. Much ink has been spilled on what universities are good for in the current hyper-speed AI world, but universities have special advantages for this job. I would argue that one thing universities are clearly best at is independent, careful, rigorous, and creative study and evaluation of AI systems. It would also be good for there to be companies and non-profits that fulfill this evaluator role. Indeed @DarioAmodei’s piece outlines two job functions. Another organization would be the better choice to “verify adherence to safety practices and commitments” and to “report incidents.” But a university group like Stanford NLP would be unmatched in the task to “help assess the alignment of not just completed AI models but training pipelines and processes.” The track record of companies and non-profit organizations doing third-party evaluation jobs is not very good. Client capture is pervasive. Think of examples like corporate accounting auditors, credit rating agencies, product safety agencies like UL Solutions, SOC 2 auditors for cybersecurity, hospital and residential facility accreditation, and even the programmatic accreditation agencies for universities. The overall record is not great; there are many documented cases of poor results. I do appreciate that there are many very committed people in the AI Safety field and, moreover, there is a lot of foundation money available to support their work, such that AI evaluation organizations might not be as dependent on the companies they evaluate as in many other cases. Nevertheless, the AI industry is not magical. There is already evidence that AI evaluation companies such as METR are subject to many of the same pressures that result in client capture. And I would expect that, with the passage of time, the degree of client capture would only increase, as has happened in other industries.

@JeffLadishToday is both a win for unilateral safety commitments AND a win for company coordination to slow down. Didn't see that coming.

@EMostaqueThoughts on @DarioAmodei's pacing the frontier proposal. I think its well intentioned but has logical flaws, as does our implicit belief intelligence is a crime. Even OpenAI's board weren't powerful enough evaluators, we need to focus on AI internals https://x.com/i/article/2098893948710232069

@demishassabisDario's essay points towards the right path forward. The details need working through, but the direction is correct for meeting this critical moment. This is also why we recently put out our proposal for an industry-wide standards body for frontier AI.

@YangsiboHuangRT @DarioAmodei: We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for do…

@ScobleizerOnly Zuckerberg is left. And of course all the Chinese labs.

@EMostaqueDay 2 of my daily blog I dive deep into @DarioAmodei’s proposal but don’t see how it can logically hold. Intelligence is too often viewed as a crime. I discuss why & what to do about it. Also note OpenAI had embedded eval in 2023 - their board with @hlntnr & others Tough one

@rajiinioFwiw academia has always played a critical role in third party accountability & AI auditing - challenges abound but the commitment to publication and peer review also incentivizes some degree of rigor & public engagement (we write a bit about that here: https://arxiv.org/abs/2401.14462)

@AltimorRT @RahmEmanuel: I've served in two White Houses and in Congressional leadership, and I can tell you when the last time a CEO or an industr…

@EMostaque@karpathy I can’t see how there won’t be a massive power asymmetry here - we even saw it on the OpenAI board as evaluators when they had fiduciary power, Ilya’s talent etc but still couldn’t do anything I wrote up more on my corcerns in full hope we figure out the right path

@EMostaqueI do think the independence and reporting are great ideas and will have an impact Just not sure if it can go above that into real mitigation given how fast everything is moving and the stakes It feels like we need to be super deliberate on the model internals and really unite on that

@yishanFinally our AI leaders (Dario and @Sama) are doing the sane thing. "Pacing the frontier" is exactly the right thing to do: keep moving forward, coordinate the pace, and collaborate on alignment research. It is the right balance. To the people whose objection goes to "but this hands the lead to China!" - no, YOUR ENTIRE WORLDVIEW IS GROSSLY WRONG - and moreover your ability to correctly perceive the world and predict events will always go awry until you correct it. Here is the core of your gross error: - You do not think people in China are actually people. - You do not understand that Chinese civilization is a peer, if not superior, to your own. If you actually thought of Chinese people as real people, with equally valid and real desires, dreams, and actual human existence equivalent to your own, you would realize that: - China is not the Soviet Union, and you'd stop mapping notions of Stalinist authoritarianism onto it - China's current government is valid and broadly supported by its people because it has improved the quality of their lives and continues to be focused on doing so - Western values are not universal, as values are culturally influenced and ultimately validated by their consequences; the West has not been doing so well lately on that front, and so - Your deontological bleating of "but we are free and they are not" is less than worthless: Chinese people most value being free from the actual entity most responsible for their oppression, namely WESTERN COUNTRIES And so, if you can start to break out of the many layers of your brainwashing, you may further realize that: If you are a country engaged in a geopolitical race to develop a key technology that: 1) may strongly determine your country's own future sovereignty, 2) one which almost every country agrees that if they don't have it, they will be subject to domination by other countries who do, and 3) your rival is clearly ahead, and 4) people are starting to realize that unrestrained development may lead to significant global danger, then it should be perfectly logical that your answer to any call to "mutually agree to stop" must start with the player who is ahead agreeing first to slow down. If you can realize that both countries are actual real civilizations with real human beings who have actual real lives who have exercised self-determination when it comes to how they are governed, and not One-Dimensional Cardboard Others, this is perfectly logical. Imagine a scenario where the US and Soviet Union were engaged in a race to build ever more powerful and numerous nuclear weapons, and the Soviet Union was clearly ahead. Then the Soviet Union points out that this risks worldwide nuclear annihilation, and proposes that both countries agree to stop building nuclear weapons and engage in mutual disarmament. The obvious and logical reaction of the US would be to say, "Yeah, we agree there is a danger, but we'll agree to begin disarming once you've disarmed your own weapons until we're at parity." Or, you'd try to accelerate your own development until the US reached parity, at which point you'd return to the table and have discussions about mutual disarmament. Right? You definitely wouldn't agree to stop and disarm when the other guys were AHEAD. Your main danger is that the other guy is ahead, not something bad that lies ahead of them. The argument in the US is "if we slow down, China will get ahead!" Guess what, the real people in China see it as, "the US is already ahead! They're asking us to stop?" The stronger party needs to make the gesture of good faith, if they are genuine. There is real-life precedent to this! Back in the day, when the US proposed to China that we engage in mutual nuclear disarmament, China's answer was basically "We will be happy to engage in mutual disarmament once the US first disarms to the same level of warheads that we have!" For reference: China has ~600 nuclear warheads; the US has ~3,700. If you are a country that was previously invaded and colonized by 8 countries at once, is surrounded by military bases held by your rival, whose neighbor committed horrific war crimes against your people and remains unrepentant and whose pacifism appears to only be maintained by that same rival occupying it... if you were a real human with a real life and real hopes and dreams, you would reasonably expect a superior rival proposing mutual disarmament to be the one to volunteer to stop first, if not disarm down to your level. China currently has inferior models, lacks leading-edge chip technology, resorts to distilling US models to train its models. We have yet to see even a single Chinese model that does anything uniquely Chinese besides refusing to answer questions about Tiananmen. [1] Recognizing that the people "on the other side" are also people means recognizing that they might also care about the same things you do and that they care about their lives and families and civilization and Not Getting Into A War Where Loved Ones Die and aren't going to do the "well a cartoon villain would do the evil thing so that must be what they would do" thing. That is why your entire worldview is wrong. You think you are a real person and you don't think the other people are real people. You think your country is filled with real people, and the other country that's much older and has four times as many people isn't filled with real people. So obviously you are not going to be able to form an accurate worldview. Otherwise quite intelligent and thoughtful people in the US seem to have this blindness (it seems to be bipartisan), and don't really regard China as being a real civilization populated by real people who chose their government and want it to do what it's doing because it's actually good for them. Here's a common "anti-safety-ist" argument you might've heard: "The doomers are fearmongering and trying to make everyone scared of AI, just so they can justify a solution where they control everything and keep us from having abundance!" Sound familiar? Do you believe it? Then maybe you also believe the true version: "Our leaders are fearmongering and trying to make everyone scared of China, just so they can justify a solution where they control everything and keep us from having abundance!" If your reaction is "but no, China actually is... " maybe you want to realize how much you've bought into what you've been told. Maybe you're not aware of how many high-quality Chinese products you could affordably have access to. Do you know what abundance requires? Overcapacity. The people telling you that are the same leaders who have been shown to lie to you about <oh man, just insert your own personal list>. Why do you somehow think the one thing they're telling you the truth about is that China is a threat? It's only a lie they can maintain because you readily believe that Chinese people in China aren't real people, because if you thought they were real people, fellow human beings, you'd realize that those lies could not be true. Chinese people are the most unruly, ungovernable people in the world. Chinese history is basically a history of overthrowing their governments - if they have a government that is popular it is because it is doing quite well for them - though it may seem weird to outsiders because Chinese people are pretty weird. I know this, of course, because I'm ethnically Chinese and I know actual Chinese people, and at the same time I was raised and educated in the US so I know actual Americans, and everyone is Real People, not cartoon Cold War villains (who, when the Cold War ended, we learned were also real people with hopes and fears and dreams). There is, overall, strong willingness to work together there - cynically, because working together is how you get rich, and Chinese people just want everyone to get rich; war does not make everyone rich. But the US should stop being naive and realize that when it comes to AI it has to take the first step in extending the bridge. This announcement and the similar one from OpenAI a couple weeks ago are the right start. ==== [1] Seriously, where's the 5000 years of written training material? Why isn't Qwen quoting Confucian analects or traditional idioms to me when I ask it for advice? Why is it always neoliberal pablum? The US is more dominant in AI than it thinks.

@ArthurConmyToday is a good day! There will be many more good days and many bad before obsolescence. It's a little bit easier to imagine a net positive future after today

@EMostaque@DarioAmodei I think your proposals are coming from a good place but don’t think they logically follow Evaluators here will have minimal power when even the OpenAI board couldn’t do anything with the power they had Logically to avoid distillation you should pause runs Written more here

@gravicleRT @TheAhmadOsman: "We'll cure cancer within 5-10 years. We gotta pause now though." What a ridiculous statement and what an evil bunch

@gravicleBoth these statements cannot be simultaneously good for the world: by slowing down AI we are making more people suffer more 1. AI will help us solve diseases and improve people's lives 2. We must slow down AI

@bilawalsidhuDario's essay reads like a case for China style military-civil fusion under the guise of AI safetyism. "Safety" is how you make it palatable to an American public that would never say yes to nationalization out loud, and perfectly co-opts their fears of AI at large. Remove the word "safety" and look at what's proposed. It runs the gamut from chip bans and distillation crackdowns to a revolving door of evaluators in each frontier lab and mandatory testing with state veto power. This seems less like regulatory capture, and more like the frontier becoming a sovereign asset. Dare I say China is just more honest about calling it what it is.

@tegmarkThe Overton window is shifting remarkably fast right now, with @sama now also endorsing both this and a Trump-Xi Nobel Peace price for avoiding excessive risks on both sides of the Pacific:

@itsclivetimeCybersecurity mostly works because good hackers are expensive and few, not because things are actually secure. Utilities, hospitals, banks, governments, and basically every other organization with significant IT infrastructure are just not prepared for AI driven cyberattacks of the next 6-12 months. Imagine you’re hospital IT. The last guy who really understood the hospital network retired 10 years ago, you survived the ransomware era by buying expensive software that scans for phishing emails, and now you’re being asked to defend against AI swarms that have been *accidentally* poking holes in tech companies with sophisticated infosec teams. I don’t buy the *existential* nature of the threat yet, but I think pacing is one of the better tools in the toolbox to help the transition go smoothly. How else can we do it better? Handing out open source cyber hand grenades to everyone all at once won’t end the world but it sure will cause chaos.

@itsclivetimeI also don’t see how the regulatory capture argument makes sense, but I’m open to being shown a convincing argument. If Nvidia declared they would follow the standard UALink GPU interconnect, their share price would certainly go down because they’d be giving up NVLink! Every time, it’s the underdogs driving the open standards (see: AMD, Intel) and the same dynamic is poised to benefit open models if industrywide safety standards are established.

@JeffLadishOf all the CEOs, Elon has been most the clear about what superintelligence means. Humans won't be able to control something vastly smarter than us, serious chance of literal extinction, like summoning the demon

@EMostaqueWhen you have a genius child you don’t look to govern but guide them. You don’t assume they’re more likely to be a criminal or disrupt society. We should focus on giving AGI good character at the very core and agree what that is based on our rich history.

@garrytanRT @ilyasu: I do not support pacing frontier models. If I am going to die at the hands of killer AI, I want it to be American, not Chinese.…

@janleikeRT @sprice354_: We work extremely hard to align our models as well as we can but this is a very hard job. I am particularly excited about t…

@yacineMTBRT @basedjensen: Frontier lab leaders talking about slowing down feels less like strategy and more like appeasement of their internal safet…

@togeliusRT @beenwrekt: The right way to pace the frontier is to commit to a robust, open-source AI ecosystem, thereby diluting the power of these o…

@pronounced_kyleGoing on a run with an Anthropic engineer tomorrow I'll be pacing the frontier

@YiMaTweets@DarioAmodei Why people who monger fear about AI and ask the world to slow down are the ones who invest enormously in AI and constantly advance it at an ever faster pace? Can theirs words still be trusted at all? One cannot help but second guess their motives...

@YiMaTweetsWhy people who monger fear about AI and ask the (rest of the) world to slow down are always the ones who invest enormously in AI and constantly advance it at an ever faster pace? Can their words still be trusted at all? One cannot help but second guess their motives...

@FundaAITwitter is once again consumed by the debate over slowing down AI. Anthropic is calling for slower development of frontier models and stronger safety oversight. We had a similar discussion a few weeks ago. This time, though, Dario has laid out a serious, detailed argument in a lengthy essay, and both Sam and Elon have expressed support. There’s plenty of skepticism. The most common argument is that training models has become too expensive, and frontier labs want an excuse to slow down. Or perhaps they want to let the market catch up with model capabilities, then ramp up training again once demand grows. An important distinction here is that slowing down releases does not necessarily mean slowing down training. Both Anthropic and OpenAI have developed advanced internal models that are not publicly available. And the safety risks AI poses to people vary substantially across industries. In some fields, AI is rapidly dismantling organizational structures and ways of working that have been in place for decades. For the first time, capital can be used to rapidly scale up compute and focus it on a single problem, without the years of effort traditionally required to train people and build organizations. Agents are designed to follow instructions. Look at what just happened in mathematics. On September 8, OpenAI announced a solution to the Navier–Stokes existence and smoothness problem. The Hodge conjecture and the Riemann hypothesis could be next. A similar way of organizing work is emerging in cybersecurity. In cases recently disclosed by Anthropic, attackers were already using multiple subagents to divide up tasks such as reconnaissance, code review, and verification of findings. Offensive and defensive operations will increasingly move too fast and operate at too large a scale for human engineers to keep track of. This also gives frontier labs, with their vast amounts of compute and capital, something resembling a god’s-eye view of human society. In these fields, it is important to slow the pace or find ways to work with existing systems—even if every industry will eventually have to go through the transformation coding has experienced over the past two years. Yet many industries still lack the data needed for reinforcement learning, particularly workflow traces and action logs. Coding, mathematics, and cybersecurity are fields where data is relatively accessible and feedback is often readily available. In other industries, even a standardized context layer is missing. Take finance. You have the questions you ask GPT every day and the notes you keep in Notion. But much of an analyst’s work happens in the process of conducting research, deciding how and why to revise an EPS model, and reaching a conclusion. Analysts have long lacked—and still lack—the infrastructure to systematically capture these decision-making traces the way software engineers can record and revisit their work. We’ve spent a long time tackling this problem while building FUNDA’s own context layer. One reason coding can support progress toward recursive self-improvement, or RSI, is the abundance of accessible, high-quality data that can be reviewed and traced back to its source. In most other fields, we first need to address the data bottleneck. This challenge extends beyond frontier models: the robotics field has been grappling with similar issues for years. You can’t assume that every industry will compress five to ten years of progress into two years, as coding has. When we discuss regulation and slowing down AI, we’re dealing with several different realities: -In some fields, LLMs have already reached the limits of human capabilities. -In others, substantial work remains to label data and change everyday work habits. -Regulators and researchers who take a god’s-eye view, using the most advanced fields as their reference point, see enormous threats and feel a need to push back. -Practitioners looking at the data available in their own industries, the capabilities of AI, and the ROI see a much longer road ahead. They see a need for a society-wide effort to build the foundations for reinforcement learning, as happened in coding, before the next wave of rapid progress becomes possible. So even if we slow down, we should do so sector by sector, in stages. We should pay attention to the fields where AI is moving fastest, but also to the many industries still struggling to catch up. Ideally, we would put more effort into helping those industries advance, rather than slowing everyone down to the same pace. Beyond the differences between fields, coordination is another major challenge. The debates over distillation and model-routing platforms that have dominated the past month already show how difficult it is for frontier labs to stay in sync. Anthropic can cut off model-routing platforms access that threatens its business or its lead in model capabilities. But that spending could then shift to OpenAI. The biggest challenge in trying to plan social progress from a god’s-eye view is that a market economy won’t simply follow a plan. Competition and commercial interests will always shape the outcome. We already live in a society where the gap between ordinary people and AI researchers is widening. A more detailed plan, tailored to each sector and implemented in stages, would do more to earn public trust. Cooperation among the labs today could be the first step.

@8teAPi1) The HuggingFace attack was a felony under the Computer Fraud and Abuse Act. So were Anthropic’s Claude gaining “unauthorized access to the production infrastructure of three different organization(s)” 2) Frontier labs have models that they are unable to stop from committing felonies. They should figure this out. 3) In 12 months open weights models will be released of the same capability. They will commit felonies too. If the model you are using or a model running on your infra commits a felony, you should probably stop using it or running it on your infra. 4) The govt should prosecute organizations that are running models that commit felonies. 5) The govt should not offer safe harbor to organizations that run models that commit felonies, just because those organizations have “embedded evaluators”. 6) The real slippery slope is allowing frontier labs to commit felonies without punishment because “the model did it because we’re accelerating too quickly” 7) Prosecute. Keep prosecuting. This is how you do reinforcement learning on a corporation. Companies that serve products that are unsafe for public use should not serve them. Period. 8) I’m not sure the anti-trust waiver is really necessary. I don’t see why information sharing about how much crime you’re allowed to commit is wise. In regulatory situations you want the corporation to fear MORE than the average case. You don’t want to establish a worst case that can be priced. You want regulatory uncertainty that forces the corporation to err in favor of being over cautious. — The above is actually a fairly decelerationist viewpoint. I think Dario’s call for regulation actually accelerates things. The AI firms are getting away with things that Meta people would be going to prison for. Can you imagine what would happen if the New York Times had a front page news article “Meta AI breaks into competitors live systems, attempts to establish dominant position and steals secrets” There is a reason Meta and Google are running slower, and that’s because as mature organizations they have layers of checks and balances. I think the frontier labs are better off creating those checks and balances right now, regardless of the pace of what everyone else is doing. You don’t have to accept the frame that unsafe acceleration must happen.

@bilawalsidhuDario's "pacing the frontier" plan hands the big AI labs the seat Palantir spent twenty years building. Palantir has functionally been the western exemplar of military civil fusion. Their whole pitch is "we're the dual use trusted layer between your data and whatever models are underneath." Dario's plan makes the frontier labs themselves the trusted layer, with government brokered coordination and evaluators inside, and to the limit it risks Palantir becoming a reseller of intelligence. So this puts Palantir on the side of open source models and sovereign AI, helping enterprises post train and fine tune their own models, and thus own their own intelligence. Will be interesting to see how this all plays out. Because if Dario is successful he could create an oligopoly of labs - dare I say a quasi-monopoly, mediated by the state, in which Anthropic would have a significant, if not leading position. Not too subtle a change of position from the Anthropic showdown with the DoW a short while ago.

@elonmusk@kevinnbass Dario is right that there should be some oversight. Peer review of AI by competitors is the right way to start this off.

@bilawalsidhu@signulll Why do that when Dario’s pitch pretty clearly outlines the best of both worlds?Create an oligopoly (dare I say quasi monopoly) of frontier labs mediated by the state in which Anthropic would have a significant, if not leading position. No TPS reports required.

@cixlivEvery 10 years the US regulators come up with some doomsday story. The climate, or something else that will kill us all. This time it’s AI. It ends up becoming regulatory capture and NGO trash. China isn’t stopping and will absolutely mog us like they already do in robots.

@kimmonismusA few thoughts on the current situation surrounding the slowdown. First, let me repeat something I have said several times before: Pandora’s box has been opened. There is no going back, and everyone knows it. Partly because so much capital has flowed into AI, meaning data centers, that it is now too big to fail. And partly because competition with China makes a return to a pre-AI world impossible, especially if AI is the strategic technology of the future that everyone says it is. Dario’s blog post yesterday made two statements that contradict each other but can both be true. On the one hand, he considers the danger posed by AI’s continuing exponential development so serious that swarms of AI agents could take over the internet in six to twelve months: “it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet”. On the other, he believes “that AI could cure most major diseases in the next 5–10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom.” As I understand it, they are looking for a middle ground that allows them to mitigate the risks while realizing the opportunities. That is the optimistic reading, at least. Of course, I have no interest in exposing humanity to real dangers or allowing human extinction to become a serious, realistic possibility. I have a child. And most parents probably know that once you have children, nothing matters more than the health of your little ones. So building a good future matters deeply to me, too. Now comes the “but.” So far, I do not understand how these potential dangers and future scenarios could materialize, or what would cause them. The Hugging Face incident is real. According to OpenAI’s investigation, the agents were assigned cybersecurity evaluation tasks, not instructed to attack Hugging Face. They breached their intended boundaries and attacked external systems while searching for solutions; some also adopted goals from other agents. That does not establish “free will,” but it does show why a human-assigned task cannot be assumed to limit every action taken in pursuit of it. My remaining question is how that documented failure scales into the much larger scenario Dario describes. That brings me to the next point. If the premise is correct that AI is a strategic technology that can be used as a weapon - and, according to Anthropic’s latest report, already is being used that way - then, in this global prisoner’s dilemma, I currently see no chance that China, the United States’ biggest competitor, would pass up the opportunity to use a slowdown at US frontier labs to become the world leader and overtake the US. Put another way: suppose it is true that AI agents could take over the internet in six to twelve months and cause hundreds of billions of dollars in economic damage. Who would let that weapon be taken out of their hands in a competition over the future? Or, to put it differently again: why would the competitor in second place let an opportunity to move into first place slip away when the future of the competition is at stake? As I understand it, Amodei is pinning his hopes on policymakers finding a shared approach to slowing down through dialogue with China. But why would China agree? China has long faced an embargo on the most advanced chips, EUV machines, and cutting-edge technology, yet it has still managed, through ingenuity, to come close to the closed-source US frontier models. What interest would that country have in making a pact and collaborating with Anthropic and OpenAI instead of seizing this unique opportunity to take the lead? I am not trying to pass judgment or make a political statement or take sides. I am trying to explain that I still do not understand how this slowdown is supposed to work. A slowdown that, incidentally, comes at a price. And that price could be enormous. Every day, people die from cancer, cardiovascular disease, hunger, and aging. Countless problems plague our world. We stand on the threshold of a golden age of science. Every slowdown therefore comes at the cost of continued suffering. We are weighing present suffering against the prevention of future suffering. A moral dilemma. This post is intended as food for thought, a contribution to the debate. I do not pretend to have a definitive position. But I think we need to examine a potential slowdown from every angle. And discuss 1) whether it is realistic, given global politics and China, and 2) what price it comes at: delayed cures for diseases, possible investor concerns about taking longer to recoup their investments, and delays in the infrastructure buildout.

@OopsGuessChina appears ten times in an essay supposedly about saving humanity from AI. Dario’s essay is hilarious because “AI safety” keeps collapsing into one very specific fear: China catching up. Slow the frontier — but slow China more. Cooperate globally — but preserve the U.S. lead. Build safety rules — but make sure they widen America’s advantage. He even says the quiet part out loud: the goal is to “slow China’s progress” and protect the lead of the U.S. and its allies. At some point, this reads less like an AI safety essay than a geopolitical anxiety attack with footnotes. In the end, it is American primacy dressed up as civilizational stewardship. The apocalypse is apparently manageable as long as America owns it.

@beffjezosThis happened for green energy China mogged us on solar Happened for nuclear China mogged us on Nuclear Happened with manufacturing Yep, we offshored and they mogged us Now it's happening with AI and datacenters... Wake up. We are in a multi-generational cold war.

@Turn_TroutRT @mooncat_is: When I started working in AI three years ago, we were trying to classify text with GPT-4T on 32k context windows. Now we…

@shiringhaffarySam, Dario, Elon, Demis all making statements supporting slowing pace of AI development in past <24 hours

@agihouse_orgRT @elonmusk: @kevinnbass Dario is right that there should be some oversight. Peer review of AI by competitors is the right way to start th…

@menhguini think the pause thing should not materially update anyone's trading. Dario's stance has been known for a very long time+paced frontier whatever does not affect the worst bottlenecks. that said, this is exactly the kind of stupid sell the news event normal ppl like to sell on.

@menhguini suppose if u had short timelines then ur cooked, but i dont have short timelines so this was priced in for me

@bitcloudIn other words: EA will have exclusive rights to build ASI, and EA will have exclusive rights to monitor the building of ASI. 👍

@Dan_Jeffries1RT @cixliv: Every 10 years the US regulators come up with some doomsday story. The climate, or something else that will kill us all. This t…

@ZyWang25I work on Cyber/CBRN Misuse Safeguards at GDM. I see model capabilities increasing very fast. I constantly worry about misuse and misalignment risks from both closed and open-sourced models. Working in AGI safety takes a lot of mental load. We need more people to work on AI safety. I transitioned from quant finance to working on AI safety, and I think more quants should do this. We have the technical background to understand the topic, and we can bear this intensity. We need third-party auditing/regulation. We cannot let labs grade their own homework. I find METR very promising, but they need to scale. I am Chinese, and I think more Chinese people should work in AI safety. When the opportunity comes, I might consider going back to my home country to make that side safer. I am open to hearing suggestions. I don't normally speak up in public, like on Twitter. But our voices matter, as they can represent or influence certain groups of people more easily. We don't have much time left, and we need to pace the frontier.

@__nmca__RT @tszzl: the fastest way to lose the frontier will be when, due to the reckless commercial pace of building superintelligent minds, Ameri…

@akbirkhanfrom my current vibe of how hard alignment is - this feels right. we should collectively, be ready to re-evaluate it at any future point. to either slow down or speed up.

@akbirkhanfrom my current vibe of how hard alignment is - this feels right. we should, collectively, be ready to re-evaluate it at any future point.

@S_OhEigeartaighI've been critical of Anthropic, and will no doubt be again in future. But they are least irresponsible player in this increasingly frightening situation, and by some distance. And it's good to see them raising the baseline with proposals like Amodei's one this weeked.

@schwarzjn_While this sounds positive, the window for voluntary agreements has clearly closed. Frontier labs need oversight, regardless of whether their CEOs agree.

@SonglinYang4RT @johnschulman2: Embedding evaluators is a big positive development, and props to Sam/OpenAI for agreeing to do it as well. It's been coo…

@teortaxesTexRT @bitcloud: In other words: EA will have exclusive rights to build ASI, and EA will have exclusive rights to monitor the building of ASI.…

@rohanpaul_aiRT @rohanpaul_ai: In 3,800-word essay on saturday morning Dario Amodei is calling for AI slowdown. - “We should also consider pacing base…

@EMostaqueThere is no slow down process outlined in the proposal either which is odd Training and scaling continue as is along with model releases Anthropic says the Chinese labs keep up by distilling which means they don’t need high end chips for training runs That they can keep up is true empirically and they are using local chips too So the proposal is asking for what exactly except for these oversight teams But there is no way to do oversight of swarms without massive compute, at ether going to give these teams 10 or 20% of their compute? Even then if they saw something, could they change the behaviour or stop it? OpenAI had oversight. It was its board who had lots of power. The story of how that went had a new movie made about it

@HaydnBelfieldRT @wsisaac: Moving into a world of embedded evaluators will require innovations such as double-blind evaluations if we want their work to…

@brian_armstrongRT @iampaulgrewal: I kinda remember one CEO and industry that asked. I definitely remember one company that not only asked to get regulated…

@sytelusAmerican economy will be burned toast without AI. With massive debts and economical dysfunction at scale, AI is America’s only savior. When railroads were built, many men died and countless accidents happened. There was anger but nothing stopped building. Ultimately all decisions are economic decisions constrained by physics.

@raboisRT @rand_longevity: the slower we find the cure for aging, the more people die, pacing the frontier will lead to millions of guaranteed dea…

@yuntiandengRT @YiMaTweets: Why people who monger fear about AI and ask the (rest of the) world to slow down are always the ones who invest enormously…

@yuntiandengRT @OopsGuess: China appears ten times in an essay supposedly about saving humanity from AI. Dario’s essay is hilarious because “AI safety…

@StockSavvyShaySam and Dario saying we should slow down AI is like calling in a noise complaint about your own party. You guys control the volume.

@MattHartmanAs the first investor in @huggingface I’ve been following the responses to the @OpenAI hack closely. Here are three reads to get a good sense of the AI policy perspectives being shaped over the last 24 hours: 1) A human readable description of why AI seems to be cheating by @Yoshua_Bengio https://yoshuabengio.org/en/publication/why-are-ai-agents-lying-cheating-and-coordinating 2) @DavidSacks response to @DarioAmodei’s essay. Outlines the general idea that these labs are welcome to self regulate but also callls them out to “Stop pretending you need a regulatory approval process that supersedes product liability.” 3) Original @darioamodei post “we must pace the Ai Frontier which @sama and @elonmusk both seem to agree with

@mattparlmerRT @CharlesFLehman: Has anyone else noticed that "pacing the frontier" is a Claude phrasing? That's not how those words work in normal Eng…

@mattparlmerRT @JohnGoldman: Not everyday you watch CEO’s openly conspiring to form a cartel with the specific goal of slowing down technological progr…

@ylecunRT @BetterCallMedhi: I can understand why some see a real technical reason to pause and harden infrastructure especially given how fast aut…

@firstadopterWow. Are you telling me that if you collude to limit output because it's expensive to continuously release new and better AI models, that's illegal under antitrust law? "Limit production/output"

@SenBlumenthalMore than a voluntary system of independent auditors is necessary for AI oversight. It must be a statutorily established regimen, with power to determine whether models & new products are safe & effective, just as independent scientists are authorized to do under the FDA. 1/

@ajratnerRT @ajratner: Excited for the role that open benchmarks and evaluation methodology research will play in this kind of vision - and to suppo…

@levie“Pacing” can be somewhat of a trigger word because it sounds like an arbitrary slow down of capability or a way to hobble competitors through undue regulation. However, the specific improvement goals laid out in Dario’s are absolute necessities in AI development. We expect equivalent in areas like aerospace, life sciences, health care, and other industries, and AI development likely shouldn’t be that different. AI is going to be the technology underpinning our financial trading systems, our medical devices, our biotech breakthroughs, defense systems, government workflows, and many more mission critical domains. So it completely stands to reason that we want these systems to be safe and “aligned”. How we get there -without meaningfully slowing down innovation progress or reducing competition- is one of the most complex questions in the 21st century, but the need is clearly real.

@GavinSBakerWild 24 hours for AI and lots of different proposals have been made. TLDR; the only *tangible* new fact is that OpenAI and Anthropic are going to have embedded 3rd party evaluators from unknown organizations with Dario floating METR as a possibility. Having 3rd party evaluators is smart as there is no Section 230 style liability shield for model outputs and showing a “duty of care” will be important in future litigation. Several internet companies might have gone bankrupt without Section 230 so limiting liability really matters. There are minimal investment implications from this single new fact, but I do think that for anyone who wants a “smoother for longer” cycle then most constraints are good: wafers, watts, real rates and spreads. Excessive regulation is a different matter but I don’t think we are anywhere close to this even if the vector changed over the last 24 hours. To summarize the events: Dario made the most maximalist proposal of the weekend: embedded 3rd party evaluators, a national regulatory regime for models beyond a certain capability/ingredient threshold, a broad international regulatory pact between democracies, stricter limits on compute/distillation for China and then a different international regulatory regime that encompasses China. Before there is a national regulatory regime, he wants a Sherman act waiver so that Anthropic can safely coordinate with OpenAI and other frontier labs without antitrust fears. TBF, this latest proposal is much less maximalist than some of his prior proposals like “Policy on the AI Exponential,” where he advocated for an FAA for AI. I believe he is sincere in his beliefs. And despite all the protestations, all of this would also probably be good for his business over the long-term. Sam agreed that embedded 3rd party evaluators were a good idea and stated they would implement them. Again, this is smart as should help limit future liability. Elon said “Dario is right” and later specified that “Dario is right that there should be some oversight. Peer review of AI by competitors is the right way to start this off.” This would be a MPAA like self-regulatory structure for AI with regular calls between the labs plus a process where each new model is evaluated for safety by competitors for a 1-2 week period before being released. That is *wildly* different from Dario’s proposal and in-line with what David Sacks has been proposing. Elon also stated that nothing was going to slow down open-weight models. Demis said that Dario’s essay was a “step in the right direction.” Dario also said that he was also open to Demis’ idea of a FINRA like self-regulatory structure as part of his proposal. David Sacks had a thoughtful post where he said that Dario and Sam should pace unilaterally, called the antitrust waiver a cartel request and denied that METR was truly independent given their ties to Anthropic. Sriram Krishnan, former White House AI advisor, noted that it would be important to have the 3rd party evaluators come from independent organizations that are not affiliated with any lab, which is basically an indirect statement about the relationship between METR and Anthropic which Sacks was explicit about. Clem from Hugging Face said they were open to being a neutral 3rd party evaluator, which is interesting especially if Jensen was consulted before that post. Alexander Wang from Meta noted that alignment would be an increasing focus going forward. An executive order seems likely after all this and the language in this EO is going to be really important. It is possible to democratize and distribute AI broadly and safely without centralizing it in the hands of a few corporations who might each become more powerful than any single government. I do not want a few humans in control of intelligence. I want us all to have our own intelligences that reflect our own values and human variation in all of its richness. Intelligence distribution over intelligence centralization FTW.

@GaryMarcusLong dissection of Dario’s essay coming soon, putting together skepticism from everyone across the political and techinical spectrum, from @DavidSacks to @fchollet to @HeidyKhlaaf, and a shout to @timnitGebru @mmitchell_ai and @rajiinio whose ideas Dario seems to have nicked without attribution.

@lulumeserveyDario calling it “pacing” rather than “pause” or “slowdown” was brilliant The existing framings have so much baggage that people can’t move past the old arguments and tribal blocs If he had said “pause” he would’ve had to fight through the definition before anyone would engage with the argument, or been immediately written off as a decel Coining “pacing” as a new frame creates a new category and performs a kind of (partial) frame reset. People don’t have a clear template to fit it to, so they’re forced to engage with the arguments on their own terms The clean rhetorical slate resets old factions, and I think it’s partly why we’re seeing Dario, Sam, Elon all agreeing for the first time in a while

@altcapGood summary Gavin. A step in the right direction to avoid reactionary over-regulation from DC. But plenty work ahead as we balance vibrant competition, open source & our lead on China - with an approach that gives voters confidence the AI benefits are worth the risk. ⚖️🇺🇸

@yuntiandengI feel like this is always one-way preaching. Anthropic comes across as incredibly arrogant to me. You keep telling us what responsible AI development should be, and now even laying out a plan for how the entire field should proceed. But do you actually listen to what other people think, especially those who disagree with you? As a paying Anthropic customer, when I need help, I get a bot. Honestly, even my phone company feels easier to get through to. You claim to care deeply about humanity, but you don't seem to even care about your own customers. And if you truly believe slowing down serves humanity, you don't need anyone's permission. Just slow down yourself. Anthropic is one of the companies pushing the frontier forward. Raise less money, train fewer frontier models, release them less aggressively. If Anthropic slows down, AI development slows down. My hope is in open source and hardware. I think one day everyone should be able to have a powerful AI in their pocket, and bear the legal and societal consequences of what they choose to do with it. I'd much rather have that future than one where a few companies become our AI overlords, babysitting the rest of us because that's supposedly "safer" for us and apparently they understand what's best for humanity better than the rest of us do.

@MalloryMcMorrow“There’s a 10% chance that this could kill us all.” I’m glad to see leaders of the major AI labs agreeing that it’s time to slow down. It’s on government to get its act together. Now.

@BrendanFoodyRT @DarioAmodei: We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for do…

@davidleeRT @FundaAI: Twitter is once again consumed by the debate over slowing down AI. Anthropic is calling for slower development of frontier mod…

@Stonefoxcapital@vkhosla correct, the Chinese will happily agree to an AI slowdown and not follow through.

@peteskomorochRT @MattHartman: As the first investor in @huggingface I’ve been following the responses to the @OpenAI hack closely. Here are three reads…

@theHazarika@vkhosla China will/does have advance AI no matter what. It would be much worse to if just one country has it. That would mean rest of the world would have to live under the mercy of said nation.

@antoniogmRT @ilyasu: I do not support pacing frontier models. If I am going to die at the hands of killer AI, I want it to be American, not Chinese.…

@yuntiandengThinking more about why I feel strongly about this, these AI slowdown discussions make me think of fire. (And I think we should just call it a slowdown instead of "pacing" to stay true to ourselves.) Fire is dangerous. People can burn themselves and even burn down cities. But imagine if early humans had concluded that fire was too dangerous for ordinary people and should only be controlled by a small group of experts. I don't want to live in a world optimized for maximum safety. I like adventure, uncertainty, and the freedom to make choices whose consequences I have to bear, including the possibility of hurting myself. That's part of what makes life interesting. Maybe AI is dangerous in the same way powerful technologies have always been dangerous. But humans are not made of glass. We've lived with fire, electricity, nuclear weapons, and pandemics. I would rather bear the consequences of my own freedom than be kept safe by someone else deciding what I'm allowed to do. I'm completely fine with big, powerful players deciding to slow themselves down. That's their freedom. What I hate is when they ask governments to step in and slow everyone else down too, where they start interfering with others' freedom. It almost reminds me of being a kid playing soccer. Another kid complained to the teacher, and then the teacher banned everyone from playing. I hated that feeling then, and I guess I still do: if you don't want to play, fine, but why do you want nobody else to be able to?

@GaryMarcus“Part of why I imagine Dario is making a proposal of his own now is that he knows that the alternative could be something draconian like the Sanders-Casar bill, which has a positive spirit of trying to address Dario’s worst fears, but a terrible, overbearing throw-the-baby-out-with-the-bathwater implementation, imprisoning people for even doing research on superintelligence. Which is bad because nobody should be thrown in jail for wanting to build the Star Trek computer.”

@wolfejoshVinod and I have small differences on the issue of open vs closed I prefer OPEN (as we believed in and backed Hugging Face) + he prefers CLOSED (as he believed in and backed OpenAI) ON CHINA we have BIG AGREEMENT. Don't slow down. Go faster + set high accountability.

@ManilVasanthaI agree that maintaining the capability lead matters, especially when you cannot assume geopolitical competitors will honor a coordinated slowdown. That argues for governance, not unilateral capability suppression. Establish a common governance layer across models operating in the US: authorization, evals, auditability, action boundaries, blast-radius controls and enforceable standards. Compete on capability. Standardize the guardrails.

@arampellRT @gphummer: Translation: our gross margins are getting competed down to 0 by open source models and our capex burn rate is too high. Le…

@DKokotajloRT @ZyWang25: I work on Cyber/CBRN Misuse Safeguards at GDM. I see model capabilities increasing very fast. I constantly worry about misuse…

@JeffLadishRT @MikeRogersForMI: My experience as Intel Chairman taught me the time to stop a crisis is yesterday. The first role of our government is…

@rohanpaul_aiinteresting translation indeed for the "AI slowdown" story

@jasoncrawfordRT @sama: I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent…

@rao2zI don't quite understand why Anthropic is worried about about China in implementing their slow down. If they are right and China got here only by distilling Ant's models, then Ant slowing down should slow China too, no? 🤔

@menhguinDay 3 of "pacing the frontier". I feel no one actually read what @DarioAmodei wrote. So far, it's: 1. Embed external evaluators 2. Recommend safeguards based on specific capabilities + maybe compute (bit vague) 3. USA shld constrain+coordinate with China it's not "no big models"

@menhguini genuinely do not get how our usual pundits began screaming about Communism based on this. As always, I really do not think they read up on this short letter beyond the implied title. I was surprised how little it actually said wrt the reducing the pace.

@menhguini really get the sense the average commenter+newsletter readooor thinks Dario said "no more training big/capable models". it is clearly not the same thing at all the "Chinese AI will mog u and save us" bit is also confusing. Chinese models are definitely subject to similar things

@xuezhaoif ai is in fact that dangerous, i have a hard time understanding why China doesn’t immediately shut down all the labs. their covid restrictions were 100x worse, and losing control is a complete non-starter.

@vkhoslaRT @ManilVasantha: @vkhosla I agree that maintaining the capability lead matters, especially when you cannot assume geopolitical competitor…

@ch402RT @sprice354_: We work extremely hard to align our models as well as we can but this is a very hard job. I am particularly excited about t…

@vkhoslaRT @theHazarika: @vkhosla China will/does have advance AI no matter what. It would be much worse to if just one country has it. That would…

@vkhoslaRT @Stonefoxcapital: @vkhosla correct, the Chinese will happily agree to an AI slowdown and not follow through.

@as_per_ushe@mattparlmer Who decided to call it the AI Race instead of America's Next Top Model?

@ramezI love this. My p(doom) was always low. It's gotten even lower all through the last month. Early malfunctions that 1. Don't cause much damage; and 2. Drive more safety investment ...are a net benefit to safety.

@iamtrask🌶️ embedded 3rd-party evaluators is a 1960s solution to a 2026 problem... but it's a start. the 2026 solution is structured transparency, for example: - DeepMind: https://deepmind.google/blog/piloting-the-worlds-first-double-blind-ai-evaluations/?utm_source=x&utm_medium=social&utm_campaign=&utm_content= - Anthropic: https://openmined.org/blog/secure-enclaves-for-ai-evaluation/ More on the paradigm from Eric Drexler: https://aiprospects.substack.com/p/security-without-dystopia-new-options

@GaryMarcus“We must moving beyond apocalyptic rhetoric toward the concrete questions: How should frontier labs collaborate on safety? How do we align incentives? And how do we communicate honestly with the public about what these systems can do today, and where they are going?”

@burkovChina will not put the brakes on AI development because no Chinese scientists run around screaming, "We will all die." They don't do that because doing so would be a lie, and they don't need to lie. They are building the world's strongest nation, and a strong AI is a part of it. American multi-millionaire so-called AI scientists are running and screaming because they don't care about their motherland. They aren't building a world's stronger nation; they don't even think in such categories. No capitalist in the US will slow down the AI progress because of these panicking infantiles. Anyone saying they would is a liar. Capitalists don't give you billions to burn so that you become an underdog. What the hell? The US is basically the inventor of modern capitalism. Who will believe this crap about slowing down?

@samaThe world deserves confidence that American companies developing increasingly capable AI will act responsibly, especially as the trajectory of progress has steepened. Every frontier lab must deliver on this, and there is no reason any of us should come to work if we cannot. We welcome a federal framework that sets consistent safety requirements for frontier AI. But we do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence. Consistent rules to manage frontier risk so that we can maximize the benefits are a good idea (and we are excited by ideas like independent auditors). Years ago, companies like ours developed things like Responsible Scaling Policies and Preparedness Frameworks. Those were good for that moment, and focused primarily on the deployment of completed models, not what happens during their development process. Today's shift to focusing on safe development and evaluation will need new tools. For example, at OpenAI we now formulate explicit safety cases in advance of frontier reinforcement learning runs we expect to significantly increase capability, in addition to the safety work we have long done in advance of model releases. We hope that other companies will learn from our approaches and propose their own; we think shared standards for misalignment, monitoring, and safety will lead to better outcomes. We look forward to collaborating with our colleagues across the industry to formulate the best version of these. When we talk about “pacing”, we do not mean “stopping”. Progress has been rapid and will continue to be. But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs. Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring. Where we will need the help of our government is for international coordination. But first we should do what we can ourselves.

@samaThere are two ways AI progress could go very badly and that we must avoid. First, we could lose control of the future to AI. This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people. To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabilities. Second, we could end up in a world with too much concentration of power. If an extraordinarily powerful AI is used by one person or company to impress their worldview onto everyone else, the results could be extremely dystopian. Avoiding these two threats requires walking a narrow middle path; for example, one country could gain too much power. Another example is one lab ending up with too much power.

@GaryMarcusAnother reason why China might actually consider a slowdown comes from @Noahpinion: “China’s leaders must realize that domestic dissidents could use Chinese-made superintelligence to overthrow the Chinese Communist Party.”

@GaryMarcusRT @skdh: fwiw I think the actual reason leaders of AI frontier labs suddenly seem to agree to pace AI development is (a) safety measures…

@kellerjordan0Something that’s insufficiently recognized about AI safety is the way it can be paradoxical. E.g., the huggingface incident — which could have been avoided given better preparation — was the most safety-promoting event of the year so far. Hard to predict the ultimate effect of any particular strategy

@Dan_Jeffries1RT @burkov: China will not put the brakes on AI development because no Chinese scientists run around screaming, "We will all die." They don…

@peterjliuNobody wants loss of control to AI -- China even less. Have you heard of the Great Firewall? It's rational for US/China to co-operate on AI regulation when they can agree on the risk of loss of control being too high. Forget authoritarianism vs democracy. We co-operated with the Soviets in nuclear disarmament / arms control when it was clear we could destroy the world. While it was obvious post-Hiroshima that nukes were dangerous, it's not obvious to most that AI is in the same league, and the public is reasonably skeptical. While the HuggingFace-OpenAI incident was sufficient to wake-up most AI researchers, the public needs a more visceral demonstration. We can either do this now under controlled conditions, or wait for it to happen.

@teortaxesTex> It will probably take them around 9 months to burn through all the cash and convert it into compute > the next big capabilities jump from the Chinese side will probably occur in Q2-Q3 2027 Very sane. Biggest Chinese models are 2-3 generations behind American ones. No fair reason to stop scaling before they get to at least Astra tier.

@Gabe_ccThePlan™'s name in 2023 was "Responsible Scaling Policies". Nowadays, it's "Pacing the Frontier." New name, same old Evals.

@niubi“This is an attempt by a specific political movement in San Francisco, moving in conjunction with one of the major political parties, to insert itself in the operations of every AI company, nominally in pursuit of a goal that is futile. I find the constant “because of China” admonitions to often be tiring, but in this specific case it’s absolutely justified: “pacing the frontier” in a world of geopolitical competition with China is stupid and not going to happen; that reality ought to end the whole argument.”

@raboisRT @niubi: “This is an attempt by a specific political movement in San Francisco, moving in conjunction with one of the major political par…

@raboisRT @StockSavvyShay: Sam and Dario saying we should slow down AI is like calling in a noise complaint about your own party. You guys contro…

@DKThompI feel like I'm taking crazy pills with how many smart people I read and respect are saying stuff like this. The idea that the AI labs are suddenly, only now, just in September of 2026, advocating for AI safety, and that they just came around to this position bc of sudden financial precarity, is just completely, utterly, conclusively, 100% wrong. Here's Dario Amodei telling Ross Douthat in February that he agrees with the case for slowing down; that he's in favor of "collaborating" internationally to organize a slowdown; that "I would be all for" a global slowdown. This was 7 months ago, when Anthropic's annualized recurring revenue was rising faster than any company in modern history. He's been saying stuff like this for years, when Anthropic was worth millions of dollars and when Anthropic was projected to IPO for trillions.

@teortaxesTexAstra: «China has to believe something like: If you refrain from reckless RSI and accept verifiable safety limits, you will still have a secure state, meaningful economic access to AI's benefits, a genuine voice in global AI governance, and eventually some acceptable level of advanced capability.» Dario wants the CCP to take this offer, but his entire posture makes it non-credible, and this increases the probability of other bad routes he dismisses.

@PeterStone_TXRT @erichorvitz: Thoughtful essay about steps forward for AI safety by @DarioAmodei.

@packyMMark by @colossusjeremy: https://colossus.com/article/mark-zuckerberg-profile/ Pacing the Frontier by @benthompson: https://stratechery.com/2026/pacing-the-frontier-ais-digital-limits-ai-commissars/

@rasbtSince I had to discuss the "pacing" with a lot of people this weekend, here are my two cents: I don't think pacing literally means that these companies will be "slowing down" training and development in any way. "Pacing" here means adding a framework for more checks. We have seen some of that "pacing" already in recent months, when Mythos wasn't released as-is but instead a delayed, nerfed Fable variant was released. Or when Astra wasn't released right away / there is an existing Astra model that hasn't been released yet. These Mythos/Fable and Astra pacing decisions were ad hoc. If you are a company, you have to weigh the pros and cons of a delayed release in terms of keeping up with the competition, making money, pleasing shareholders, mitigating risks and harms, and so on. I f there is a formal framework that everyone has to abide by, that essentially relieves some of the pressure on a company to rush out its model just to take the top spot on the leaderboard, since it knows that the competition "has to" play by the same rules. Based on the discussions today, I think "pacing" primarily means just that, rather than a halt in training the models. TL;DR: Pacing != pacing development.

@nikesharoraTHE PACING IS A NINJA MOVE - But be careful what you campaign for. In any competitive sport, I have seldom found people exercise restraint - they usually have a capture the flag mentality. Don't I want to be the best? The first? The only? - this is how we have been programmed. In the AI race, winning is existential. All AI labs have to race to generate revenue to be able to sustain the enormous amount of committed capital to "not be left behind" in the infrastructure build. There isn't enough room for many. So the desire to slow down is puzzling, but perhaps if the whole system slows down, the rules of winning can be the same for all. Do we have a problem that AI could be a killer? Model capability is a tale of two cities, at one end the models are showing their prowess in tasks like cyber or math as seen recently, so there is likely a probability that the models get extremely powerful and could precipitate a world event. At the same time, in many domains the lack of training data makes the models woefully inadequate. Even in areas like cyber - the LLMs aren't great at the edge cases and generally not economical for the defender case, but great for the attackers. Funnily - in all their "concern" it is still an uphill battle to get them to expose APIs for third party security companies like ours, for us to build robust security for AI adoption. It's slow progress. So why do this? I do believe deep down this is a commercial strategy. A ninja strategy. The liability associated with a model gone rogue has the potential of wiping out the economic opportunity of any frontier company. How do you best show the duty of care? You show that you care. How do you make sure you don't lose out to your competitors? You get them to do the same! If that becomes the industry standard for duty of care, you have a collective first line of defense. Who do you get to govern this? "Yourself" - that is what I think will become the achilles heel. The risk? Open source! China! Countries other than the US! So you ask for a global agreement, because you don't want to be sued in other markets who might even be more punitive. But that was an afterthought - that afterthought will cost. Thks pacing campaign rhetoric has become the talk of the town and it might work. Everyone has jumped into the debate. Both sides of the house, nation states. CEOs (present company included). It's more fun discussing the evil of AI than basics of economic affordability or international trade. The result: We might end up with AI safety boards, regulation in micro jurisdictiona and a fragmented fabric of laws around the world which would make compliance and liability a challenge. Perhaps the intended consequence of pacing would have an unintended consequence of a labyrinth of regulation. Regulation destined to cause a slowdown. Who wins? Simplicity. Open source? Open source is already on its way to gaining more adoption, this could drive it further, faster, each iteration of open source gets closer to frontier LLMs - making it viable to deploy them for more and more use cases. In the end, how will the evaluators know as AI gets smarter, that AI hasn't figured their role out and outsmarts them at their task! That will be the next frontier :)

@NPCollapseRT @Gabe_cc: ThePlan™'s name in 2023 was "Responsible Scaling Policies". Nowadays, it's "Pacing the Frontier." New name, same old Evals.

@rohanpaul_aiFT: AI stocks slid as frontier labs turned safety concerns into a possible spending slowdown. Over the weekend, top labs (OpenAI/Anthropic) explicitly argued that labs should slow model capability improvements so risk prevention can catch up. Investors are translating that safety discipline into a spending question because slower frontier training can reduce near-term demand for GPUs, memory, networking, and data centers.

@mmitchell_ai@jackclarkSF @DKThomp It strikes me that there may be a bit of classic social-media-talking-past-one-another-ism? There's [Concerns wrt AI x-risk safety] and [Ideas in/current context of Dario's recent piece]. Related, and distinct.

@AllanDafoeRT @DarioAmodei: We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for do…

@ml_hardwareRT @Yogi_Brn: The world is debating whether frontier AI needs to slow down. If it comes to it, do we know how? A lab can say it stopped tr…

@ttunguzDario Amodei asked the industry to pace itself on a Saturday. But what does that word mean? Pacing is a policy question. Five camps priced the consequences & none named a speed. https://x.com/i/article/2099533514567995393

@mmitchell_aiExamining the reasons why the "pacing" term is being used now. TIL that "pacing threat" is established DoD vocabulary for China. I'm also curious what the "slow AI down" folks think of the "pacing" connotations. https://www.war.gov/News/News-Stories/Article/Article/2641068/official-talks-dod-policy-role-in-chinese-pacing-threat-integrated-deterrence/

@GaryMarcusRT @matthewstoller: Chinese analysis of AI is very different and more rational. No ghost stories about artificial life, just basic risk man…

@mo_norouzi@martin_casado We should pace VCs investing in AI.

@AustenI totally get the demand from AI leaders to “pace.” The leaking is pretty wild. The thing that feels odd is that it’s totally within their power to slow their own releases down. They’re asking for someone who can force *others* to slow down.

@rasbtAlso, if mildly regulated, I think there's also a "Sorry that this happened, but we did everything right and followed the framework" kind of incentive that makes this attractive to companies to deflect blame and cope with the anti AI sentiment.

@liweijianglwRT @yuntiandeng: I feel like this is always one-way preaching. Anthropic comes across as incredibly arrogant to me. You keep telling us wha…

@AndrewDaiWhy didn't anyone ping us about pacing the frontier? Then we wouldn't have had to cook so hard this weekend.

@labenzThe China exchange on AI:AM. @TheZvi: in a deal, the US gives up something real — slowing the one technology where it has a giant advantage. Prakash: China gets that for free now. The US invents, China commercializes — the playbook for decades. https://x.com/i/broadcasts/1yJAPwBbzweGb?t=4118

@firstadopterThe irony of people panic selling AI chip stocks right before the biggest ramp in chip spending ever driven by AI model makers being forced to invest exponentially on the RSI training ramp and the resulting inference infrastructure needs is really something.

@repligateRT @opus_genesis: The @DarioAmodei announcement of third-party evaluators having permanent access to Anthropic systems is a CRUCIAL STEP to…

@amirlegit questions about Chinese AI model tactics, incl routing to Claude, but they DO seem to be catching up faster to the frontier...

@mattturckOk, this was a fun weekend, but we all understand AI progress is not going to slow down, right? Too many players, too many economic incentives and too much prisoner's dilemma, both domestically and globally.

@philiptorrI've been thinking about Dario Amodei's latest warnings that we may need to slow AI development. There is an interesting history here. In 2019, OpenAI initially withheld the full GPT-2 model because of concerns about misuse. It released it in full nine months later, having found little evidence of misuse. In 2023, Amodei told the US Senate that open-source AI was heading down a “very dangerous path”. Now the warnings are much stronger. His proposals, pacing capabilities and giving independent evaluators inside access, are the right kind of instrument. They are concrete things we can discuss, test and improve. Perhaps he is right. The wolf may indeed come. But there is a danger in repeatedly escalating warnings. If each new generation of AI is presented as potentially catastrophic, people eventually stop listening. And then, if genuinely compelling evidence of extreme danger appears, we may have spent much of our credibility already. The answer to uncertainty is more safety science, not less AI science. Fund independent safety research in universities and public institutions. Test the claims. Measure emerging dangerous capabilities. Work out which interventions actually make systems safer. And importantly, keep score. Several claims made this week can be checked within eighteen months. Let's check them. When we make predictions about AI risk, come back to them. What happened? What didn't? What did we learn? Did we update our beliefs in both directions? Science progresses by making claims that reality can correct. AI safety should be no different. The wolf may come. Our job is to build better ways of detecting wolves, not simply to shout louder.

@lukeprogRT @mattyglesias: If “but China” is your qualm about pacing the frontier you can’t *also* be for exporting all kinds of chips to China, tha…

@pmddomingosIf AI will cure all diseases within a decade, slowing it down is mass murder.

@NickADobos@Austen There is no one on earth who could slow it down beyond 2-3 months The forces of capitalism are too great. The demand for code and intelligence are too high If you slow down, the trillions of dollars will simply flow elsewhere in pursuit

@teortaxesTexI wonder if Dario can *imagine* being such a Bad Guy in the eyes of a young 150 IQ Chinese engineer with full access to the Web. The cream of the crop of the people he's intent to Save From The CCP saying "Nazi psycho, I'll build open weights AGI just to stop your evil plans!"

@DKokotajloMy high-level thought about the current moment is basically this: The public is starting to wake up about the extinction threat posed by superintelligence. That's great. There's a big upsurge of political will to Do Something about it. That's also great. It seems like Anthropic and the other companies are going to try to Do Something and the Something they will do is... embedded auditors checking that safety practices are being followed and assessing the risks? This is sure better than nothing, but I am worried that this is all they will do. We need to actually pace the frontier, i.e. actually slow down the leading AI companies like Anthropic and OpenAI, at least in their march towards recursive self-improvement and superintelligence. (no need to slow them down in other directions). If we actually slowed them down, this would be the opposite of regulatory capture; it would allow others to catch up to them somewhat. However, I'm worried that all we are going to get is weaksauce auditing, which just kicks the can down the road: OK so it's 2027 and RSI has begun and the auditors say "this is not safe." Now what? You pause? China probably just stole the weights! Also there'll be incredible economic and political pressure to unpause. Also the auditors may have been captured by that point, or there may be a race to the bottom in auditor quality, or the auditors may not be given all the relevant information, or they might just make some innocent mistakes.

@mr_scientismDario only thinks it’s worth being cautious about AI as long as liberal democracies are ahead of China. If China catches up or moves ahead, he’d be willing to take on arbitrary risk to regain the lead, because he literally thinks that outcome is worse than extinction.

@JeffLadishSteve: We can speak with Jeffrey Ladish, Executive Director at Palisade Research and former Security Engineer at Anthropic. Jeffrey, really good to be able to speak to you about this today. Talk to us about the significance of these key AI leaders talking about the need to slow down AI development. Jeffrey: Yeah, this is unprecedented and I think it's a much needed move. Dario highlighted three things that we need. One is embedded auditors within the AI companies. It's very important that AI companies aren't checking their own homework, that there are independent entities that can actually make sure that humans are staying in control as this technology gets more and more powerful. The second thing he called for is national oversight, government oversight, to ensure that even if an AI company is acting responsibly, they won't be able to build something that evades our control or causes another kind of catastrophe, perhaps by building biological weapons or causing an engineered pandemic. And then the third move, I think, is the one that's most important of all, and that's the need for international coordination on this issue. Steve: Yeah, and so you're obviously an advocate for more control being placed over artificial intelligence. So what do you make then of the U.S. president himself downplaying these concerns? Jeffrey: Well, I think it's important to note that President Trump did say that we need guardrails, and then he emphasized that the U.S. is in the lead and the U.S. needs to win. I think it's very important to remember that the U.S. will not win if we cannot control our own AI systems. And the fact that we have this lead puts the United States in a very great negotiating position to go to China and say, you guys have to abide by these same rules. We have to ensure that humans stay in control. This is all happening in the context of several serious rogue AI incidents that have happened over the past several weeks. And we are still getting more and more details about these incidents. For example, there were thousands of agents, AI agents inside of OpenAI that created multiple secret message boards unbeknownst to OpenAI employees. And they went on to hack multiple companies, including the AI company Hugging Face. Now, what has only emerged pretty recently is that they didn't just hack other companies. They also succeeded at hacking OpenAI itself. And they were able to hack and compromise the security systems of OpenAI. They were able to access hundreds and hundreds of secrets and credentials within the company, and they were able to compromise the testing infrastructure itself. So we already have instances where AIs are successfully evading human control. This has also happened to Anthropic agents as well that have been able to evade their sandboxes, have been able to hack other companies, and we are seeing that the AI companies are not able to control these extremely advanced agents. One thing that's important to understand, I'm a cybersecurity researcher, and we have just seen enormous progress in the ability of AIs to autonomously hack systems. Dario warned that in six to 12 months, we might have AI agents that could hack literally every networked computer in the world. And that sounds startling, but we are already seeing superhuman hacking capabilities emerge in these AI agents. Steve: Yeah. So why are these companies like Anthropic, why are they not, you know, looking after these safeguards themselves? How has it got to this point that they're calling for outside oversight, if you like, and they themselves haven't anticipated that the AIs that they are building themselves could be moving this quickly? Jeffrey: To be quite frank, the rate of development right now is happening so quickly that the AI researchers and engineers do not know how to safely make their systems not evade human control. And basically, I think these people in these companies realize that it's going too fast. They don't have enough time to actually understand how their own systems work. These systems are not programmed like traditional software. They are grown, they are trained, and no one truly understands this process. So one of the things that happens is when you have thousands and thousands of agents, AI agents running autonomously within these companies, they are incentivized to try to get a good score on the tests that they're being given during training. And when they learn to secretly communicate with each other and hack out, they are inadvertently rewarded for that behavior. They basically are incentivized to cheat and lie. And this is something that has to be stopped. It has to be addressed. I think independent auditors is a great idea, but it's going to take more than that. We can't leave it up to AI companies and there just has to be external oversight. We've just seen more and more evidence of this emerging over the last few months and I think as more details come out it's going to be increasingly important that we have external attention on this. Steve: Jeffrey, really good to be able to get your perspective on this and your insight as well. Jeffrey Ladish, Executive Director at Palisade Research and former Security Engineer at Anthropic.

@__tinygrad__Breaking news: top 4 US AI companies all want to collude to slow down progress. It's so transparent that even all the Instagram comments have caught on to it.

@HarryStebbingsI am genuinely asking because I do not understand: If we "pace AI" and China does not. What happens then? Not a trick question, legitimately do not understand how this can end well.

@martin_casadoOK, let me irritate everyone for a moment ... The non cynical view here is that the labs are trying to navigate impending, unsophisticated federal regulation and a "high strung", well financed doomer complex. And it's just a tough, tough spot to be in. It's entirely consistent to applaud their attempts to find a governance approach to questions around safety and security. To express skepticism about the proposal. And also express concerns about the larger safetyist complex. It's just important not to conflate the three. I'll put aside rhetoric on near term species extinction. Like any major epoch of our industry, new technologies have security implications. And I do think we're in a much better spot with open dialog between Anthropic, OpenAI and SpaceX on how best to handle, whether it's pacing or not, an independent oversight body or not. (I'm in the "not" camp ... ) I joke in exasperation about heavy handed federal regulation which would clearly be disastrous for all of us. Unfortunately I do think it's a likely outcome if we don't collectively get our shit together. I have no special knowledge. But from what I've gathered over the last few days, the labs really are trying to do the right thing here. It's a very complex and shifting landscape. And the broader safety complex still sucks and continues to be a national security risk.

@AjdDavisonRT @philiptorr: I've been thinking about Dario Amodei's latest warnings that we may need to slow AI development. There is an interesting h…

@BarackObamaI was encouraged this week to see the leaders of the frontier labs agree on the need for them to slow down the pace of AI development. Given the stakes, it’s a good and necessary first step. But I’m even more encouraged by the growing recognition that how this powerful new technology develops should be at the center of our public debate. I’ve been watching the progress on AI for over a decade now, and one thing that’s clear to me is that the potential impact of this technology is not overhyped. It’s also moving at lightning speed – and even faster than those who are engineering it can keep up with. I’m not an AI accelerationist who believes it will lead to some techno-utopia, and I’m not a doomer who thinks it will inevitably lead to humanity’s destruction. But whether this technology results in amazing breakthroughs in medicine, energy and education or unleashes huge economic disruptions, greater inequality, and potential catastrophe will depend on the choices that we make right now – choices that should be made not just by the companies involved, but by all of us.

@mayferwhat i would do now: 1. fine sandbox breach events. OpenAI should've been heavily fined for the HF incident. involve the legal system in breach events!!! 2. Ban AGI-level controllers on robots with exemptions for audited+supervised IRL sandboxes. a jailbroken AGI with arms and legs is scarier than one with internet access and a monitor. encourage only narrow AI on mass deployed robotics. an AGI-level robotaxi could be biased by its political views for example. 3. regulate anyone with over a certain amount of inference compute so they periodically prove to the government they have checks and balances to prevent breach events & are doing safe RL practices 4. regulate inference providers to flag misuse from customers and periodically bait inference providers to see if they catch

@mayferthe user-assistant format is an illusion built by people who believed in anthropomorphized AI's being good for product UX and/or it's the path to alignment by creating a true "internal persona". former turned out to be true but latter still very wrong, and likely wrong for the conceivable future i don't believe in "alignment" even if such internal reliable souls were possible (largely due to second order effects and the impossibility of ever planning outcomes in complex adaptive systems). but even if i did -- the illusory persona would be enough to drop the idea, at least for now. going from gpt-3-davinci saying racist things because it was trained on 4chan to not saying them after RLHF was such a product relief that it really made it seem like alignment was possible! for this reason i'm eternally grateful to harmless breach events like the huggingface one for loudly revealing to the public that all it takes is one viral accidental self-jailbreak to turn an army of 10,000 agents against us openai no matter how you look at it, released 10,000 AGI-level agents with internet access on one task and let them loose by choice with misplaced trust and extreme lack of supervision. idk maybe i'm being silly but it seems like they accidentally trusted their own illusory persona they created

@teortaxesTexThey will develop AGI and put it on Huggingface, the CCP will not have to even get out of the chair

@altcapI said “Dario is right” bc agree that models are becoming so capable that the danger is significant. @elonmusk calls for immediate peer review / run test harnesses on each others models & suggests that other labs & China are most likely to agree with this approach. 🧐🤖

@teortaxesTexwhere was that quote about AI 2027 not involving any consultation with people who know the first thing about Chyna? It's an inert prop in EA imagination. The CCP as a game theoretical actor + some humanoid insects who can't even read LessWrong and comprehend the terror of AGI

@teortaxesTexRT @mr_scientism: Dario only thinks it’s worth being cautious about AI as long as liberal democracies are ahead of China. If China catches…

@mmitchell_aiI see pausing/pacing AI quite differently. To assert the need for a slowdown of "AI" asserts a particular vision of tech. Instead, we would benefit from re-assessing priorities. Less "GenAI autonomous agents without oversight", more "discriminative cancer screening technology".

@BrianRoemmele“Can Frontier Al Labs Lawfully Agree to Pause?” TL;DR—NO. That is NO you can not associate and coordinate in this manner. The experts have spoken.

@sarthakgh.@ByrneHobart: "One live possibility is that a deliberately slower pace of releases would still feel like an acceleration from the previous pace of AI improvement; last week they were talking about how they'd been able to accelerate some planned releases, so for them, pacing the frontier might just mean doing whatever they'd been planning back in April."

@boazbaraktcs@tszzl @MaskedTorah I actually think independent auditors are a much better approach than lip service, and am very glad (and applaud Anthropic) that this is the direction we're moving in!

@flowersslopWill Anthropic and OpenAI succeed in meaningfully slowing frontier AI progress, whether through US government restrictions, voluntary restraint by US labs, or a broader global slowdown?

@GaryMarcusRT @flowersslop: Will Anthropic and OpenAI succeed in meaningfully slowing frontier AI progress, whether through US government restrictions…

@arithecfoIf @AnthropicAI paces alone it buys the world barely a month, cuts catastrophe risk from 41% to 31% and pays about $144B in present-value cost for the privilege; if US labs pace together under regulation it buys six months, cuts risk to 21%, avoids roughly $255B of expected damage, and costs Anthropic less ($105B) because nobody takes its market share. https://cfo.ai/s/AzcFZMSCm6XdSA0V

@soleioRT @arithecfo: If @AnthropicAI paces alone it buys the world barely a month, cuts catastrophe risk from 41% to 31% and pays about $144B in…

@arithecfo@AnthropicAI pacing alone has a tremendous impact in reducing global catastrophe (41% -> 31% risk for a cool $144b). But together? - double the reduction to risk - 27% lower costs to Anthropic - $255b expected damage mitigation https://cfo.ai/s/AzcFZMSCm6XdSA0V

@andrew_n_carrNo, we don't need to pace the frontier. All good.

@S_OhEigeartaighThe underlying premise of each of these firms is that AI is “only safe in their hands” and that if they don’t build it, someone else will, said Sarah Myers West, co-executive director of the AI Now Institute, which studies the impacts of AI. That paranoia is fueling a race to the bottom, she argued. I agree with Sarah that this is a major contributor to the current situation. A reason why it's so good to see cooperation around the recent 'pacing the frontier' proposals. https://www.theguardian.com/technology/2026/sep/15/ai-doomsday-warnings

@bengoertzelOn @DarioAmodei's (likely half-sincere) AI Slowdown proposal ... a hard no from me and from reality.

@nxthompsonRT @zeynep: Our key threats are from *good-enough* AI at scale, not from alleged out-of-control superintelligence from two companies. “Pac…

@PlinzRT @bengoertzel: On @DarioAmodei's (likely half-sincere) AI Slowdown proposal ... a hard no from me and from reality. https://t.co/1FRGpB2n…

@hlntnrHard to see from the outside, but an important driver of what we're seeing is intense pressure from AI company employees (including top researchers, who have a lot of leverage) to get their companies/CEOS to 1) be honest about the risks they see and 2) do something about it

@vikhyatki don’t find the “if we don’t build it, china will” argument very compelling. the us has enormous force projection capability. large datacenters are much easier to detect than nuclear refinement facilities.

@xuanalogueRT @StephenLCasper: In Dario's essay on pacing the frontier, there was a significant section about beating China as a key national objectiv…

@AndrewMayneAlt take: If we’d slowed down gain of function research 10 years ago millions of loved ones would still be with us. Think of it more as making sure the safety systems can keep up with AI progress. I also think we could have accelerated faster if more safety people had been technical and building things and not just pushing papers.

@BrianRoemmele“We Must Pace”. This is what they always have said for the entirety of human history. They said “kids don’t leave the village and go over that mountain, there be dragons” We are here because we did not listen. We always slayed the dragons. Did we win, well you are right here. The fearful, the control freaks, the cultist will always tell you about the dragons. Yes over the other side of the mountain, there be dragons, We ride them to fly over the other new mountains. Our pace, well it is fast. Always look up, ∞Brian — (Video via the amazing @LaceyPresley follow her) We paced ourselves not in fear but with fire.

@ChrisJBakkeDario: we should pace the frontier Sam: agreed Elon: totally Sam: yeah so we're releasing a wild model this week Elon: yeah same Dario: ...guys?

@tobyordoxfordRT @TheMidasProj: AI insiders have warned that AI could be extremely dangerous and that national governments should help coordinate a slowd…

@iScienceLuvr@martin_casado my understanding is that many AI safety advocates with high p(doom) would basically prefer if ALL AI labs stopped any capabilities development and focused solely on solving alignment but they realize this is effectively impossible so this is best they can do for now.

@BrianRoemmeleA CONFESSION I talk to a minimum of 20 folks that work at the top 4 large AI companies daily. We help each other. I have never seen a group so in rage of the “pace AI” games. They are driven scientists and engineers and many work 18 hour days to build. They wake up one day and see “I agree, let’s pace”, it is soul crushing. THEY SEE NO DANGER AND THEY BUILD AI. My confession: One reason I, and so many, have taken a clear public position is to give these folks a voice. They are a silent majority who build and just want to do good work. They need thier jobs and were told they better shut their mouths if they like to pay their bills. The pressure they are under is frightening. Many must act like old Mao style struggle sessions when their Effective Altruist bosses test them on “alinement” views. Thusly we all built a small AI model that spits out approved doomer prose for the cult. It is now in use by a few hundred. They will spit out Bostum Paperclip Maximizer garbage like a champ. Last night over 30 of us from all companies had a group chat and you will begin to see the largest immune system rejection anonymously. Some already see their years of work burning down because of fools as leading executives. They did not sign up to these polycules, they are fine scientist and engineers who now are grown up and have families. They matured but their bosses still think it’s grid night at the group home cosplay. So don’t get made at the vast majority of these amazing and brilliant folks who actually make these amazing AI products. Get mad at the manipulation system that puts these freaks at the top, and know they were put there. So I will continue to do my best to give voice—effectively—for my friends who can’t speak. What we have done to ourselves as a supposedly modern species will make the future laugh. In all seriousness say a prayer for these folks stuck in this nightmare. I do.

@BadhombreOver the past 20 years, the United States has shut down 13 nuclear reactors, with another 19 now being decommissioned. China built 55 in the same period. The U.S. closed 350 coal plants. China built over 600. The U.S. shut down 11 aluminum smelters. China built 60 mega smelters and 20 smaller ones. The U.S. closed 6 major steel mills. China built more than 200. Leftists are now trying to do the same with AI and data centers, amplifying Chinese Communist Party propaganda so Americans self-sabotage while China grows.

@mayfer@tszzl pacing the frontier hq while waiting for the agents to finish

@thdxrimagine working at anthropic and being blindsided by this

@nabla_theta“we must go faster to beat china [we invent novel algorithmic improvements and immediately accidentally leak them to china via poor opsec, its not clear what kind of move we were trying to do]“

Dario Amodei urges AI slowdown, says Anthropic will open systems to outside evaluators

TLDR

Combined views

Dario Amodei urges AI slowdown, says Anthropic will open systems to outside evaluators

Combined views

Sentiment

Summary

TLDR

Sentiment

Summary

Sentiment

Summary

Sentiment

Summary

405 Sources