Anthropic released a paper outlining US strategies to maintain frontier AI lead over China through advanced chip controls and sustained competition
MIRI's Nate Soares questioned its alignment with safety priorities.
Do you disagree with the article?
As far as I can tell, it's making the correct point that America shouldn't be leaking chips to China. Fighting this smuggling is the correct move even within (especially within!) a high-pdoom worldview.
If negotiation is possible, China is more likely to negotiate when they're losing (or when we have a carrot to offer them, in the form of chips that we're not giving for free).
If negotiation is impossible, then it's better to have all the AI development concentrated in one country. That country then at least has the option to pause/slowdown AI for however long it takes the other countries to catch up, even if it can't do so permanently. Or it can regulate AI without having to worry about losing the race. I tried to make this case at https://www.astralcodexten.com/p/why-ai-safety-wont-make-america-lose , which I think makes the same anti-compute proliferation arguments Anthropic is making on their blog post, from a specifically safety-oriented perspective.
I think attacking Anthropic for fighting compute proliferation is a net negative even within what I think is your own world-model. Any successful slowdown will come from a hundred small things going right beforehand that convince everyone it's in their best interest (like the US cracking down on compute leaking to other countries). If you attack every attempt to make small things go right because it's not the big thing you want, you're decreasing the chance of ever getting the big thing.
Nate Soares ⏹️@So8resAnthropic encourages racing without even acknowledging the possibility of global coordination (below). They hire top scientists (Karpathy) to work on the most dangerous tech (recursive self-improvement). This is not "good guys" behavior.
@slatestarcodex @So8res Scott, one question. Have you considered a scenario where you "win the AI race" but it doesn't actually translate into any victory over Chyna, and instead you've just permanently pissed them off with this decapitation strike scheming, and they have like 70% of world industry.
Scott Alexander@slatestarcodexDo you disagree with the article? As far as I can tell, it's making the correct point that America shouldn't be leaking chips to China. Fighting this smuggling is the correct move even within (especially within!) a high-pdoom worldview. If negotiation is possible, China is more likely to negotiate when they're losing (or when we have a carrot to offer them, in the form of chips that we're not giving for free). If negotiation is impossible, then it's better to have all the AI development concentrated in one country. That country then at least has the option to pause/slowdown AI for however long it takes the other countries to catch up, even if it can't do so permanently. Or it can regulate AI without having to worry about losing the race. I tried to make this case at https://www.astralcodexten.com/p/why-ai-safety-wont-make-america-lose , which I think makes the same anti-compute proliferation arguments Anthropic is making on their blog post, from a specifically safety-oriented perspective. I think attacking Anthropic for fighting compute proliferation is a net negative even within what I think is your own world-model. Any successful slowdown will come from a hundred small things going right beforehand that convince everyone it's in their best interest (like the US cracking down on compute leaking to other countries). If you attack every attempt to make small things go right because it's not the big thing you want, you're decreasing the chance of ever getting the big thing.
@slatestarcodex @So8res Sentiments aside, you forever lose IC market foothold in China and its economic sphere which will likely expand. I get that you're a man of letters but you probably can comprehend how *puny* American industry is, and how Wunderwaffe-like these dreams of AI leapfrogging are
Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTex@slatestarcodex @So8res Scott, one question. Have you considered a scenario where you "win the AI race" but it doesn't actually translate into any victory over Chyna, and instead you've just permanently pissed them off with this decapitation strike scheming, and they have like 70% of world industry.
@slatestarcodex @So8res puny in relative terms ofc the US is a big country, inhabited by many Americans etc etc
Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTex@slatestarcodex @So8res Sentiments aside, you forever lose IC market foothold in China and its economic sphere which will likely expand. I get that you're a man of letters but you probably can comprehend how *puny* American industry is, and how Wunderwaffe-like these dreams of AI leapfrogging are
Nov 25 2025 I'd say looks more like 6 months still. Shouldn't it have expanded to like 3 years by now? anyway, all reasonable for a Patriot
Scott Alexander@slatestarcodexDo you disagree with the article? As far as I can tell, it's making the correct point that America shouldn't be leaking chips to China. Fighting this smuggling is the correct move even within (especially within!) a high-pdoom worldview. If negotiation is possible, China is more likely to negotiate when they're losing (or when we have a carrot to offer them, in the form of chips that we're not giving for free). If negotiation is impossible, then it's better to have all the AI development concentrated in one country. That country then at least has the option to pause/slowdown AI for however long it takes the other countries to catch up, even if it can't do so permanently. Or it can regulate AI without having to worry about losing the race. I tried to make this case at https://www.astralcodexten.com/p/why-ai-safety-wont-make-america-lose , which I think makes the same anti-compute proliferation arguments Anthropic is making on their blog post, from a specifically safety-oriented perspective. I think attacking Anthropic for fighting compute proliferation is a net negative even within what I think is your own world-model. Any successful slowdown will come from a hundred small things going right beforehand that convince everyone it's in their best interest (like the US cracking down on compute leaking to other countries). If you attack every attempt to make small things go right because it's not the big thing you want, you're decreasing the chance of ever getting the big thing.
#1442Eli Lifland@ELI_LIFLANDDisappointed to see Anthropic’s US/China scenarios have essentially no mention of a coordinated slowdown.
“US/China slowdown is obviously infeasible” attitude is self-fulfilling. (This piece doesn’t explicitly argue it, but its focus is telling and Dario has explicitly said it.)
Anthropic@AnthropicAIWe've published a paper that explains our views on AI competition between the US and China. The US and democratic allies hold the lead in frontier AI today. Read more on what it’ll take to keep that lead: https://www.anthropic.com/research/2028-ai-leadership
There is something here:
Nate Soares ⏹️@So8resAnthropic encourages racing without even acknowledging the possibility of global coordination (below). They hire top scientists (Karpathy) to work on the most dangerous tech (recursive self-improvement). This is not "good guys" behavior.
@So8res I think i’d say that pushes forward the frontier, rather than most dangerous tech. I think RSI is justifiable if everyone is doing it, but bad if you’re pushing the frontier.
Nate Soares ⏹️@So8resAnthropic encourages racing without even acknowledging the possibility of global coordination (below). They hire top scientists (Karpathy) to work on the most dangerous tech (recursive self-improvement). This is not "good guys" behavior.
@So8res I agree.
Nate Soares ⏹️@So8res@NathanpmYoung "We must do this thing that horribly endangers you bc if we don't the next guy will do it even more dangerously" is a possible justification, but it comes with a solemn responsibility to do everything in your power to help Earth find a third alternative. Anthropic fails that test
I don’t think the articles most important point is that it’s about not smuggling chips to China.
The article’s most important point is that America should aim for a 24 month lead within 24 months, which basically precludes any interest in a mutual slowdown or pause.
It also directly threatens the rest of the world that America should should use AI technologies offensively or as an active threat in negotiations, which of course will predictably produce an arms race.
Yes, probably someone at Anthropic wrote this article with an aim to push the USG into doing more chip controls, but in the process of doing that they described a catastrophic AI policy that extends substantially beyond that.
Scott Alexander@slatestarcodexDo you disagree with the article? As far as I can tell, it's making the correct point that America shouldn't be leaking chips to China. Fighting this smuggling is the correct move even within (especially within!) a high-pdoom worldview. If negotiation is possible, China is more likely to negotiate when they're losing (or when we have a carrot to offer them, in the form of chips that we're not giving for free). If negotiation is impossible, then it's better to have all the AI development concentrated in one country. That country then at least has the option to pause/slowdown AI for however long it takes the other countries to catch up, even if it can't do so permanently. Or it can regulate AI without having to worry about losing the race. I tried to make this case at https://www.astralcodexten.com/p/why-ai-safety-wont-make-america-lose , which I think makes the same anti-compute proliferation arguments Anthropic is making on their blog post, from a specifically safety-oriented perspective. I think attacking Anthropic for fighting compute proliferation is a net negative even within what I think is your own world-model. Any successful slowdown will come from a hundred small things going right beforehand that convince everyone it's in their best interest (like the US cracking down on compute leaking to other countries). If you attack every attempt to make small things go right because it's not the big thing you want, you're decreasing the chance of ever getting the big thing.
They pretend like they're a "safe" AI company; a "good" company; but insofar as they're racing to enrich themselves by endangering your life *without* also begging the world to please find some other way, they're betraying that mantle.
Nate Soares ⏹️@So8resAnthropic encourages racing without even acknowledging the possibility of global coordination (below). They hire top scientists (Karpathy) to work on the most dangerous tech (recursive self-improvement). This is not "good guys" behavior.
@NathanpmYoung "We must do this thing that horribly endangers you bc if we don't the next guy will do it even more dangerously" is a possible justification, but it comes with a solemn responsibility to do everything in your power to help Earth find a third alternative. Anthropic fails that test
Nathan is in Berkeley 🔎@NathanpmYoung@So8res I think i’d say that pushes forward the frontier, rather than most dangerous tech. I think RSI is justifiable if everyone is doing it, but bad if you’re pushing the frontier.
I'm not trying to highlight inaccuracies; I'm trying to highlight a missing mood.
I think any attempt to say "we're forced into doing this horribly reckless thing that might kill you and your family, because if we don't then the next guy will do it even more dangerously" comes with a solemn responsibility to do everything in your power to help the world find some third alternative. I think Anthropic fails this test pretty badly, e.g. as evidenced here: https://x.com/AnthropicAI/status/2031674089035714578 and as Rob documented a bit here: https://x.com/robbensinger/status/2027848301236912525.
Over the last few months, reporters have asked me some variant of "but what about Anthropic? Aren't they a safe company? Do you hope that they win, as the good guys?" a handful of times. This causes me to think that a bunch of people are moved by the "we're the good guys" act.
I think it matters, strategically, as to whether all the world needs right now is the Right Company to Win, or whether we need something more like a global shutdown. So I think it's important to correct what seems to me like a common misconception around anthropic. I also think a lot of locals are loathe to criticize anthropic for one reason or another (they work there; their friends work there; they think they're better than OpenAI; ...). Thus, it looks to me like I can probably make a positive difference by highlighting ways that Anthropic is (afaict) dramatically failing to carry the "safe/good AI company" mantle.
(I tend to think it's even more important to communicate how even a company that *was* living up to the mantle still wouldn't have much of a chance, and how the real solution is an international shutdown. But I don't have to pick just one. When current events evidence some of the difference between the niche Anthropic pretends to occupy and the niche Anthropic actually occupies, I try to take those opportunities.)
Scott Alexander@slatestarcodexDo you disagree with the article? As far as I can tell, it's making the correct point that America shouldn't be leaking chips to China. Fighting this smuggling is the correct move even within (especially within!) a high-pdoom worldview. If negotiation is possible, China is more likely to negotiate when they're losing (or when we have a carrot to offer them, in the form of chips that we're not giving for free). If negotiation is impossible, then it's better to have all the AI development concentrated in one country. That country then at least has the option to pause/slowdown AI for however long it takes the other countries to catch up, even if it can't do so permanently. Or it can regulate AI without having to worry about losing the race. I tried to make this case at https://www.astralcodexten.com/p/why-ai-safety-wont-make-america-lose , which I think makes the same anti-compute proliferation arguments Anthropic is making on their blog post, from a specifically safety-oriented perspective. I think attacking Anthropic for fighting compute proliferation is a net negative even within what I think is your own world-model. Any successful slowdown will come from a hundred small things going right beforehand that convince everyone it's in their best interest (like the US cracking down on compute leaking to other countries). If you attack every attempt to make small things go right because it's not the big thing you want, you're decreasing the chance of ever getting the big thing.