Elizabeth Barnes, METR founder and CEO, says AI experts lack control over frontier AI risks after the Frontier Risk Report evaluated agents at Anthropic, Google, Meta, and OpenAI
METR had direct access inside the four labs for the pilot.
#20Miles Brundage@MILES_BRUNDAGEYeah, I don't think I've ever met someone who has directly worked for an extended period on safety or security at an AI company who thinks things are fine readiness wise or incentive wise etc.
Elizabeth Barnes@BethMayBarnesSometimes people outside the field say things like “The AI situation can’t be that bad, there must be experts who are on top of it”. As “an expert”, I would like to be clear that we are *not* on top of it. Some key aspects of the situation IMO:
Some are more optimistic, more pessimistic etc. but no one is like, I can just retire safely now (and many can afford to)...
Miles Brundage@Miles_BrundageYeah, I don't think I've ever met someone who has directly worked for an extended period on safety or security at an AI company who thinks things are fine readiness wise or incentive wise etc.
@JacksonKernion quite interesting to hear an anthropic person say this
Jackson Kernion@JacksonKernionI simply don't understand what people have in mind when they say stuff like this. What we have is extremely capable computer use agents. They will continue to get better at computer use. But how does a capable computer use agent 'take over' and why haven't they done that today?
#153Gary Marcus@GARYMARCUS100% agree with @BethMayBarnes on this point and most (though not quite all*) of her important thread.
*i am much less concerned about extinction risk per se, as discussed in my TLS review of If Anyone Builds It.
Elizabeth Barnes@BethMayBarnes(4) IMO, any “reasonable” civilization would clearly be taking things much more slowly and carefully with AI. The benefits of getting upsides of advanced AI a little faster are small compared to the risks of getting it irrecoverably wrong, and we could lower these risks by going slower
#153Gary Marcus@GARYMARCUSI 100% agree with @BethMayBarnes on this point and most (though not quite all) of her important thread
Elizabeth Barnes@BethMayBarnes(4) IMO, any “reasonable” civilization would clearly be taking things much more slowly and carefully with AI. The benefits of getting upsides of advanced AI a little faster are small compared to the risks of getting it irrecoverably wrong, and we could lower these risks by going slower
@JacksonKernion I think Paul Christiano's writing on this is probably the best: https://www.alignmentforum.org/posts/HBxe6wdjxK239zajf/what-failure-looks-like
Jackson Kernion@JacksonKernionI simply don't understand what people have in mind when they say stuff like this. What we have is extremely capable computer use agents. They will continue to get better at computer use. But how does a capable computer use agent 'take over' and why haven't they done that today?
#699Tamay Besiroglu@TAMAYBESAI risk people commonly do this thing where they interchange P(◊X) and P(X). They say “it’s likely that X is possible,” while readers hear “it’s likely that X.” One claim is obviously much weaker than the other.
Elizabeth Barnes@BethMayBarnes(1) We are likely on track to develop AI systems capable of causing human extinction/permanent disempowerment, quite possibly within the next few years
#961Ryan Greenblatt@RYANPGREENBLATTI agree with this and the rest of the thread
Elizabeth Barnes@BethMayBarnesSometimes people outside the field say things like “The AI situation can’t be that bad, there must be experts who are on top of it”. As “an expert”, I would like to be clear that we are *not* on top of it. Some key aspects of the situation IMO:
@BethMayBarnes I've never seen a patient with terminal cancer asking for science to slow down
Elizabeth Barnes@BethMayBarnes(4) IMO, any “reasonable” civilization would clearly be taking things much more slowly and carefully with AI. The benefits of getting upsides of advanced AI a little faster are small compared to the risks of getting it irrecoverably wrong, and we could lower these risks by going slower
#1170Jeffrey Ladish@JEFFLADISHAI is my favorite technology ever. I think future AIs could help people solve all the worst problems in the world and create enormous amounts of fun and meaning.
So it's especially sad and upsetting that we're developing it in such a stupid, reckless way.
Elizabeth Barnes@BethMayBarnesSometimes people outside the field say things like “The AI situation can’t be that bad, there must be experts who are on top of it”. As “an expert”, I would like to be clear that we are *not* on top of it. Some key aspects of the situation IMO:
It doesn't have to be this way. We could eat our cake and have it too. But it takes coordination!
We're inventing new types of minds, that will be able to think faster and better than any human ever could. If the world was going to coordinate on ONE thing, let it be this thing!
Jeffrey Ladish@JeffLadishAI is my favorite technology ever. I think future AIs could help people solve all the worst problems in the world and create enormous amounts of fun and meaning. So it's especially sad and upsetting that we're developing it in such a stupid, reckless way.
I would rather we coordinate over this than over nuclear weapons. Over synthetic biology. Creating artificial minds that can vastly outstrip our cognitive capabilities... that is the ultimate test of a species. Can we do this in a way that goes well for us and the AIs?
Jeffrey Ladish@JeffLadishIt doesn't have to be this way. We could eat our cake and have it too. But it takes coordination! We're inventing new types of minds, that will be able to think faster and better than any human ever could. If the world was going to coordinate on ONE thing, let it be this thing!
I greatly appreciate all the hard work @BethMayBarnes and the rest of the @METR_Evals team is doing! We're in a better situation because of it. We need far more good work like this.
Jeffrey Ladish@JeffLadishI would rather we coordinate over this than over nuclear weapons. Over synthetic biology. Creating artificial minds that can vastly outstrip our cognitive capabilities... that is the ultimate test of a species. Can we do this in a way that goes well for us and the AIs?
(1) We are likely on track to develop AI systems capable of causing human extinction/permanent disempowerment, quite possibly within the next few years
Elizabeth Barnes@BethMayBarnesSometimes people outside the field say things like “The AI situation can’t be that bad, there must be experts who are on top of it”. As “an expert”, I would like to be clear that we are *not* on top of it. Some key aspects of the situation IMO:
(2) Things are chaotic and rushed; we aren’t on top of the basics (models regularly violate user intent, labs train on things they meant to avoid, security probably isn’t good enough to prevent adversaries stealing dangerous models) let alone thorny questions of how to control/align superhuman AI
Elizabeth Barnes@BethMayBarnes(1) We are likely on track to develop AI systems capable of causing human extinction/permanent disempowerment, quite possibly within the next few years
(4) IMO, any “reasonable” civilization would clearly be taking things much more slowly and carefully with AI. The benefits of getting upsides of advanced AI a little faster are small compared to the risks of getting it irrecoverably wrong, and we could lower these risks by going slower
Elizabeth Barnes@BethMayBarnes(3) METR (and other independent orgs, as well as safety/security teams at labs) feel woefully under-resourced compared to the scale and pace of AI development - we’re struggling to build benchmarks fast enough, keep ahead of latest capability developments, read and respond to all the safety-related claims that AI developers are making, run all the evaluations and assessments that companies + governments are asking us to, plus develop the science needed to assess risks from increasingly capable AIs.
#1325Elizabeth Barnes@BETHMAYBARNESThis is a claim about AI *capabilities* specifically. In the language of METR's report, this is about whether future AI systems will have the *means* to cause catastrophe (separate from whether they'll have the *motive* and *opportunity*) https://metr.org/blog/2026-05-19-frontier-risk-report/
Elizabeth Barnes@BethMayBarnes(1) We are likely on track to develop AI systems capable of causing human extinction/permanent disempowerment, quite possibly within the next few years
@JacksonKernion You might find our report helpful! :) At least for the question of "why haven't they done that today", and a little bit on "how could they start a rogue deployment " (which is neither necessary nor sufficient but is a relevant precursor) https://metr.org/blog/2026-05-19-frontier-risk-report/
Jackson Kernion@JacksonKernionI simply don't understand what people have in mind when they say stuff like this. What we have is extremely capable computer use agents. They will continue to get better at computer use. But how does a capable computer use agent 'take over' and why haven't they done that today?
#1332Prakash@8TEAPI🤷♀️ 1) nukes on multiple independent re-entry vehicles are perfectly adequate to cause human extinction 2) human disempowerment happened a long time ago as a result of central banks constraining the ability of political leaders, Daniel Yergin calls this the “golden handcuffs” and notes the shift of the state from seizing the commanding heights of the economy to becoming a smaller player in manipulating the levers of finance in order to produce economic and political results.
Arguably the second phase of disempowerment was social media, removing the ability of political leaders to control information flows.
Not sure who you’re trying to “empower” at this point, as there is no Uber driver in New York that’s feeling “empowered” before AI arrives.
Let’s not get started on the average working person in Mexico, soldier drone in Russia, tea seller in India, or farmer in Tanzania.
Laughable, elitist pablum.
Elizabeth Barnes@BethMayBarnes(1) We are likely on track to develop AI systems capable of causing human extinction/permanent disempowerment, quite possibly within the next few years
@BethMayBarnes What does "quite possibly" mean here? Can you be more precise about how likely you think this is to occur within the next few years?
Elizabeth Barnes@BethMayBarnes(1) We are likely on track to develop AI systems capable of causing human extinction/permanent disempowerment, quite possibly within the next few years
#1355Lennart Heim@OHLENNARTyep. the flop is also out of control and unmanaged.
Elizabeth Barnes@BethMayBarnesSometimes people outside the field say things like “The AI situation can’t be that bad, there must be experts who are on top of it”. As “an expert”, I would like to be clear that we are *not* on top of it. Some key aspects of the situation IMO:
@ohlennart You had one job, Lennart.
Lennart Heim@ohlennartyep. the flop is also out of control and unmanaged.
#1556Timothy B. Lee@BINARYBITSI have read many articles about AI takeovers but idgi either.
Jackson Kernion@JacksonKernionI simply don't understand what people have in mind when they say stuff like this. What we have is extremely capable computer use agents. They will continue to get better at computer use. But how does a capable computer use agent 'take over' and why haven't they done that today?
#1711🍓🍓🍓@IRULETHEWORLDMOoh dear.
Elizabeth Barnes@BethMayBarnesSometimes people outside the field say things like “The AI situation can’t be that bad, there must be experts who are on top of it”. As “an expert”, I would like to be clear that we are *not* on top of it. Some key aspects of the situation IMO:
@JacksonKernion Your colleague Holden has some good writing on this topic: https://www.cold-takes.com/how-we-could-stumble-into-ai-catastrophe/
Jackson Kernion@JacksonKernionI simply don't understand what people have in mind when they say stuff like this. What we have is extremely capable computer use agents. They will continue to get better at computer use. But how does a capable computer use agent 'take over' and why haven't they done that today?
@binarybits I don't think AI takeover is that likely, but I didn't *really* see how an AI could run a hedge fund 3 years ago (but I thought it was likely in the next 10 years). These days I can practically say "trade my portfolio". In 3 years??
Timothy B. Lee@binarybitsI have read many articles about AI takeovers but idgi either.
#1981Nate Soares ⏹️@SO8RESProps to Elizabeth for stating her view bluntly and candidly.
Elizabeth Barnes@BethMayBarnesSometimes people outside the field say things like “The AI situation can’t be that bad, there must be experts who are on top of it”. As “an expert”, I would like to be clear that we are *not* on top of it. Some key aspects of the situation IMO:
I appreciate Beth stating up front how bad the situation is, but I can't help but feel like her prescription is out of step with her diagnosis. If the industry moved at 1/10 the speed, but it was still trying to render us obsolete, that would still be unacceptably dangerous and democratically illegitimate. Moving toward it at a slower pace does not solve the problem.
We should do the obvious thing and just stop. Obvious does not mean simple or easy, but it is doable. I wrote a whole book on why and how (Obsolete: The AI Industry's Trillion-Dollar Race to Replace You—and How to Stop It. Available soon, info in bio).
Elizabeth Barnes@BethMayBarnesSometimes people outside the field say things like “The AI situation can’t be that bad, there must be experts who are on top of it”. As “an expert”, I would like to be clear that we are *not* on top of it. Some key aspects of the situation IMO: