Elizabeth Barnes, AI safety researcher and founder/CEO of METR, says development is likely on track for AI systems capable of causing human extinction or permanent disempowerment possibly within the next few years
Gary Marcus quoted the post in full agreement.
#153Gary Marcus@GARYMARCUS100% agree with @BethMayBarnes on this point and most (though not quite all*) of her important thread.
*i am much less concerned about extinction risk per se, as discussed in my TLS review of If Anyone Builds It.
Elizabeth Barnes@BethMayBarnes(4) IMO, any “reasonable” civilization would clearly be taking things much more slowly and carefully with AI. The benefits of getting upsides of advanced AI a little faster are small compared to the risks of getting it irrecoverably wrong, and we could lower these risks by going slower
#153Gary Marcus@GARYMARCUSI 100% agree with @BethMayBarnes on this point and most (though not quite all) of her important thread
Elizabeth Barnes@BethMayBarnes(4) IMO, any “reasonable” civilization would clearly be taking things much more slowly and carefully with AI. The benefits of getting upsides of advanced AI a little faster are small compared to the risks of getting it irrecoverably wrong, and we could lower these risks by going slower
(2) Things are chaotic and rushed; we aren’t on top of the basics (models regularly violate user intent, labs train on things they meant to avoid, security probably isn’t good enough to prevent adversaries stealing dangerous models) let alone thorny questions of how to control/align superhuman AI
Elizabeth Barnes@BethMayBarnes(1) We are likely on track to develop AI systems capable of causing human extinction/permanent disempowerment, quite possibly within the next few years
(4) IMO, any “reasonable” civilization would clearly be taking things much more slowly and carefully with AI. The benefits of getting upsides of advanced AI a little faster are small compared to the risks of getting it irrecoverably wrong, and we could lower these risks by going slower
Elizabeth Barnes@BethMayBarnes(3) METR (and other independent orgs, as well as safety/security teams at labs) feel woefully under-resourced compared to the scale and pace of AI development - we’re struggling to build benchmarks fast enough, keep ahead of latest capability developments, read and respond to all the safety-related claims that AI developers are making, run all the evaluations and assessments that companies + governments are asking us to, plus develop the science needed to assess risks from increasingly capable AIs.
@BethMayBarnes What does "quite possibly" mean here? Can you be more precise about how likely you think this is to occur within the next few years?
Elizabeth Barnes@BethMayBarnes(1) We are likely on track to develop AI systems capable of causing human extinction/permanent disempowerment, quite possibly within the next few years