Superintelligence's existential risk lies in its raw power, Michael Nielsen argues

In a text for a talk, Michael Nielsen explores why experts disagree about whether artificial superintelligence (ASI) poses an existential risk to humanity. He argues that the fundamental danger is the power ASI would confer and the lower barriers to creating dangerous technologies—not whether a rogue ASI escapes control. In his view, finding rogue ASI implausible has led many people to mistakenly dismiss existential risk. He also argues that much AI alignment work accelerates catastrophic capabilities without addressing the world's potential vulnerability to dangerous technologies.