“P(doom)” and two distinct questions about superintelligence
A critic argues that “P(doom)” conflates the odds of artificial superintelligence being built with the risk of ruin if it is—and says policy choices matter to the first question.
TLDR
One post argues that “P(doom)” bundles together two different probabilities. The author treats ruin as certain if artificial superintelligence (ASI) is built using anything resembling current techniques, or new techniques invented and managed by language models resembling Astra/Fable. But they express much less confidence about whether ASI will be built, citing uncertainty about future policy choices. They argue for a more hardline policy to prevent ASI and accuse accelerationists of overstating its inevitability.
Combined views
1.7K
1 Source, first seen 4h ago
“P(doom)” and two distinct questions about superintelligence
A critic argues that “P(doom)” conflates the odds of artificial superintelligence being built with the risk of ruin if it is—and says policy choices matter to the first question.