• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    The argument that many would prefer extinction risk to an AI-fueled world government

    The post imagines a single AI-fueled global government combining what its author considers the worst regulatory and social policies of the past 20 years.

    Yann LeCunYL
    Miles BrundageMB
    Richard SocherRS
    63 Sources, ,

    TLDR

    One post argues that many people would find a 10% chance of ending humanity far preferable to a roughly 100% chance of living under an AI-fueled global government combining what the author calls the worst regulatory and social policies of the past 20 years. The author criticizes others on their timeline for being unable to entertain that preference.

    Combined views

    1.5M

    63 Sources, first seen 27d ago

    Combined views

    1.5M

    63 Sources, first seen 27d ago

    11.6K likes
    27d ago
    first seen 27d ago
    11.6K likes
    843 comments
    1.9K saves
    3.7K reposts
    843 comments
    1.9K saves
    3.7K reposts

    63 Sources

    Samuel Ratnam@eterecursionFor those that don't know, this is the former Chief Scientist at the UK AI Security Institute. Before that, he worked at Google DeepMind, OpenAI, and Google Brain.27d
    Nathan 🔎@NathanpmYoungA ranty tweet in defence of using numbers to quantify risk: What might happen if there were no AI forecasts? Would people say "wow we don't know what is going to happen with AI, I guess we should be ambivalent between speeding up and slowing down"? No! It would probably (in my view) be full speed ahead. This has forced making poorly justified but precise forecasts to illustrate levels of concern. There are flaws in this, sure, but note that signifcant groups of people in the field don't talk remotely like this about most other things. 1000s of plastic researchers aren't like "there is a 10% chance plastic kills everyone". Oil CEOs are not like "we might end human life". But 1000s of AI researchers do think that. We have polled them, 10% is the median response. So I don't love P(doom) and I would prefer we were clear that 3+ year forecasts are very inaccurate. But let us not pretend that if people had said "I am very worried" this would have been understood as those people intended. They think there is a plausible imminent risk to everyone alive. Some think the risk is likely, some almost certain. I am glad they can convey that, using numbers if they have to.27d
    rat king 🐀@MikeIsaaca frustrating thing about all the "TK percent chance we die" headlines is that researchers can say any number and non-technical people will just be like "damn well that guy's in tech, im worried" show your work! is this about scaling laws or some arxiv file ive yet to read?26d
    Nate Silver@NateSilver538The AI risk people actually do show their work! There are hundreds of working papers, thousands of LessWrong posts, literally entire best-selling books about this.26d
    Geoffrey Irving@geoffreyirvingRT @NateSilver538: The AI risk people actually do show their work! There are hundreds of working papers, thousands of LessWrong posts, lite…26d
    David Krueger 🦥 ⏸️ ⏹️ ⏪@DavidSKruegerRT @NathanpmYoung: Here is a simple idea for AI safety comms people. Say what you think is true. Just be honest.26d
    Tristan Harris@tristanharrisRT @eterecursion: For those that don't know, this is the former Chief Scientist at the UK AI Security Institute. Before that, he worked at…26d
    rohit@krishnanrohitWhen you're talking about doom, you should make it exceedingly clear that the chain of events is: 1. Fable 5.2 -> 2. Fable 7 -> 3. SuperDangerousASI 4. Doom And that you assume this will happen spontaneously and over v short timeframe (days/weeks). Otherwise you're misleading.26d
    Jakob Foerster@j_foerstPrediction markets usually break down for existential risks - e.g. if everyone gets wiped out by a comet then nobody is there to win the bet (and therefore nobody will take the other side). P(doom) is different: once there is an entity meanfigfully taking the other side of the bet that becomes informative in its own right: https://manifold.markets/MartinRandall/will-ai-wipe-out-humanity-before-th-d8733b2114a826d
    Ben Recht@beenwrektTed Chiang wrote the definitive essay on p(doom) in 2017. It's worth reading again in light of this week's events. https://www.buzzfeednews.com/article/tedchiang/the-real-danger-to-civilization-isnt-ai-its-runaway26d

    Sentiment

    Positive16.6%83.4%Negative

    Summary

    Replies rejected high AI extinction forecasts like 70% or 18% p(doom) as clickbait lacking any modeling and rooted in Hollywood neuroticism rather than evidence.

    Based on 202 sentiment-bearing replies from 175 accounts across 11 conversations.

    Sentiment

    Positive16.6%83.4%Negative

    Summary

    Replies rejected high AI extinction forecasts like 70% or 18% p(doom) as clickbait lacking any modeling and rooted in Hollywood neuroticism rather than evidence.

    Based on 202 sentiment-bearing replies from 175 accounts across 11 conversations.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    63 Sources

    Samuel Ratnam@eterecursionFor those that don't know, this is the former Chief Scientist at the UK AI Security Institute. Before that, he worked at Google DeepMind, OpenAI, and Google Brain.27d
    Nathan 🔎@NathanpmYoungA ranty tweet in defence of using numbers to quantify risk: What might happen if there were no AI forecasts? Would people say "wow we don't know what is going to happen with AI, I guess we should be ambivalent between speeding up and slowing down"? No! It would probably (in my view) be full speed ahead. This has forced making poorly justified but precise forecasts to illustrate levels of concern. There are flaws in this, sure, but note that signifcant groups of people in the field don't talk remotely like this about most other things. 1000s of plastic researchers aren't like "there is a 10% chance plastic kills everyone". Oil CEOs are not like "we might end human life". But 1000s of AI researchers do think that. We have polled them, 10% is the median response. So I don't love P(doom) and I would prefer we were clear that 3+ year forecasts are very inaccurate. But let us not pretend that if people had said "I am very worried" this would have been understood as those people intended. They think there is a plausible imminent risk to everyone alive. Some think the risk is likely, some almost certain. I am glad they can convey that, using numbers if they have to.27d
    rat king 🐀@MikeIsaaca frustrating thing about all the "TK percent chance we die" headlines is that researchers can say any number and non-technical people will just be like "damn well that guy's in tech, im worried" show your work! is this about scaling laws or some arxiv file ive yet to read?26d
    Nate Silver@NateSilver538The AI risk people actually do show their work! There are hundreds of working papers, thousands of LessWrong posts, literally entire best-selling books about this.26d
    Geoffrey Irving@geoffreyirvingRT @NateSilver538: The AI risk people actually do show their work! There are hundreds of working papers, thousands of LessWrong posts, lite…26d
    David Krueger 🦥 ⏸️ ⏹️ ⏪@DavidSKruegerRT @NathanpmYoung: Here is a simple idea for AI safety comms people. Say what you think is true. Just be honest.26d
    Tristan Harris@tristanharrisRT @eterecursion: For those that don't know, this is the former Chief Scientist at the UK AI Security Institute. Before that, he worked at…26d
    rohit@krishnanrohitWhen you're talking about doom, you should make it exceedingly clear that the chain of events is: 1. Fable 5.2 -> 2. Fable 7 -> 3. SuperDangerousASI 4. Doom And that you assume this will happen spontaneously and over v short timeframe (days/weeks). Otherwise you're misleading.26d
    Jakob Foerster@j_foerstPrediction markets usually break down for existential risks - e.g. if everyone gets wiped out by a comet then nobody is there to win the bet (and therefore nobody will take the other side). P(doom) is different: once there is an entity meanfigfully taking the other side of the bet that becomes informative in its own right: https://manifold.markets/MartinRandall/will-ai-wipe-out-humanity-before-th-d8733b2114a826d
    Ben Recht@beenwrektTed Chiang wrote the definitive essay on p(doom) in 2017. It's worth reading again in light of this week's events. https://www.buzzfeednews.com/article/tedchiang/the-real-danger-to-civilization-isnt-ai-its-runaway26d