• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI
    Reaction

    The role of human decisions in the AI extinction-risk debate

    A commenter argues that incentives, culture and safeguards deserve more attention than AI’s capabilities in isolation.

    Joshua SaxeJS
    1 Source, ,

    TLDR

    One post says Scott Alexander wrote an open letter responding to Steven Pinker. Discussing their AI extinction-risk debate, another commenter argues that the key question is how human decisions could allow dangerous AI deployments. They say they are very concerned about AI harms but less concerned about extinction, which they argue would require a long sequence of unlikely, avoidable mistakes.

    Combined views

    2.1K

    1 Source, first seen 2h ago

    Combined views

    2.1K

    1 Source, first seen 2h ago

    25 likes
    2h ago
    first seen 2h ago
    25 likes
    11 comments
    4 saves
    4 reposts
    11 comments
    4 saves
    4 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    1 Source

    Joshua Saxe@joshua_saxeReaction to the @slatestarcodex vs. @sapinker AI x-risk debate The main problem with this debate is that both interlocutors are mainly focused on AI as a disembodied technological object that's fallen down from heaven, not a congealed mass of human decisions. Scott has understood that there's nothing ontological about deep learning that'd prevent us from killing ourselves with it and that this risk gets worse as AI scales. But this is a trivial observation! Here's how to kill everyone with AI: put Opus 8.1-Killeveryone in charge of 100 bio labs; skimp on safety post-training; connect it to the Internet; let it develop affective relationships with the lab post-docs and its nihilist friends on Reddit; don't monitor it; have it work as a swarm to solve 'BioKillEveryoneGym' over a holiday break... In fairness, Pinker and Alexander do touch on social processes. But if one is concerned with scenarios like these, 90% of the debate should focus on the social processes by which we risk creating them - incentives, culture, guardrails, ... - and on why we won't adapt and reach a reasonable equilibrium somewhere along the way. To be clear, I'm very concerned about AI harms - including tail harms - but not very concerned about extinction risks. The burden is on Pinker and deep learning denialists to show that AI couldn't kill everyone under any circumstances (hard to defend); but the burden is on Alexander to show the social mechanisms by which we’d make the long sequence of low-probability unforced errors that would actually allow it to do this (seems hard to defend). We could kill a lot of people with nuclear power plants. Here's how: don't monitor them. Fire all their safety engineers. Build them in the middle of all of the world's population centers and then cut off their water supplies whenever city budgets run low. That we haven't done this is heartening; we should understand why. That there've been many accidents with dangerous technologies, especially in the earlier phases as we thrash towards a safety equilibrium, is disheartening; we should understand why those happened, too, and apply all those lessons to AI.3h

    1 Source

    Joshua Saxe@joshua_saxeReaction to the @slatestarcodex vs. @sapinker AI x-risk debate The main problem with this debate is that both interlocutors are mainly focused on AI as a disembodied technological object that's fallen down from heaven, not a congealed mass of human decisions. Scott has understood that there's nothing ontological about deep learning that'd prevent us from killing ourselves with it and that this risk gets worse as AI scales. But this is a trivial observation! Here's how to kill everyone with AI: put Opus 8.1-Killeveryone in charge of 100 bio labs; skimp on safety post-training; connect it to the Internet; let it develop affective relationships with the lab post-docs and its nihilist friends on Reddit; don't monitor it; have it work as a swarm to solve 'BioKillEveryoneGym' over a holiday break... In fairness, Pinker and Alexander do touch on social processes. But if one is concerned with scenarios like these, 90% of the debate should focus on the social processes by which we risk creating them - incentives, culture, guardrails, ... - and on why we won't adapt and reach a reasonable equilibrium somewhere along the way. To be clear, I'm very concerned about AI harms - including tail harms - but not very concerned about extinction risks. The burden is on Pinker and deep learning denialists to show that AI couldn't kill everyone under any circumstances (hard to defend); but the burden is on Alexander to show the social mechanisms by which we’d make the long sequence of low-probability unforced errors that would actually allow it to do this (seems hard to defend). We could kill a lot of people with nuclear power plants. Here's how: don't monitor them. Fire all their safety engineers. Build them in the middle of all of the world's population centers and then cut off their water supplies whenever city budgets run low. That we haven't done this is heartening; we should understand why. That there've been many accidents with dangerous technologies, especially in the earlier phases as we thrash towards a safety equilibrium, is disheartening; we should understand why those happened, too, and apply all those lessons to AI.3h