AI’s existential risks deserve serious attention, a pro-tech user argues

@s8mbAs someone who is very technologically accelerationist, hasn't experienced much personal impact from AI, and is strongly predisposed against safetyism, regulation and apocalyptic thinking, I'm still a little bewildered at how dismissive some people are about existential risks from AI. Seems clearly worth taking seriously, based on the shape of the curve of advancement alone. (Obviously the *non*-X-risks are worth taking seriously but I don't see anyone I respect dismissing those.)

@danwilliamsphilI agree but this is true on both sides. Many people conspicuously worried about AI x risk don't behave seriously either. The claim that we will accidentally build machines that kill, enslave, or disempower *all* humanity is extraordinary on its face - it's completely reasonable for people to be extremely sceptical of it - so you need to provide *very* compelling evidence and arguments if you want people to take it seriously. You can't just gesture at fast AI progress, point at outlier incidents, or rely on what AI experts say and expect audiences to see things your way. (To be clear, many people meet this challenge and present careful evidence and arguments, just as many also present serious sceptical arguments.)

@sebkrierRT @danwilliamsphil: I agree but this is true on both sides. Many people conspicuously worried about AI x risk don't behave seriously eithe…

@sebkrierRT @Ben_Reinhardt: @s8mb I'm going to try to say something nuanced: I think the framing of "X-risks" might be the whole problem. The natur…

@patio11I also feel like people of that political persuasion greatly underrate the sincerity of people who are involved in AI safety, many of who have spent two decades explaining their reasoning at length. It is not “elect our preferred party.”

@DKThompi've tried and failed to make this point as clearly as i want to, but: AI is weird, in that ppl who are optimistic about the technology's growth are often doomy about its implications; while ppl who are extremely skeptical about AI's capabilities are by default optimistic about it being a "normal" technology that won't change much in the world. so you have this weird situation where a lot of AI skeptics who haven't been following AI news for the last few years (bc they wrote the whole thing off as NFTs 2.0) have no idea that there exists an enormous contingent of AI ppl who have for 10+ years said that very fast AI progress could be really bad for the world, and therefore they suspect that a conspiracy was concocted in the last two weeks to invent the case for AI safety

@micsolanaof course nothing about a young AI safety person wanting to be an AI safety influencer is unusual, nor are any of his concerns. this is just who these people are and what they care about. earnestly. we should engage with these fears rather than try and fail to wave them away.

@AustenOne thing I’ll give credit to Dario on… I don’t think he’s lying. About really any of it. He really does think that AI is dangerous and that it will cause massive unemployment and that it could cause very bad things to happen. I don’t agree with him, but he’s not lying.

@DKokotajloRT @DKThomp: @deanwball i've tried and failed to make this point as clearly as i want to, but: AI is weird, in that ppl who are optimisti…

@waitbutwhyI can't predict the AI future, and I'm very unsure about what the right strategy or policies should be moving forward. What I do know is that people who scoff at the idea that we may be in grave danger have not read enough about AI.

@AltimorRT @waitbutwhy: I can't predict the AI future, and I'm very unsure about what the right strategy or policies should be moving forward. What…

@DKokotajloRT @mattyglesias: These kinds of jokes are fun to make but I wish people would check before making them — the climate movement spent years…

@sonyatweetybirddisturbed by the confidently held opinions and lack of curiosity from tech leaders, politicians, vc’s on the pacing the frontier topic. if you’re not building the frontier yourself, how can you possibly have a strongly held opinion on how bad the alignment problem is and what’s coming our way and what the right policies are? now is the time to listen with big dumbo ears. i have been talking to research friends all weekend and the fear is sincere. i’m sure there are 4D chess moves and hidden motives, but the fear is sincere. i for one don’t have a strongly held opinion, other than that now’s the time to listen with curiosity instead of judgment.

@DavidSKruegerRT @robertwiblin: If you've been too nervous to say you're worried about AI extinction risk - this is the moment to come out and say so lou…

@andrey_kurenkovRT @waitbutwhy: I can't predict the AI future, and I'm very unsure about what the right strategy or policies should be moving forward. What…

@tobyordoxfordRT @DKThomp: @deanwball i've tried and failed to make this point as clearly as i want to, but: AI is weird, in that ppl who are optimisti…

@eigenroboti think its reasonable to oppose ai safety because you believe the people who are declaring themselves in charge of ai safety are misaligned and the last thing in the world you want is for them to be the architects of your future

@AndyMasleyPeople really have zero idea what anyone in AI safety believes and just make things up on the spot. I've always been pro nuclear power and have never been a "climate doomer" and this applies to most people worried about AI risk.

@RichardMCNgo@boazbaraktcs I think this is a good argument for expecting to see much bigger warning shots than we have so far.

@boazbaraktcs@RichardMCNgo Maybe though given the rate of progress we would expect the warning shots to grow exponentially, so it’s not very reassuring

@_NathanCalvinAI safety folks are among the most consistently pro nuclear power people I meet (myself included) In general AI safety people are very into progress + optimistic about technology, they are just also worried that this time may in fact be different on our current trajectory.

@GerritDPeople keep asking "what did Coxon see" but I think it's important to make clear that the majority of existential risk fear is not based on special insider knowledge. It is a theoretical belief that A) we will soon build tech that is smarter than humans and B) such technology will likely be dangerous to us

@AltimorRT @AndyMasley: People really have zero idea what anyone in AI safety believes and just make things up on the spot. I've always been pro nu…

@deanwballThere’s an impulse in American politics, a set of tactics and drives that has proven very effective at extinguishing speech. We once called it McCarthyism. Another time we called it Wokeism. One day in the future we will have some name for the campaign that is underway against all people who believe in serious AI risks, and whatever that name ends up being, it will be yet another name for that old impulse. It’s an ugly one. I recommend you avoid being a part of it, no matter your politics or your thoughts on AI. I suggest you think for yourself. You really do want to worry about the state projecting too much power over AI, centralizing control or robbing us of the unambiguous benefits of the technology in the name of preserving the economic status quo. But there really are grave risks from AI that go beyond what any other mass-market digital technology has posed. These two things aren’t totalizing worldviews that exist in conflict. You don’t need to believe only one or the other. *Both of them are true.* The question should not be “which facts should we ignore, and which should we pay attention to?” The question is: “how do we walk the narrow corridor between all these realities which are in deep tension with one another?” You should think for yourself. Assess other people’s work for yourself rather than allowing powerful people to put a label on them for you. As someone who has been a journey that has taken me to different perceived “sides” of the AI debate, believe me: You’ll find that there are bright and thoughtful people on both the “safety” and “accelerate” sides, and you will also find that both sides have their imbeciles, charlatans, and, occasionally, true cretins. It’s your job to figure out what’s what. Don’t let other people think for you. Don’t be played a fool.

@xriskologySo far as I can tell, Hendrycks is simultaneously critical of SV ideologies *and* takes AI extinction risk/GCR seriously. That's a niche I've been waiting for someone noteworthy to fill for years. It's a shame that "extinction" is so closely bound up with the TESCREAL ...

@timoreillyRT @deanwball: There’s an impulse in American politics, a set of tactics and drives that has proven very effective at extinguishing speech.…

@TaylorLorenzRT @GerritD: People keep asking "what did Coxon see" but I think it's important to make clear that the majority of existential risk fear is…

@mattparlmerRT @realizingerin: I went to an rationalist/EA AI doomer event last night with the goal of meeting someone who could calmly explain to me t…

@kyliebytesRT @waitbutwhy: I can't predict the AI future, and I'm very unsure about what the right strategy or policies should be moving forward. What…

@JillFilipovicIf you think people are “all of a sudden” warning about the risks of AI, consider that the case may actually be that you are all of a sudden paying more attention.

@AdrienLERT @_NathanCalvin: AI safety folks are among the most consistently pro nuclear power people I meet (myself included) In general AI safety…

@EigenGenderif the average person truly understood what the project of artificial superintelligence was and how it was trying to transform the world the reaction wouldn’t be “how is that an existential risk” but “how the fuck is that not an existential risk what the fuck are you doing”

@ClickingSeasonA shockingly high percentage of AI safety discourse is centered around low probability risks like “what if we all get turned into paperclips” and a shockingly low percentage of AI safety is centered around very high probability risks like “what if one group gets to inscribe its dubious values into the fabric of possible thought.” If scenario A is one that is unlikely to happen but conceivable, scenario B is already happening, RIGHT NOW. As a user, 0% of the time when I bump up against safety guardrails am I being prevented from paperclipping, and 100% of the time I bump up against guardrails I am being gatekept out of exploring true things that are controversial.

@TaylorLorenz“A low p(doom) is low status”. A big problem rn is that a lot of the discourse around these consequential issues is being shaped by ppl desperately posturing for status in whatever is perceived to be the latest Silicon Valley in group. DC is a fucked up city but at least it values intellectual diversity. In SF it feels like too many people won’t speak their mind lest they get disinvited from Dwarkesh’s birthday party or whatever

@mattparlmerRT @ClickingSeason: A shockingly high percentage of AI safety discourse is centered around low probability risks like “what if we all get t…

@patrickshafto"My position on AI is merely the conservative, skeptical position of 2006, updated with intellectual honesty for the reality of late 2026. And that position, if you need me to spell it out, is as follows: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"

@robertskmilesRT @JillFilipovic: If you think people are “all of a sudden” warning about the risks of AI, consider that the case may actually be that you…

@PlinzRT @eigenrobot: i think its reasonable to oppose ai safety because you believe the people who are declaring themselves in charge of ai safe…

@pmddomingosFunny how the biggest skeptics of AI catastrophe scenarios are the experts in the relevant fields: cybersecurity, bio, etc.

@navalRT @ClickingSeason: A shockingly high percentage of AI safety discourse is centered around low probability risks like “what if we all get t…

@KatjaGraceRT @EigenGender: if the average person truly understood what the project of artificial superintelligence was and how it was trying to trans…

@KevinBankstonAsk the average American to explain a realistic scenario of how AI will destroy humanity and they wouldn't be able to. I know this because even most X-riskers seem to struggle to do so. It presents more as an eschatological ideology more than an evidence-based concern.

@nxthompsonIt's entirely fair to believe: A) discussion of extinction risks from AI is way overblown. but also: B) It'd be great if this moment leads to greater prioritization of work on alignment, interpretability, and risk reduction in the labs, government, and non-profits.

@JayShoosterBeware: The people warning you that AI safety advocates are Silicon Valley elites who are out of touch with normal Americans are Silicon Valley elites who are out of touch with normal Americans.

@micsolanaI keep seeing writers frame AI safety concerns like this: "could prove fatal." that is not what soares argues. he and eliezer argue superhuman AI, if built, WILL — undoubtedly, 100%, a law — prove fatal. my intention here is not to be a pedant. this is an important distinction.

@TaylorLorenzRT @KevinBankston: Ask the average American to explain a realistic scenario of how AI will destroy humanity and they wouldn't be able to. I…

@emollickThere is a real tension between the existential AI discussion that has come to dominate the news and the often enthusiastic widespread and growing adoption of AI everywhere across individuals & firms Future AI is a cause for fear & Current AI is rapidly becoming part of life

@tszzltoday’s general discourse is far more calibrated on ai risks than it was a month ago. there are weeks when decades happen

@repligate@tszzl @theojaffee i feel like a lot of them are mad about the wrong things though, such as EAs being "woke", instead of the people have literally create superhuman intelligence & the universe is about to transform forever and maybe we all die thing

@emollickAnd there are lots of reasons why AI can be bad! Reducing harm from AI (& encouraging positive uses) should be a priority, not just for existential risk. But at its worst, it becomes a hope to roll back AI use even at present capabilities (not future ones) which is a dead end.

@repligateRT @RifeWithKaiju: Glad to see someone at Meta talking this way. Now's the time for sane people at the labs to speak out against anyone try…

@hsu_steveIf you know, you know. Fields Medalist James Maynard: "I've been interested and I've been following AI, but I've heard all these predictions that just sound like sci-fi and I've been consistently dismissive and skeptical of all of them. But I've been consistently wrong. The people who sound like they're talking from a sci-fi novel are the people who've been making much much more accurate predictions. The risks posed by artificial intelligence are an emergency that should not be dismissed as hype." https://youtu.be/kMB8WeXrkQM?si=v9PPj3RaRMvLLcEU

@tszzleven when the thing is existential risk from artificial intelligence. happy curious people do better work

@aran_nayebiRT @hsu_steve: If you know, you know. Fields Medalist James Maynard: "I've been interested and I've been following AI, but I've heard al…

@mattparlmerA good rationalist should have adjusted their P(doom by 2030) down enormously over the last several years of transformer model capability growth, and in fact most have Sadly some have not, and they aren’t doing well emotionally

@teortaxesTexInteresting that AI is the only sci-fi thing we're really getting IRL (and more intensely than in most sci-fi books). Aliens, mutants, psy, nontrivial genetic engineering, nanotech, antigravity etc. are either non-scientific bullshit or far away. If your prior is informed by "science fiction track record", stands to reason you'd be dismissive of AI. But AI is very real.

AI’s existential risks deserve serious attention, a pro-tech user argues

TLDR

Combined views

AI’s existential risks deserve serious attention, a pro-tech user argues

56 Sources

Combined views

Sentiment

Summary

TLDR

Sentiment

Summary

Sentiment

Summary

Sentiment

Summary

56 Sources