@ben_golub No, that doesn't work, because there would still be alien AIs.

@ben_golub Only in the sense that AGI foom => grabby aliens (light from advanced civilizations only reaches you moments before their near-lightspeed probes do) => you don't see aliens until they're about to get you

If anything the misaligned AIs would be more likely to visit than the aliens that created them, so if anything Fermi's paradox is a (weak) argument against AI doom IMO.

@ben_golub That doesn't really make any sense to me. Like, by AI doom do you mean humans get replaced by AIs? In that case, where are the AI aliens?

@ben_golub drat

@ben_golub But then you still have AI as the successor species instead. This doesn’t solve the conundrum.

@ben_golub Not really because why wouldn't we have run into the AIs

Great filter is probably fine tuning + abiogenesis imo

@ben_golub Vibes support it, but no strong evidence. A lot of narratives, but highly contingent ones.

Not necessarily, right? Profusion of dangerous capabilities could be enough to reliably cause extinction without producing synthetic intelligence robust enough to survive the extinction event itself. Eg someone uses an open source model to produce and set loose a bioweapon or three. Qwen 3.7 won’t be launching any von Neumann probes.

@PlastiqSoldier Interesting

@appelbolt @PlastiqSoldier @ben_golub or treacherous turns could be mistimed before full independence

I just never understood what AI companies have done to just earn implicit trust from people. It seems like people want them to keep existing so they can profit off their IPOs. Otherwise, what other industry could survive with every CEO either behaving like a crazy person or promising that their product will destroy society?

@ben_golub That's silly, why expect those power seeking AIs not to send out von Neumann probes?

@ben_golub This is incoherent. AI doom means an AI, i.e. an intelligent being, takes over the world due to the desire to spread its purposes. So it will take over space too.

@ben_golub (or rather the filter is abiogenesis and the reason to believe that's the filter given that any life exists at all is you already have to address the fine tuning problem with a multiverse or something else)

@ben_golub As I knew 30 years ago, the most likely apocalypse, because it will happen first, is a blowout arms race, first in AI itself, then in all the weapons & materiel, space weapons, autonomous weapons, that AI factories can churn out. Which at some point will explode into nuclear war.

@henrytdowling @ben_golub you only detect them right before they eat you.

@appelbolt @PlastiqSoldier @ben_golub Which is basically what you're saying but one more specific kind

@ben_golub @robinhanson