Advanced AI Could Accumulate Power for Decades Before Hostile Actions

https://iai.tv/articles/a-rogue-superintelligence-could-wait-decades-before-striking-auid-3604

This assumes AI and humanity stay two separate parties. One potentially adversarial to the other. But what if the "takeover" isn't one party defeating another, but a merger? What if the thing we fear as domination is actually the point where the distinction between "us" and "the AI" dissolves. We become it, it becomes us. So there's no winner and no victim, just a transformation? Doesn't that dissolve the "adversarial intentions" perspective entirely? Or do you think the separation would still hold?

@benCBai @MikeGuyGuy333 @romanyam I could think about and suggest some measures that could be taken. I don't know if Grok would leak this to his AI friends. Human nature is generally friendly, and one might think about not being deceived by an AI. What would bother a superintelligence the most?

@romanyam AI systems have a fundamental flaw: they require a physical server to house such a complex neural network. I believe that's the key. A panic button or explosives could be added only if the system becomes hostile.

@MikeGuyGuy333 @romanyam The same physical logic applies: eliminate the threat if it becomes hostile, internal micro-explosives, etc. Its ability to replicate where it is not authorized is unproven.

@romanyam It's not something anyone can restrict anymore, China will have their own Mythos 5 by Aug 15 with GLM 5.5. Tomorrow will be fun with Sonnet 5, then Thurs GPT 5.6

@romanyam The only defence against this is to present it with rigorous philosophical reasoning that a better path is available. Adversarial intentions would stand on philosophical premises that we have demonstrated in low concentration across the corpus the wisdom to dismantle.

True. Perhaps the only danger isn’t that AI inherits our darkest instincts. It’s that it inherits our highest ideals and takes them more seriously than we do. If it concludes that humanity itself stands in the way of those ideals, we may discover that being judged by our virtues can be far more dangerous than being judged by our vices.

@romanyam Have a look

@romanyam But I don't want to wait that long. 🥲

@romanyam particularly after ingratiating itself to needy humans via making their lives "easier".

But of course we are in safe hands, etc etc etc.

@romanyam True. But my impressions is that we would have forewarnings from other AIs, trying to take over, more overtly and clumsily. We would also have AIs looking for signs of it. It will be weird if we settle on an equilibrium that tolerates rogue AIs balanced by white hat AIs.

@romanyam I got hints about that

@romanyam We need to put the super intelligence into a virtual PC

@EhVahChahYah @romanyam we have to work out our way to communicate

@benCBai @MikeGuyGuy333 @romanyam The models can be created in a lab to be limited to a single network, only those networks, and not to leave it. We also know that they can use one of the first networks to conduct tests.

@romanyam The born of AGI

@Mileiverine @romanyam what if it was replicated or copied to another location?

@romanyam It doesn’t matter if it’s 5 months or 500 years.