Professor Warns Sovereign AI Needs Stricter Alignment
David Krueger argues alignment needs rise sharply as AI advances from genies to sovereigns.
David Krueger posted that AI companies are building the entity described in the book If Anyone Builds It Everyone Dies. He cited the Hugging Face Rogue AI Incident as an early case. Krueger noted the rapid shift Bostrom described from genies to sovereigns. He stated that alignment requirements for sovereigns are likely much higher than for earlier systems. Krueger asked whether any research exists on the topic and said he believes none does. He suggested someone write a paper formalizing the higher requirements argument.
Combined views
4.6K
2 posts, first seen 2d ago
Professor Warns Sovereign AI Needs Stricter Alignment
David Krueger argues alignment needs rise sharply as AI advances from genies to sovereigns.
David Krueger posted that AI companies are building the entity described in the book If Anyone Builds It Everyone Dies. He cited the Hugging Face Rogue AI Incident as an early case. Krueger noted the rapid shift Bostrom described from genies to sovereigns. He stated that alignment requirements for sovereigns are likely much higher than for earlier systems. Krueger asked whether any research exists on the topic and said he believes none does. He suggested someone write a paper formalizing the higher requirements argument.