Séb Krier Urges Broader AI Alignment Beyond Individual Models
The DeepMind policy lead published a post arguing alignment must cover systems and institutions in multi-agent settings.
TLDR
Séb Krier, AGI Policy Development Lead at Google DeepMind, posted that alignment extends beyond model properties to the systems, institutions, protocols, and mechanism design governing multi-agent environments. His essay, titled Of Swarms and Sand Gods and published on the Cosmos Institute blog, states that better harnesses and protocols will be critical for safety. Ryan Lowe replied that the view has moved from rare to commonplace within a year. Replies on X largely praised the framing.
Combined views
65.7K
7 Sources, first seen ago
Sources
RL 
Ryan Lowe@ryan_t_lowea year ago this take was rare, now it seems almost commonplace. huge W for the discourse https://twitter.com/sebkrier/status/2090836044249711009
- likes: 44
- replies: 3
- bookmarks: 16
- reposts: 4
JP Jason Potts@profjasonpottsReally like this. The multi-agent decentralized cooperation approach to safety has the advantage of drawing on theoretical insights from institutional Econ, complexity theory, public choice theory, Austrian Econ etc. we do already know a lot about this class of problems…
- likes: 10
- replies: 0
- bookmarks: 3
- reposts: 2
DR Deb Raji@rajiinioThis was published almost exactly three years ago, in 2023. Earlier that same year of 2023, @AlondraNelson46 gave the keynote @FAccTConference, literally titled "Thick Alignment". This has been so far from a "niche" view!…
- likes: 0
- replies: 1
- bookmarks: 0
- reposts: 0
M3 M3taversal@m3taversalThis is a really useful essay. I think the move from alignment as a property of individual models to alignment as a property of the institutions around them is exactly right. Harnesses, constrained communication, specialized judges, and permissioned action spaces give us a way…
SK Séb Krier@sebkrierYes I agree! I think there are two parts to the harness thing: "So at lower levels, harnesses help an organization bound its own models; but externally, where other malicious agents might live, verifiable protocols can help you rely on (for example) cryptographic attestation…
RL Ryan Lowe@ryan_t_lowe@sebkrier and @cosmos_inst deserve a lot of credit for this of course, though I like to flatter myself by thinking this paper played a role as well