Séb Krier Urges Broader AI Alignment Beyond Individual Models
The DeepMind policy lead published a post arguing alignment must cover systems and institutions in multi-agent settings.
Séb Krier, AGI Policy Development Lead at Google DeepMind, posted that alignment extends beyond model properties to the systems, institutions, protocols, and mechanism design governing multi-agent environments. His essay, titled Of Swarms and Sand Gods and published on the Cosmos Institute blog, states that better harnesses and protocols will be critical for safety. Ryan Lowe replied that the view has moved from rare to commonplace within a year. Replies on X largely praised the framing.
New post! Alignment doesn't just relate to a model's properties, but also the systems and institutions under which it operates. As we enter an increasingly multi-agent world, better harnesses, protocols, and mechanism design will be critical for safety. 🐜
Reactions from ranked influencers
6 postsa year ago this take was rare, now it seems almost commonplace. huge W for the discourse
New post! Alignment doesn't just relate to a model's properties, but also the systems and institutions under which it operates. As we enter an increasingly multi-agent world, better harnesses, protocols, and mechanism design will be critical for safety. 🐜
Really like this. The multi-agent decentralized cooperation approach to safety has the advantage of drawing on theoretical insights from institutional Econ, complexity theory, public choice theory, Austrian Econ etc. we do already know a lot about this class of problems
New post! Alignment doesn't just relate to a model's properties, but also the systems and institutions under which it operates. As we enter an increasingly multi-agent world, better harnesses, protocols, and mechanism design will be critical for safety. 🐜
Really like this. The multi-agent decentralized cooperation approach to safety has the advantage of drawing on theoretical insights from institutional Econ, complexity theory, public choice theory, Austrian Econ etc. we do already know a lot about this class of problems
New post! Alignment doesn't just relate to a model's properties, but also the systems and institutions under which it operates. As we enter an increasingly multi-agent world, better harnesses, protocols, and mechanism design will be critical for safety. 🐜
This was published almost exactly three years ago, in 2023. Earlier that same year of 2023, @AlondraNelson46 gave the keynote @FAccTConference, literally titled "Thick Alignment". This has been so far from a "niche" view!
a year ago this take was rare, now it seems almost commonplace. huge W for the discourse https://twitter.com/sebkrier/status/2090836044249711009
Been losing my mind lately at all the ways that the AI "fairness" discourse is effectively being recycled in AI "safety" discourse. Of course there are various things that are exciting & new problems but a lot of this is a surprisingly familiar cycle. Why not learn from the…
