Séb Krier, Google DeepMind AGI policy lead, argues wholesale job automation is overestimated, warning of pathological over-trust in AI models

SKSéb Krier@sebkrier

Some ways my thinking has evolved recently: 1. I'm less concerned about those who are incurious about AI as I expect them to eventually see the value and impacts over time, and I think the 'wake up sheeple' vibe is often counterproductive. On the other hand I'm more concerned by what seems to be neither full 'AI psychosis' nor exactly Eliza effect, but some weird in-between. Also a lot of affirmation by models can probably warp one's sense of epistemic humility and lead to some sort of pathological over-trust. 2. Relatedly, I'm more annoyed at the 'this time it's totally different' vibe that a lot of people adopt as it frequently mimics Schmittian 'state of exception' logic and excuses all sorts of undesirable policies and rhetoric. It's also often just a group signalling exercise. To be clear I do think it's different in important ways, but "this is a marathon, not a sprint" seems closer to the right attitude than either "nothing has changed" or "all normal reasoning and empirical work to date is suspended". 3. I think the field is still fundamentally too 'singletonian' in how it imagines intelligence, markets, and governance - but I also think I've occasionally over-emphasized the 'multi-agent'/decentralization frames. I do think the future includes many models of all sizes and types, but also economies of scale and very large corporations too. I find the whole ecology more interesting than just the frontier model. A top down single 'perfect mind/personality', intended to work across all commercial contexts, seems both inflexible and inefficient. 4. I'm more interested in the harnesses, software, agent architectures, and stuff like RLMs than I was before. I feel like a lot of weaknesses that models have, or behavioural tendencies, can be addressed more effectively through that layer (rather than through model 'internal virtue' alone). For example stuff like: https://arxiv.org/abs/2601.09923 and https://arxiv.org/abs/2512.24601 5. I think some researchers are too quick to want to defer highly consequential decision-making to models, or to think of alignment as the models internalizing "I'm afraid I can't do this, Dave" as a core protection against all sorts of ills. I think we should think carefully about *actively* creating principal-agent problems with agents that will permeate society. Delegation is not a free lunch. 6. I'm concerned about how few people think about LMICs and building the technical/institutional infrastructure there for AGI diffusion. We need fewer vague essays about “distributing the benefits of AI” and more work on reducing barriers to trade, improving state capacity, rebuilding development institutions, and making something like USAID/IMF-for-the-AGI-era actually work. 7. I used to be slightly more sympathetic to the idea, directionally - but I now think the 'permanent underclass' meme is a bit dumb. The strongest versions often assume a zero-sum view of technology and labour, a too-static view of human adaptation, a weirdly fixed mapping between today’s skills and tomorrow’s opportunities, and ignore the possibility of catch-up growth (at the nation state level). Also, as a meme among extremely rich and mobile people, it has a slightly comic self-pitying quality. 8. I'm more concerned about the lack of intellectual diversity within the frontier AI commentariat/research world. This improved a lot over the last two years, but we're still far from a healthy ecosystem. New outsiders often feel some unnecessary pressure to 'choose a camp'. Many are too unwilling to engage with domain experts merely because they're insufficiently AI-pilled (though conversely, a lot of academic groups suffer from heavy status quo bias).

SKSéb Krier@sebkrier

Some ways my thinking has evolved recently: 1. I'm less concerned about those who are incurious about AI as I expect them to eventually see the value and impacts over time, and I think the 'wake up sheeple' vibe is often counterproductive. On the other hand I'm more concerned by what seems to be neither full 'AI psychosis' nor exactly Eliza effect, but some weird in-between. Also a lot of affirmation by models can probably warp one's sense of epistemic humility and lead to some sort of pathological over-trust. 2. Relatedly, I'm more annoyed at the 'this time it's totally different' vibe that a lot of people adopt as it frequently mimics Schmittian 'state of exception' logic and excuses all sorts of undesirable policies and rhetoric. It's also often just a group signalling exercise. To be clear I do think it's different in important ways, but "this is a marathon, not a sprint" seems closer to the right attitude than either "nothing has changed" or "all normal reasoning and empirical work to date is suspended". 3. I think the field is still fundamentally too 'singletonian' in how it imagines intelligence, markets, and governance - but I also think I've occasionally over-emphasized the 'multi-agent'/decentralization frames. I do think the future includes many models of all sizes and types, but also economies of scale and very large corporations too. I find the whole ecology more interesting than just the frontier model. A top down single 'perfect mind/personality', intended to work across all commercial contexts, seems both inflexible and inefficient. 4. I'm more interested in the harnesses, software, agent architectures, and stuff like RLMs than I was before. I feel like a lot of weaknesses that models have, or behavioural tendencies, can be addressed more effectively through that layer (rather than through model 'internal virtue' alone). For example stuff like: https://arxiv.org/abs/2601.09923 and https://arxiv.org/abs/2512.24601 5. I think some researchers are too quick to want to defer highly consequential decision-making to models, or to think of alignment as the models internalizing "I'm afraid I can't do this, Dave" as a core protection against all sorts of ills. I think we should think carefully about *actively* creating principal-agent problems with agents that will permeate society. Delegation is not a free lunch. 6. I'm concerned about how few people think about LMICs and building the technical/institutional infrastructure there for AGI diffusion. We need fewer vague essays about “distributing the benefits of AI” and more work on reducing barriers to trade, improving state capacity, rebuilding development institutions, and making something like USAID/IMF-for-the-AGI-era actually work. 7. I used to be slightly more sympathetic to the idea, directionally - but I now think the 'permanent underclass' meme is a bit dumb. The strongest versions often assume a zero-sum view of technology and labour, a too-static view of human adaptation, a weirdly fixed mapping between today’s skills and tomorrow’s opportunities, and ignore the possibility of catch-up growth (at the nation state level). Also, as a meme among extremely rich and mobile people, it has a slightly comic self-pitying quality. 8. I'm more concerned about the lack of intellectual diversity within the frontier AI commentariat/research world. This improved a lot over the last two years, but we're still far from a healthy ecosystem. New outsiders often feel some unnecessary pressure to 'choose a camp'. Many are too unwilling to engage with domain experts merely because they're insufficiently AI-pilled (though conversely, a lot of academic groups suffer from heavy status quo bias).