Satya Nadella wants companies to treat frontier AI models less like ordinary software and more like powerful insiders: useful, but capable of making mistakes or being compromised.
In an article published Saturday, the Microsoft CEO argues that organizations can no longer trace an AI system's behavior to a specific code path or reliably connect an output to particular training data and model weights. As agents gain access to sensitive information and permission to take important actions, he says assurances from the model provider are not enough.
Put the controls outside the model
Nadella's proposal separates the supply of intelligence from the authority over it. Organizations would set identity and access rules, limit privileges, log activity and define containment boundaries outside the model and the software harness directing its work.
He also calls for model diversity, continuous testing and independent audits. No model should be the sole dependency for an important outcome or verify its own work. Every meaningful action should leave tamper-proof, human-readable evidence, and an authorized person should be able to pause or shut down a model mid-task.
Nadella says chain-of-thought transparency can help but is not sufficient because model explanations are not consistently faithful. Having one opaque model check another can simply create layers of black boxes.
The framing drew enterprise-focused reactions. Box CEO Aaron Levie compared the idea to a zero-trust approach for AI, while investor Chamath Palihapitiya predicted that governance systems tracking changes against policies and procedures could become a new kind of system of record.
