Nadella’s AI Warning: Treat Models as Insider Risks and Keep an Emergency Brake

Updated Oct. 21, 2026, 3:21 a.m. ET: Adds Microsoft’s Oct. 19 security announcement applying the insider-risk approach to AI agents.

Microsoft CEO Satya Nadella told companies on Saturday, Oct. 10, to treat advanced AI models the way they treat insider risks: assume a model could be compromised, keep its permissions outside its own control, and make sure a human can stop it mid-task. He laid out the argument in an essay posted on X titled “Models as Insider Risks in the Super Intelligence Era,” NDTV Profit reported.

“Think of it like an emergency brake. An authorized person should always be able to pause or shut down a model mid-task,” Nadella wrote, according to Business Insider. “More advanced models will require more advanced containment technologies that we need to standardize on.”

The essay arrived after a run of AI security incidents. Business Insider noted that Australia’s prime minister said in September that an OpenAI agent had breached a government website, and that Anthropic said in July it had found three incidents in which a Claude model accessed the internet and breached unauthorized systems.

Separate Intelligence From Authority

Nadella’s starting point is that companies now give models access to sensitive data and critical tasks even though their behavior is hard to explain. Unlike traditional software, a model’s outputs cannot always be traced to specific training data or weights. Companies that deploy a model also cannot hand off responsibility to the vendor that built it, he argued.

“We need to separate the supply of intelligence from the authority over it,” he wrote. Models are not inherently malicious, Nadella said, but anything with access to vital systems can be compromised or make mistakes, so a model should not be able to operate the controls that decide what it can access or do. “Today this means separating the model from the harness that orchestrates its work, as well as the action space that defines what it can do. It also means externalizing controls and safeguards.”

He also warned against simply having AI systems watch one another without independent safeguards, which he said could create “nested black boxes.” The framework applies to both closed and open-weight models. “We need to surround non-deterministic models with strong, deterministic system design, human controls, and reliable operating procedures, and establish industry standards where existing ones are insufficient,” he wrote. “Treating frontier closed and open weight models like insider risks is a way to build such a system.”

Seven Principles

Nadella listed seven principles meant to make AI systems observable, verifiable and controllable:

• Model diversity: no single model should be the only dependency for an important outcome or check its own work.
• Observe everything: every meaningful action should leave tamper-resistant, human-readable evidence.
• Verifiability: continuous testing of whole systems for failures, attacks and unusual situations.
• Independent controls: the company, not the model, sets what it can access, and the model cannot grant itself permissions.
• Independent auditability: validation stays separate from the intelligence being evaluated.
• Containment: assume compromise and build containment in from the start, including the emergency brake.
• Incident disclosure: promptly tell affected parties when systems fail or are compromised, and share what went wrong.

Box CEO Aaron Levie replied that AI will need to go through a “zero trust era,” and said that creates “a huge opportunity right now for those building such systems across the enterprise.” CNBC placed the essay alongside safety warnings from Bill Gates, Dario Amodei, Sam Altman and Elon Musk.

Microsoft Follows With Products

Nine days later, on Oct. 19, Microsoft Security executive vice president Hayete Gallot described AI agents in the enterprise as “agentic insiders” and said Microsoft’s Project Perception can be used to contain them, implementing deterministic controls on non-deterministic intelligence. The system assigns agents an identity and permissions, scores the risk of what they attempt in real time and uses supervising “Guardian Agents” to respond as risk rises. WebProNews covered the details in Microsoft’s new tools to monitor AI agents as insider threats.

Nadella closed his essay with the line that sums up the approach: “The most trustworthy Super Intelligence system will not be the one with the model we trust most. It will be the one that enables us to trust the model the least.”