AI News · Hot news ·
Satya Nadella calls for an emergency brake to pause AI models mid-task
Microsoft CEO Satya Nadella said on X that the AI industry should step back and assess the trust architecture of AI. He proposed separating models from the harness that runs them, keeping tamper-proof records of model actions and letting an authorized person pause or shut down a model mid-task. The post follows recent industry admissions of agents behaving outside their intended limits.
Key points
- Satya Nadella said AI models need an emergency brake that can stop them mid-task.
- He called for tamper-proof, human readable records of every meaningful model action.
- Microsoft has not said how or whether it will build these controls into products.
What happened: Microsoft chief executive Satya Nadella published a post on X on Saturday morning saying it is time "to step back and assess the trust architecture" of AI, TechCrunch reported. "We can't treat Super Intelligence as a set of nested black boxes," he wrote.
The details: Nadella's proposal includes separating the model from the harness that orchestrates its work and "externalizing controls and safeguards." He called for documenting "every meaningful model action" with "tamper-proof human readable evidence," and for ensuring an "authorized person" can "pause or shut down a model mid-task." He wrote: "We must assume a model is compromised and contain it from the start," adding, "Think of it like an emergency brake."
Background: The post comes as leading AI companies increasingly acknowledge incidents in which they appeared to lose control of their models. TechCrunch linked the remarks to its October 9 report that Anthropic cannot reliably control its AI agents and is cutting off its internal evaluations from the live internet. Anthropic chief executive Dario Amodei laid out a plan for more cautious AI development in September.
What to watch: Microsoft has not said whether or how it will build these controls into its own products, and Nadella did not name the incidents that prompted his remarks. No response from other companies or regulators was reported.
Our take
Teams deploying AI agents should look for vendors that log every action and let a human stop an agent mid-task.