AI News · Hot news ·
Anthropic drafts AI safety bill that could halt models posing catastrophic risk
Anthropic has been shopping a draft federal AI safety bill to Congress that would let a court block a model's release if it poses a foreseeable, imminent risk of more than 50 deaths or over $1 billion in damage, Politico first reported. Independent evaluators registered with the Commerce Department would assess the risk. The bill would also bar federal agencies from halting a model on their own.
Key points
- The draft targets imminent risks of over 50 deaths or over $1 billion in property damage.
- Evaluators registered with the Commerce Department would have 72 hours to report a catastrophic risk.
- A D.C. judge, not a federal agency, would decide whether to halt a model's release.
What happened: Anthropic has proposed a draft AI safety bill that would allow a model's release to be blocked in certain catastrophic risk scenarios, TechCrunch reported on October 20, citing Politico, which first reported the draft. A company spokesperson confirmed the draft to Politico and said it "represents the proposals outlined in Anthropic's Advanced AI Framework," which the company has promoted to federal and state lawmakers since June.
The details: The bill covers a "foreseeable, imminent risk" where more than 50 people could die or more than $1 billion in property damage could result. The risk must come from the creation or release of a biological weapon, a model evading its developer's controls, or a cyberattack. Independent evaluators registered with the US Commerce Department would assess the risk, and labs could pay them as long as payment does not depend on the results, according to Politico. If evaluators find an imminent catastrophic risk, they would have 72 hours to notify Commerce, which would then ask a judge in Washington, D.C., to halt the release. Federal agencies would not be able to stop a model's distribution on their own.
Background: TechCrunch said the thresholds resemble those in California's Senate Bill 53. Labs have generally resisted mandatory government oversight, and Nvidia CEO Jensen Huang has argued the industry needs no new AI laws.
What to watch: The bill's status in Congress has not been disclosed. TechCrunch also noted it is unclear how the payment model would keep evaluations independent.
Our take
If a bill like this advances, release timelines for frontier models could start to depend on third-party evaluators and court review.