AI News · Good news ·
Anthropic gives METR researchers employee-level access to assess its AI
Anthropic is giving six researchers from the nonprofit METR employee-level access to independently assess its models, safety practices and risk management. METR gets company laptops, office space and internal tools, and Anthropic has given up the right to block its published findings. Redactions are allowed only in narrow, predefined categories.
Key points
- Six METR researchers get laptops, office space and internal tool access at Anthropic.
- Anthropic cannot block METR's publications but may redact narrow categories such as customer data.
- Anthropic, OpenAI, Google, Meta, Nvidia and xAI signed a White House accord on independent review.
What happened: Anthropic said in a blog post on Thursday that it is giving six researchers from METR, a nonprofit that tests frontier AI models, employee-level access to independently assess its models, safety practices and risk management, TechCrunch reported. The researchers receive Anthropic-issued laptops, office space and access to internal tools roughly on par with those used by Anthropic's own risk assessment teams.
The details: METR will publish its own assessments of Anthropic's risks, incidents and practices. Anthropic has given up the right to block publication, but may require redactions in narrow, predefined categories such as customer data, attorney-client privileged material and model architecture details. "METR is free to characterize our redactions publicly," the company said. Anthropic runs a three-tier evaluator system, and METR is currently the only evaluator at the top tier. Lower-tier evaluators, such as biosecurity nonprofit SecureBio, need approval to publish.
Background: The move follows a promise by CEO Dario Amodei in a September essay to embed outside evaluators inside AI labs. Anthropic compared the setup to banking regulators placing resident examiners inside large banks. OpenAI pledged the same, and Anthropic, OpenAI, Google, Meta, Nvidia and xAI have since signed a White House accord committing to independent review.
What to watch: Anthropic has not disclosed how long the arrangement lasts or which models METR will assess. The first METR reports will show how much detail outside evaluators can actually share.
Our take
Independent, publishable audits of AI vendors give buyers better evidence when judging the safety and risk claims of the tools they adopt.