Six evaluators with company laptops
Anthropic has given a team from METR, the nonprofit that evaluates frontier AI models, what it calls “employee-like access” to the company. Six METR employees now have Anthropic hardware, office space and internal tools roughly comparable to those used by Anthropic’s own risk assessment teams, and they will publish the results of their assessments. Anthropic said it cannot veto those publications.
The company frames the move as a response to a problem it created for itself: it wants to be publicly accountable for its safety work, but much of the evidence cannot be published, either because it would help the people the safeguards are meant to stop or because it is commercially sensitive. Embedding evaluators, Anthropic argues, closes the gap between what it knows and what anyone outside can verify. It compares the arrangement to the resident examiners banking regulators place inside banks, while noting the analogy is imperfect.
What the evaluators can and cannot see
The access is broad but bounded. Anthropic lists three exclusions: customer data, attorney-client privileged material, and model architecture details, which it says carry a high leak risk and are usually of low relevance to risk assessment. Where architecture does turn out to matter, it can be shared with explicit internal approval.
On publication, Anthropic says it cannot block METR’s output, but it can require redaction within narrow, predefined categories. METR is free to say publicly whether it considers a redaction justified, without revealing what was removed. Only a small, contractually bound group inside METR works on the arrangement; the rest of the organisation is walled off from it.
Three tiers, and who sits in each
Anthropic has formalised the access into three tiers. Tier 1 covers narrowly scoped investigations, where Anthropic chooses what the evaluator sees and has to approve anything published. Tier 2 widens the information and drops the approval requirement, keeping only the redaction categories. Tier 3 is the METR arrangement — employee-like access with Tier 2 publication rights.
SecureBio sits in the first tier and is reviewing the chemical and biological weapons chapter of Anthropic’s August Risk Report, having already published a pilot review of the previous one. Accenture and Faculty are in talks over a broader partnership covering model evaluation, red-teaming, alignment assessments, safeguard testing and capability building. Hugging Face is among several organisations in early discussions. Anthropic says it hopes several evaluators eventually reach Tier 3, each publishing its own reading of the company’s risk posture.
The standard Anthropic is setting for itself
The company sets out four things it thinks an evaluator needs: expertise in assessing frontier models, independence — no equity in the company, no financial or operational dependence on it — enough access to evaluate training, safeguards and models, and the freedom to publish what it finds.
Anthropic signed the White House Accord on Super Intelligence, in which companies pledged to work with independent external auditors, and committed last month to stronger third-party assessment. This is the first concrete shape that commitment has taken. The company closes by saying it is doing this voluntarily and that “every frontier AI company should do the same.”
The test of the arrangement is what METR publishes, and whether its first report reads differently from the ones Anthropic writes about itself. Nothing in the announcement says when that report is due.
