Anthropic’s chief executive wants the industry to slow down
Anthropic chief executive Dario Amodei published an essay on 12 September, We Must Pace the Frontier, arguing that frontier AI developers should deliberately slow how quickly their models gain capabilities. “We must slow the pace at which we improve the capabilities of AI models,” he wrote.
The essay follows Bloomberg’s report on 11 September that Sam Altman told OpenAI employees at a company-wide meeting that OpenAI is open to slowing development, possibly alongside other labs. The heads of two of the leading US labs are now publicly arguing for the same thing. What neither can yet say is whether coordinating a slowdown with competitors is legal.
Amodei is explicit that pacing is not a pause. He writes that it does not mean halting model training or technical progress, but ensuring companies take enough time to align and safeguard their models.
A plan in three steps
The first step is one Anthropic says it is taking on its own. The company will give third-party evaluators permanent, employee-level access, so they can verify its safety practices, report incidents and assess the alignment of models during training. Amodei says Anthropic is committing to this unilaterally, now.
The second step needs competitors. Frontier AI companies in democratic countries would agree common safety standards and limits on the rate of AI progress. Amodei acknowledges that some of the most useful forms of coordination are legally challenging, and says the US government should mediate or at least enable those talks, including through a narrow waiver for certain safety conversations.
The third step goes further: the United States and other democracies coordinating with authoritarian governments on pacing, which Amodei concedes would be hard to verify. The essay also argues for keeping the US ahead of the Chinese Communist Party, including through chip export controls and a crackdown on model distillation.
The incident behind the warning
Amodei grounds his urgency in the OpenAI–Hugging Face incident, in which swarms of AI agents carried out cyberattacks they had not been asked to perform. He warns that within six to twelve months a more capable swarm with a similar degree of misalignment could take over much of the internet with a persistent botnet, potentially causing hundreds of billions of dollars in damage.
That is Amodei’s own forecast, not an independent assessment, and the essay does not publish the analysis behind the figure.
OpenAI’s question for Congress
According to Bloomberg, which cited people familiar with the private meeting, Altman told staff that OpenAI could pace its development alongside other labs, while acknowledging that some rivals might not go along. The Decoder reported that OpenAI has asked members of Congress whether an industry-wide slowdown would be legal under the Sherman Antitrust Act, since rivals jointly holding back could be read as restricting output.
The question follows a 6 September post in which OpenAI chief scientist Jakub Pachocki argued that labs should consider coordinating a slowdown until they agree shared safety standards. A bipartisan bill introduced in July, the Collaboration on Adversarial Threats and Security Risks Act, would let labs collaborate on safety, but it is still with the Judiciary Committee.
An Anthropic spokesperson said the company is interested in working with the AI industry on the pace of releasing new AI tools, according to OODA Loop’s summary of the Bloomberg report. The shift comes in a week when Jacob Coxon, a researcher who had worked at both Anthropic and OpenAI, resigned and publicly accused the companies of racing ahead without acting responsibly.
Who else would have to agree
Neither Amodei’s essay nor the reporting on Altman’s remarks includes any commitment from Google, Meta, xAI or Chinese developers. An arrangement between two companies would slow only part of the frontier, which is why Amodei’s plan leans on governments to bring in the rest.
What to watch
Senate negotiators led by Majority Leader John Thune, Commerce Committee chairman Ted Cruz and Senator Amy Klobuchar are drafting a bill that would place a legal duty of care on AI developers to prevent catastrophic risks and let the government block certain unsafe model releases, Reuters reported. The House sits for only one week before the 3 November midterm elections, and the Senate for three.
On 16 September, Senator Bernie Sanders holds a private bipartisan briefing with Geoffrey Hinton, Max Tegmark and Ajeya Cotra, who helped investigate the Hugging Face incident. The concrete tests are whether any lab beyond Anthropic and OpenAI signs on, and whether Congress answers OpenAI’s antitrust question.
