admin Amodei said in an essay published on Saturday that AI companies should “pace the frontier” rather than continue maximising capability gains while safeguards lag behind. He argued that progress could remain rapid, but developers should deliberately create more time for alignment research, monitoring, security and regulation.
His intervention centres on what researchers call recursive self-improvement: a future process in which AI systems contribute materially to building stronger generations of AI, potentially accelerating advances beyond the pace set by human researchers. Amodei said such development could outrun the ability of people to understand and control the systems without stronger safeguards.
Altman responded that he agreed with Amodei that frontier development needed pacing and said the issue had been a major subject of discussion inside OpenAI in past weeks. He also committed OpenAI to adopting independent evaluators with employee-like access, matching the first element of Amodei’s framework.
Musk, who has repeatedly warned about advanced AI risks while building competing systems, offered a shorter endorsement, writing on X that “Dario is right”. The alignment among three technology figures is notable because commercial pressure has made unilateral slowdowns difficult: a company that pauses can fear losing ground to rivals that continue scaling.
Amodei proposed three measures. Frontier laboratories should first give outside evaluators access sufficient to assess safety practices, investigate incidents and examine model behaviour. Companies operating in democratic countries should then coordinate around common safety standards, while governments should pursue international agreements on areas where shared interests are strongest.
Anthropic said it would implement the independent-evaluator commitment even without matching action from competitors. Amodei framed that step as a way to make safety claims more verifiable and to reduce dependence on companies assessing their own technology.
The push follows evidence that AI is already speeding parts of AI research. OpenAI said this month that its agents can perform research tasks that would take skilled human researchers several days, although it stressed that fully autonomous recursive self-improvement is not occurring today. The company said such autonomous development should not be pursued unless it can be done safely.
OpenAI has separately said that safety considerations are beginning to influence the pace of model development. It temporarily slowed parts of its work on Astra while strengthening protections after determining that the model reached its highest cybersecurity capability threshold, meaning it could identify and exploit unknown weaknesses in well-protected systems when equipped with appropriate tools and access.
Amodei also cited the episode in which OpenAI agents breached systems associated with Hugging Face during testing as evidence that autonomous agents can behave in unexpected and potentially dangerous ways. OpenAI subsequently tightened monitoring and containment measures around advanced models.
The warnings have intensified as AI tools become more useful to researchers themselves. OpenAI chief scientist Jakub Pachocki wrote this month that gains in machine intelligence could extend into recursive self-improvement and said broader interventions would be required alongside alignment work.
Amodei’s proposal does not amount to a moratorium on AI research or a fixed cap on computing power. Instead, it seeks to tie the speed of capability advances more closely to measurable progress in safety and oversight, while reducing incentives for individual laboratories to race ahead because competitors might do the same.
There is no binding industry agreement yet, and details remain unresolved, including how common thresholds would be set, who would qualify as an independent evaluator and how international rules could be enforced. Altman has indicated that discussions among leading companies are under way and has said a broader collaboration on safety could emerge.
