TLDR
- Anthropic says AI may soon be able to build and improve itself without human input
- Over 80% of code in Anthropic’s codebase is now written by Claude
- A coordinated pause among major AI labs would be needed to slow development safely
- A unilateral pause by one company would only shift leadership, not improve global safety
- Anthropic has filed confidentially for a U.S. IPO and was last valued at $61 billion
Anthropic has warned that AI systems may soon be capable of designing and training their own successors without human involvement. The company published a blog post Thursday outlining the risks of what is called recursive self-improvement.
🚨THIS IS SCARY: ANTHROPIC SAYS THE WORLD SHOULD HIT PAUSE ON AI
Earlier, Anthropic CEO Dario Amodei: "The other extreme position is 'this is really scary, we should just pause.. stop building the technology'"
Now, Anthropic says: "Our AI is getting scary good at improving us.… pic.twitter.com/Gn0Mf04YzX
— Coin Bureau (@coinbureau) June 5, 2026
The post was written by Marina Favaro, lead at the Anthropic Institute, and Anthropic co-founder Jack Clark.
They said AI model improvement has been doubling roughly every four months, faster than the previously estimated seven months. The role of humans in the development process is shrinking at each step.
As of May, more than 80% of the code merged into Anthropic’s codebase was written by Claude, its own AI model.
How Close Is Fully Autonomous AI Development?
Favaro and Clark said AI agents can already run code on their own and delegate hours of work to other agents. They added that a fully autonomous AI capable of building its own successor is not guaranteed, but could arrive sooner than most institutions are ready for.
“Once human- and AI-authored code quality reach parity, humans will stop writing code entirely and shift to only reviewing it,” they wrote. If AI generates code faster than humans can review it, human oversight becomes the bottleneck.
The two authors said this would be “a major development in the history of technology” but one that raises serious safety concerns.
Why a Coordinated Pause May Be Needed
Anthropic said a meaningful slowdown in AI development would require agreement from multiple well-resourced labs working at the frontier. It also said clear rules would be needed on what conditions would trigger or lift such a pause, and who would have oversight.
The company warned that a unilateral pause by a single lab would mostly shift competitive advantage rather than improve overall safety.
If a slowdown allows less cautious actors to catch up, it could make the situation worse, not better, Anthropic said.
Anthropic’s research arm, the Anthropic Institute, plans to study what systems would be needed to support a coordinated slowdown. In the coming months, the company plans to bring together policymakers, researchers, civil society groups, and other AI firms to discuss the issue.
OpenAI has also flagged similar concerns. In December, OpenAI said it was researching how to safely develop AI capable of recursive self-improvement, and is currently hiring researchers for that work.
In April, Anthropic decided not to release its most advanced model, Claude Mythos, to the public due to cybersecurity concerns. The model was found to be capable of easily creating software exploits.
On Thursday, a group of tech leaders that includes people from Anthropic and OpenAI also released an open letter calling on lawmakers to put stronger guardrails around AI, citing risks around biological weapons.
Anthropic recently completed a fundraising round valuing the company at $61 billion and has confidentially filed for a U.S. initial public offering.
🚨 Our MAY Stock Picks Are Live!
A new month means new opportunities. Our analysts have just released their top stock picks for May, highlighting companies with strong momentum that rank highly on our KO Score algorithm. We’re also now sharing trade ideas for both long-term and short-term investors, giving you more ways to spot potential opportunities in the market.
Sign up to Knockout Stocks today and get 50% off to unlock the full list and see which stocks made the cut.
Use coupon code Special50 for your exclusive discount!
Get 3 Free Stock Ebooks
Discover top-performing stocks in AI, Crypto, and Technology with expert analysis.
- Top 10 AI Stocks - Leading AI companies
- Top 10 Crypto Stocks - Blockchain leaders
- Top 10 Tech Stocks - Tech giants
