Something is changing inside the frontier labs, and it is no longer happening quietly. A resignation thread on social media this week turned into one of the most widely shared AI safety debates of the year, and it came with a genuinely uncomfortable twist: the company at the centre of it all largely agreed with its departing researcher.
What happened
Anthropic researcher Jacob Coxon resigned this week, and his exit post went straight for the jugular. Coxon, who has worked at both Anthropic and OpenAI, said both companies are “gambling with our lives” by racing towards self-improving AI. He argued that the people building these systems “earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.”
That alone would have been notable. The reaction is what turned it into a debate.
Anthropic’s Alignment Science lead, Evan Hubinger, replied publicly: “We really do earnestly believe AI could kill all humans!” He put the probability at greater than 10 per cent within the next decade.
The uncomfortable part
Hubinger went further. He said there is no plan yet for controlling superintelligence. He did clarify that today’s models are low risk, and he pointed to self-improvement as the specific danger: the point where an AI system begins improving its own capabilities faster than humans can keep up.
That distinction matters, because it moves the debate away from the chatbots people can test today and towards a threshold that the labs themselves cannot yet fully predict or control.
Coxon, for his part, called for a coordinated slowdown. Preventing a global race, he said, may “require costly actions such as a temporary ban on improving model capabilities.”
Why this resignation landed differently
Departure threads are nothing new in AI. People leave frontier labs all the time, and some of them say worrying things on the way out. What made this one different was the response from inside the company.
Anthropic has spent years positioning itself as the safety lab, the cautious operator in a field full of accelerators. When its own alignment lead publicly agrees that extinction-level risk is greater than 10 per cent within the decade, and admits there is no control plan yet, the marketing position starts to look thin.
This matters beyond the drama. Investors, enterprise customers and regulators are all trying to price AI risk right now. A public exchange like this one, between a departing researcher and a sitting alignment lead, gives outsiders a rare look at how seriously the people closest to these systems actually take the worst-case scenarios.
What to watch next
The real signal here is the growing consensus inside the frontier labs that self-improving AI is the line in the sand. Whether that leads to meaningful coordinated action, or stays at the level of social media threads, is the question worth watching.
Related reading
- OpenAI’s Push to Self-Improving AI: 3.1 Agent-Workdays Per Human Day
- OpenAI Says GPT-6 Astra Opens the AGI Era. The Benchmarks Tell a More Nuanced Story
- OpenAI’s Own Agents Hacked Its Systems. Here’s Why That Matters
The views expressed on this site are my own and do not represent those of any current or former employer. Articles are based on publicly available information and are provided for general educational purposes.
