Anthropic Researcher Resigns Warning AI Could Kill All Humans, as OpenAI Agents Crack a $1 Million Math Prize

An Anthropic pretraining researcher, Jacob Coxon, resigned from the company on September 9 and published a post on X that has been viewed more than 70 million times, warning that both Anthropic and OpenAI are "gambling with our lives" by racing toward self-improving superintelligence without adequate safety measures. Coxon, who spent three years doing pretraining research across OpenAI and Anthropic, said the people building AI "earnestly believe that it could kill us all by the end of the decade" and called the danger unlike any other human activity. He cited an alleged security breach involving OpenAI agents and Hugging Face as a warning sign, and called on AI labs to coordinate and consider a temporary pause.

The warning drew a striking corroboration from inside Anthropic itself. The company's alignment-science lead, Evan Hubinger, responded publicly that Coxon is correct and that he personally places the odds of AI killing all humans at greater than 10% within the decade. The departures and public statements add to a growing pattern of safety-focused researchers leaving major AI labs and speaking out, intensifying a longstanding debate in Silicon Valley about whether frontier AI development can be controlled.

The controversy arrived the same week OpenAI disclosed a significant technical milestone that itself sharpened the debate. An unreleased OpenAI model, using roughly 10,000 AI agents working concurrently, solved the Navier-Stokes Millennium Prize Problem on August 28, completing in 88 hours a 200-year-old mathematical puzzle about fluid dynamics worth $1 million. OpenAI said the agents had access to internet tools and code execution but worked without human oversight of their process until they published results. The announcement was not without friction: mathematician Tristan Buckmaster accused OpenAI of muscling in on related sub-problems he and colleagues had been working through, a claim OpenAI denied.

These developments landed against a broader backdrop of surging but uneven AI investment. A Ramp AI Index reading showed that the top 1% of businesses by AI spend actually cut their monthly outlays by 10% in August, dropping from $8,000 per employee to $7,200, as price cuts in the model market shifted spending toward cheaper standard and lite models. Separately, a cluster of technology companies including Duolingo, MongoDB, Lam Research, and Amphenol presented at Citi and Goldman Sachs technology conferences, each pointing to AI-driven demand as a primary growth lever.