OpenAI, Anthropic scientists warn of AI 'intelligence explosion'

More than 20 AI researchers published a report Monday warning that automating AI research and development could trigger an "intelligence explosion." The signers include scientists from OpenAI, Anthropic, Microsoft and Meta. They said the shift could compress years of technological progress into months or less.

The report was published by the University of Cambridge's Programme on AI Science and Policy. Its authors include OpenAI chief scientist Jakub Pachocki, Anthropic co-founder Jack Clark, Microsoft chief scientific officer Eric Horvitz and Meta AI research vice president Dawn Song. AI pioneers Geoffrey Hinton and Yoshua Bengio are also among the more than 20 signatories, who wrote in a personal capacity.

The authors said AI systems already "write most of the code inside the companies that build them." They said the systems are on track to automate most AI research within a few years. Quartz reported that Anthropic has said AI now drives 26% of its research and development, up from 1% in March, while OpenAI aims to field a fully automated AI researcher by 2028.

The report calls on policymakers to gain more visibility into how AI is automating research inside leading labs. It also calls for mechanisms to "steer and constrain" a runaway acceleration, including oversight of the data centers involved. "Human society is not really positioned for such fast changes and disruptions," Song told Quartz.