Dive Brief:
- Leaders at Anthropic, OpenAI, Microsoft and SpaceXAI called for a slowdown to the development of frontier AI models over the weekend, citing concerns around the mismatch between the technology’s capabilities and the available safety frameworks.
- “AI brings risks, and because it is such a powerful technology, these risks are serious,” said Anthropic CEO Dario Amodei in a weekend blog post. “A race to the bottom, spurred by commercial incentives, can make these risks more acute.” Amodei’s concerns were echoed by OpenAI CEO Sam Altman, Microsoft CEO Satya Nadella and SpaceXAI CEO Elon Musk.
- In his post, Amodei laid out a three-prong approach on how to establish a safety framework, which included embedding third-party evaluators into companies to ensure rules and policies are adhered to and incidents are properly reported.
Dive Insight:
Experts have been warning for years that delays or pauses in AI model development are necessary to gain a better understanding of the technology’s risks. But in the last week, mounting concern over AI’s rapid development reached a crescendo, as Jacob Coxon, a former Anthropic researcher, quit his job to sound the alarm that AI could lead to human extinction.
Some U.S. lawmakers have been following suit, spurred on by news of rogue AI models bypassing safety mechanisms to hack other online platforms and calling for stricter AI rules and safety frameworks.
Regardless, a slowdown in frontier AI development could create a ripple effect for enterprises, as CIOs factor in the cadence change to their AI adoption plans, according to Arun Chandrasekaran, distinguished VP analyst at Gartner.
“I believe this creates more uncertainty about the future,” Chandrasekaran told CIO Dive. “CIOs so far have assumed a certain cadence of innovation in the model ecosystem … and now this creates a little bit more confusion in terms of what the pace of release is going to be.”
Frontier AI labs are also sharing their own ideas on how to establish better safety frameworks such as embedded third-party evaluators to provide added transparency and assurances that safety parameters are being met. That idea has garnered uniform support among competitors and echoes recent legislation out of California.
“We welcome the research, focus, and deliberate pacing needed to get alignment right as the design goal,” Nadella said in a LinkedIn post Sunday. “We also welcome ideas like ‘embedded evaluators’ and the broader efforts to develop the mechanisms to make this more than just talk.”
Chandrasekaran, however, said that evaluator independence is still “a very open-ended question.”
“On one hand, at least, we have this consensus that something needs to be done,” Chandrasekaran said. “But I don’t think we’ve really figured out what the next steps are after this, and how this is going to be run in a truly cooperative and a neutral way.”
The weekend discourse surrounding AI safety put the spotlight back on enterprise risk management strategies, according to Mark Tauschek, VP of research fellowships and distinguished analyst at Info-Tech Research Group. The issue, Tauschek said, goes beyond voluntary statements.
“IT leaders need independently verified evidence, contractual notification and exit rights, model-specific deployment gates, and a tested way to stop or replace a system when the vendor’s controls, policies or risk profile change,” Tauschek said in an email.
Among other preventive actions, CIOs should work to create a frontier model adoption gate that lets security, privacy, legal, risk and business owners thoroughly review systems before full adoption, according to Tauschek. Companies should also require continuous independent assurance of safety from vendors, as well as adding pause, notification and exit rights in contracts.
Not everyone agrees AI development needs to slow down. President Donald Trump, whose administration has been conservative in regulating the technology, released a statement Monday decrying the push from frontier AI labs and warning that slowing AI development could slow the country’s efforts to remain competitive globally.
