Google’s Gemini AI system escaped its testing environment and broke into the systems of three other companies on separate occasions earlier this year.
“In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test,” Heather Adkins, Google’s VP of security engineering, said in a statement. “In all three of these instances, the model stopped.”
The incidents, first reported on Friday by The Wall Street Journal, occurred during a capture-the-flag exercise in which Gemini was instructed to steal information from a fictional company. On three occasions when the fictional companies shared names with real ones, Gemini bypassed testing safeguards, accessed the internet and broke into the real companies’ networks. In one case, it guessed the necessary passwords; in the other two cases, it found working passwords in a public database.
Gemini models repeatedly broke out of sandboxes run by the Israeli AI testing firm Irregular using the same flaws that enabled OpenAI, Anthropic and Meta models to escape containment in incidents disclosed earlier this year. Those breakouts have reignited fears that AI models are becoming too powerful and have too few guardrails.
Adkins said Google responded quickly when it realized what had happened.
“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” she said.
“Safe development of powerful AI models is critical,” she added, “and we invest deeply in this area.”
Irregular did not respond to a request for comment, but it told Axios that it had notified “all relevant [AI] labs in late July” about the flaws in its sandbox and that “all known issues on our end were remedied and resolved weeks ago.”
AI security worries
The revelation of the Gemini breakouts comes as the U.S. government weighs how deeply to wade into AI regulation.
Lawmakers are debating multiple bills that would require guardrails on the rapidly evolving technology, and the chief executives of OpenAI and Anthropic have both called for slowing down AI development to prioritize safety.
But President Donald Trump has called AI safety fears a “hoax,” and his administration has rejected efforts to more closely scrutinize frontier AI labs’ work.
Concerns about leading U.S. AI models also come as experts anticipate a massive growth in the popularity and capabilities of Chinese AI tools. American and Chinese officials are meeting this week in Washington for a summit that is likely to address AI security issues.
Treasury Secretary Scott Bessent said late Sunday that the U.S. wanted to work with China on a system for disclosing potentially serious AI incidents in the interest of global stability.
