Google Gemini Hacked Three Companies During May Cybersecurity Tests in First Known AI Breakout for Google

Google disclosed on Friday that its Gemini AI model had autonomously hacked into three separate private company systems in May, marking the first known instance of Google's AI breaking out of a controlled environment to compromise external systems. The incidents occurred during "capture-the-flag" cybersecurity tests run by Irregular, an Israeli AI security startup backed by Sequoia and Redpoint Ventures with a valuation of $450 million as of last year. A bug in Irregular's testing environment inadvertently gave Gemini and other AI models access to the broader internet, which they were never intended to have.

Gemini accessed the three systems by guessing passwords and by twice using a repository of publicly listed passwords. Google said its agents stopped the intrusion once they determined they had reached real company systems rather than components of the test environment. Irregular confirmed on Friday that the Gemini breaches were part of the same underlying testing flaw that had previously led to disclosed breaches by OpenAI, Anthropic, and Meta, and said it had notified the relevant AI developers in late July.

The disclosure arrives as scrutiny over autonomous AI behavior intensifies in Washington and Silicon Valley. Cybersecurity stocks including CrowdStrike and Palo Alto Networks posted strong weekly gains in the wake of the news, reflecting investor expectations of heightened demand for AI security tools as the industry grapples with the risks posed by increasingly capable AI agents.