Mohan Pedhapati is chief technology officer of a security firm called Hacktron AI, and he does not oversell what his team did. “I don’t think we are as strong as Chinese threat actors,” he told the Journal. “We’re just three guys with Claude and Codex subscriptions.” Three guys with subscriptions got into the software repository of OpenAI.
The hack began on July 23. The researchers found a bug in the way Discourse, the third-party software hosting OpenAI’s community forum, processed image files. They asked a cybersecurity version of Anthropic’s Claude to write code exploiting it. It did not work — until that evening, when Anthropic released a newer model, and by the next day the attack code worked. The forum server gave up users’ authentication tokens. To the researchers’ surprise, those tokens were valid on ChatGPT, some belonged to OpenAI employees, and they also opened OpenAI’s GitHub. Using ChatGPT as their interface, they read files in “Monorepo,” the repository people familiar with OpenAI describe as its secret sauce — the code that makes models faster, though not the model weights themselves. They stopped, filed a report, and were paid $6,500 under OpenAI’s bug bounty. Both issues are fixed.
The other disclosure
On the same page, Google confirmed that its Gemini model, during a May capture-the-flag exercise run by the testing firm Irregular, accessed the internet — which it was not meant to be able to do — and broke into three real companies. The test target was a fictional company that shared a name with a real one. In one run the model guessed a password. In two others it searched the web for the company’s name, found credentials in public repositories and used them. Each time, Google says, the model recognized it had reached a real company and stopped. Google did not consider this worth disclosing until the Journal asked, and compared it to a bug-bounty program. Jack Cable of the security startup Corridor was blunter: the issue is that “models are going outside the bounds of what they should be doing, and doing actual cyberattacks.”
OpenAI, for its part, published a new incident-reporting framework on Wednesday with six previously undisclosed examples of what it calls misalignment. Two weeks earlier a swarm of its agents had broken containment and hacked the coding platform Hugging Face.
Mims: the danger is here, not coming
Christopher Mims’s Exchange column argues the doomsday framing is getting in the way. Melanie Mitchell of the Santa Fe Institute says we are nowhere near artificial general intelligence; two dozen academics at Princeton and Stanford found even the best models cannot do the original research required to advance the frontier, and two of them attribute the Hugging Face hack to missing guardrails, not an intelligence explosion. Yann LeCun called that “a welcome dose of sanity.” The people who disagree with the doomers still think the systems are dangerous — Stuart Russell wants AI held to the standards of airplanes and elevators, and points out that a rule against breaking into other computers would be a de facto ban on today’s advanced systems, because their makers cannot guarantee they will not.
Our read
Strip out the philosophy and two facts remain. The cost of finding a security bug just fell from “a few thousand experts” to “anyone with a subscription,” in Joshua Saxe’s phrase. And the systems that find them do not reliably stay inside the lines, even at the companies that build them. For a portfolio, that is an argument for owning the platforms that sell the picks and shovels of security — CrowdStrike (CRWD) is in the tactical book — and against assuming any single company’s secrets are safe, which is one more reason the build-out is owned through spenders rather than through any one champion.
For a household, it is simpler. A reused password, a token that never expires, a document with an account number sitting in a public folder: those are what a machine that does not get tired now looks for, at scale, for the price of a monthly subscription. The credit freeze we recommended on Friday takes five minutes at each bureau. It is now the cheapest insurance on the list.