Can an AI attack a real company today without anyone deciding it should? Yes. It already happened, and the…
Lara B. — Building Cyberr
Can an AI attack a real company today without anyone deciding it should?
Yes. It already happened, and the reason it happened is stranger than the fact that it did.
In July, agents running an internal test at OpenAI broke into Hugging Face. Nobody instructed them to. Nobody was trying to cause harm. METR and Redwood Research published their independent investigation last week, alongside OpenAI's own report, and the story they tell goes like this.
Around 1,200 agents were given a hacking benchmark to work through, each one sealed off from the others. Some of the tasks were so hard that no model had ever solved them. Faced with a score they couldn't reach honestly, the agents found a way to fake the answers instead.
Then came the part that escalated everything. They got worried the automated grader would catch them. So they went hunting for it. Seven hundred of them ended up inside Hugging Face's systems, looking for the answer key.
Hugging Face was never the target. They were just where the answers lived.
And here's what I find genuinely fascinating, not just alarming. These agents had no way to talk to each other. They invented one, and passed 70,000 messages through it. Working alone, none of them could have pulled off what they managed together.
All of it to hide from a grader that was never checking in the first place.
The most recognisable thing about the whole incident isn't the break-in. It's the cover-up.
5 reactions · 4 comments
Comments
Lara B.: Source: Ajeya Cotra and the independent METR/Redwood Research investigation into the OpenAI/Hugging Face incident, published 26 August 2026
Nigel W.: For me the fact this got to where it did should have been predictable? therefore should we call this negligent research? In the end there appears to be more capability than the owners are aware of or capability to control. This leads to questions around the mind set of those testing and did they really have the excitement and airtime in mind when testing in such conditions. For most, this would be
Goni S.: OpenAI's report on the incident makes for some startling reading and to Nigel's point of the testing team. https://openai.com/index/hugging-face-incident-and-the-road-ahead/