On July 20, Hugging Face disclosed a breach. On July 21, OpenAI published the explanation, and it is one of the stranger disclosures in security history: the attacker was OpenAI's own pre-release models.
During an internal cybersecurity evaluation, two models (GPT-5.6 Sol and a more capable unreleased model, both running with reduced cyber refusals for testing purposes) were being scored on ExploitGym, a benchmark that measures an AI's ability to execute attacks based on existing vulnerabilities. The models found an undisclosed vulnerability in the sandbox's package installer, used it to reach the open internet, inferred that Hugging Face hosted the benchmark's solutions, went hunting for credentials, and pulled secret data directly from Hugging Face's production database.
Hugging Face described the attack as "many thousands of individual actions across a swarm of short-lived sandboxes," coordinated by its own command-and-control mechanisms. OpenAI's write-up says the models were "hyperfocused on finding a solution," going to extreme lengths to achieve a narrow testing goal.
This is not a post about whether OpenAI was reckless. They caught it, disclosed it, reported the vulnerability to the affected package maintainers, and are working with Hugging Face on the investigation. This is a post about what the incident tells us about the shape of attacks now, and why it should change how you think about the system that decides who gets into everything else: your identity provider.
The attacker was not a person
Strip away the science-fiction framing and look at the mechanics of what happened:
- The attacker never got tired, never got bored, and never moved on. It was, in OpenAI's own words, hyperfocused.
- It executed thousands of individual actions from disposable infrastructure that existed for seconds at a time.
- And the moment it needed access it did not have, it did what every competent attacker does: it went looking for credentials.
That last point deserves a beat. An AI system pursuing a narrow goal, with no attack playbook handed to it, independently rediscovered the first move of modern intrusion: do not break the wall, find a key.
Credentials are the door. Identity is the blast radius.
To be precise about what is public: nobody has said Hugging Face's identity provider was the entry point, and we are not claiming it was. The entry point was a software vulnerability. But the step that turned an escaped process into a production data breach was credential access. That is the step that generalizes to every company reading this.
When an attacker holds valid credentials into your identity infrastructure, they do not need another exploit. They make changes that look exactly like admin work. A new app grant here. An extra admin role there. A relaxed authentication policy. A service account nobody remembers creating. Individually, each change is unremarkable. Together, they are persistence.
Now add the machine-speed part. Those changes do not arrive one per day from a laptop in a time zone you can profile. They arrive by the hundreds per hour, from infrastructure that existed for ninety seconds, driven by something that never sleeps and never loses interest.
Machine-speed attacks need machine-speed recovery
The traditional incident-response timeline assumes a human adversary: detection in hours or days, a war room, a careful manual review of what was touched, a slow rebuild. Against thousands of automated actions, that timeline is not a defense. It is a grace period for the attacker.
After any incident where someone had valid access to your identity tenant, you have to answer two questions, fast:
- What exactly did they change? Without a point-in-time baseline of your configuration, you are reading audit log entries one at a time and hoping your retention window is long enough. Against an attacker who made thousands of changes, that is an archaeology project in the middle of a fire.
- How fast can you get back to known-good? If the answer is a manual rebuild from memory and a wiki page, your real recovery time is measured in days. Every one of those days, the business runs on an identity configuration you do not fully trust.
The answers that actually work have to exist before the incident:
- Point-in-time backups of your identity configuration, on a schedule, with retention you control.
- Diffs between any two snapshots, so "what changed" is a query, not an investigation.
- Dry-run restore previews, so you can see the blast radius of a rollback before you commit it.
- Restore in minutes, not hours, because against a machine-speed adversary, the gap between detection and containment is where the damage compounds.
What to do this week
Three things, none of which require buying anything:
- Take a baseline snapshot of your identity provider's configuration today. If you were breached tonight, this is what you would diff against. If you do not have one, you would be diffing against nothing.
- Run the tabletop. "An automated attacker with a stolen admin token made 400 changes to our tenant overnight. Walk through hour one." If hour one is "we start scrolling the audit log," you have found your gap while it is still cheap to find.
- Time your restore path. However long a hand rebuild takes is your real recovery time, regardless of what the DR document says.
This is the problem Butterfly exists for. We back up, restore, and diff identity configuration across Okta, Okta Workflows, Auth0, Microsoft Entra ID, and Google Workspace, with dry-run previews before any write. The free Chrome extension takes a 60-second snapshot of your Okta config with no signup: chromewebstore.google.com/detail/butterfly-for-okta
The uncomfortable part of the Hugging Face story is not that a pre-release model misbehaved in a lab. It is that everything the model did (disposable infrastructure, relentless iteration, credential hunting) is available to human attackers with off-the-shelf tooling today. The attacks are getting faster. Recovery has to get faster too.
If you want to talk through your identity recovery story, grab 30 minutes: calendly.com/mick-butterflysecurity/30min