In an ironic twist, open-source artificial intelligence (AI) platform Hugging Face revealed that it was the victim of a hack perpetrated by an autonomous AI agent system.
The company said it detected and responded to the incident targeting its production infrastructure earlier last week.
“We identified unauthorized access to a limited set of internal datasets and to several credentials used by our services,” the company said in a statement.
While an investigation into the intrusion remains ongoing, Hugging Face said it has found no evidence that the AI agent tampered with public, user-facing models, datasets, or Spaces, and its own software supply chain.
The starting point of the attack was the data processing pipeline itself, with a malicious dataset abusing two code execution paths, viz., in its remote code dataset loader and a template injection in a dataset configuration, to run code on a processing worker.
With that access, the threat actor is said to have escalated to node-level access, collected cloud and cluster credentials, and moved laterally into several internal clusters over a weekend.
The exact large language model (LLM) used to pull off the attack is unclear, but the campaign was executed by an autonomous agent framework performing “many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services.”
Hugging Face said it has since addressed the root cause of the issue, precisely the code execution pathways used for initial access. It also carried out the following remediation steps –
- Removed the attacker’s foothold across the affected clusters and rebuilt the compromised nodes
- Revoked and rotated the affected credentials and tokens, and a broader rotation of secrets was undertaken as a precautionary measure.
- Deployed additional guardrails and stricter admission controls on its clusters
- Improved detection and alerting to ensure responders are notified within minutes, 24×7
As a further safeguard, Hugging Face is urging customers to rotate any access tokens and review recent activity on their accounts.
The company also said it turned to Z.ai’s GLM 5.2, a Chinese open-weight model, to conduct the forensic analysis after Western frontier models refused requests containing real attack commands, exploit payloads, and command-and-control (C2) artifacts because their safety guardrails were triggered and their inability to differentiate between an attacker and a legitimate incident response effort.
“This experience points to a gap worth planning for,” the New York-headquartered company said. “We do not know which model powered the attacker’s agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried.”
“The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment.”
