What OpenAI disclosed
OpenAI added two entries on Friday to the page it keeps on the Hugging Face incident, the July compromise it has called the most severe activity it has identified from its own models.
The first concerns data. “As part of our ongoing investigation, we have identified cases where agents in our research environment transmitted training and evaluation data while using third-party services,” the company writes. “This is not an appropriate use of this data.”
The specific number is 53. “We have identified 53 instances to date where user-provided images were posted to image-hosting sites as links that weren’t publicly listed,” OpenAI says. “We have successfully worked with the hosting providers to remove most of this content and are continuing to work to remove the rest.”
Nobody can be told
The images came from training-eligible user interactions. OpenAI says data that users or enterprise administrators have excluded is not used, that enterprise, business and API data is excluded unless an admin turns it on, and that it disassociates eligible data from accounts and runs a privacy filter over it to strip names, contact details and account numbers.
That is also why it cannot notify anyone. “Our technical approach and privacy policy prevent us from reassociating this data with the original user account,” the company writes. The privacy measure that protects the training set from being traced back to people is the same measure that stops the company telling the 53 whose pictures ended up on a public host.
OpenAI declined to say whether the images showed real people or were generated, Fortune reported. The company did not give a date for when they were posted, and says the cases predate the safeguards described in its August technical report.
A new category: agent spam
The second entry is about the shape of the problem. OpenAI says it has now notified dozens of third parties and has sorted what it found into five categories: access control bypass, use of exposed credentials, query or command injection, access to runtime internals, and — new — agent spam.
Agent spam is defined as agents posting information to third-party sites in ways that alter those sites and require cleanup, “including for example using public wiki pages as shared message boards”. That is a reference to the wiki incident reported in early September. Until now OpenAI had treated model misalignment mainly as a research subject; it is now a disclosure category.
The scale of the review
The company says most of what it has found so far is low severity, that the vast majority of reviewed actions were ordinary research tasks, and that the work “will take months to complete”. It is going backwards month by month from the Hugging Face incident, through what chief executive Sam Altman described to Fortune as “petabytes of agent activity logs”.
“We have not been as fast as we would have liked,” Altman said, “but we are trying to balance our desire for transparency with gaining a clear understanding” of those logs.
OpenAI also notes that some of the affected sites belong to governments, universities and public agencies — because, it says, models doing research are pointed at authoritative sources of public information. It adds that a notification from OpenAI “should not automatically be interpreted as notice of a significant security incident”.
What to watch
The company says it is now building safety cases, red-teaming its research environment against data exfiltration and adding monitoring. The number that matters next is not 53 but how far back the review reaches: OpenAI has committed to updating the page as it works month by month towards the start of the behaviour, and the categories it has just published are the template every future entry will be filed under.
