A safety case for keeping reasoning private

OpenAI has published an account of how it detects and disrupts adversarial distillation — the practice of using a frontier model’s outputs, without authorisation, to train a second model that inherits the capability but not the safeguards. The company frames the problem as a safety one rather than only a commercial one: an adversary could distil a model’s cyber ability and then deploy it without the refusals that constrained the original.

The headline number concerns protected reasoning, the intermediate chain-of-thought a model generates while working toward an answer. In an internal adaptive black-box evaluation using automated red-teamers, OpenAI says the average attack success rate for recovering reasoning the model had generated while answering an earlier prompt fell from 54.0% against GPT‑6 Astra with its original safeguards to 0.2% against GPT‑6.2 Astra with updated ones. For reasoning from the current request, the rate went from 22.9% to 0.9%. Both figures are OpenAI’s own, aggregated across six attacker models.

Why the traces matter more than the answers

OpenAI also describes an internal study on how cyber capability transfers. Using the content a user normally sees — final responses, reasoning summaries and tool interactions — as a baseline, it fine-tuned student models on trajectories from Astra-level teachers working on coding and defensive security tasks, then measured what changed when the student could also see the teacher’s exact reasoning traces.

A rusted bolted flange joining two large steel pipes
Output classifiers also check tool-call arguments for leaks. Illustrative image. David Brown · pexels · Pexels License

After supervised fine-tuning, the student trained on exact traces generally beat the one trained on summaries. After a round of reinforcement learning, the exact-trace student still scored higher at the largest inference budget tested on both benchmarks shown, while the summary-trained student did better at lower budgets. OpenAI notes in a footnote that Anthropic reported a similar advantage for full transcripts over summaries on SWE-bench Verified, and argues its own result extends that finding to offensive cyber work.

Classifiers, API boundaries and account bans

The defences described sit at two levels. At the request level: adversarial training so models refuse extraction attempts, input classifiers to spot those attempts, and output classifiers that check assistant messages and tool-call arguments — the latter because a model can leak reasoning in a tool’s arguments without putting it in the final answer.

At the API level, GPT‑6.2 Astra and Sol tie protected reasoning to the conversation history that produced it. If an earlier message is rewritten, the API drops the reasoning that followed it. Older model families with weaker safeguards are blocked from reading the newer models’ protected reasoning. OpenAI says a September disclosure covered attacks that moved encrypted reasoning from one conversation into another, and that it has since closed a pathway that could expose another user’s encrypted reasoning.

A chain-link fence in close focus against a hazy sunset
Weekly enforcement reaches tens of thousands of accounts, OpenAI says. Illustrative image. Chris F · pexels · Pexels License

Named actors, and a legislative ask

The enforcement numbers are blunt. OpenAI says weekly actions against distillation campaigns “have ranged in scale from thousands to tens of thousands of accounts, with the rate increasing significantly this past summer”.

It names two parties. In February it briefed Congress on activity indicating DeepSeek was attempting to distil US frontier models. In September it reported disrupting a campaign to extract protected reasoning and attributed a core part of that activity to individuals associated with Moonshot AI, the developer of Kimi. More disclosures are promised.

The post closes with policy. OpenAI says it supports legislation giving the US government more tools to identify and sanction foreign actors that extract capabilities from American models, and names the Chip Security Act, the AI OVERWATCH Act and the MATCH Act. It also offers four practices to other labs: protect reasoning across the whole interaction, keep successful attacks as regression tests, combine request-level mitigations with campaign investigations, and coordinate safeguard requirements with deployment partners. Whether rivals publish comparable extraction rates is the obvious next test.