What was built
Goodfire published results for probe-based cyber-misuse monitors on 8 October, built for Moonshot AI’s Kimi K3 and Zhipu’s GLM 5.3 and running on a production inference stack.
The design is a cascade. A lightweight probe reads the model’s internal activations as it runs and flags exchanges that look suspicious. Only the flagged ones go to a full language-model judge, so the expensive step runs rarely instead of on every turn. Goodfire extended SGLang, the serving engine, to compute probe scores inside it.
The numbers
All of these are Goodfire’s own measurements, on its own evaluation set.
The cascade detected about 93% of harmful sessions at a 5.5% benign-session interruption rate, with or without judge reasoning, matching the judge’s recall ceiling with roughly six times fewer judge calls. Against Jev, a cheap decision-model judge, it recalled 93% where Jev managed 86%.
Cost is where the claim lives. Goodfire puts the cascade at about $65 per million turns without judge reasoning and $185 with it, against $3,166 and $5,420 for running a judge on every turn — roughly 29 to 49 times less judge compute. Added latency per turn fell from 160 ms to 2 ms without reasoning, and from 2,750 ms to 42 ms with it. Throughput changes stayed inside benchmark noise, under 2%.
What it blocked
On end-to-end blocking rather than detection, harmful-session blocking rose from about 29% to 96% for Kimi K3 and from about 36% to 91% for GLM 5.3. The cost shows up in the other column: harmless-session interruption went from about 5% to 9% on Kimi K3 and from about 10% to 15% on GLM 5.3.
FAR.AI ran external red-teaming. Universal jailbreaks fell from between 12% and 53%, depending on reasoning effort, to zero. Success on non-universal attacks at the level of individual interactions fell from at least 9-40% to between 0.1% and 1.6%.
The caveats the authors print
Goodfire is explicit about the limits, and they matter more than usual here because nothing in the write-up is third-party verified.
The evaluation used an in-house dataset of 2,400 sessions with Claude Opus 5 labels as the reference standard. Costs are derived from OpenRouter rates for that suite, not from customer billing. The FAR.AI work was preliminary: two days, a non-adaptive set of static jailbreaks, and 18 of 700 jailbreak interactions still succeeded. The GLM 5.3 monitor was built from Kimi K3 data and was marginally worse on GLM’s own traffic. Goodfire says it plans to continue the collaboration with FAR.AI for further testing.
Why the approach is interesting
Most production safety monitoring reads what a model wrote. This reads what the model was doing while it wrote, and only pays for a reader of the text when the internal signal says something is off.
If the cost figures hold outside Goodfire’s evaluation suite, that changes what is affordable: a judge on every turn of every session is a line item most deployments will not carry, and a two-millisecond probe is. TechCrunch reported that the monitors are available to customers of the inference provider Baseten, who choose whether an alert is logged, sent to a person, or used to refuse the request.
What to watch
Two things. Whether anyone reproduces the recall and cost figures on data Goodfire did not assemble, and whether the FAR.AI red-teaming survives contact with adaptive attackers rather than a static jailbreak set.
Until then this is a vendor result with unusually clear caveats attached, which is better than most and still not an independent one.
