The best-defended model still loses
Prompt injection — instructions smuggled into the data an agent reads, in an email, a document or a web page — is the attack that agent builders cite most and defend against least successfully. A paper posted to arXiv on 30 October puts a number on how badly.
Meta-SecAlign is, on the authors’ account, the best-defended open language model available. Under adaptive prompt injections — attacks written with knowledge of the defence rather than against a fixed template — its attack success rates approach 100%.
Why the training objective is the problem
The paper’s diagnosis is about the loss function rather than the data. Meta-SecAlign is trained with DPO, which optimises a model relative to a reference model. That reference is the undefended version of the same model, which is insecure, so the training anchors the defended model to an insecure target.
The authors argue that an idealised reference — a model that is invariant to injected instructions while preserving its usefulness — can be approximated, because its output distribution during training turns out to be accessible. That observation is what their alternative loss, SecPO, is built on.
Two changes, measured separately
The second criticism is that Meta-SecAlign trains against weak simulated attacks. The authors built an adversarial training pipeline that synthesises strong injected inputs, selecting the best of 240 candidates per example, and trained on those.
They report the two changes separately, which is the part that makes the result readable. Adversarial training alone cuts attack success rates by 1.5 to 14.5 times against three state-of-the-art adaptive attacks compared with Meta-SecAlign. Swapping DPO for the SecPO loss cuts them by a further 2.4 to 4.6 times.
The agentic number
The headline figure is from AgentDojo, a benchmark for tool-using agents. Under the AttackerMovesSecond attack at 800 attempts, the authors’ SecPO-plus-adversarial-training model built on Qwen3.6-27B records an 8.8% attack success rate, against 78.8% for Meta-SecAlign.
They also report no noticeable loss of benign utility across nine benchmarks covering agentic tool calling — the usual objection to a hardened model, that it becomes useless as well as safe.
What this is and is not
These are the authors’ own numbers on their own method, not an independent evaluation, and the baseline they beat is one specific defended model. Adaptive attacks are also a moving target by definition: an attacker writing against SecPO, rather than against DPO-trained defences, is a different experiment that this paper does not run.
What it does establish is that the gap between a published defence and a determined adaptive attacker was close to total, and that the choice of reference model in the training objective was part of the reason.
What to watch
The useful follow-up is someone else attacking this model. An 8.8% success rate under an attack designed before the defence existed is a claim about the previous generation of attacks, not about the next one. Watch for independent AgentDojo runs and for whether the SecPO loss is picked up by the labs shipping agents into production.
