The footnote is the story

Microsoft launched MAI Cyber 2 Flash on 17 November with a headline claim: on CyberGym-E2E, a benchmark for real-world vulnerability discovery, the model scored 92.3% against GPT-6 Astra’s 91.5%, using half the reasoning budget and less than 2% of the cost per task.

Underneath that claim, in a section headed “About these results”, Microsoft says something that vendors almost never volunteer. While reviewing its CyberGym-E2E results, the company found that 92 of the benchmark’s 905 tasks are based on the same open-source projects and vulnerability reports as some of its training data. It says the benchmark’s reference solutions were never used in training, and that it excluded the 92 tasks for every model and reported results on the remaining 813.

Why that matters

A benchmark loses its meaning when a model has seen the answers. The usual failure mode is silent: a vendor runs the suite, posts the number, and nobody outside can tell which items were memorised. Microsoft’s disclosure describes the weaker form of the problem — overlapping source projects rather than leaked reference solutions — and it rescored symmetrically, applying the exclusion to competitors as well as to its own model.

A green circuit board photographed close up, dense with surface-mounted components
The benchmark tests vulnerability discovery in real software projects. Illustrative stock image. Júlio Riccó · pexels · Pexels License

It is also a 10% cut to the task set, taken voluntarily, on the benchmark that carries the launch’s most aggressive cost comparison. The same-day arXiv preprint on benchmark contamination argues that exactly this kind of disclosure should become standard practice rather than a footnote.

The rest of the claims

Microsoft says MAI Cyber 2 Flash also outperforms GPT-5.6 Cyber on CyberGym-E2E and on WebExploitBench at the same reasoning budget, scoring 40.4% against 38.4% on the latter. It gives the model a one-million-token context window, four times larger than MAI Cyber 1.1 Flash, which it says lets the model hold more of a codebase during multi-step investigations.

The cost comparison rests on list prices Microsoft publishes in its own footnotes: $0.10 per million input tokens, $0.01 cached and $0.50 output for MAI Cyber 2 Flash, against $10, $1 and $50 for GPT-6 Astra on Azure OpenAI Global. Microsoft notes the figures reflect offline evaluation cost, not customer spend. A customer-reported claim from Insight Enterprises puts MAI Cyber models 20% ahead on token efficiency, measured against GPT 5.4, GPT 5.3 Codex and GPT 5.4 Mini.

An aisle between tall steel storage racks in a warehouse
The model is coming to Microsoft's own security products rather than the API. Illustrative stock image. Adrien Olichon · pexels · Pexels License

What is missing

Microsoft has not compared the model with OpenAI’s current dedicated security model. It says GPT-6 Cyber is available only to select customers, and that evaluating it on agentic, tool-using benchmarks requires additional safety safeguards that are still being put in place. The company says it plans to share comparative results in the coming weeks.

That gap matters because the comparison Microsoft does make is with GPT-6 Astra, a general frontier model, rather than with a model built for the same job. OpenAI said on the same day that both GPT-6.2 models meet the Critical threshold for cybersecurity under its Preparedness Framework and that it has tightened default restrictions on higher-risk security assistance as a result — which is part of why a like-for-like test is hard to run.

MAI Cyber 2 Flash is not sold as an API model. Microsoft says it is coming exclusively to its own security products, including one it refers to by the codename MDASH, where it will automatically replace the existing model in the MAI profile for eligible customers starting in mid-December. What to watch is whether any outside evaluator reruns CyberGym-E2E on the full 905 tasks and publishes the difference.