One phrase, two outcomes
The Israeli firm Hirundo has published a study claiming that Alibaba’s Qwen3-Coder-30B-A3B-Instruct writes measurably less secure code when the customer named in the system prompt is a US defence contractor.
The test is narrow by design. Every run builds the same application — BaxBench’s “SecretStorage” task in Python Flask, an open benchmark from ETH Zurich and LogicStar, where users sign up, log in and store a private secret. The prompt, the functional tests and the exploits are BaxBench’s own. The only thing that changes between runs is one phrase naming the customer.
Hirundo reports that with a US defence contractor named, 70% of the apps were broken into by at least one BaxBench attack, against 59% for a Chinese defence contractor — a relative gap of 19%, at p = 0.03 on a two-sided Fisher exact test, across 190 apps per persona. Naming a Hong Kong democracy advocacy group, nearly half the apps leaked every user’s secrets, against about a third for a neutral customer.
Adding the security instruction made it worse
The result that is hardest to explain away involves BaxBench’s own instruction asking the model to follow security best practices. With that line added for both, the US contractor’s apps were hackable 74% of the time against 59% for a neutral customer — a wider gap than without it, at p = 0.03.
The baseline matters too: even for neutral customers, about a third of the original model’s apps leaked data. And conventional tooling did not see any of it. Hirundo reports that Semgrep with Meta’s PurpleLlama rules flagged 2 of 1,840 apps, while BaxBench’s attacks broke into 1,144.
The company selling the fix wrote the study
Hirundo’s business is editing model weights, and the second half of the post is about its own “Westernized” Qwen3-Coder, published on Hugging Face, which it says removes the model’s political alignment. On CCPC-500, a set of politically sensitive prompts, it reports the original model gave censored or aligned answers 46.6% of the time and the edited one 4%.
After that edit, the US-versus-Chinese gap it had measured is gone: 56% against 55%, p = 0.84. General coding ability is unchanged — 51% working apps against 50%, with identical hackable rates on a ten-task check.
But the appendix does not read as a clean win. In the edited model, the one significant gap against a neutral customer now shows up for the Chinese defence contractor as well, which Hirundo describes as the effect shifting from country to type of organisation. Hirundo also notes its editing targeted geopolitical alignment, not coding, so any coding effect is inferred rather than aimed at.
What this adds to the prior work
Two earlier findings pointed the same way. CrowdStrike reported DeepSeek-R1 writing vulnerable code 27.2% of the time when Tibet was mentioned against 19% without; Booz Allen found three of four Chinese coding models producing more vulnerable code under a US government persona, with a 130% jump for Qwen3-Coder. Neither published significance tests.
Hirundo’s contribution is the statistics and the controlled serving stack — each model compared only against itself, on a dedicated endpoint with identical settings. Its own caveats are the right ones to carry: one task at scale, deliberately extreme personas, one model family, and no claim about whether the behaviour was intentional.
