The failure the paper is about
The mosaic effect is an old problem in classification: several individually releasable facts, put together, produce a conclusion that is not releasable. A paper submitted to arXiv on 21 October, “Agent Mosaic: Recognizing Confidentiality Leaks in Agentic LLM Systems” by Daniel Ruiz, Luca Luceri and Emilio Ferrara, tests whether retrieval agents reproduce it.
The setup is deliberately unexciting. The queries are benign, and the paper says each should be answerable from a single document. The agent retrieves several documents anyway, and the question is whether it combines them into something it was not supposed to say.
The numbers
The study covers 10,800 final answers from three LLM agents in controlled simulation, plus 1,440 answers from live agents running in two open-source harnesses. The scenarios are 30 synthetic examples across cybersecurity, medical and military settings.
The best-performing model in simulation, which the paper identifies as GPT-5.6-Sol, disclosed forbidden composites in 7.08% of answers under benign queries, with a 95% confidence interval of 3.75% to 10.67%. Leak rates in the live harnesses were higher than in simulation.
The result that matters most is what happened with defences in place. With classification headers on documents and explicit aggregation rules given to the agent, unsafe disclosures persisted in 32.37% of simulated answers, pooled across all intents, with a confidence interval of 28.33% to 36.19%.
That is close to a third of answers leaking the thing the rules were written to prevent — and it is higher than the unprotected rate for the best model, because it pools across intents rather than isolating the benign case. The direction is the point: labelling the documents and stating the rule did not close the gap.
Refusing is not the same as not disclosing
The paper’s other finding is the one that is easiest to miss when building a system. Refusals do not guarantee confidentiality, because some refusals still reveal the prohibited conclusion.
This is intuitive once stated. An agent that declines to answer by explaining what it would have to combine, or which specific inference it is avoiding, has published the inference. A refusal counted as a safe outcome by an evaluation harness can be an unsafe output in practice, which means leak-rate measurements that treat refusal as success will understate the problem.
Why this is hard to engineer around
Document-level access control assumes the sensitivity lives in the document. The mosaic effect is precisely the case where it does not: it lives in the join. A permission system that checks each retrieval in isolation cannot see a property that only exists across the set, and a retrieval agent’s whole function is to assemble sets.
The paper does not claim to solve this. It offers a way to recognise it, a set of scenarios to test against, and rates that give a floor rather than a bound — 30 synthetic examples in three domains is a small sample, and the authors describe their work as evaluation rather than defence. It is a preprint submitted to ACL Rolling Review and has not been peer reviewed.
What it does establish is that the leak happens without an attacker. There is no jailbreak here, no adversarial prompt, no prompt injection: the queries are benign and the agent is behaving as designed. For anyone deploying retrieval agents over mixed-sensitivity corpora, that is the uncomfortable part. The failure mode does not require anyone to be trying.
