Why the usual experiment proves little
Whether a model was trained on a particular document is the question underneath several arguments at once: copyright claims, privacy complaints, and whether a benchmark score means anything. Membership inference is the technique used to answer it, and a paper posted to arXiv on 30 October argues that most evaluations of it are not evidence either way.
The flaw is in how test sets are assembled. Researchers usually collect members and non-members after training has happened, which means the two groups can differ in topic, style or era as well as in membership. A method that appears to succeed may be detecting that difference, and a method that fails has not been shown to be useless — it may simply have been given an impossible pairing.
A re-run with a coin flip
The authors removed that ambiguity by rebuilding the training run. They re-ran the official 50-billion-token mid-training stage of OLMo 2 1B, replacing selected training chunks with documents drawn from 7,814 pairs matched on domain and difficulty. For each pair, a coin flip decided which document went into training and which was held out.
The counterfactual is the part that makes this work. The run released by the Allen Institute for AI from the same seed saw identical data everywhere else, so it serves as a reference model that differs from the experimental run only in the inserted documents.
One exposure is not enough
The headline result is a negative one, stated precisely. A single exposure does raise a previously unseen document’s likelihood on average — but it rarely identifies it.
Using the counterfactual as the reference, the reference-ratio method reaches an area under the ROC curve of 0.767, which sounds respectable. Applied as a practical test, with one threshold held across domains, it catches 3.8% of members at a 1% false-positive rate. That is the number that matters to anyone who wants to make a claim about a specific document, and it is close to useless.
Thirteen documents are enough
The same data supports a far stronger claim at the level of a collection. With 13 matched pairs, a paired statistical test reaches 80% power at the 5% level to detect that a collection was trained on.
The conditions are stated: same-domain non-members, and a close reference model. Both are real constraints — an outsider examining a commercial model has neither a counterfactual run nor, usually, matched non-members.
What this means for the arguments it touches
The result cuts in two directions and does not favour either side cleanly. A rights holder asking whether their specific book was in a training set is asking the question the evidence cannot answer from one exposure. A rights holder asking whether their catalogue was is asking a question that thirteen documents can settle, if the comparison is constructed properly.
For benchmark contamination the implication is the same: individual item-level detection is weak, collection-level detection is not.
What to watch
The experiment was possible because OLMo 2 is an open model with a published training recipe, released seeds and reproducible stages. No frontier lab offers that, so the method cannot be pointed at the models the legal arguments are actually about. Watch for whether anyone builds an approximation of a counterfactual reference for a closed model — and for whether the thirteen-pair result is used in a filing.
