Auditing the name against the code
Agent systems built on local language models frequently advertise cognitive-architecture modules: a global workspace, theory of mind, causal reasoning, active inference. A paper posted to arXiv on 30 October by Peter Chen, Ni Zhang, Ying Xu and Siyuan Zhang argues that the module’s name is routinely treated as evidence that the mechanism exists, and sets out to check whether the named computation actually runs.
The method is deliberately boring and transferable: a four-question static audit, with line-number evidence for every answer, followed by call-chain tracing to see where a computed value goes. The authors apply it to 25 claimed mechanisms in one agent system, pre-register predictions for 11 of those modules, and then run 12 interventions to test them.
The number
In 18 of the 25 modules — 72% — the value the module computes never reaches any production output. The code runs. It produces a result. Nothing downstream consumes it.
The authors name three new patterns for this, which they call idle wiring: ways a module can be connected to the system closely enough to execute, and loosely enough to have no effect. Of 61 criteria frozen before the audit ran, 53 were confirmed, five were falsified, two were partially adjudicated and one was ruled adjudication-ambiguous.
The repairs are the proof
A null result of this kind invites the obvious objection: perhaps the module’s output was never meant to be consumed directly, and its influence is indirect. The authors answer it with two repair experiments run as positive controls — connect the dead path, and see whether anything changes.
One did, dramatically. A planner’s task success rate went from 0.028 to 1.000 once its module’s output was actually routed into the decision it was supposed to inform. In the second repair, a state controller’s error fell by 52%. If the modules had been contributing indirectly, wiring them in properly would not have produced a change of that size.
The authors also replicate the protocol on a second, independent system — auditing its integrated-information-theory phi core against that system’s own pre-committed falsification criteria — and report a 52-check frozen-source audit across 12 files drawn from two hybrid runtimes and one public authorization substrate.
What it means for reading agent papers
The conclusion the authors draw is narrow and worth stating plainly: a module’s name and a module’s computation are two separate claims, and they should be audited separately. A system card, a repository README or an architecture diagram that lists “theory of mind” is reporting the first claim. Whether the second holds is a question about call chains, and the paper’s contribution is a cheap way to ask it.
The audit covers two systems, both built on local models, so it says nothing directly about frontier commercial agents whose code is not available. That is also the point: the same question cannot currently be asked of them.
