What the method does
A paper posted to arXiv on 28 September describes LEGO-Anything, a framework in which a coding agent reconstructs a 3D scene from a single photograph by writing and running Blender code, looking at the result, and revising the program.
The output is not a rendering or a mesh. It is a scene program: executable code whose objects, geometry and camera position are explicit and can be inspected, edited and queried afterwards. The authors make the case for that representation directly, arguing a reconstruction “is most useful when represented not as a rendering or a fixed 3D output, but as an explicit scene program whose execution yields a scene that can be inspected, edited, and queried”.
The benchmark, and why its scoring matters
The same work introduces LEGO-Bench: 208 images drawn from 104 indoor and outdoor scenes, scored separately on three axes — whether the artifact is valid at all, whether the visible-surface geometry matches, and whether the render looks like the photograph.
Keeping those three apart is the design decision that gives the paper its result. A single blended score would let a convincing render paper over geometry that is wrong, which is precisely the failure these systems make.
On that benchmark, GPT-6 Astra was the strongest agent tested, at 53.4% indoors and 39.6% outdoors. LEGO-Plugin, a training-free addition, improved results across six models with gains of up to 62.7%.
The finding worth keeping
The headline number is the self-assessment one. When the agents were asked to judge the geometric accuracy of their own reconstructions, they performed near or below chance.
That is a different failure from being inaccurate. An agent that builds a wrong scene and knows it is wrong can iterate toward a right one; an agent that cannot tell has no gradient to follow. The authors identify three recurring problems in the construction traces — weak initialisation of the scene, edits during iteration that make things worse rather than better, and unreliable self-evaluation — and the third is what makes the second possible.
The practical reading is that the loop these agents run, write code then look at the result then revise, only closes if the looking step works. Here it does not.
What to watch
The paper is a preprint and the benchmark is new, so the scores are a starting point rather than a ranking. The more useful thing to watch is whether a separate verifier — a geometric check the agent does not perform on itself — closes the gap. That is the same shape as the acceptance test other agentic science work has converged on: the model proposes, something else decides whether it is right.
