An agent that writes down the rules as it plays
Researchers at University College London, Huawei’s Noah’s Ark Lab in the UK and the University of Liverpool have published a paper describing an agent that clears every level of all 25 public games in ARC-AGI-3, the interactive version of the ARC benchmark, and does it in 44% of the actions a human player needed. The paper was posted to arXiv on 8 October.
ARC-AGI-3 drops an agent into a grid-world game with no instructions. It is not told the goal, the controls or the rules, and it cannot read the environment’s source. It has to work all of that out by playing. Scores are reported as Relative Human Action Efficiency, or RHAE, which weights each level by how many actions a human baseline needed and tops out at 100.
The agent, called Memento 3, reaches that ceiling: a mean RHAE of 100.0 across all 25 games and 183 levels. What makes the result unusual is that the language model underneath never changes. The authors run the system in Claude Code with Claude Opus 5 as the backbone and reasoning effort set to extra-high, and the weights stay frozen throughout.
The learning happens in a file, not in the weights
Instead of training, the agent keeps a rulebook: a natural-language document of what it currently believes about how the environment works, with the parts it has not figured out left deliberately underspecified. It then compiles that rulebook into executable Python and uses the program to predict what the game will do next and to plan.
When a prediction fails, the agent revises the rulebook and recompiles. A revision is only accepted on two conditions: the model has to judge the new code faithful to the rulebook, and a cell-exact replay of what actually happened in the game has to reproduce the observed transitions. The whole workspace — rulebook and compiled modules — lives in a Git repository, committed and tagged after every iteration, which is how the agent can walk back a hypothesis a later level disproves. The authors frame this as a model-based route to recursive self-improvement, and describe a population extension that keeps several competing world models running in parallel and shares evidence between them.
The honest comparison is the action count
A ceiling score invites a careless headline, so it is worth being precise about what is new. Memento 3 is not the first system to clear every public ARC-AGI-3 game. The closest entry in the paper’s own comparison table, baseline1, scored 99.0 and also finished all 25. The next systems down are further back: NOOA at 85.1 with 19 games cleared, OPINE-World at 78.4 with 20, DreamTeam at 38.1 with 6 and Continual Harness at 20.5 with 3.
Where the gap opens is efficiency. Memento 3 finishes the set in 7,518 actions against baseline1’s 8,347 and a human baseline of 17,135, a ratio of 0.44. Per-game ratios run from 0.20 to 0.67, with a median of 0.42. Against a plain Claude Opus 5 running the ARC Prize Standard harness, the paper reports a mean RHAE 59.3 points higher — which says less about Memento 3 than about how much of the result comes from the scaffolding rather than the model.
All the baseline figures, including the human action baseline, are taken from each system’s official ARC Prize scorecard, so the columns are counted the same way.
What the paper does not claim
The 100.0 is on the 25 public games. The paper does not report a result on a held-out set, and the ARC Prize Foundation’s own measurement is on games the entrant has not seen. Saturating the public split is a real engineering result; it is not the same as the benchmark being solved, and the benchmark’s maintainers have been explicit that the private set is what counts.
The authors also tested whether the same mechanism works outside discrete planning. In an Atari Pong case study, the agent learned a feedback controller that won 21:0 in each of three evaluated episodes with different openings, with no further calls to the language model once the controller was in place.
What to watch is whether an ARC Prize scorecard appears for Memento 3 on the held-out games, and what the efficiency number looks like when the environment is one the rulebook has never had a chance to be wrong about.
