What the paper does
A paper posted to arXiv on 5 October, H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning, trains a stack of action-conditioned JEPA world models instead of one.
The idea is a division of labour in time. Each level predicts farther ahead than the one below it, in its own learned latent space. Planning then runs top-down: the top level sketches a coarse route, and what it predicts becomes the subgoal the level beneath it has to reach, down to the level that emits actions. Nothing is hand-specified; the hierarchy is learned end to end.
Joint-embedding predictive architectures already avoid predicting pixels, which is what makes them cheap enough to plan with. The contribution here is that one flat latent space turns out to be the wrong granularity for a long task, and that the fix is more levels rather than a bigger model.
The numbers
The headline result is on Visual AntMaze, where a three-level hierarchy raised the success rate from 18% to 73% against a flat JEPA, and did it with less planner compute. That is the comparison the authors lead with on the project page and in the abstract.
Across the four simulated environments they test — AntMaze, FourRoom, Push-T and the OGBench Cube task — adding levels up to three improves the trade-off between success and compute. The gains are not uniform: on Push-T a two-level model matches or exceeds the flat baseline, which is the kind of result that tells you the hierarchy earns its keep on long-horizon tasks rather than everywhere.
VentureBeat reports that in AntMaze the three-level model reached nearly twice the success rate of HWM, another hierarchical world model, which is the comparison that matters more than the flat baseline.
The part that points at real robots
The simulated results are the easy half. The authors also apply the method to DROID, a dataset of real robot manipulation video, and add an inverse-dynamics objective — training the model to recover which action connects two observations. They report that this keeps action-relevant information in the representation that would otherwise be discarded, and improves offline planning fidelity with less planner compute. The project page states this qualitatively rather than with a table.
Who wrote it
The authors are Wancong Zhang, Basile Terver, Michael Rabbat, Yann LeCun and Randall Balestriero, with affiliations listed as NYU, Advanced Machine Intelligence, INRIA Paris and Brown University. Zhang and Terver are equal contributors; LeCun and Balestriero are equal advisors. VentureBeat describes AMI as LeCun’s Advanced Machine Intelligence Labs. Code is published at github.com/kevinghst/H-JEPA.
The lineage matters for reading the claim. LeCun has argued for years that predicting in a learned representation, not in pixels, is the route to planning agents, and that the missing piece is abstraction at more than one timescale. This is that argument with numbers attached.
What to watch
Whether anyone reproduces the AntMaze jump outside the authors’ own setup, and whether the DROID extension turns into a quantitative result on a real robot rather than offline planning fidelity on recorded video. Those are different claims, and only the first one is currently measured.
