LeCun's Startup Unveils H-JEPA A Hierarchical World Model for Visual Planning

A first result from LeCun's startup

AMI (Advanced Machine Intelligence), Yann LeCun's world-model startup, has released its debut research paper. The paper introduces H-JEPA, a hierarchical world model for visual planning, co-authored with researchers from New York University, INRIA Paris, and Brown University. Alongside the paper, the team has open-sourced the full code and pre-trained model weights on Hugging Face.

The idea had been brewing since LeCun first proposed hierarchical JEPA (Joint Embedding Predictive Architecture) back in 2022 during his time at Meta. The core insight is simple: instead of asking one flat model to handle everything from big-picture goals to tiny physical movements, you stack multiple layers, each responsible for a different time scale.

How hierarchy makes planning work

In H-JEPA, each layer has its own state encoder, action encoder, and predictor. The lowest layer reads raw images; higher layers abstract further, keeping only the information relevant at their time scale. When planning, the top layer formulates a rough, long-term plan and hands down a predicted intermediate state as a sub-goal. Lower layers then refine that sub-goal into actual executable actions.

This division of labor mirrors how humans plan a trip: first you decide on the destination and flight, then you figure out which taxi to take, and finally you focus on where to place each foot on the stairs.

The results are striking. In the Visual AntMaze simulation maze task, a three-layer H-JEPA achieved a 73 percent success rate, compared to just 18 percent for a single-layer model ¹. The hierarchical approach also delivered better performance with less planning computation across multiple simulation tasks.

Solving a classic AI problem

Training hierarchical world models comes with a well-known risk called representation collapse, where the model compresses all inputs into an identical vector and effectively learns nothing useful. To counter this, H-JEPA builds on SIGReg, a technique previously proposed by LeCun's team in LeJEPA, which constrains how representations are distributed so different states stay distinguishable ⁴.

The layers are trained together end-to-end, linked by those sub-goals: the high layer predicts a future state, the low layer maps that prediction into the high-level space, compares it to its assigned sub-goal, and adjusts its actions accordingly. Interestingly, in the maze experiments researchers observed that the high layer gradually stopped tracking low-level details like leg posture, focusing instead on the task-relevant structure of the environment.

Why open source matters

LeCun made open sourcing a founding principle of AMI, and H-JEPA follows through on that commitment. Researchers can download the code and weights directly and reproduce the planning experiments without needing proprietary access. For a field still dominated by closed research from large labs, that transparency is notable.

The paper lists authors Wancong Zhang, Basile Terver, Mike Rabbat, Yann LeCun, and Randall Balestriero, signaling that this is a collaboration between industry and academia rather than a solo lab effort.

As world-model research moves from theory toward practical planning systems, H-JEPA gives the community its first open, hierarchical blueprint to build on.

Key facts

• H-JEPA achieves 73% success in Visual AntMaze vs 18% for a single-layer model
• Full code and pre-trained weights are open-sourced on Hugging Face
• The model stacks multiple JEPA layers for planning at different time scales
• SIGReg is used to prevent representation collapse during training
• Co-authored with researchers from NYU, INRIA Paris, and Brown University

Sources

1. H-JEPA: LeCun’s World Model Startup Unveils Groundbreaking Debut Results with Full Code and Weights Open-Sourced — eu.36kr.com
2. H-JEPA: LeCun's World Model Startup Delivers, Code and Weights Fully ... — htx.com
3. jepa-world-models/h-jepa · Hugging Face
4. LeCun's AMI debuts H-JEPA: hierarchical world… · AGI Hunt — agihunt.info