The term “world model” is tossed around a lot but this is the real deal. From video and actions, the model learns a consistent representation of the physics, 3D environment, and game state of Rocket League. The multi-player formulation demonstrates that the model maintains a consistent representation of the world over long time horizons.
I am excited to have been part of this project because I see this a as step towards learning world models that can generalized to the real world. I’m a believer in learning from synthetic data (Sintel, SURREAL, AGORA, BEDLAM, BEDLAM2) and I’ve seen how models trained on synthetic data can generalize.
MIRA is an important step because it’s learned from video gameplay generated completely by bots. This provides a pathway to scaling and scaling is key to real-world generality. Traditional graphics and games provide a path to learning rich models of the world.
Check out the detailed tech report: https://mira-wm.com/paper/
Introducing MIRA.
A playable, multiplayer world model. A dream of Rocket League.
Trained on 10k hours of data collected with publicly available bots, MIRA learns the dynamics of a four-player game. The model runs in real time at 20 fps, based on the keys you and the other players press.
Built by General Intuition and @kyutai_labs, in collaboration with Epic Games. Not used to develop Rocket League.
▶️ Play the demo, read the technical report, and explore the open-source code at http://generalintuition.com/mira
At ICML? Find us at Booth 111 to try it yourself and dig into the results with the team.
