Super excited to share the last paper of my PhD: "Hallucination in World Models is Predictable and Preventable"✨
We train a 350M-param generative world model on a large dataset w/ 210 tasks and show that we can predict *when* hallucination happens and use that to fix it!
🧵1/n
Users praised the research showing hallucinations in world models are predictable and preventable, calling the paper and interactive site very nice work after trying it themselves.
No Digg Deeper questions have been answered for this story yet.
Very proud of @ncklashansen . I started working with Nick when I was a postdoc at Berkeley. I think one day Yu Sun told me there is a guy seems sharp and we should get him to do test time training, and that is how everything starts.
Nick’s first work is continuous learning of agents during deployment, that is 2019, and I think it still is the right thing to do even now. Or maybe just getting caught up?
Since then, Nick has been pushing the direction of world model, the line of TD-MPC, long before world model is popular. I guess our work is a bit RL focus and do not have so many fancy pixels.
Nick’s final work at PhD kind of connects the pixels prediction back and does a comprehensive analysis of the hallucinations in world models and how it affects decision making. This draws a good conclusion to his PhD works.
Super excited to share the last paper of my PhD: "Hallucination in World Models is Predictable and Preventable"✨
We train a 350M-param generative world model on a large dataset w/ 210 tasks and show that we can predict *when* hallucination happens and use that to fix it!
🧵1/n
💡Importantly, we find that not all data is created equal; some data collection strategies are more effective at improving the model than others! Our curiosity approach nearly matches expert policy and human data collection while being completely autonomous.
6/n
We find that hallucination in world models is, first and foremost, a *data coverage* problem.
Below viz shows visited state density (blue) and our developed hallucination predictor u_r (purple) for four different tasks. Hallucinations tend to happen in less visited regions!
4/n
We built a super cool interactive web-based paper (with a live demo!) so definitely check that out!
Everything is open-source: code, model, dataset
🌐 Interactive paper: https://www.nicklashansen.com/mmbench2 📄 Paper: https://arxiv.org/abs/2606.27326
Work with my PhD advisor @xiaolonw 🙌
7/n
Modern generative WMs render realistic, action-controllable futures, but they also frequently *hallucinate*: generating visually fluent futures while drifting from the gt dynamics. Why?
To answer this question, we need to control every part of the 🌎 training pipeline.
2/n
We train a 350M-parameter generative WM following the Dreamer 4 recipe: a video tokenizer + symmetric decoder is trained via masked autoencoding, and a latent dynamics model is then trained on top of frozen tokenizer latents via flow matching.
3/n
A simple yet effective strategy for correcting the model is *targeted* data collection. During live environment interaction, we roll out candidate trajs in the WM, score them by predicted hallucination, and execute the most hallucination-prone one.
Example: an unseen maze
5/n
@ncklashansen @xiaolonw Very nice work. Saw your paper when it showed up in arxiv and played around with your interactive site!