Robotics Teams Weigh World Models Against Direct Robot Data Collection
Reactions from ranked influencers
2 posts
Many robotics teams are investing heavily in world models for reasons such as better generalization, improved sample efficiency, and the ability to leverage internet-scale video data. But I've been wondering about the economics. Given a fixed total budget (compute + data collection), which investment leads to better robotic capabilities? 1. Train a large world model on internet-scale video and adapt it to robotics. 2. Spend the same resources collecting as much diverse, high-quality robot interaction data as possible (e.g., UMI-style), covering a wide range of tasks, environments, and edge cases, and train policies directly. At what scale does a world model become more cost-effective than simply collecting additional robot data?
Many robotics teams are investing heavily in world models for reasons such as better generalization, improved sample efficiency, and the ability to leverage internet-scale video data. But I've been wondering about the economics. Given a fixed total budget (compute + data collection), which investment leads to better robotic capabilities? 1. Train a large world model on internet-scale video and adapt it to robotics. 2. Spend the same resources collecting as much diverse, high-quality robot interaction data as possible, covering a wide range of tasks, environments, and edge cases, and train policies directly. At what scale does a world model become more cost-effective than simply collecting additional robot data?