2/5 📊 Data.

Data is the key to success🔑 We built a large-scale, standardized data pipeline that curates, filters, and dedups broad public robotics and egocentric human datasets. This gives us a clean training dataset with diverse tasks, scenarios, actions, and embodiments.

impressive WM evaluation alignment! real ones know that this is the most impressive result

I think one of the cool things about world models is just how much more efficient they seem to be in both data and compute.

3/5 🦾 Method.

We leverage a 2D kinematic skeleton rendering as a unified control signal. From robot arms to human hands ✋, every embodiment is rendered as an image-aligned 2D skeleton. This enables high-precision action conditioning for video models. 🎯

4/5 📈 Results.

With our cleaned data and kinematic skeleton conditioning, we finetune the Cosmos2.5-2B model on a single GH200 GPU ⚡. Our method delivers superior quality on action following, appearance, and motion consistency, pushing SOTA performance with a fraction of the compute. 🏆

@Haotianxue_GT @wuzy2115 No new modules is one part of the efficiency. The most useful things are: (i) curating a balanced and diverse dataset, (ii) skeleton conditioning as a unified action representation, and (iii) the base model is only a 2B model (Cosmos 2.5)

5/5 🧪 Robot policy evaluation.

Finally, we deploy OSCAR for real-world robot policy evaluation using the episodes from RoboArena. Our extensive experiments show a high correlation ✅ between virtual assessment in our world model and real-world results, paving the way for a future where robot policies are evaluated entirely within generated environments!!

@ericjang11 Thank you for the nice words, Eric! Yes, we believe the most immediate application of world models in robotics is to evaluate different policies at scale, accelerating development speed before cumbersome (and easy to break) real-world deployment.

@JunGao33210520 @wuzy2115 Impressive, it seems much more efficient. Is that because it introduces no new modules?

@JunGao33210520 would you mind uploading the real world policy eval rollouts to huggingface datasets as well?

@JunGao33210520 @wuzy2115 the skeleton is an interesting representation, i wonder if there are any ambiguity problems when the three red arrows are colinear (their plane intersects camera center)?

@JunGao33210520 @wuzy2115 Have you tried to compare to the famous JEPA? I am looking for independent people that has tried it and said it works better than diffusion

@ericjang11 what's the evaluation actually testing? i've watched too many impressive lab benchmarks fail when you hit real-world edge cases. curious if this one avoids that.

@JunGao33210520 @wuzy2115 a world model is the agent rehearsing what happens if i do x before committing. video prediction is just what that rehearsal looks like from outside

@ericjang11 'real ones know' is tech's most efficient shortcut: declare something the best without all that tedious comparing

@ericjang11 licensing is the real wall. you can run evals on anything. but sharing it? ip/privacy/liability questions pop up and labs can't afford the legal review. seen this kill so many promising datasets.

@JunGao33210520 @wuzy2115 The combination of cross-embodiment generalization and full reproducibility stands out here. It's encouraging to see strong robotics results paired with open data, open weights, and a compute budget that remains accessible to the broader research community.

@chris_j_paxton the compute part is the real standout to me, single GH200 doing the lifting