A post argues that a “Chronos Policy” should model physical time from observation through inference to action execution.
The author calls for every robot policy to be a “Chronos Policy,” explicitly modeling physical time across the system stack. The argument is that the physical world runs on real time, not sequence indices.
6.6K
2 Sources, first seen ago
@SongShuranEvery robot policy should be a 🕒 Chronos Policy — one that explicitly models physical time across the entire system stack, from observation to inference to action execution. Because the physical world runs on real time, not sequence indices.
@vigram_voidRobot policies have a hilariously basic blind spot: they know which camera frame came first, but not necessarily how much time passed between frames. Imagine an object moves 10cm between two images. Did that take 20ms or 200ms? That's a 10x difference in velocity, yet a fixed-rate diffusion policy can see the same two adjacent tokens. @SongShuran's @Stanford team built Chronos Policy around this problem. Instead of treating token position as a clock, they give every observation its actual capture time and every predicted action its intended execution time. The neat part is HOW time enters the transformer. They replace index-based RoPE with time-aware RoPE, so attention uses the physical interval between tokens. Camera frames, proprioception, force readings and future actions can all live on different clocks without being forced onto one artificial sampling grid. Then they randomize sensor rates, delays and action start times during training, while preserving the timestamps. And the ablation is fascinating: on simulated multisensory Pivoting, timing randomization WITHOUT timestamps actually drops average perturbed-condition success from 48.4% to 21.6%. More diverse data made the model worse because it couldn't tell what time the data represented. With time-aware RoPE + randomization: 55.3%. On a simulated moving conveyor, adding up to 200ms of inference delay drops standard Diffusion Policy to 1.8% success. Chronos retains 76.4%. They also tested real manipulation. Under combined sensing and inference disturbances, placing a cup on a moving saucer scored 55% vs 30% for standard Diffusion Policy. Important caveat: that real-world score includes half-credit for partial success, and the cup result is only 10 trials per condition. Code is still forthcoming. But I love the broader implication. We've spent years making robot models understand WHERE things are. A physical system also needs to know WHEN that information was true, and WHEN its decision will actually touch reality. A 200ms-old observation isn't just an old image. It's a different world. https://arxiv.org/abs/2610.13929 #RobotLearning #EmbodiedAI #PhysicalAI@SongShuranEvery robot policy should be a 🕒 Chronos Policy — one that explicitly models physical time across the entire system stack, from observation to inference to action execution. Because the physical world runs on real time, not sequence indices.@vigram_voidRobot policies have a hilariously basic blind spot: they know which camera frame came first, but not necessarily how much time passed between frames. Imagine an object moves 10cm between two images. Did that take 20ms or 200ms? That's a 10x difference in velocity, yet a fixed-rate diffusion policy can see the same two adjacent tokens. @SongShuran's @Stanford team built Chronos Policy around this problem. Instead of treating token position as a clock, they give every observation its actual capture time and every predicted action its intended execution time. The neat part is HOW time enters the transformer. They replace index-based RoPE with time-aware RoPE, so attention uses the physical interval between tokens. Camera frames, proprioception, force readings and future actions can all live on different clocks without being forced onto one artificial sampling grid. Then they randomize sensor rates, delays and action start times during training, while preserving the timestamps. And the ablation is fascinating: on simulated multisensory Pivoting, timing randomization WITHOUT timestamps actually drops average perturbed-condition success from 48.4% to 21.6%. More diverse data made the model worse because it couldn't tell what time the data represented. With time-aware RoPE + randomization: 55.3%. On a simulated moving conveyor, adding up to 200ms of inference delay drops standard Diffusion Policy to 1.8% success. Chronos retains 76.4%. They also tested real manipulation. Under combined sensing and inference disturbances, placing a cup on a moving saucer scored 55% vs 30% for standard Diffusion Policy. Important caveat: that real-world score includes half-credit for partial success, and the cup result is only 10 trials per condition. Code is still forthcoming. But I love the broader implication. We've spent years making robot models understand WHERE things are. A physical system also needs to know WHEN that information was true, and WHEN its decision will actually touch reality. A 200ms-old observation isn't just an old image. It's a different world. https://arxiv.org/abs/2610.13929 #RobotLearning #EmbodiedAI #PhysicalAI@SongShuranEvery robot policy should be a 🕒 Chronos Policy — one that explicitly models physical time across the entire system stack, from observation to inference to action execution. Because the physical world runs on real time, not sequence indices.@vigram_voidRobot policies have a hilariously basic blind spot: they know which camera frame came first, but not necessarily how much time passed between frames. Imagine an object moves 10cm between two images. Did that take 20ms or 200ms? That's a 10x difference in velocity, yet a fixed-rate diffusion policy can see the same two adjacent tokens. @SongShuran's @Stanford team built Chronos Policy around this problem. Instead of treating token position as a clock, they give every observation its actual capture time and every predicted action its intended execution time. The neat part is HOW time enters the transformer. They replace index-based RoPE with time-aware RoPE, so attention uses the physical interval between tokens. Camera frames, proprioception, force readings and future actions can all live on different clocks without being forced onto one artificial sampling grid. Then they randomize sensor rates, delays and action start times during training, while preserving the timestamps. And the ablation is fascinating: on simulated multisensory Pivoting, timing randomization WITHOUT timestamps actually drops average perturbed-condition success from 48.4% to 21.6%. More diverse data made the model worse because it couldn't tell what time the data represented. With time-aware RoPE + randomization: 55.3%. On a simulated moving conveyor, adding up to 200ms of inference delay drops standard Diffusion Policy to 1.8% success. Chronos retains 76.4%. They also tested real manipulation. Under combined sensing and inference disturbances, placing a cup on a moving saucer scored 55% vs 30% for standard Diffusion Policy. Important caveat: that real-world score includes half-credit for partial success, and the cup result is only 10 trials per condition. Code is still forthcoming. But I love the broader implication. We've spent years making robot models understand WHERE things are. A physical system also needs to know WHEN that information was true, and WHEN its decision will actually touch reality. A 200ms-old observation isn't just an old image. It's a different world. https://arxiv.org/abs/2610.13929 #RobotLearning #EmbodiedAI #PhysicalAI@SongShuranEvery robot policy should be a 🕒 Chronos Policy — one that explicitly models physical time across the entire system stack, from observation to inference to action execution. Because the physical world runs on real time, not sequence indices.@vigram_voidRobot policies have a hilariously basic blind spot: they know which camera frame came first, but not necessarily how much time passed between frames. Imagine an object moves 10cm between two images. Did that take 20ms or 200ms? That's a 10x difference in velocity, yet a fixed-rate diffusion policy can see the same two adjacent tokens. @SongShuran's @Stanford team built Chronos Policy around this problem. Instead of treating token position as a clock, they give every observation its actual capture time and every predicted action its intended execution time. The neat part is HOW time enters the transformer. They replace index-based RoPE with time-aware RoPE, so attention uses the physical interval between tokens. Camera frames, proprioception, force readings and future actions can all live on different clocks without being forced onto one artificial sampling grid. Then they randomize sensor rates, delays and action start times during training, while preserving the timestamps. And the ablation is fascinating: on simulated multisensory Pivoting, timing randomization WITHOUT timestamps actually drops average perturbed-condition success from 48.4% to 21.6%. More diverse data made the model worse because it couldn't tell what time the data represented. With time-aware RoPE + randomization: 55.3%. On a simulated moving conveyor, adding up to 200ms of inference delay drops standard Diffusion Policy to 1.8% success. Chronos retains 76.4%. They also tested real manipulation. Under combined sensing and inference disturbances, placing a cup on a moving saucer scored 55% vs 30% for standard Diffusion Policy. Important caveat: that real-world score includes half-credit for partial success, and the cup result is only 10 trials per condition. Code is still forthcoming. But I love the broader implication. We've spent years making robot models understand WHERE things are. A physical system also needs to know WHEN that information was true, and WHEN its decision will actually touch reality. A 200ms-old observation isn't just an old image. It's a different world. https://arxiv.org/abs/2610.13929 #RobotLearning #EmbodiedAI #PhysicalAI—
Not ranked yet
—
Not ranked yet
