Training required only 15 hours on a single GPU.
New Paper: Human-like Autonomy Emerges from Self-Play and a Pinch of Human Data.
We trained self-play RL on 60 years of simulation on 1 GPU in ~15 hours. Regularizing with 30 minutes of demonstration data produces much more human-like driving policies!
Users are praising the self-play RL paper on human-like driving policies with minimal data because they find it awesome, coherent, well-written, interesting, and superhuman.
No Digg Deeper questions have been answered for this story yet.
@daphne_cor Unregularized is a better driver. Deploy it in real life. Superhuman
Project page: https://spiced-self-play.com/ Arxiv: https://arxiv.org/abs/2606.19370
New Paper: Human-like Autonomy Emerges from Self-Play and a Pinch of Human Data.
We trained self-play RL on 60 years of simulation on 1 GPU in ~15 hours. Regularizing with 30 minutes of demonstration data produces much more human-like driving policies!
@daphne_cor The reason unregularized drives the way it does is because it has god sight. Give it only raytraced pixels by simulating lidar
A big thank you to my collaborators for their unique contributions to this work: @julianh651, Zixu Zhang, Waël Doulazmi, @kev_joseph_, Jaime Fernández Fisac, and @EugeneVinitsky!
@daphne_cor @yacineMTB I want to see the alien driving policies
@daphne_cor @julianh651 @kev_joseph_ @EugeneVinitsky very nice. thank you for a coherent, well-written paper! actually interesting research too.
@DanAdvantage @julianh651 @kev_joseph_ @EugeneVinitsky Thank you for the kind words! I hope it led to some high-advantage transitions
@daphne_cor This is awesome Daphne!!
@daphne_cor Cool!!!
@daphne_cor regularized respects traffic, unregularized respects gradients:)
@yacineMTB @daphne_cor dont forget to put in human road-rage
@marcospereeira @yacineMTB We provide a couple of comparisons on the webpage! https://spiced-self-play.com/
e.g., see yellow car here
@daphne_cor @julianh651 @kev_joseph_ @EugeneVinitsky haha thanks for the irl chuckle
@yacineMTB @daphne_cor what they don't show is regularized still driving to point b after 10000000 hours meanwhile unreg is at the cabana sipping champagne
Hi! You could make the policies more cautious by intentionally creating more uncertainty in the environment. But that alone won't give you human-like policies. For context, the reward we specify is high-level and sparse (by design) - it only tells the agent: get to the goal destination safely. Since there are multiple possible ways to solve this perfectly (solution space is large), there is no guarantee that the RL policy converges to the same solution as people do. We talk about this more in the introduction, if you're interested. 🙂
