Kushal K. and Tyler Lum develop SimTooReal to train robotic tool manipulation without human demonstration data
The policy transferred directly to a physical KUKA arm.
#732Chris Paxton@CHRIS_J_PAXTONA general-purpose robot will need to be able to use tools, as the hand isnt the optimal interface to the world for every task.
We talked to @kushalk_ and @tylerlum23 about their new work SimTooReal on robopapers, which uses sim to train general-purpose tool use ->
RoboPapers@RoboPapersHumans use tools to perform almost all of the physical work that we do from day to day. However, tools come in many different sizes and shapes, and it’s very difficult to collect human data for them in general. What about training in simulation? SimTooReal aims to address this through, unsurprisingly, sim-to-real learning. @kushalk_ and @tylerlum23 talk about how it works: they procedurally generate tool-like objects, and then train with the universal objective of moving objects around to different locations. This creates a general-purpose model which can manipulate various tools to perform a variety of tasks in the real world. Watch episode #82 of RoboPapers, hosted by @micoolcho and @DJiafei, now to learn more!