The framework eliminates the need for robot-specific training data.
This is the strongest argument in favor of humanoid hands
Turns out you can train humanoid hands without any robot data.
The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.
Users praise the HUG method for training humanoid robot hands on human grasp data from smart glasses because it enables effective zero-shot grasping with thorough real-world benchmarks and highlights promising industry partnerships.
No Digg Deeper questions have been answered for this story yet.
Many more interesting insights and information is on @kevin_y_wu 's post: https://x.com/kevin_y_wu/status/2066924653600227550?s=20 . Many congrats to @kevin_y_wu, @DandanShan_ and the team! You can try HUG yourself at https://github.com/KevinyWu/hug and get demos that look like this.
2/n With 1M-HUGs, HUG is trained to predict MANO hand grasps through a flow-matching transformer, given RGB-D input and a query point on the target object.
In my opinion; one of the most interesting parts is the benchmark (HUG-Bench). We built digital twins of 90 real-world objects and validated HUG in both sim and the real world using this. This is, to our knowledge, one of the most thorough grasping benchmarks out there.
🧐A question I've long been interested in: how can we learn from human hands and transfer that directly to robots?
Our new work, HUG, makes it possible in three simple steps: (1) collect human grasps at scale, (2) learn from them, and (3) retarget for deployment.
4/n Additionally, to support reproducible and more challenging evaluation, we curate HUG-Bench: a set of 90 objects diverse in size and shape (available on Amazon for ~$250), along with their reconstructed 3D counterparts, for evaluation in both simulation and the real world.
1/n Smart glasses like Aria make data collection easier, less tiring, and more scalable.
We start by collecting 1M-HUGs: an egocentric dataset of human grasps spanning 1M frames (27.8 hrs) and 6,707 object instances across 41 buildings.
3/n The predicted MANO human grasps can be retargeted directly to downstream robot hands.
We tested this on the WUJI, Ability, and Inspire hands, and all worked out smoothly, with no per-embodiment retraining.
@chris_j_paxton @chris_j_paxton Curious what kind of sensors they use in tectile? Any guideline or recommendations?
n/n Code, datasets, and checkpoints will be open-sourced!
A big milestone for @kevin_y_wu leading his 1st PhD project, along with the amazing team @Will_Zhou2025, Issac Tu, @BillyYYan, @irmakkguzey, David Fouhey, and @LerrelPinto!
More on our website: http://grasping.io
@chris_j_paxton also a very strong argument for humanoid form factor more generally
It's more than just an argument
This week ABB announced a partnership with @PSYONICinc which flew under the radar but was big news if you read beyond the headline at what it meant
One of Europe's biggest industrial robotics companies is seeing the hand as the preferred method to advance physical AI, not a gripper.
@chris_j_paxton could have a clumsy human vs. non-clumsy human dataset and see which one recovers better
@erpriyankpatel Not sure