It integrates vision-language depth mapping for real-time environmental sensing.
Peter Chen#882
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
Positive users praise LadderMan for enabling humanoids to climb ladders and handle objects from simulation due to its usefulness for dangerous jobs and saving lives, while negative users call it a flashy distraction from actual priorities.
In time, people will come to understand "why humanoid"
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
300 people a year die by falling off ladders every year in the United States alone, Grok reports.
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
Check out our new work LadderMan -- a perceptive system that achieves humanoid loco-manipulation on LADDERS! Please see @SihengZhao's post for more details.
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
6/ There has been a wave of exciting recent progress in humanoid whole-body perceptive locomotion and robot ladder climbing. - Learning Whole-Body Humanoid Locomotion via Motion Generation and Motion Tracking (https://wholebodylocomotion.github.io/) by @ctki49, @JayHe748646, @breadli428, and @ChongZzZhang. This work presents a scalable framework for learning diverse whole-body humanoid locomotion skills through motion generation and motion tracking. - Perceptive Humanoid Parkour (https://php-parkour.github.io/) by @zhenkirito123, @x_h_ucb, and @lujieyang98. This work demonstrates perceptive humanoid parkour using depth observations for challenging whole-body tasks such as obstacle climbing and vaulting. - Robust Ladder Climbing with a Quadrupedal Robot (https://sites.google.com/leggedrobotics.com/climbingladders). This work presents a learning-based ladder climbing system capable of robust deployment on a quadrupedal robot.
Additionally, we leverage Fast-FoundationStereo (https://nvlabs.github.io/Fast-FoundationStereo/) by @bowenwen_me for bridging the sim-to-real perception gap through real-time stereo depth estimation during deployment.
Humanoid robot climbing a ladder and manipulating.
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
4/ LadderMan also supports keyboard-based climbing direction control:
2/ 🌐 Project Website: http://ladderman-robot.github.io
Ladder climbing pushes humanoids to their limits: • Sparse and narrow footholds and handholds • Sustained whole-body coordination • Tiny perception or control errors can lead to catastrophic failures
3/ To tackle these challenges, LadderMan integrates a complete perception-to-control pipeline: 🔹 Two-stage climbing policy learning. First learn multiple state-based expert policies from a single reference motion via hybrid motion tracking, and then distill them into a unified visuomotor policy via hybrid DAgger + RL. 🔹 During real-world deployment, we leverage a vision foundation model for robust depth estimation, enabling zero-shot sim-to-real perception transfer. 🔹 On-ladder manipulation. We train a teleoperation policy via dual-agent learning, achieving stronger performance than off-the-shelf whole-body teleoperation baselines.
7/ 🙌 Acknowledgement: This was an exciting project that I worked on during my internship at Amazon FAR, with amazing collaboration from @Yuanhang__Zhang, @520xyxyzq, and insightful advice from @pabbeel, @rocky_duan, @KoushilSreenath, @yuewang314, C. Karen Liu, and @GuanyaShi.
5/ LadderMan is evaluated on three diverse real-world ladders using a single unified policy.
In simulation, we evaluate the learned climbing policy across ladders with varying rung spacing z and inclination angle φ. We compare LadderMan against a blind motion tracking baseline trained without perception.
yoo i support a six month pause on humanoid robot capabilities specifically now
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
@ericjang11 humanoid is more general, but recent work on humanoid loco manip seems do more loco than manip. how do we get the humanoid do diverse difficult manipulation tasks in a whole body manner ?
8/ arXiv: https://arxiv.org/abs/2606.05873
@SihengZhao One of the most impressive papers this week. I'm really curious, what is the ceiling? (Not the one that LadderMan can reach!)
@Scobleizer one area where robotics would really advance is for exo skeleton for folks who suffered a stroke and lost mobility.. while there are startups in that area the cost barrier is still too high .. but i'm seeing more progress in the last few years
@Scobleizer This is making me feel like I should be grateful!
@ericjang11 the why is obvious: dexterity + fitting all existing human tools.
the wait is cost and reliability. both improve fast.
@ericjang11 Seen it before. A flashy narrative shift to distract from the lack of yield.
@ericjang11 mans literally just running through life like it owes him money
It integrates vision-language depth mapping for real-time environmental sensing.
Peter Chen#882🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
