Siheng Zhao's LadderMan system trains a single visuomotor policy entirely in simulation then runs it directly on the Unitree G1, letting the 1.3 m humanoid climb ladders, stabilize itself, and handle objects without any real-world fine-tuning or extra data.
Peter Chen#719
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
A vision foundation model supplies depth estimates from the onboard RealSense camera while the policy learned via hybrid DAgger and RL keeps locomotion, balance, and manipulation inside one loop.
The authors intend to release training code, inference code, and deployable models, though results so far cover only the G1 and the specific ladders shown in the demonstrations.
Positive users call the LadderMan humanoid robot paper on zero-shot ladder climbing and manipulation impressive and amazing, while negative users dismiss it as a flashy narrative shift seen before to distract from lack of yield.
In time, people will come to understand "why humanoid"
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
Humanoid🪜
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
6/ There has been a wave of exciting recent progress in humanoid whole-body perceptive locomotion and robot ladder climbing. - Learning Whole-Body Humanoid Locomotion via Motion Generation and Motion Tracking (https://wholebodylocomotion.github.io/) by @ctki49, @JayHe748646, @breadli428, and @ChongZzZhang. This work presents a scalable framework for learning diverse whole-body humanoid locomotion skills through motion generation and motion tracking. - Perceptive Humanoid Parkour (https://php-parkour.github.io/) by @zhenkirito123, @x_h_ucb, and @lujieyang98. This work demonstrates perceptive humanoid parkour using depth observations for challenging whole-body tasks such as obstacle climbing and vaulting. - Robust Ladder Climbing with a Quadrupedal Robot (https://sites.google.com/leggedrobotics.com/climbingladders). This work presents a learning-based ladder climbing system capable of robust deployment on a quadrupedal robot.
Additionally, we leverage Fast-FoundationStereo (https://nvlabs.github.io/Fast-FoundationStereo/) by @bowenwen_me for bridging the sim-to-real perception gap through real-time stereo depth estimation during deployment.
4/ LadderMan also supports keyboard-based climbing direction control:
2/ 🌐 Project Website: http://ladderman-robot.github.io
Ladder climbing pushes humanoids to their limits: • Sparse and narrow footholds and handholds • Sustained whole-body coordination • Tiny perception or control errors can lead to catastrophic failures
3/ To tackle these challenges, LadderMan integrates a complete perception-to-control pipeline: 🔹 Two-stage climbing policy learning. First learn multiple state-based expert policies from a single reference motion via hybrid motion tracking, and then distill them into a unified visuomotor policy via hybrid DAgger + RL. 🔹 During real-world deployment, we leverage a vision foundation model for robust depth estimation, enabling zero-shot sim-to-real perception transfer. 🔹 On-ladder manipulation. We train a teleoperation policy via dual-agent learning, achieving stronger performance than off-the-shelf whole-body teleoperation baselines.
7/ 🙌 Acknowledgement: This was an exciting project that I worked on during my internship at Amazon FAR, with amazing collaboration from @Yuanhang__Zhang, @520xyxyzq, and insightful advice from @pabbeel, @rocky_duan, @KoushilSreenath, @yuewang314, C. Karen Liu, and @GuanyaShi.
5/ LadderMan is evaluated on three diverse real-world ladders using a single unified policy.
In simulation, we evaluate the learned climbing policy across ladders with varying rung spacing z and inclination angle φ. We compare LadderMan against a blind motion tracking baseline trained without perception.
yoo i support a six month pause on humanoid robot capabilities specifically now
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
Humanoid robot climbing a ladder and manipulating.
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
@ericjang11 humanoid is more general, but recent work on humanoid loco manip seems do more loco than manip. how do we get the humanoid do diverse difficult manipulation tasks in a whole body manner ?
@SihengZhao cool!
@SihengZhao amazing works!
8/ arXiv: https://arxiv.org/abs/2606.05873
@SihengZhao One of the most impressive papers this week. I'm really curious, what is the ceiling? (Not the one that LadderMan can reach!)
@SihengZhao To the moon 🚀
@SihengZhao wireman :)
@ericjang11 Seen it before. A flashy narrative shift to distract from the lack of yield.
@ericjang11 mans literally just running through life like it owes him money
Siheng Zhao's LadderMan system trains a single visuomotor policy entirely in simulation then runs it directly on the Unitree G1, letting the 1.3 m humanoid climb ladders, stabilize itself, and handle objects without any real-world fine-tuning or extra data.
Peter Chen#719🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
