Siheng Zhao's LadderMan system trains a single visuomotor policy entirely in simulation then runs it directly on the Unitree G1, letting the 1.3 m humanoid climb ladders, stabilize itself, and handle objects without any real-world fine-tuning or extra data.
Peter Chen#765
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
A vision foundation model supplies depth estimates from the onboard RealSense camera while the policy learned via hybrid DAgger and RL keeps locomotion, balance, and manipulation inside one loop.
The authors intend to release training code, inference code, and deployable models, though results so far cover only the G1 and the specific ladders shown in the demonstrations.
Positive users praise LadderMan for its impressive zero-shot ladder climbing and object manipulation by humanoid robots, while the negative reply dismisses the work as flashy narrative meant to distract from yield shortcomings.
In time, people will come to understand "why humanoid"
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
300 people a year die by falling off ladders every year in the United States alone, Grok reports.
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
"why humanoid" gets another case study? Expansion of capability envelope of humanoids and importantly in a very data and compute efficient manner.
the recipe that worked for locomotion in increasingly working on loco-manipulation and perceptive locomotion.
- data to generate reference ("what") - RL in sim to create experts ("how") - Distillation to add visual observations - Deployment on real env.
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
Humanoid🪜
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
Check out our new work LadderMan -- a perceptive system that achieves humanoid loco-manipulation on LADDERS! Please see @SihengZhao's post for more details.
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
Humanoid robot climbing a ladder and manipulating.
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
6/ There has been a wave of exciting recent progress in humanoid whole-body perceptive locomotion and robot ladder climbing. - Learning Whole-Body Humanoid Locomotion via Motion Generation and Motion Tracking (https://wholebodylocomotion.github.io/) by @ctki49, @JayHe748646, @breadli428, and @ChongZzZhang. This work presents a scalable framework for learning diverse whole-body humanoid locomotion skills through motion generation and motion tracking. - Perceptive Humanoid Parkour (https://php-parkour.github.io/) by @zhenkirito123, @x_h_ucb, and @lujieyang98. This work demonstrates perceptive humanoid parkour using depth observations for challenging whole-body tasks such as obstacle climbing and vaulting. - Robust Ladder Climbing with a Quadrupedal Robot (https://sites.google.com/leggedrobotics.com/climbingladders). This work presents a learning-based ladder climbing system capable of robust deployment on a quadrupedal robot.
Additionally, we leverage Fast-FoundationStereo (https://nvlabs.github.io/Fast-FoundationStereo/) by @bowenwen_me for bridging the sim-to-real perception gap through real-time stereo depth estimation during deployment.
4/ LadderMan also supports keyboard-based climbing direction control:
2/ 🌐 Project Website: http://ladderman-robot.github.io
Ladder climbing pushes humanoids to their limits: • Sparse and narrow footholds and handholds • Sustained whole-body coordination • Tiny perception or control errors can lead to catastrophic failures
3/ To tackle these challenges, LadderMan integrates a complete perception-to-control pipeline: 🔹 Two-stage climbing policy learning. First learn multiple state-based expert policies from a single reference motion via hybrid motion tracking, and then distill them into a unified visuomotor policy via hybrid DAgger + RL. 🔹 During real-world deployment, we leverage a vision foundation model for robust depth estimation, enabling zero-shot sim-to-real perception transfer. 🔹 On-ladder manipulation. We train a teleoperation policy via dual-agent learning, achieving stronger performance than off-the-shelf whole-body teleoperation baselines.
yoo i support a six month pause on humanoid robot capabilities specifically now
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
7/ 🙌 Acknowledgement: This was an exciting project that I worked on during my internship at Amazon FAR, with amazing collaboration from @Yuanhang__Zhang, @520xyxyzq, and insightful advice from @pabbeel, @rocky_duan, @KoushilSreenath, @yuewang314, C. Karen Liu, and @GuanyaShi.
5/ LadderMan is evaluated on three diverse real-world ladders using a single unified policy.
In simulation, we evaluate the learned climbing policy across ladders with varying rung spacing z and inclination angle φ. We compare LadderMan against a blind motion tracking baseline trained without perception.
@Scobleizer right becase it HAS to be bipedal...lol
@ericjang11 humanoid is more general, but recent work on humanoid loco manip seems do more loco than manip. how do we get the humanoid do diverse difficult manipulation tasks in a whole body manner ?
@SihengZhao cool!
@SihengZhao amazing works!
@SihengZhao Really impressive
8/ arXiv: https://arxiv.org/abs/2606.05873
Siheng Zhao's LadderMan system trains a single visuomotor policy entirely in simulation then runs it directly on the Unitree G1, letting the 1.3 m humanoid climb ladders, stabilize itself, and handle objects without any real-world fine-tuning or extra data.
Peter Chen#765🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
