A new pair of methods lets standard robot arms sense external forces through their existing joint sensors alone, opening force-aware teleoperation and contact-rich training to lower-cost platforms that previously lacked dedicated torque hardware.
What if some parts of a robot demonstration are more important than others?
Most of a trajectory is free-space motion. But success or failure is often determined by a few critical moments around contact.
In FACTR 2, we use force to find these moments and prioritize them for training. We find this helps policies learn better alignment and recovery behaviors, like the example below.
w/ @StevenOh_ @JasonJZLiu
🧵(1/N)
The approach flags critical pre-contact and touch segments from estimated force signals then upsamples them during behavior cloning, which helps policies learn recovery and alignment faster on tasks such as block stacking and bottle capping.
Evaluations cover both low-cost arms around $2500 and pricier ones near $30000, yet full training code and model weights remain referenced rather than confirmed as publicly available.
Users praise FACTR 2 for adding external force sensing to commodity robot arms without extra hardware or sensors, calling the approach brilliant and a game-changer for data-efficient training.
Force is arguably the most overlooked ingredient in modern robot learning.
Introducing FACTR 2: it turns *any* commodity robot into a force-aware system with no force sensors required.
Train a tiny force network in <1min with <10mins of data and drop it into any existing teleop pipelines:
✅ Free force sensing for both the robot and the operator arm ✅ Makes demos higher-quality → fewer of them needed. ✅ A new force-aware learning algorithm (FIRST) uses those recovered forces to figure out which parts of a demo actually matter, making learning data-efficient. ✅ Strong performance on complex tasks with fewer demos and even no pretraining!
More details below.
💥Introducing FACTR 2, learning external force sensing on commodity robot arms without needing dedicated sensors.
We show that learned force signals enable force-feedback teleop on low-cost arms and improve BC policies.
FACTR 2 consists of: 1. Neural External Torque (NEXT): learns external forces without needing dedicated force sensors. 2. Force-Informed Re-Sampling Training (FIRST): uses the learned force signal to identify task-critical regions and upsample them during training.
w/ @StevenOh_ @_tonytao_
🧵(1/N)
Contact is the hard part
You can often tell who is faking it based on how much theyre avoiding making sustained contact with stuff
Cool work
What if some parts of a robot demonstration are more important than others?
Most of a trajectory is free-space motion. But success or failure is often determined by a few critical moments around contact.
In FACTR 2, we use force to find these moments and prioritize them for training. We find this helps policies learn better alignment and recovery behaviors, like the example below.
w/ @StevenOh_ @JasonJZLiu
🧵(1/N)
New work on FACTR 2: Learning External Force Sensing for Commodity Robot Arms Improves Policy Learning:
Paper: https://arxiv.org/abs/2606.12406 Web: https://jasonjzliu.com/factr2/
FACTR 2 shows that learned force signals can both enable force-feedback teleoperation on low-cost manipulators and improve behavior cloning (BC) policies for contact-rich tasks. It consists of two components:
1. Neural External Torque Estimation (NEXT): A lightweight model that infers external joint torques without dedicated force sensors. 2. Force-Informed Re-Sampling Training (FIRST): A training strategy that uses the learned force signal to identify and upsample task-critical moments.
The key insight is simple: policy failures rarely occur in free space, they occur during brief pre-contact alignment and contact-rich interactions, where precise corrections matter most.
Together, NEXT and FIRST bring force-aware teleoperation and robust long-horizon contact-rich policy learning to off-the-shelf robot arms, without requiring additional sensing hardware.
See a more detailed thread by @JasonJZLiu.
💥Introducing FACTR 2, learning external force sensing on commodity robot arms without needing dedicated sensors.
We show that learned force signals enable force-feedback teleop on low-cost arms and improve BC policies.
FACTR 2 consists of: 1. Neural External Torque (NEXT): learns external forces without needing dedicated force sensors. 2. Force-Informed Re-Sampling Training (FIRST): uses the learned force signal to identify task-critical regions and upsample them during training.
w/ @StevenOh_ @_tonytao_
🧵(1/N)
We first introduce NEXT: Neural External Torque, which learns external joint torque without dedicated force sensors.
With just 10 min of free-space data and 1 min of training, NEXT learns to predict the torque needed for contact-free motion.
At runtime, subtracting this prediction from measured motor torque gives external torque.
🧵(2/N)
Special shoutout to @StevenOh_.
Steven has been visiting us at CMU from Japan for the past few months and worked incredibly hard on this project. I’m very proud of what he has accomplished, and excited for him to start his PhD at UChicago.
Please check out his thread for more details.
Leveraging this, we introduce FIRST: Force-Informed Re-Sampling Training.
FIRST simply upsamples the contact-relevant parts of demonstrations during behavior cloning, biasing training on the moments that matter most.
Validation curves show that contact phases are harder than free-space motion, and FIRST specifically reduces error in these phases.
🧵(3/N)
This work was done @CMU_Robotics with co-lead @StevenOh_ and @_tonytao_ as well as @yangphiliphan, @kenny__shaw, @funabashihand, @rsalakhu, @pathak2206.
Website: http://jasonjzliu.com/factr2 Paper: https://arxiv.org/abs/2606.12406
🧵(6/N)
We validate NEXT on the Franka, an arm with dedicated force sensing.
Despite using no force sensors, NEXT closely matches Franka’s factory external torque estimates.
🧵(3/N)
We then up-sample pre-contact and contact data during training to improve policy performance.
This is intuitive: most failures do not happen in free space. They happen near contact, where precise alignment, small error recovery, and force-sensitive interaction matter most.
🧵(5/N)
We further introduce FIRST: Force-Informed Resampling Training.
Using the learned force signal, we can automatically segment demonstrations into free-space, pre-contact, and contact regions.
🧵(4/N)
@CMU_Robotics @StevenOh_ @_tonytao_ @yangphiliphan @kenny__shaw @funabashihand @rsalakhu @pathak2206 Also check out @_tonytao_’s thread as well!
@JasonJZLiu Please get back on we need you for pantheon
Behavior cloning sees many frames where the robot is just moving through space, and relatively few where precise contact matters.
To identify those important moments, we use learned external torque (a method we call NEXT) to automatically segment demonstrations into:
1. free-space motion 2. pre-contact alignment 3. contact-rich interaction
🧵(2/N)
Work done with co-leads @StevenOh_ and @JasonJZLiu, plus @yangphiliphan, @kenny__shaw, @funabashihand, @rsalakhu, @pathak2206
Website: https://jasonjzliu.com/factr2 Paper: https://arxiv.org/abs/2606.12406
@0fir0z Harder rules for the model to learn. Usually you have a very straightforward relationship between action and state, not true when you start making lots of contact.
@chris_j_paxton Why is it tho? I mean training a light network around parameters that express contact e.g. current consumption to predict areas of contact should be pretty straightforward no?
Special shoutout to @StevenOh_ It was awesome working with you and amazing to see how much you have grown as a researcher the last few months.
I will be sure to remember all our late nights spent at the lab.
Check out his thread to understand how we get force information, without force sensors!
And check out @JasonJZLiu for an overview of the project.
@lumbogg 😅
@JasonJZLiu Congrats on the release!
A new pair of methods lets standard robot arms sense external forces through their existing joint sensors alone, opening force-aware teleoperation and contact-rich training to lower-cost platforms that previously lacked dedicated torque hardware.
What if some parts of a robot demonstration are more important than others?
Most of a trajectory is free-space motion. But success or failure is often determined by a few critical moments around contact.
In FACTR 2, we use force to find these moments and prioritize them for training. We find this helps policies learn better alignment and recovery behaviors, like the example below.
w/ @StevenOh_ @JasonJZLiu
🧵(1/N)