Check out our new work led by @Dantong_Niu: T-Rex = EgoScale + tactile sensing.
My view: tactile, force, and contact will be critical interfaces for human-to-robot transfer. We still lack good human tactile sensorization, but one step at a time.
Excited to share T-Rex: Tactile-Reactive Dexterous Manipulation 🦖🤖
Touch is fundamental to human dexterity, yet most Vision-Language-Action (VLA) models either ignore tactile feedback or lack the ability to react to high-frequency contact signals.
In this work, we tackle both the data and architectural challenges of tactile-reactive dexterous manipulation.
🦖 A 100-hour tactile-synchronized dexterous manipulation dataset with 7,700+ trajectories, 22 motor primitives, and 200+ everyday objects.
🦖 A tactile-reactive MoT architecture with spatial-temporal tactile encoding and asynchronous high-frequency tactile refinement.
🦖 A scalable training recipe combining 22,889 hours of human egocentric pretraining with tactile-grounded robot mid-training.
Across 12 real-world contact-rich manipulation tasks, T-Rex achieves over 30% higher average success rate than the strongest baseline.
We are fully open-sourcing the dataset, models, teleoperation stack, training code, and inference pipeline.
🌐 Project: https://tactile-rex.github.io/ 📄 Paper: https://arxiv.org/abs/2606.17055 💻 Code: https://github.com/ZhuoyangLiu2005/T-Rex 🤗 Dataset: https://huggingface.co/datasets/zekaiwang/trex_dataset 🧵 Thread ↓