The @ArayaGlobal RL team's latest work, "An Augmented Reality Brain-Robot Interface for Generalist Robot Arm Manipulation" has been accepted at @ieeeroman 🧠🤖
As part of @moonshot_IoB, we've been pushing BRIs, and the combination of AR and foundation models is really enabling
Users are excited about the AR Brain-Robot Interface accepted at IEEE ROMAN because studies showed near-100% success rates with high usability and a cool sci-fi experience controlling robots via thoughts and eyes.
No Digg Deeper questions have been answered for this story yet.
How can you control a robot if you cannot use your limbs and cannot even speak? The solution - gaze for object selection, and EEG for action selection. AR provides visual feedback
And with VLAs like π0.5, assistive robotics can go beyond single-task control
The @ArayaGlobal RL team's latest work, "An Augmented Reality Brain-Robot Interface for Generalist Robot Arm Manipulation" has been accepted at @ieeeroman 🧠🤖
As part of @moonshot_IoB, we've been pushing BRIs, and the combination of AR and foundation models is really enabling
On a side note, the experience of sitting there and using just your eyes and thoughts to control a robot to perform different tasks is really cool and sci-fi
We hope that this encourages future work on more general-purpose assistive robotics. Challenges remain, but I believe this is the path forwards 🚀
Project website: https://ar-bri-manip.github.io/ Preprint: https://arxiv.org/abs/2606.16413
We ran a feasibility user study with 18 healthy participants, achieved close to 100% success rates, and scored highly on usability (SUS) with a low-moderate workload (NASA-TLX) - all very promising for this approach!
We designed 3 multi-step tasks inspired by activities of daily living (ADLs): putting cutlery in a drawer, putting food in an oven, and taking a drink
Although VLAs are capable of even more, our goal here is a PoC for AR BRIs/assistive robotics
We hope that this encourages future work on more general-purpose assistive robotics. Challenges remain, but I believe this is the path forwards 🚀
Project website: https://ar-bri-manip.github.io/ Preprint: https://arxiv.org/abs/2606.16413
We ran a feasibility user study with 18 healthy participants, achieved close to 100% success rates, and scored highly on usability (SUS) with a low-moderate workload (NASA-TLX) - all very promising for this approach!
We designed 3 multi-step tasks inspired by activities of daily living (ADLs): putting cutlery in a drawer, putting food in an oven, and taking a drink
Although VLAs are capable of even more, our goal here is a PoC for AR BRIs/assistive robotics
How can you control a robot if you cannot use your limbs and cannot even speak? The solution - gaze for object selection, and EEG for action selection. AR provides visual feedback
And with VLAs like π0.5, assistive robotics can go beyond single-task control
