New Paper Shows Mouse And Eye Data Can Improve LLMs

It demonstrates that explicit user preferences can be inferred using only implicit signals such as mouse movements and eye gaze captured via webcams, i.e., signals that are noisy but widely available.

those of us in the information retrieval community have long known that implicit feedback, including cursor movement, is a strong way to derive preferences. this very nice work from umass brings some of this to contemporary conversational systems.

The paper further shows that reward models trained on such implicit feedback can lead to improved alignment.

💥 I cannot overstate the significance of these findings. 💥

you can find more information about implicit feedback in our neurips 2020 tutorial,

https://github.com/pchandar/beyond-accuracy-tutorial