Kosta Derpanis (sabbatical in Zurich)#1869
🚨Introducing: Ego-MC-Bench (Mistake Corrections) benchmark and Ego-CoMist (Counterfactual Mistakes) dataset.
🎯Ego-MC-Bench: Where AI assistants need to intervene at the right time (when) and with the right feedback (what) to prevent mistakes.
👉https://tinyurl.com/y7y9mwrs
1/4
❗️Ego-MC-Bench contains instruction feedback pairs provided by an expert in real-world kitchen scenarios.
⏹️Current SOTA video LLMs show very poor mistake intervention capabilities.
👉Even Gemini-3-Flash manages to get an mistake intervention F1 score of only 0.18.
2/4
🎯A major bottleneck is the lack of appropriate video data of procedural activities with mistakes.
This is in spite of abundance of procedural activity datasets.
Therefore, we propose a synthetic data generation process with counterfactual mistakes: Ego-CoMist.
3/4
👉This leads to a significant improvement in mistake intervention capabilities, especially for small models ideal for edge deployment.
📜Paper: https://arxiv.org/abs/2606.09547
4/4
Joint work with: @Matewhs @SanjayHaresh @RolandMemisevic
No Digg Deeper questions have been answered for this story yet.
Kosta Derpanis (sabbatical in Zurich)#1869🚨Introducing: Ego-MC-Bench (Mistake Corrections) benchmark and Ego-CoMist (Counterfactual Mistakes) dataset.
🎯Ego-MC-Bench: Where AI assistants need to intervene at the right time (when) and with the right feedback (what) to prevent mistakes.
👉https://tinyurl.com/y7y9mwrs
1/4