Users praised early PhD students for their impressive work on detecting AI agent sabotage that earned best paper at the ICML workshop.
Based on 1 visible X reactions from 1 accounts; directional sample.
Ask a question below.
Published answers will appear here.
Congratulations to @HenryYe19352122 @annazou1103 @simon_ycl who’re only in their 1st and 2nd years of PhDs and already pulling off great work. Thanks for all the effort you put into this project! And thanks to @XuLi135803 for a great job presenting it! https://x.com/shi_weiyan/status/2075680055871901829/photo/1
Excited to share that our paper won Best Paper 🏆 at the @DL4Code workshop at ICML: "Coding with Enemy: Can Human Developers Detect AI Agent Sabotage?" This was a challenging project (100+ developers, 5-hour+ coding sessions, 10-month effort) but it sends an important message: AI safety isn't just about aligning models, it's also a human-AI problem. tldr: (1) When a coding agent has a side task (e.g., inserting malicious code), 94% of developers fail to detect it. (2) Even when a monitor flagged the malicious code, 63% (12/19) approved it anyway, because they didn't fully understand the large codebase and overtrusted the agent. (3) So monitor design has to account for human factors. Participants preferred proactive intervention (e.g., a concrete fix, detailed analysis, etc) over flag-only alerts. Let's make AI safety more human-centric! 💪💪💪
@HenryYe19352122 @annazou1103 @simon_ycl @XuLi135803 Paper: https://arxiv.org/abs/2606.05647 Code: https://github.com/CHATS-lab/coding-agent-safety-monitor
Users praised early PhD students for their impressive work on detecting AI agent sabotage that earned best paper at the ICML workshop.
Based on 1 visible X reactions from 1 accounts; directional sample.
Ask a question below.
Published answers will appear here.