Coding agents failed most research-extension tasks in RExBench tests
RExBench’s researchers report that the best of 12 agents achieved around a 33% success rate.
TLDR
RExBench’s authors say its tasks extend 12 research papers and codebases, with domain expert-written instructions and automated checks for success. They tested 12 LLM agents built with aider and OpenHands; all failed most extensions autonomously. The best reached around a 33% success rate. With additional human-written hints, the best performance remained below 44%.
Combined views
—
1 Source, first seen ago
— likes— comments— saves— reposts
