AI skill retrieval can look helpful even when it hurts, a post warns

A post summarizing a paper warns that comparing tasks where an AI agent retrieved skills with tasks where it didn’t can be misleading. Those may be different kinds of tasks: if the agent retrieves skills on easier problems, retrieval can look helpful even when it does nothing or makes answers worse. The post describes the paper’s fix, RAE, as rerunning the exact task without skill access whenever retrieval happens, allowing a same-task comparison.