The case for coding-agent benchmarks that match developer task flows

A post describes a JetBrains Research paper arguing that benchmarks should reflect how coding-agent users switch tasks mid-session. It says researchers studied 4,782 real agent sessions, where feature and bug-fix requests were well under half, compared with 93.8% of SWE-Bench Pro tasks. The paper’s SWE-TaskFlow splits issues into turns and adds checkable code questions; on 700 SWE-Bench Pro tasks, the post says that split roughly doubled agent cost without a stable change in success rate.