Wharton's Ethan Mollick says academic literature lacks empirical data on the productivity gains of autonomous AI coding agents
Rapid AI adoption forced METR to redesign its productivity experiments.
The closes thing is METR's very tentative update in February 2026, but the data is not high quality enough to know much of anything. https://metr.org/blog/2026-02-24-uplift-update/#other-means-of-measuring-productivity
Seems like a really high stakes question, and we have very little data (but lots of speculative Substack essays)
Ethan Mollick@emollickWe have, as far as I can tell, no good tests of the productivity impact of the autonomous coding tools that appeared starting in December 2025. Every paper out there is from prior to the Claude Code/Codex revolution. A huge gap in our knowledge about what is happening in coding.
#612Ravid Shwartz Ziv@ZIV_RAVIDTotally agree, the true is that we don't know yet how much productivity gain we have because of coding agents and my guess it will take many more months till we get good data about it
Ethan Mollick@emollickWe have, as far as I can tell, no good tests of the productivity impact of the autonomous coding tools that appeared starting in December 2025. Every paper out there is from prior to the Claude Code/Codex revolution. A huge gap in our knowledge about what is happening in coding.