Users praised the study on AI boosting research submissions while lowering quality as convincing and highlighted AI's potential to greatly advance science over the medium to long term.
AI-heavy reviews aren't affecting editorial decisions much (yet), likely because editors discount them. Important caveat to keep in mind re: studies showing AI-evaluation systems prefer AI-produced stuff. (6/n)
Are these AI-augmented papers any good? 1. Writing quality seems bad -- I didn't put much faith into these generic, automated measures before seeing the results, but it's clear they are catching *something*. AI is making texts harder to read, more jargon (3/n)
2. Editorial judgment: More importantly/directly, papers with more AI use more likely to get desk rejected! This is the crucial piece of evidence. Fully AI written papers --> 70% desk rejection rate (4/n)
Org Sci submissions went up 40% since ChatGPT. Fully human-written papers are *already* in the minority (2/n)
Already 30% of reviews show *substantial* AI use. These reviews have lower writing quality (again using the generic automated measures), and, most interestingly: - focus more on theory/framing - focus less on data - have a narrower focus (5/n)
Overall, a really nice and convincing paper, props to the authors. AI will undoubtedly benefit science a LOT in the medium-to-long run, but in the short run, and at least in management scholarship, the benefits are ... fuzzy. (7/7)
https://pubsonline.informs.org/doi/10.1287/orsc.2026.ed.v37.n3
Consistent with ICLR