The paper uses metascience to address automated review challenges.
🔥🔥
Using AI to support peer review seems unavoidable, but what quality checks should AI implement? We can take some lessons from metascience on the hard reward design problem that is AI review.
I wrote a paper synthesizing a few points the emerging lit is at risk of confusing. 1/
Users praise the nuanced points in papers on quality checks and reward design for AI peer review, especially as AI agents integrate into research workflows.
No Digg Deeper questions have been answered for this story yet.
More in the paper, including an empirically-informed demo of how much relevant signal statistical significance and exact replication success provide for estimating true signal-to-noise ratio & effect direction: https://users.eecs.northwestern.edu/~jhullman/AI_metascience_position.pdf Blog post: https://statmodeling.stat.columbia.edu/2026/06/24/treating-ai-review-like-the-contentious-policy-design-problem-it-is/
@JessicaHullman I realize of course the correctness isn't as black and white an issue when you step away from math.
Using AI to support peer review seems unavoidable, but what quality checks should AI implement? We can take some lessons from metascience on the hard reward design problem that is AI review.
I wrote a paper synthesizing a few points the emerging lit is at risk of confusing. 1/
@JessicaHullman How do you think about this for math papers? It seems like there is a useful compartmentalization into: Is the theorem interesting? and Is the theorem correct? A tool that could answer the second question and leave the first question to the tastes of the reviewer seems great.
Bottom line is there's no stable non-conflicting set of checks that distinguish good from bad science. We should be careful not to blur distinction between basic integrity checks (plagiarism, fake citations, data/code to reproduce) & "epistemic filters" like robustness tests. 2/
What's the review decision problem a proxy is meant to inform? What's the proxy, what's the target latent state capturing quality, under what assumptions are they linked? This provides the minimal info the community needs to weigh in on tradeoffs & possible incentive effects. 5/
Former asks, Is it what it claims to be? Latter asks, Is it sound? We may get value from OpenSci proposals like preregistration or multiverse robustness, but shouldn't assume they're necessary or sufficient. This can shift incentives away from exploration, theory building, etc 3/
Banning AI in review until we can show its not gameable is not a viable path forward--any automated review policy that allocates attention will be gameable. Best we can do is put the onus on those proposing new review tools to specify their target & assumptions. For example.. 4/
@Aaroth Yes, I think in general AI is better reserved for correctness & other signals that can be verified. The risk with epistemic filters like multiverse robustness is that it can seem 'objective' but how you define the space of sensitivity tests encodes a lot of domain knowledge
@JessicaHullman This is such a nuanced point, especially as AI agents become more integrated into our research workflows!
@Aaroth Not to mention a lot of breakthrough ideas from the history of science would not have made it to light if we were filtering everything based on how robust it appeared at the time of discovery.
