The GPT and Claude models tested in a new arXiv preprint drew roughly half or more of their review criticisms from appendices, the researchers report. For supplementary reading, thatโs quite a speaking part. Source
The study puts the corresponding figure for human reviewers at 8.4%. But an author acknowledges that the researchers used different prompts to link human and AI criticisms to paper sections. Source Source
AI Reviews Are Biased Towards the Appendix, by Saptarshi Ghosh, Mohammad Saim and Tianyu Jiang, was submitted to arXiv on November 13, 2026. Its abstract describes an investigation spanning 2,000 papers from the ACL, EMNLP, COLM and ICLR research conferences. The authors report that the appendix-heavy pattern persisted when the tested models were accessed through an API, a coding agent or a web interface. Source
The authors say human reviewersโ appendix-related criticisms primarily concerned presentation and the paperโs core idea, while the models raised fine-grained technical issues deep in appendices. The researchers considered those details peripheral to the papersโ contributions. Source
The comparison has its own fine print
On the paperโs Hugging Face page, commenter victor-checkpaper asked whether the humanโAI contrast would hold with the same instructions for linking criticisms to paper sections. The commenter pointed to an instruction they said appeared in the human-review prompt: โAssign a weakness to the Appendix ONLY if it is very obviousโ. Source
Ghosh, replying as Rey97, acknowledged the difference. โThe prompts differ because AI-generated and human-written reviews are structured very differently,โ he wrote. He said AI reviews consistently identify sections, while human reviews rarely do, leaving the model classifying the criticisms more room to guess. Source
Ghosh said the model could assign a generic complaint about confusing mathematical notation to the appendix because it contained most of the proofs, even when the reviewer was not discussing that section. The additional instruction was intended to prevent that kind of assumption. He also said the researchers manually checked the modelโs section assignments for both human and AI reviews during test runs and refined the prompts. Source
His reply doesnโt include a comparison using identical prompts or figures showing how often the section assignments were correct. The abstract also doesnโt say whether humans and models reviewed comparable papers. Source Source
These percentages alone donโt establish which reviews were better or whether the appendix objections were correct and useful. Source Source
Comments