LLM checklists may reduce variation in human review outcomes

Announcing a new paper, Jessica Hullman asks whether AI could address weaknesses in human review rather than automate it. She describes “lottery variance,” where different reviewer selections change outcomes, and says checklists learned from prior reviews can reduce that variance with limited added bias. A separate post praised the checklist idea.