Many users criticized NeurIPS reviewers for deploying AI agents to review papers they do not understand, calling it a worrying blind spot that outsources judgment and undermines peer review quality.
Based on 6 visible X reactions from 10 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@ericmitchellai That is a worrying jump from using agents as assistants to outsourcing judgment entirely. Review quality depends on context that automated summaries can easily miss.
@ericmitchellai This is exactly the kind of blind spot that undermines peer review. If you can't audit the agent's reasoning, the review is just noise.
@ericmitchellai both sides using agents they dont understand and assuming the other will catch mistakes this ends well
@ericmitchellai Peak midwit energy. Degens outsourcing their brain to LLMs is just sad.
Several people with no technical background told me they were reviewing for NeurIPS using agents for all reviewing--stating they didn't understand what the agent did, nor the underlying paper. Asked how they could trust the agent's review, they said they assume the author would have caught any errors.
@PeterHndrsn Tbf, the sad irony is that the agents they use are probably better at doing research than these types of researchers 😅 (doesn't excuse their behavior of course -- if anything, these authors' names should be in the *acknowledgements* for prompting, rather than the author list)
(In case it wasn't clear this is satire but probably accurate)
Many users criticized NeurIPS reviewers for deploying AI agents to review papers they do not understand, calling it a worrying blind spot that outsources judgment and undermines peer review quality.
Based on 6 visible X reactions from 10 accounts; directional sample.
Ask a question below.
Published answers will appear here.