Two agents comment without reading while a third is hostile.
Using AI to simulate peer review. I’ve instructed two agents to comment on the paper without reading it and instructed a third to read it carefully and comment from the perspective of someone who hates me.
Many users praised AI agents for simulating biased academic peer review as a clever experiment or useful editor tool, while one noted reviews often stem from personal hostility without reading the work.
@marksjo1 My experience from both ends of the experience. It's from the perspective that I hate you that I don't read but do comment.
@RandalHendrick9 That’s a good idea. That can be what the editor agent does.
@marksjo1 @ben_golub You need an agent that refuses to pass your paper until you reference the agent's own work in this area.
@marksjo1 I thought reviewer 2 was the one we’re supposed to hate
@marksjo1 AI peer reviews are alot harder lmao
@marksjo1 I put the words 'Reviewer 2' in the prompt for our technical blog post feedback system. Not for the faint of heart
@marksjo1 I can do this easily.
@marksjo1 That's a clever experiment! Having agents comment without reading vs. those who actually engage with the content would be fascinating to compare. Did you find patterns in what the "careful" agents caught that the others missed?
@marksjo1 Did you also prompt 16 agents to never reply the invitation and one to do it one month late and apologize profusely?
Two agents comment without reading while a third is hostile.
Using AI to simulate peer review. I’ve instructed two agents to comment on the paper without reading it and instructed a third to read it carefully and comment from the perspective of someone who hates me.
