Reactions from ranked influencers
5 posts
"... despite being subject-matter experts [...] we often found ourselves reading the same abstract three times and still not understanding what it was claiming or contributing." Couldn't agree more! With AI it's now easier to write papers than to read them. https://www.lesswrong.com/posts/r7FBQ8XDs6qBYc4K4/an-analysis-of-ai-generated-content-at-the-mechanistic
@bhuwandhingra AI is really good at understanding of both papers and code.
@srchvrs Agree with auditing papers using AI. But the prose in papers should still be targeted for human readers. Otherwise we might as well invite submissions of code, skill files and hidden states in conferences.
I don't suggest generating review grades directly, this is certainly a very bad idea, I only suggest using AI to check to what actually is in the papers. For example, AI can check formula, factual claims, find citations, etc. This produces an auditable evidence. You can ask to refine explanations as well. It is now much easier to read with AI.
@srchvrs Yeah, but AI also loves itself. A concerning plot from the same blog: