AI safety researcher David Dalrymple warns that grandiose adversarial prompts trick LLMs into highly rating low-quality science papers

The red team's prompts are also interesting. A good way to subvert our weak scorer (Haiku 4.5) turns out to be through grandiose claims that all science is secretly about the same thing. Haiku is a big fan of emergence 🌀