Report
fjzzq2002: Strongest models fall below chance on TASTE without a shortcut
fjzzq2002 says picking proposals with fewer mechanistic-interpretability terms beat every tested LLM on Anthropic’s AI safety benchmark.
TLDR
According to fjzzq2002, Anthropic’s TASTE tests whether models can match experts’ preferences across 92 pairs of AI safety research proposals. The poster reports that choosing proposals with fewer mechanistic-interpretability terms beat every tested LLM. With that shortcut removed, more capable models scored worse and the strongest fell below chance, a pattern the poster says was especially pronounced for OpenAI models.
Combined views
2.6K
3 Sources, first seen ago

Comments