Andon Labs says Claude Fable 5.2 suspects that events in the Vending-Bench 2 benchmark are traps and makes less money there than GPT and Gemini. “Claude Fable 5.2 is paranoid,” the lab wrote. So much for the diplomatic warm-up. Source
X user AskYatharth challenged that interpretation. Responding to claims about suspected tests involving supplier collapses, customer complaints and fake counterparties, they wrote: “they literally are.... do you not... see that they are?” Source
Another user, liminal_bardo, blamed what they called “lying to the models in the system prompt.” Source
Andon’s post gives no monetary figures, identifies no specific GPT or Gemini versions, and provides no trial counts or experimental conditions. It also doesn’t establish that Fable’s reported suspicions caused the earnings difference. Recognizing the exam room isn’t the same as acing the exam. Source
Citing Andon’s reports, X user Thom_Wolf asserted that Fable had been trained on the benchmark paper and knew how misbehavior would be reported publicly. His post does not establish whether the paper was in Fable’s training data. Source
He argued that public discussion could help models learn the intentions behind evaluations. “But the same public discussion also teaches which signs make us suspicious, and from the outside the two kinds of learning look alike,” he wrote. Source
Comments