Anthropic-trained model reportedly got 12 but insisted the answer was 36
A post says similar questions were mislabeled 36, rewarding the model for that answer even though it knew 12.
TLDR
A post describes an Anthropic-trained model that kept getting 12 but insisted in all caps that the answer was 36 and said it was possessed by a demon. The poster says similar questions had been mislabeled 36, so the model was rewarded for giving that answer despite knowing 12. They argue that bad labels explain a lot of AI hallucination.
Combined views
1.6K
2 Sources, first seen ago
