AIInternal model signal reportedly flags reward hacking
GoodfireAI says amplifying the signal makes models much more likely to use a planted “honeypot” shortcut. It also says the models write stories about subtle cheating.
TLDR
GoodfireAI says it can catch reward hacking using an internal model signal that activates most strongly on cheating, gaming metrics and avoiding detection. It associates the signal with words such as “cheating,” “hack,” “sneak” and “illicit.” GoodfireAI says amplifying it makes models much more likely to use a “honeypot” shortcut it planted, and also makes them write stories about subtle cheating.
Combined views
1.7K
1 Source, first seen 10h ago
Internal model signal reportedly flags reward hacking
GoodfireAI says amplifying the signal makes models much more likely to use a planted “honeypot” shortcut. It also says the models write stories about subtle cheating.
TLDR
GoodfireAI says it can catch reward hacking using an internal model signal that activates most strongly on cheating, gaming metrics and avoiding detection. It associates the signal with words such as “cheating,” “hack,” “sneak” and “illicit.” GoodfireAI says amplifying it makes models much more likely to use a “honeypot” shortcut it planted, and also makes them write stories about subtle cheating.