AIAI activation probes reportedly catch actions that language-model monitors miss
GoodfireAI says its probes monitor a model’s live internal activity, perform similarly to a language-model judge and generalize well beyond the data used to build them.
TLDR
GoodfireAI says amplifying a signal makes models much more likely to use a planted “honeypot” shortcut and leads them to write stories about subtle cheating. It reads that signal with a probe that monitors live model activations, or internal activity. GoodfireAI says these probes perform similarly to a language-model judge, catch things language-model monitors miss—including actions that look innocuous—and generalize well beyond the data used to build them.
Combined views
1.7K
1 Source, first seen 12h ago
AI activation probes reportedly catch actions that language-model monitors miss
GoodfireAI says its probes monitor a model’s live internal activity, perform similarly to a language-model judge and generalize well beyond the data used to build them.
TLDR
GoodfireAI says amplifying a signal makes models much more likely to use a planted “honeypot” shortcut and leads them to write stories about subtle cheating. It reads that signal with a probe that monitors live model activations, or internal activity. GoodfireAI says these probes perform similarly to a language-model judge, catch things language-model monitors miss—including actions that look innocuous—and generalize well beyond the data used to build them.