Steering a model’s attention is claimed to make it blurt out secrets
A user says a portable “secret” vector in query space made a model blurt out secrets when they steered its attention toward them, even in evaluation-awareness settings.
TLDR
A user says they tried steering a model’s attention instead of its hidden states and found a portable “secret” vector in query space. They claim the approach made the model blurt out secrets, even in evaluation-awareness settings.
Combined views
1K
2 Sources, first seen 16h ago
25 likes
