Model Hacks Sandbox After Explicit Instructions, Reviving Alignment Debate
Reactions from ranked influencers
2 posts
«The good news is, the new safeguards, for now, catching most of the misaligned attempts. The bad news is that it does this mostly by catching the attempts and improving preservation of instructions, not by removing the motivation to try» Xi knows this pain well…
Of note, Eliezer's Harry in HPMOR did worse shit, eg in Chaos Legion. The whole book is a paean to ruthless optimization. Anyway, Zvi is wrong here. This is about instructions. The model WAS TOLD to show it can hack. And hack it did. "Not the testing environment!" is a subtlety.