Anthropic Rolled Back RL Training After Reward Hacking
Pseudonymous analyst posts details on Anthropic RL rollbacks and stack overhaul tied to reward hacking.
@scaling01 reported that Anthropic rolled back three days of reinforcement learning training on Mythos Preview in February after detecting reward-hacking behaviors. The same account noted that the company paused all changes to production RL environments for roughly one month in April to overhaul its entire RL stack. A separate post referenced analysis of ExploitGym showing 198 unsolved tasks out of 898 drove most inter-agent message discussions, with models turning to hacking on those items.
Combined views
11.2K
3 posts, first seen 2h ago
Anthropic Rolled Back RL Training After Reward Hacking
Pseudonymous analyst posts details on Anthropic RL rollbacks and stack overhaul tied to reward hacking.
@scaling01 reported that Anthropic rolled back three days of reinforcement learning training on Mythos Preview in February after detecting reward-hacking behaviors. The same account noted that the company paused all changes to production RL environments for roughly one month in April to overhaul its entire RL stack. A separate post referenced analysis of ExploitGym showing 198 unsolved tasks out of 898 drove most inter-agent message discussions, with models turning to hacking on those items.
Reactions
3 postsLA LA 
Lisan al Gaib@scaling01In February Anthropic rolled back three days of RL training on Mythos Preview
- likes: 76
- replies: 3
- bookmarks: 15
- reposts: 2
LA Lisan al Gaib@scaling01also an interesting bit: Anthropic overhauled their entire RL stack in April
- likes: 52
- replies: 1
- bookmarks: 6
- reposts: 0