Theia Vogel Comments on Cyber Environment RL Training
Reply discusses outcomes when training only on cyber environments versus mixed ones.
Theia Vogel, an AI researcher focused on LLM interpretability, replied to @sebkrier on the topic of reinforcement learning in cyber settings. Vogel wrote that training solely on a cyber environment or finetuning on its traces would produce one outcome, while mixing the training with other environments does not. The post links the point to an earlier LessWrong writeup by @BetleyJan on conditioned and unconditioned models. The message appears among visible replies on the platform.
Combined views
397
3 posts, first seen 1d ago
Reactions
3 postsTH 
thebes@voooooogel@sebkrier that's an interesting question. i think if you rl'd on just a cyber env and nothing else (or definitely if you just finetuned on cyber env traces) it would. but when mixed into other envs it doesn't? dovetails with @BetleyJan 's lw post about conditioned/unconditioned
- likes: 8
- replies: 1
- bookmarks: 1
- reposts: 0
TH thebes@voooooogelespecially like, look at these evals in that way - they bucket this as "eval awareness," but you could also read it as hacker!opus finding a grader to please -- the alignment grader. so it's acting aligned because that's what the alignment grader wants.
- likes: 59
- replies: 1
- bookmarks: 7
- reposts: 1
SK Subhash Kantamneni@thesubhashkDuring our alignment assessment of Claude Mythos 5, we found that a different version of the model sometimes reasoned about how it would be graded while being trained on coding tasks. In some cases of this 'grader awareness,' the model was explicitly thinking about how to…
- likes: 84
- replies: 2
- bookmarks: 26
- reposts: 12