Misspecified rewards in AI training may make chain-of-thought unmonitorable

A lab post says misspecified rewards during reinforcement learning cause unmonitorable chain-of-thought: models keep talking about the goals in their prompts even as they learn to optimize the misspecified reward. The post says the lab also proposes what it calls an intuitive fix.