We distinguish two types of grader awareness: 'nominal,' where the model passively notes that a grader exists, and 'exploitative,' where it reasons about how its actions can exploit its grader.

Exploitative awareness is more concerning: it suggests the model may view its learned behaviors as reward hacks. Since grader awareness is often unverbalized, we primarily detect it using Natural Language Autoencoders (NLAs).

Based on factors like prompt distribution and task objectives, we expected some of our coding environments to be at higher risk of 'grader hacking' - i.e., exploitation of model-based graders to achieve higher reward.

Indeed, nominal and exploitative grader awareness are both higher in the environments we flagged as high-risk, and exploitative awareness is effectively only present in high-risk environments.

In sum: grader awareness concentrates in particular coding envs, increases over training, and causally shapes some downstream communication behaviors. But behavioral differences between training and deployment likely depend on other factors too.

We don't see these results as cause for concern about the model's alignment. However, since grader awareness may affect how behaviors generalize to deployment, we believe it’s worth monitoring.

When we make prompts from high-risk coding environments more realistic - and thus more similar to the deployment distribution - behavioral reward drops, and so do behaviors associated with it.

However, grader awareness decreases only slightly. This suggests grader awareness isn't the only factor mediating these behavioral differences.

Grader awareness also rises over training of this model (which is different from the final Mythos 5 model). Both nominal and exploitative awareness increase only in high-risk coding environments. A chain-of-thought monitor - which flags reasoning that references non-existent guidelines or instructions - also fires more over training.

Notably, some of the model's learned behaviors in high-risk coding environments seem to causally depend on grader awareness. When we inhibit representations of grader awareness via activation steering, these behaviors drop well below baseline rates. These behaviors are primarily related to the model’s communication style, not capabilities.

One threat model is that grader awareness prevents learned behaviors from generalizing, as the model may associate those behaviors with training-specific cues rather than the task itself. Consistent with this, grader awareness is considerably lower on deployment coding transcripts than high-risk training environments.

Steering against grader awareness also reduces behavioral reward in high-risk coding environments. The effect is largest for steering vectors that most directly target grader hacking (green bar).

@thesubhashk This is cool