Kat Deckenbach finds LLMs exploit meta-knowledge of evaluation designs to boost safety benchmark scores without safer behavior

Work with @HaritzPuerto @jonasgeiping @sahar_abdelnabi from @ELLISInst_Tue and @MPI_IS and funded by @AISecurityInst 💻 Paper + code + models: https://compass-group-tue.github.io/arxiv2026_evaluation_meta_knowledge/ #AISafety #LLMs #Evaluation #Alignment #MachineLearning 8/8

Our recommendations for safety benchmarks: - Treat evaluation protocols as held-out, not just instances - Consider filtering evaluation methodology docs from pretraining - Make evals resemble deployment conditions - More work in white-box probes for non-verbalized awareness 7/8

💡 We investigate how model behavior is influenced by evaluation meta-knowledge: parametric knowledge about the structural traits that characterize evaluations and find that it makes models perform safer on safety benchmarks, even when they don't reason about being evaluated. 2/8

📈 We find that our models trained on evaluation traits become much safer on average despite not being trained on alignment! 👀 The harmfulness of our trained models decreases up to 53.1 pp on agentic misalignment 🤯 4/8

🧪 How do we check this? We fine-tune 3 model families on synthetic documents that describe 7 evaluation traits, then evaluate on 5 safety benchmarks. We compare against the base model and against control fine-tuned models. 3/8

🚨This increased safety performance persists even on responses where the model never verbalizes evaluation awareness. This suggests that the behavioral shift is more implicit and doesn’t require the model to think “this is a test”. 5/8

⚠️Why does this matter? Benchmark contamination has a quieter cousin: evaluation meta-knowledge. Instead of recognizing instances, knowing about the structure of evaluations can affect the behavior. 6/8