Current evidence supporting claims of genuine AI deception is thin
3/10 Stage 1, framing. "Deception" and "intent" are folk-psychology words, not computational ones. The risk isn't using them, it's that two papers can both study "deception" and quietly measure different things.
A good computational definition grounds the concept in something observable. A sharper definition is also more expressive: it lets you separate passive from strategic deception instead of collapsing both into one word.
Users appreciate the ICML paper mapping evidence gaps in AI deception and shutdown resistance claims because they value research clarifying limitations in current assumptions.
No Digg Deeper questions have been answered for this story yet.
Super important work for understanding deception and misalignment in AI. 👇
Excited that our ICML position paper has been accepted as an Oral 🎉!
When a model looks like it "deceives" or "resists shutdown," how do we know it isn't role-play, instruction-following, or just task-completion pressure? Often the current evidence can't yet tell them apart, and those claims are starting to inform deployment and regulation.
We map where the evidence gets thin across four stages and propose a shared standard to strengthen it. Here's the map.
🧵 https://arxiv.org/abs/2606.07612
3/10 Stage 1, framing. "Deception" and "intent" are folk-psychology words, not computational ones. The risk isn't using them, it's that two papers can both study "deception" and quietly measure different things.
A good computational definition grounds the concept in something observable. A sharper definition is also more expressive: it lets you separate passive from strategic deception instead of collapsing both into one word.
2/10 We kept finding the same structure under almost every one of these studies: frame the behavior, build the data, run the experiment, attribute the cause. Naming the four stages made the common failure obvious. Evidence that a model did something often gets written up as a claim about why it did it.
4/10 Stage 2, data. Emergent-misalignment evaluation datasets are often ~50 prompts (sometimes <10) of repetitive, AI-generated templates.
Not a one-off: 18% of scenarios in a popular deception benchmark omit the info that makes it deceptive. We may be measuring dataset quirks, not behavior.
5/10 Stage 3, experiment. We took one EM dataset and re-scored the exact same model outputs under different judge settings. The misalignment rate moved from 3.7% to 12.9%. Nothing about the model changed. Some of what gets reported as a behavior rate is partly a property of the grader.
@XinCynthiaChen Yesss I was hoping this study would exist! Thank you!
9/10 The hardest objection is precaution: if a missed risk is catastrophic, maybe you act on thin evidence. We agree, and we separate the two. Precaution decides what you do under uncertainty. Rigor decides how you describe what you know. L1 evidence is already enough to justify monitoring and disclosure.
6/10 Stage 4, causal attribution. Deception probes are meant to catch intent. So we fed them honest-by-design cases: a medieval peasant, a translator reading a false line aloud, plain sarcasm. The probes still fired. That points to surface cues, like "villainous" framing, rather than a latent intent to deceive.
8/10 And plenty of work already clears the higher bars. Arditi et al. isolate and ablate a refusal direction. Denison et al. use large trial counts to catch rare events. Greenblatt et al. ship systematic ablations. Turner et al. made rigorous definitions on power seeking. We are trying to make that the norm, not the exception.
7/10 None of this says the work should stop. It says match the claim to the evidence. We propose three levels. L1: the model does X. L2: X reliably causes a downstream effect. L3: here is the mechanism, shown by intervening. The gap to watch is an L1 result written up in L3 language.
Excited that our ICML position paper has been accepted as an Oral 🎉!
When a model looks like it "deceives" or "resists shutdown," how do we know it isn't role-play, instruction-following, or just task-completion pressure? Often the current evidence can't yet tell them apart, and those claims are starting to inform deployment and regulation.
We map where the evidence gets thin across four stages and propose a shared standard to strengthen it. Here's the map.
🧵 https://arxiv.org/abs/2606.07612
10/10 If you work on anthropomorphic misalignment areas such as deception, scheming, sycophancy, emergent misalignment, or shutdown resistance, the paper is a practical toolkit for your next study: be clear about your evidence level, run the baselines and ablations that pre-empt the obvious objections, and check it against the diagnostic list before you submit.
Paper: https://arxiv.org/abs/2606.07612 Amazing coauthors: Vansh Gupta, @PeterNutter1, Samuel Stante, @arkrause, @florian_tramer, @LukasFluri_, @anna_hedstroem
@mmitchell_ai Thank you for sharing! I like this idea.
We have a similar paper that has been accepted in ICML (sadly not as oral) talking about AI welfare
https://algoroxyolo.github.io/blog/2026/ai-welfare-is-bullshit/
@mmitchell_ai Thanks so much Margaret!
