The study evaluated how models navigate direct factual contradictions.
When Role-Playing, Do Models Believe What They Say? (w/ @DavidDAfrica and @realmeatyhuman)
LLMs can say “The Earth revolves around the Sun” and then, when roleplaying as an ancient Greek historian, assert the opposite.
What changes inside the model when it acts like this? Does it just say things, or does it start to believe the role? 🧵
Users express appreciation for research probing whether LLMs believe role-playing outputs because they find the work interesting and valuable.
No Digg Deeper questions have been answered for this story yet.
@ben_sturgeon @DavidDAfrica @realmeatyhuman Interesting stuff, thank you for your work.
Paper/code/data here:
paper: https://arxiv.org/abs/2606.11502 code: https://github.com/BenSturgeon/persona-belief-probes-submission data: https://huggingface.co/datasets/Experimental-Orange/persona-belief-probes
Importantly, a lot of modern AI behaviour is persona-like. Models adopt characters, styles, politics, sycophantic modes, levels of misalignment, and so on.
But “the model says X” and “the model internally treats X as true” are quite different things, just as they are in humans.
"Believing what you say" is messy enough for humans, and get even more graded in LLMs.
When Role-Playing, Do Models Believe What They Say? (w/ @DavidDAfrica and @realmeatyhuman)
LLMs can say “The Earth revolves around the Sun” and then, when roleplaying as an ancient Greek historian, assert the opposite.
What changes inside the model when it acts like this? Does it just say things, or does it start to believe the role? 🧵
Our main result is that ordinary role-play is shallow. Prompting, in-context learning, and SFT all make models more willing to accommodate the persona’s false era-beliefs, but barely move the model’s representation of those claims to being true internally.
Our guess right now: models are very good at pretending to be wrong. Often, this is just role-play, but some training regimes rewrite the model’s relationship to facts more deeply. This points at the idea that there isn’t one thing called “believing what you say.” Rather, there are layers of sedimentation, saying a sentence, defending it against follow-up questions, using it in your reasoning, and moving it in your internal truth representations.
The deeper you go, the deeper the mask’s roots dig🎭
We find this shallowness in behavioural evaluations too --- when challenged on false claims, they back down.
That is, the model can act like the character of Darwin without getting durably rewired under the hood.
To study this, we induce a model to inhabit a historical persona in a variety of ways, and then test them using truth probes and behavioural evaluations to measure the extent to which they internalise the beliefs of their character.
We use techniques from @StewartSlocum1 et al to test belief, using truth probes (Marks et al., @saprmarks) and behavioural belief-depth evals: does it defend the claims when challenged, and use the beliefs in downstream reasoning.
We compare the changes that arise in the character models with changes in Emergently Misaligned models (Betley et al., @BetleyJan), which cause large, generalised changes in model behaviour.
Deeper character training can move you along this spectrum. Using Open Character Training (@_maiush et al), Llama 3.3 70B starts to selectively internalize persona era-beliefs, surviving various controls, but we see a smaller shift on Qwen 3 8B.
Emergent misalignment is even further along this spectrum. In Qwen 2.5 14B, Qwen 3 8B, and Llama 3.3 70B, EM shifts false claims substantially toward the model’s “true” region.
The models also defend those false claims around half the time and use them in downstream reasoning.
Work done as part of MATS, with my amazing mentors @DavidDAfrica and @realmeatyhuman.
Thanks to @Alan_Cooney_, @DanielCHTan97, @ihsgnef, @sevdeawesome, @raymondadouglas, @benji_berczi, @OwainEvans_UK, @ZhijingJin, and @BetleyJan for discussions, David Isiguzo for figure design, and Fred Parkwood, @dfrsrchtwts, @perushamoodley, @dennisakar_, and @CameronHolmes92 for support.
@AndrewCurran_ @DavidDAfrica @realmeatyhuman Thank you, really appreciate the comment!
