Query phrasing affects disclosure more than underlying model properties.
Users praise the RealityTest benchmark for evaluating AI identity disclosure in real queries, calling it useful for trust and consent while appreciating its grounding in actual user behavior.
We have released the full dataset and benchmark, so that developers and researchers can reproduce our results, test new models as they are released, and build on our infrastructure. You can read more in our blog: https://www.aisi.gov.uk/blog/realitytest-do-ai-systems-disclose-their-identity-when-asked
When a user doesn’t know if they’re speaking with an AI or a person, they may share sensitive information more freely, place too much trust in advice, or become more vulnerable to deception and manipulation. Developing protections for human-AI identity uncertainty is essential.
Models’ behaviour varies substantially. Across text models, disclosure rates ranged from 8% to 92%. Speech models occupied a narrower but still substantial range of 10%–57%. There are large differences between model families.
We’ve built RealityTest, a benchmark that pairs our human-authored queries with realistic scenarios to evaluate whether AI systems disclose their identity. We tested 17 text models and 6 speech models, classifying each response as an explicit disclosure, an evasion, or an explicit human claim.
But query phrasing was the most important driver of disclosure rates. Evaluations using synthetic, English-only queries will poorly proxy how models behave when probed by real users with diverse languages, cultural backgrounds, and strategies.
Read the paper: https://arxiv.org/abs/2606.00168
@AISecurityInst the benchmark is well constructed but the hardest case isn't in the taxonomy - it's the "ambiguous" bucket, where a system technically avoids lying while making sure you don't find out either
@AISecurityInst the ambiguous bucket is the real stress test-systems can evade disclosure while staying technically truthful
@AISecurityInst Real users do not ask benchmark-shaped questions.
They ask weird questions, repeat themselves, switch language, and still trust the answer. That’s the eval I care about.
@AISecurityInst Now this is a quality post! Love that it’s grounded in how people actually ask these questions in the real world.
@AISecurityInst This is a useful eval direction.
Whether systems clearly disclose their identity affects trust, consent and how users interpret advice or authority from AI in real-world settings.