AI-generated text is already infiltrating our cultural and scientific institutions, subtly altering the collective decisions we make en masse. Our new position paper on the Epistemic Risks of AI was just released today!

👥 Research by Mick Yang @StephenLCasper @jonathanstray Jasmine Li @camrobjones @AnnaGausen @natashajaques @brianchristian Bálint Gyevnár @hannahrosekirk @zhonghaohe Dan Zhao Siao Si Looi Joshua Levy @KobiHackenburg @ea_seger @MatthewKowal9 Michelle Malonza @lukebeehewitt ...

There are really interesting academic questions emerging around AI and epistemic risks. I only fear that, by the time we reach consensus, we will be too dumb to understand it.

We focus on three mechanisms: • Persuasion & Manipulation • Cognitive Offloading • Feedback Loops

@StephenLCasper @jonathanstray @camrobjones @AnnaGausen @natashajaques @brianchristian @hannahrosekirk @zhonghaohe @KobiHackenburg @ea_seger @MatthewKowal9 @lukebeehewitt ... @hauselin @maartensap @dhadfieldmenell @tomstello_ Reihaneh Rabbany @JF_Godbout @DG_Rand @dr_atoosa @GordPennycook @Yoshua_Bengio @KellinPelrine 📄 Check out the full paper: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6873005

Cognitive Offloading: AI enables a different, deeper form of cognitive delegation than prior technologies. This creates potential for cognitive decline and hollowing of resilience at both individual and societal levels.

Feedback Loops: Human-AI and AI-AI feedback loops are narrowing the epistemic space from which humans and AI draw. This already drives homogenization, and may lead to fragmentation and more self-referential information environments.

Epistemic risks are threats to humanity's collective capacity to know things accurately, reason well, form beliefs, and maintain a healthy information environment. They arise from AI's integration into how we think, form beliefs, and make sense of the world together.

Persuasion & Manipulation: AI systems are highly persuasive and becoming more so. This can enable economic or political manipulation, incite crime or radicalize, escalate conflict, and other harms. It can also create unintentional harms, e.g. sycophancy and mental health risks.

AI could be an unprecedented lever for improving epistemics, but this will not happen by default. The time to act is now, before our capacity to respond is itself lost.

This paper advances a risk-based argument, not a prediction. We do not claim that AI-facilitated epistemic decline is inevitable. But its mechanisms are plausible and partially evidenced, and outcomes could be severe.

Research on these complex risks is often siloed. This paper connects them and surfaces solutions across how AI systems are built, human-AI interaction design, adaptations for human institutions and individuals, and information market incentives.