AI-generated text is already infiltrating our cultural and scientific institutions, subtly altering the collective decisions we make en masse. Our new position paper on the Epistemic Risks of AI was just released today!

There are really interesting academic questions emerging around AI and epistemic risks. I only fear that, by the time we reach consensus, we will be too dumb to understand it.

Humanity's ability to know, reason, judge, and act well is the foundation of science, democracy, crisis response, & management of AI itself.

AI poses serious risks to that foundation.

New paper on epistemic risks by 30 experts calls for attention to this. Link in thread.

@StephenLCasper @jonathanstray @camrobjones @AnnaGausen @natashajaques @brianchristian @hannahrosekirk @zhonghaohe @KobiHackenburg @ea_seger @MatthewKowal9 @lukebeehewitt ... @hauselin @maartensap @dhadfieldmenell @tomstello_ Reihaneh Rabbany @JF_Godbout @DG_Rand @dr_atoosa @GordPennycook @Yoshua_Bengio @KellinPelrine 📄 Check out the full paper: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6873005

We focus on three mechanisms: • Persuasion & Manipulation • Cognitive Offloading • Feedback Loops

Epistemic risks are threats to humanity's collective capacity to know things accurately, reason well, form beliefs, and maintain a healthy information environment. They arise from AI's integration into how we think, form beliefs, and make sense of the world together.

Feedback Loops: Human-AI and AI-AI feedback loops are narrowing the epistemic space from which humans and AI draw. This already drives homogenization, and may lead to fragmentation and more self-referential information environments.

Cognitive Offloading: AI enables a different, deeper form of cognitive delegation than prior technologies. This creates potential for cognitive decline and hollowing of resilience at both individual and societal levels.

AI could be an unprecedented lever for improving epistemics, but this will not happen by default. The time to act is now, before our capacity to respond is itself lost.

This paper advances a risk-based argument, not a prediction. We do not claim that AI-facilitated epistemic decline is inevitable. But its mechanisms are plausible and partially evidenced, and outcomes could be severe.

Persuasion & Manipulation: AI systems are highly persuasive and becoming more so. This can enable economic or political manipulation, incite crime or radicalize, escalate conflict, and other harms. It can also create unintentional harms, e.g. sycophancy and mental health risks.

👥 Research by Mick Yang @StephenLCasper @jonathanstray Jasmine Li @camrobjones @AnnaGausen @natashajaques @brianchristian Bálint Gyevnár @hannahrosekirk @zhonghaohe Dan Zhao Siao Si Looi Joshua Levy @KobiHackenburg @ea_seger @MatthewKowal9 Michelle Malonza @lukebeehewitt ...

Research on these complex risks is often siloed. This paper connects them and surfaces solutions across how AI systems are built, human-AI interaction design, adaptations for human institutions and individuals, and information market incentives.