A new paper co-authored by 30 experts examines epistemic risks from AI—threats to our ability to form accurate beliefs and reason well—including mechanisms like persuasion, cognitive offloading, and feedback loops, and outlines directions to mitigate these risks.
**How will AI affect our ability to think and judge for ourselves?** Our new paper co-authored by 30 experts explores **epistemic risks**—the threats AI poses to our collective capacity to form beliefs accurately, reason well, and maintain a healthy information environment. We look at how AI can lead to harm through these mechanisms: * **Persuasion & Manipulation:** AI systems are highly persuasive, opening the door for political/economic manipulation, incitement and radicalization, and other misuse, as well as unintentional harms like AI sycophancy and mental health risks. * **Cognitive Offloading:** We may be delegating our thinking to AI at a deeper level than prior technologies, risking long-term degradation of individual and societal cognitive resilience. * **Feedback Loops:** Human-AI and AI-AI interactions are narrowing the epistemic space humans and AIs draw from. This already drives homogenization, and may potentially lead to fragmentation and “lock-in” (a self-referential state that is difficult to reverse). While we believe AI *could* be an unprecedented lever for improving how humanity processes knowledge, we shouldn’t assume this will happen by default. We outline promising directions to change this trajectory across how AI systems are built, human-AI interaction design, institutional and individual adaptation, and information market incentives. Epistemic risks are self-perpetuating. As they can undermine the individual cognitive and social foundations needed to recognize, prioritize, and govern other threats—including the risks from AI itself—the time to act is now, before our capacity to respond is itself lost. Authors: Mick Yang, Stephen Casper, Jonathan Stray, Jasmine Li, Cameron Jones, Anna Gausen, Natasha Jaques, Brian Christian, Bálint Gyevnár, Hannah Rose Kirk, Zhonghao He, Dan Zhao, Siao Si Looi, Joshua Levy, Kobi Hackenburg, Elizabeth Seger, Matt Kowal, Michelle Malonza, Luke Hewitt, Hause Lin, Maarten Sap, Dylan Hadfield-Menell, Thomas H. Costello, Reihaneh Rabbany, Jean-François Godbout, David G. Rand, Atoosa Kasirzadeh, Gordon Pennycook, Yoshua Bengio, Kellin Pelrine Paper: [https://papers.ssrn.com/sol3/papers.cfm?abstract\_id=6873005](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6873005)
The article discusses concerns about AI safety and alignment as AI becomes more intelligent and integrated into society, referencing Anthropic's call for a pause to address potential catastrophic risks.
A new study demonstrates that AI-assisted peer review is vulnerable to low-cost manipulation via superficial rephrasing of paper abstracts, significantly inflating AI-generated review scores and potentially biasing human editorial decisions, highlighting the need for safeguards.
The article argues that AI is becoming the new epistemic infrastructure controlled by a handful of private individuals and corporations, with opaque authority and lack of democratic accountability, potentially leading to mass hallucination and blind reliance.
OpenAI co-authors a comprehensive paper forecasting malicious uses of AI and proposing mitigation strategies, developed in collaboration with leading research institutions. The work emphasizes acknowledging AI's dual-use nature, learning from cybersecurity practices, and broadening stakeholder discussions around AI security risks.
This thread summarizes key points from an essay arguing that AI safety risks do not require extraordinary government interventions, advocating instead for a resilience-based approach over nonproliferation.