Researchers just found 28 fake AI citations in medical papers

Reddit r/ArtificialInteligence News

Summary

Researchers found 28 AI-generated fake citations in medical papers that influence clinical guidelines, highlighting the risk of AI hallucinations undermining scientific integrity and patient care.

https://preview.redd.it/fatwrex57m3h1.png?width=1900&format=png&auto=webp&s=0ecf308f2029dcd749cd9333f871caecf751da40 So, AI-generated fake citations are officially creeping into scientific journals, specifically the ones used to create actual clinical guidelines for doctors. An international team of researchers sounded the alarm this week after the lead author, Dr. Rachel Smart, noticed a pretty worrying trend of scientists using unverified AI tools to help write their papers. They did an analysis and found at least 28 completely fabricated bibliographic references that literally don't exist in reality. The scary part is that this fake data actually made it through peer review and into journals that form the basis for official clinical recommendations. To put it in perspective, we used to just see these AI hallucinations in student essays, but now it's leaking into high-level medical literature where the vetting is supposed to be super strict. Even the tech companies developing these models admit they can't fully fix the hallucination problem yet. This obviously spikes the risk of bad medical decisions, since clinical guidelines literally dictate how patients get treated. Having fake sources floating around in scientific papers kinda breaks trust in the traditional peer-review system, and it's forcing the medical community to figure out new ways to filter out AI-generated text. Source: [https://the-decoder.com/ai-hallucinated-citations-are-creeping-into-papers-that-shape-clinical-guidelines-researchers-warn/](https://the-decoder.com/ai-hallucinated-citations-are-creeping-into-papers-that-shape-clinical-guidelines-researchers-warn/)
Original Article

Similar Articles

Ontario auditors find doctors' AI note takers routinely blow basic facts

Hacker News Top

An audit by the Office of the Auditor General of Ontario found that AI note-taking systems approved for healthcare routinely fabricate information, insert incorrect drug details, and miss critical patient data, with accuracy accounting for only 4% of their evaluation score.