RAG hallucinations are annoying AF

Reddit r/AI_Agents News
rag hallucinations retrieval-augmented-generation evaluation debugging braintrust

Summary

The team discovered that 80% of RAG hallucinations were caused by poor retrieval, not the generation model, emphasizing the need to evaluate retrieval and generation separately to effectively debug bad answers.

We've got a RAG setup answering questions over our own docs, and we kept getting these confidently wrong answers. Obviously, this isn’t super surprising within this space, but we’ve personally been struggling to solve this for a while. Initially, the team's instinct was "the model is hallucinating," so we went down the usual path, tweaked the prompt, tried turning the temp down, even tested swapping models. But even those steps felt like either temporary fixes, or wouldn’t make a meaningful impact to our outputs. Finally we started diving deeper into our traces instead of just the final output. Once we could see the retrieved context that got fed into the model for each bad answer, it was pretty obvious the model wasn't really the problem. Our retrieval was handing it garbage, and we were getting garbage back out. And honestly, the model was doing a reasonable job answering based on the trash it was given haha. The mental shift that helped us was Retrieval: did we even pull the right context? Generation: given the right context, did the model actually use it correctly? Once we started scoring those two separately it got much much much easier to know where to spend time. We got it set up on our eval platform (Braintrust) so each has its own score, and now a bad answer points us straight at the layer that broke instead of us guessing. Turned out like 80% of our issues were retrieval, not generation, which is the opposite of what the team assumed going in.
Original Article

Similar Articles

How Much Do Legal RAG Systems Still Hallucinate?

arXiv cs.CL

This research analyzes hallucination in legal RAG systems across eight models and two legal corpora, finding that hallucinations persist with rates ranging from under 10% to nearly half, particularly for false-premise questions.

RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration

arXiv cs.CL

RAGognizer introduces a hallucination-aware fine-tuning approach that integrates a lightweight detection head into LLMs for joint optimization of language modeling and hallucination detection in RAG systems. The paper presents RAGognize, a dataset of naturally occurring closed-domain hallucinations with token-level annotations, and demonstrates state-of-the-art hallucination detection while reducing hallucination rates without degrading language quality.

Most RAG apps in production are confidently wrong and nobody talks about this enough

Reddit r/ArtificialInteligence

The article highlights a critical failure mode in production RAG systems where confident but incorrect answers arise from versioning issues and lack of uncertainty mechanisms. It proposes architectural improvements like routing layers, retrieval scoring, and hallucination checks to mitigate these errors.

Most agent RAG problems I see are retrieval problems, not model problems

Reddit r/AI_Agents

The author argues that most agent RAG failures are due to retrieval problems—specifically chunking errors, lack of freshness signals, and reliance on pure vector search—rather than the LLM, and recommends structural chunking, decay-based ranking, and hybrid BM25+vector search.