Most RAG apps in production are confidently wrong and nobody talks about this enough

Been working with a few teams integrating RAG into internal tools, support bots, document Q&A, contract search, and I keep running into the same thing nobody warns you about when you're following tutorials. The basic retrieve-then-generate pipeline looks fine in demos. Clean question, clean doc, clean answer. Then real users show up. The failure mode that gets me is this: the system pulls chunks from different versions of the same policy document, has no way to know they're from different versions, blends them together, and returns an answer with full confidence. No caveat, no "I'm not sure," nothing. Just fluent and wrong. The deeper issue is that standard RAG has no mechanism for uncertainty. It retrieves, it generates, it moves on, same confidence level whether it nailed it or completely fabricated something plausible. What actually fixes this (at least in the systems I've worked on) isn't swapping out the model. It's the architecture: **A routing layer** — decide if retrieval is even necessary before making the call. Some questions don't need it and you're wasting tokens. **Retrieval scoring** — evaluate what came back before passing it to the model. If the context scores low, reformulate the query and try again instead of just generating garbage confidently. **A hallucination check** — second LLM call that reads both the generated answer and the retrieved docs and checks if every claim is actually traceable. Most teams aren't doing this and it's probably the highest ROI addition you can make. The retry loop especially helped in our case because users never phrase questions the way your embedding model expects. The system silently reformulates and retries, user has no idea it happened. None of this is exotic. It's just a few extra decision points in the pipeline. But if you're running plain RAG in production and wondering why users are losing trust in it, this is almost certainly why. Curious if anyone else has run into the versioning/context blending issue specifically, that one seems underreported.