A practitioner argues that RAG is no longer the automatic solution for enterprise AI, pointing out that many problems are really about data hygiene or structured queries, and that agents with tool use are often better.
8 years in software and a decent share of my revenue since 2023 has come from building the exact thing I'm about to argue against. In January a prospect opened a call with "we have budget approved for a RAG system" I asked what questions the system needed to answer and then someone repeated the sentence about the budget. No one could describe the problem although everyone could describe the architecture. I have sat through some version of that meeting maybe 6 times now. Tbh we have shipped more than a dozen of these and some earn their keep. A support team answering from a stable, tended set of product docs is a perfectly good use of it. RAG isn't dead and anyone telling you it is has something newer to sell you. I just don't think it should be the automatic move anymore, the thing an enterprise reaches for whenever AI and documents show up in the same sentence. Most of our disappointing projects turned out not to be retrieval problems at all. One client wanted 40,000 documents embedded. Before touching a vector database we listed the 20 questions employees actually asked and more than half were structured things like refund totals and headcounts that belonged in a database query. About 50 documents covered most of the rest. The other 39,000ish were old drafts and superseded policies that contradicted the live ones so embedding the full corpus would mostly have made the wrong ans arrive faster. What they needed was housekeeping which no one budgets for. Retrieval assumes somebody owns the library and in most companies nobody has owned it for years. That's usually why they want magic on top of it in the first place. Our humbling version of this was a bot confidently citing a 2019 travel policy that had been replaced twice because nothing in the pipeline knew the word "replaced" Better chunking was never going to fix that. 3 months of deleting fixed it and the model took the blame the whole time for what was really a filing problem. The architecture is also shifting under all of this. Chunk and embed was a workaround for the tiny context windows of 2023 and that constraint has mostly dissolved. A stable document set can often just ride along in context now… cached. For the rest we let an agent search roughly the way a junior analyst would running keyword queries and opening whatever looks promising. When the answer lives in a system, it asks the system directly instead of a stale copy. Retrieval is still around but it's just one tool the agent sometimes picks up rather than the whole design. So before the next RAG line item gets approved.. write down the 20 questions it has to answer and go answer 5 of them by hand. That hour will tell you what kind of problem you actually have. The January prospect ended up with no vector database and their answer quality is the best thing we have delivered this year. I'm still a little embarrassed by how that sentence sounds. RAG will keep running fine in thousands of companies for another decade and honestly that's kind of the point. Things don't become legacy by failing. They become legacy by working early and then nobody thinks to ask again.
A developer reflects on when GraphRAG is preferable to standard RAG for AI agents, noting it becomes valuable for multi-hop reasoning and relationship understanding, but adds significant complexity.
Ian Channing criticizes companies that waste money on RAG systems that perform no better than Google AI, arguing that access to a corpus doesn't equate to deep expertise and that reasoning cannot be cleanly separated from knowledge.
The article highlights a critical failure mode in production RAG systems where confident but incorrect answers arise from versioning issues and lack of uncertainty mechanisms. It proposes architectural improvements like routing layers, retrieval scoring, and hallucination checks to mitigate these errors.
A developer discusses the limitations of RAG for aggregation and counting queries over collections of documents, and asks for community advice on alternative approaches like text-to-SQL and intent routing.
The author argues that most agent RAG failures are due to retrieval problems—specifically chunking errors, lack of freshness signals, and reliance on pure vector search—rather than the LLM, and recommends structural chunking, decay-based ranking, and hybrid BM25+vector search.