@ianchanning: If your RAG system is no better than just https://google.com/ai you've wasted a ton of money (my suspicion is that ther…
Summary
Ian Channing criticizes companies that waste money on RAG systems that perform no better than Google AI, arguing that access to a corpus doesn't equate to deep expertise and that reasoning cannot be cleanly separated from knowledge.
View Cached Full Text
Cached at: 07/05/26, 12:33 PM
If your RAG system is no better than just https://google.com/ai you’ve wasted a ton of money (my suspicion is that there’s quite a few companies who’ve done this).
From the glorious Machine Studying:
“But this conflates having access to the corpus with developing deep expertise: you wouldn’t hire any of us as a lawyer just because we can Google the legal literature very intelligently. At minimum, what makes a lawyer a good lawyer is knowing what to look for, where to look, and what to do with a passage after they find it.
You could say that reasoning and search are not separable from knowledge (see below). An agent deciding what to grep for or which file to open is acting from its current weights, and those weights may actively conflict with the world encoded in the corpus.“ – https://jacobxli.com/blog/2026/machine-studying/#2-cant-the-agent-just-search-the-corpus…
Omar Khattab (@lateinteraction): You cannot separate reasoning and knowledge as cleanly as you think.
If you’d asked me what I care about in 2020/2021, I’d have said it was “decoupling the capacity that language models have for understanding text from how they store knowledge” (quote from link below this
Similar Articles
I don't think RAG is the default answer for enterprise anymore
A practitioner argues that RAG is no longer the automatic solution for enterprise AI, pointing out that many problems are really about data hygiene or structured queries, and that agents with tool use are often better.
@insomnia_vip: AN AI ENGINEER SPENT MONTHS BUILDING THE RAG STACK MOST PEOPLE TRY TO FAKE She published one open source project that t…
An AI engineer released an open-source project teaching how to build a local RAG system from scratch and a production-grade agentic architecture with LangGraph, hybrid retrieval, caching, and observability.
When is GraphRAG actually worth it for AI agents?
A developer reflects on when GraphRAG is preferable to standard RAG for AI agents, noting it becomes valuable for multi-hop reasoning and relationship understanding, but adds significant complexity.
Evaluated a RAG chatbot and the most expensive model was the worst performer. Notes on what actually moved the needle.
A detailed evaluation of a RAG customer support chatbot reveals that retrieval issues often masquerade as LLM problems, heuristic evaluators are misleading, deduplication improves quality, stricter grounding trades helpfulness for accuracy, and model sweeping can dramatically reduce cost while improving performance.
"Most RAG benchmarks lie about real-world corpora." Test data from 3 production websites.
This article argues that most RAG benchmarks are misleading because they assume uniform corpus quality, while real-world corpora vary significantly in content density. Using data from three production websites, it shows that a tiered approach and a 'yield score' can better predict retrieval effectiveness.