@simplifyinAI: RAG is officially cooked. IBM researchers built a retriever that produced 65× fewer invalid section references than a f…

X AI KOLs Timeline Papers

Summary

IBM researchers introduced STAIR, a retriever that leverages document tables of contents for structured retrieval, achieving 82.6% top-1 section accuracy on the SearchTome benchmark and 65× fewer invalid section references than a fine-tuned DSI baseline.

RAG is officially cooked. IBM researchers built a retriever that produced 65× fewer invalid section references than a fine-tuned baseline. RAG gives an AI relevant documents to read before answering. But finding the right information is still a problem. Many systems split documents into chunks, losing the chapter headings and structure that help explain where information belongs. IBM’s approach, STAIR, uses that structure. Think of opening a book’s contents page and finding the right section before reading. STAIR uses the table of contents to guide retrieval towards the section most likely to contain the answer. On SearchTome, a benchmark covering 18 books across six domains, it retrieved the correct section first 82.6% of the time. The fine-tuned DSI baseline scored 76.9%. Invalid section references fell from 3.25% to 0.05%. That’s where the 65× figure comes from. It measures retrieval references, not hallucinations in the final answer. For businesses searching long manuals, policies and technical documents, this is a promising improvement. It still needs testing on real company data, but preserving document structure could help agents find better evidence before they answer.
Original Article
View Cached Full Text

Cached at: 11/01/26, 01:55 PM

RAG is officially cooked.

IBM researchers built a retriever that produced 65× fewer invalid section references than a fine-tuned baseline.

RAG gives an AI relevant documents to read before answering. But finding the right information is still a problem.

Many systems split documents into chunks, losing the chapter headings and structure that help explain where information belongs.

IBM’s approach, STAIR, uses that structure.

Think of opening a book’s contents page and finding the right section before reading.

STAIR uses the table of contents to guide retrieval towards the section most likely to contain the answer.

On SearchTome, a benchmark covering 18 books across six domains, it retrieved the correct section first 82.6% of the time. The fine-tuned DSI baseline scored 76.9%.

Invalid section references fell from 3.25% to 0.05%. That’s where the 65× figure comes from. It measures retrieval references, not hallucinations in the final answer.

For businesses searching long manuals, policies and technical documents, this is a promising improvement.

It still needs testing on real company data, but preserving document structure could help agents find better evidence before they answer.

Similar Articles

@vintcessun: Feeding too many documents into RAG causes retrieval quality to drop from 75% to 40%? Vector search is diluted by a large amount of irrelevant content, causing a sharp drop in hit rate in real deployment. Root cause: heterogeneous documents are retrieved together, noise drowns out signal. Multi-agent orchestration seems intelligent but actually introduces a precision-fidelity paradox—poor configuration leads to failure in both aspects. The paper proposes MA…

X AI KOLs Timeline

This paper identifies 'vector search dilution' in RAG systems when scaling to large heterogeneous document collections, where accuracy dropped from 75% to 40% in a real-world deployment. The proposed MASDR-RAG method uses domain scoping via organizational metadata before retrieval, improving P@10 from 0.77 to 0.86 with low cost and easy deployment.