Rethinking Agentic Search with Pi-Serini: Is Lexical Retrieval Sufficient?

Hugging Face Daily Papers Papers
agentic-search lexical-retrieval bm25 information-retrieval ai-agents search-optimization

Summary

This paper introduces Pi-Serini, a BM25-based agentic search system that demonstrates lexical retrieval can suffice for deep search when agents refine queries, achieving high accuracy and reducing costs compared to default settings.

Does a lexical retriever suffice as large language models (LLMs) become more capable in an agentic loop? This question naturally arises when building deep research systems. We revisit it by pairing BM25 with frontier LLMs that have better reasoning and tool-use abilities. To support researchers asking the same question, we introduce Pi-Serini, a search agent equipped with three tools for retrieving, browsing, and reading documents. Our results show that, on BrowseComp-Plus, a well-configured lexical retriever with sufficient retrieval depth can support effective deep research when paired with more capable LLMs. Specifically, Pi-Serini with gpt-5.5 achieves 83.1% answer accuracy and 94.7% surfaced evidence recall, outperforming released search agents that use dense retrievers. Controlled ablations further show that BM25 tuning improves answer accuracy by 18.0% and surfaced evidence recall by 11.1% over the default BM25 setting, while increasing retrieval depth further improves surfaced evidence recall by 25.3% over the shallow-retrieval setting. Source code is available at https://github.com/justram/pi-serini.
Original Article
View Cached Full Text

Cached at: 05/13/26, 12:20 AM

Paper page - Rethinking Agentic Search with Pi-Serini: Is Lexical Retrieval Sufficient?

Source: https://huggingface.co/papers/2605.10848 Does a lexical retriever suffice for agentic search when agents can keep refining their queries?

As LLMs become more capable in agentic loops, agents can continuously refine their behavior, including tool use and reasoning, based on feedback from the environment. Motivated by this, I couldn’t help but ask the question above.

To answer it, we introducePi-Serini(= PI + Anserini), a minimal BM25-based search agent equipped withsearch,browse, andreadtools. This interface allows agents to cache retrieved rankings locally and selectively decide which content should enter their context window, much like how people use Google Search. This design enables agents to retrieve more deeply.

Results on BrowseComp-Plus: Pi-Serini with GPT-5.5 achieves 83.1% answer accuracy and 94.7% surfaced evidence recall.

Controlled ablations show that: Well-configured BM25improves answer accuracy by 18.0% and surfaced evidence recall by 11.1% over the default BM25 setting. Increasing retrieval depthimproves surfaced evidence recall by 25.3% over shallow retrieval.

Overall, our answer is:Yes, a lexical retriever can suffice for agentic search when it is well configured and the search agent is equipped with a tool interface that enables deeper retrieval.

Notably, if you are worried about the high cost of deep research, Pi-Serini may help: it reduces evaluation cost by 3.3×–10×.

Paper:https://arxiv.org/abs/2605.10848 Code:https://github.com/justram/pi-serini Project site:https://ricky42613.github.io/piserini

Similar Articles

Beyond Semantic Similarity: Performance and Costs of Agentic Retrieval for Complex Tasks

Hugging Face Daily Papers

This paper investigates agentic retrieval that combines LLM reasoning with dense retrievers in a ReAct loop for complex search tasks, showing an 8.7-point nDCG@10 improvement over standard retrieval and strong generalization on ViDoRe v3 and BRIGHT leaderboards, but at a substantial latency and token cost of 107.4 seconds and 764K input tokens per query.

Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction

Hugging Face Daily Papers

The paper introduces Direct Corpus Interaction (DCI), a novel approach allowing AI agents to query raw text directly using standard terminal tools instead of traditional embedding-based retrieval. By bypassing fixed similarity interfaces and offline indexing, DCI significantly outperforms conventional sparse, dense, and reranking baselines across multiple IR and agentic search benchmarks.

SkillSeek: Revisiting Agent Skill Retrieval at Marketplace Scale

Hugging Face Daily Papers

AACL-IJCNLP 2026 Findings paper showing that classic lexical retrieval (BM25 plus a small cross-encoder) matches or beats agentic skill retrieval on agent skill selection across 3 of 4 settings, at roughly half the cost, arguing agentic retrieval should be the fallback rather than the default.