Context Training with Active Information Seeking

Hugging Face Daily Papers Papers
context-optimization active-information-seeking large-language-models search-based-training data-efficiency robustness few-shot-learning

Summary

This paper introduces a context optimization method that uses active information seeking via Wikipedia search and browser tools, combined with a search-based training procedure, to achieve robust performance improvements across diverse domains without updating model weights.

Most existing large language models (LLMs) are expensive to adapt after deployment, especially when a task requires newly produced information or niche domain knowledge. Recent work has shown that, by manipulating and optimizing their context, LLMs can be tailored to downstream tasks without updating their weights. However, most existing methods remain closed-loop, relying solely on the model's intrinsic knowledge. In this paper, we equip these context optimizers with Wikipedia search and browser tools for active information seeking. We show that naively adding these tools to a standard sequential context optimization pipeline can actually degrade performance compared to baselines. However, when paired with a search-based training procedure that maintains and prunes multiple candidate contexts, active information seeking delivers consistent and substantial gains. We demonstrate these improvements across diverse domains, including low-resource translation (Flores+), health scenarios (HealthBench), and reasoning-heavy tasks (LiveCodeBench and Humanity's Last Exam). Furthermore, our method proves to be data-efficient, robust across different hyperparameters, and capable of generating effective textual contexts that generalize well across different models.
Original Article
View Cached Full Text

Cached at: 05/14/26, 04:17 AM

Paper page - Context Training with Active Information Seeking

Source: https://huggingface.co/papers/2605.13050

Abstract

Context optimization methods enhanced with active information seeking via search and browser tools achieve superior performance across diverse domains while maintaining data efficiency and robustness.

Most existinglarge language models(LLMs) are expensive to adapt after deployment, especially when a task requires newly produced information or niche domain knowledge. Recent work has shown that, by manipulating and optimizing their context, LLMs can be tailored todownstream taskswithout updating their weights. However, most existing methods remain closed-loop, relying solely on the model’s intrinsic knowledge. In this paper, we equip these context optimizers with Wikipedia search and browser tools foractive information seeking. We show that naively adding these tools to a standardsequential context optimizationpipeline can actually degrade performance compared to baselines. However, when paired with asearch-based trainingprocedure that maintains and prunes multiplecandidate contexts,active information seekingdelivers consistent and substantial gains. We demonstrate these improvements across diverse domains, including low-resource translation (Flores+), health scenarios (HealthBench), and reasoning-heavy tasks (LiveCodeBench and Humanity’s Last Exam). Furthermore, our method proves to be data-efficient, robust across different hyperparameters, and capable of generating effective textual contexts that generalize well across different models.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2605\.13050

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.13050 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.13050 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.13050 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

MemTrain: Self-Supervised Context Memory Training

arXiv cs.CL

MemTrain proposes a self-supervised training framework that uses masked reconstruction and intermediate memory recall proxy tasks on Wikipedia corpora to enhance LLM agents' context memory, achieving up to 17.67 point gains on downstream memory-intensive QA benchmarks.

Understanding the Behaviors of Environment-aware Information Retrieval

Hugging Face Daily Papers

This paper presents the first systematic analysis of how large language models can learn to adapt query formulation strategies for different retrievers using reinforcement learning, revealing distinct optimal query styles and introducing a branching-based rollout technique for multi-retrieval-step training stability.

@eng_khairallah1: https://x.com/eng_khairallah1/status/2053405155630936297

X AI KOLs Timeline

The article argues that context engineering, which involves structuring the information and memory available to an AI, is more critical for performance than prompt engineering alone. It provides a structured overview of a course designed to teach how to build reliable AI systems by managing context layers like session history and persistent memory.