ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL

Papers with Code Trending Papers
context-management reinforcement-learning long-horizon-reasoning agents fine-grained-rl proactive-context-management

Summary

ContextPilot introduces a proactive context management framework for long-horizon agentic reasoning, using fine-grained reinforcement learning with branch sampling to improve performance and efficiency in maintaining compact working contexts.

Long-horizon agentic tasks require large language models (LLMs) to iteratively retrieve, integrate, and maintain dispersed information across multi-turn interactions, but preserving all interaction histories leads to a continuously growing working context. Recent proactive context management methods allow models to edit their own working context with specialized tools, yet they still face three key limitations: (1) a limited toolset restricted to search, deletion, and summarization, with no support for global planning, long-term memory, and adaptive compression; (2) inefficient exploration that treats context management actions uniformly despite their heterogeneous impacts on final outcomes; and (3) coarse-grained credit assignment that assigns the final trajectory-level reward to all intermediate context editing actions during RL. To bridge these gaps, we introduce ContextPilot, a proactive context management framework for long-horizon agentic reasoning. Our approach systematically augments the toolset with planning, long-term memory, and soft context offloading tools. We further propose an RL method tailored for context management, which uses context and entropy variation to identify critical editing decisions for branch sampling and estimates action-level advantages from all branched trajectories that pass through the corresponding context editing action. Experiments on long-context QA and deep search tasks show that ContextPilot achieves stronger performance with a more compact working context, consistently outperforming existing baselines across various base models and benchmarks. Code is available at https://github.com/Tencent/ContextPilot.
Original Article
View Cached Full Text

Cached at: 09/01/26, 12:48 PM

Paper page - ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL

Source: https://huggingface.co/papers/2608.28476

Abstract

ContextPilot improves long-horizon agent reasoning by expanding context-editing tools and using reinforcement learning with branch sampling to identify critical context decisions.

Long-horizon agentic tasks require large language models (LLMs) to iteratively retrieve, integrate, and maintain dispersed information across multi-turn interactions, but preserving all interaction histories leads to a continuously growing working context. Recentproactive context managementmethods allow models to edit their own working context with specialized tools, yet they still face three key limitations: (1) a limited toolset restricted to search, deletion, and summarization, with no support for globalplanning,long-term memory, and adaptive compression; (2) inefficient exploration that treats context management actions uniformly despite their heterogeneous impacts on final outcomes; and (3) coarse-grained credit assignment that assigns the final trajectory-level reward to all intermediate context editing actions during RL. To bridge these gaps, we introduce ContextPilot, aproactive context managementframework forlong-horizon agentic reasoning. Our approach systematically augments the toolset withplanning,long-term memory, andsoft context offloadingtools. We further propose an RL method tailored for context management, which uses context andentropy variationto identify critical editing decisions forbranch samplingand estimatesaction-level advantagesfrom all branched trajectories that pass through the corresponding context editing action. Experiments on long-context QA and deep search tasks show that ContextPilot achieves stronger performance with a more compact working context, consistently outperforming existing baselines across various base models and benchmarks. Code is available at https://github.com/Tencent/ContextPilot.

View arXiv pageView PDFProject pageGitHub57Add to collection

Models citing this paper3

#### tencent/ContextPilot-14B Text Generation• 15B• Updated1 day ago • 407 • 11 #### tencent/ContextPilot-8B Text Generation• 8B• Updated1 day ago • 403 • 6 #### tencent/ContextPilot-E4B Text Generation• 8B• Updated1 day ago • 404 • 5

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.28476 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.28476 in a Space README.md to link it from this page.

Collections including this paper2

Similar Articles

ContextPilot-14B (Hugging Face Repository)

TLDR AI

ContextPilot-14B is a Qwen3-14B checkpoint that teaches language-model agents proactive context management via fine-grained reinforcement learning for long-horizon tasks.

TokenPilot: Cache-Efficient Context Management for LLM Agents

Hugging Face Daily Papers

TokenPilot is a dual-granularity context management framework that reduces inference costs in long-horizon LLM sessions by stabilizing prompt prefixes and conservatively managing context segments, achieving 61-87% cost reduction on benchmarks while maintaining competitive performance.

Context-Aware RL for Agentic and Multimodal LLMs

Hugging Face Daily Papers

Introduces ContextRL, a reinforcement learning approach that teaches LLMs to identify which context supports an answer, achieving gains on agentic and multimodal benchmarks.

Learning Agent-Compatible Context Management for Long-Horizon Tasks

arXiv cs.AI

Introduces AdaCoM, an external LLM-based context manager for frozen agents, using reinforcement learning to improve long-horizon task performance by preserving task constraints and pruning stale content, with experiments on web search and deep research benchmarks.