Mosaic: Multi-Session Agentic Reinforcement Learning

Hugging Face Daily Papers Papers
agentic-reinforcement-learning multi-session hierarchical-advantage context-compaction subagent-forking critic-free-optimization

Summary

论文提出 Multi-Session Agentic 强化学习设定，并提出 Mosaic 方法，通过层次化优势估计优化会话级贡献与分配，在 BrowseComp 上以更少会话数优于广播式优势方法。

Most existing agentic reinforcement learning methods assume single-session trajectories, where interaction proceeds within one continuously accumulated context. However, agentic techniques such as context compaction and subagent forking may produce trajectories spanning multiple sessions, creating a mismatch between the single-session training assumption and inference-time execution. Therefore, we introduce Multi-Session Agentic Reinforcement Learning, a more general setting that supports an arbitrary number of sessions within a trajectory. Under this setting, a widely adopted method is to broadcast trajectory-level advantages to all sessions, but this does not distinguish contributions from various sessions or guide session allocation within trajectories. To address this issue, we propose Mosaic, a multi-session agentic RL method with critic-free hierarchical optimization. At the trajectory level, it incorporates session-usage signals into advantage estimation. At the session level, it shapes each session's advantage according to the estimated marginal contribution. Experiments on BrowseComp with Qwen3.5-4B show that Mosaic outperforms broadcasting in both context compaction and subagent forking scenarios while using fewer sessions on average. Further analyses show that our method achieves improvements in session utility and allocation, which we suggested is a promising target for future advances in multi-session agentic RL.
Original Article
View Cached Full Text

Cached at: 10/16/26, 07:04 AM

Paper page - Mosaic: Multi-Session Agentic Reinforcement Learning

Source: https://huggingface.co/papers/2610.22174

Abstract

Multi-session agentic reinforcement learning is introduced to handle trajectories across multiple sessions, with Mosaic optimizing session-level contributions and allocation via hierarchical advantage estimation.

Most existingagentic reinforcement learningmethods assume single-session trajectories, where interaction proceeds within one continuously accumulated context. However, agentic techniques such ascontext compactionandsubagent forkingmay produce trajectories spanning multiple sessions, creating a mismatch between the single-session training assumption and inference-time execution. Therefore, we introduce Multi-SessionAgentic Reinforcement Learning, a more general setting that supports an arbitrary number of sessions within a trajectory. Under this setting, a widely adopted method is to broadcasttrajectory-level advantagesto all sessions, but this does not distinguish contributions from various sessions or guide session allocation within trajectories. To address this issue, we proposeMosaic, a multi-session agentic RL method withcritic-free hierarchical optimization. At the trajectory level, it incorporatessession-usage signalsinto advantage estimation. At the session level, it shapes each session’s advantage according to the estimatedmarginal contribution. Experiments on BrowseComp with Qwen3.5-4B show thatMosaicoutperforms broadcasting in bothcontext compactionandsubagent forkingscenarios while using fewer sessions on average. Further analyses show that our method achieves improvements in session utility and allocation, which we suggested is a promising target for future advances in multi-session agentic RL.

View arXiv pageView PDFProject pageGitHub8Add to collection

Get this paper in your agent:

hf papers read 2610\.22174

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.22174 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.22174 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.22174 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

AudioMosaic: Contrastive Masked Audio Representation Learning

arXiv cs.LG

AudioMosaic introduces a contrastive learning-based audio encoder that uses structured time-frequency masking on spectrogram patches for efficient large-batch training, achieving state-of-the-art performance on audio benchmarks and improving audio-language models.

Accurate and Efficient Long-Term Memory for LLM Agents

arXiv cs.AI

MOSAIC is a structured, conflict-aware long-term memory framework for LLM agents that uses entity-typed graph storage, hash-accelerated retrieval, and active conflict detection to achieve high accuracy and efficiency on long-conversation QA and factual conflict detection tasks.