Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning
Summary
Introduces a three-stage training paradigm to internalize world model planning in LLM agents, enabling future-aware decision-making. Outperforms baselines on search and mathematical reasoning tasks.
View Cached Full Text
Cached at: 06/29/26, 05:27 AM
# Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning Source: [https://arxiv.org/abs/2606.27483](https://arxiv.org/abs/2606.27483) [View PDF](https://arxiv.org/pdf/2606.27483) > Abstract:Large language model \(LLM\) agents have demonstrated strong capability in sequential decision\-making, yet they remains fundamentally reactive in long\-horizon tasks\. Unlike humans who employ "what\-if" reasoning to evaluate potential plans before commitment, standard agents lack an internal world model to simulate future outcomes\. Therefore, we propose to internalize future\-aware planning by training a single autoregressive model to verbalize both a prospective state rollout and a plan\-conditioned success estimate\-a textual analogue of the Q\-value\. Crucially, we identify a format\-capability gap: simply fine\-tuning agents on look\-ahead traces during post\-training leads to superficial mimicry of foresight without genuine predictive grounding\. To bridge this gap, we introduce a three\-stage training paradigm: \(i\) World Model Agentic Mid\-Training \(WM\-AMT\) to inject latent predictive capabilities into the policy; \(ii\) Format\-Eliciting SFT \(FE\-SFT\) to structure this injected capability; and \(iii\) Foresight\-Conditioned Reinforcement Learning \(FC\-RL\) to refine the calibration and utility of the generated simulations\. Evaluated on search and mathematical reasoning tasks, our approach consistently outperforms other training baselines\. Our results demonstrate that effective internal world modeling in LLM agents requires a capability\-first training pipeline to achieve grounded and calibrated foresight\. ## Submission history From: Xuan Zhang \[[view email](https://arxiv.org/show-email/a2f10c0a/2606.27483)\] **\[v1\]**Thu, 25 Jun 2026 19:05:44 UTC \(19,366 KB\)
Similar Articles
Bridging the Agent-World Gap: Text World Models for LLM-based Agents
This paper systematically reviews text world models for LLM-based agents, covering foundations, construction paradigms, applications in planning and training, and evaluation methods.
Beyond Next-Observation Prediction: Agent-Authored World Modeling for Sequential Decision Making
This paper introduces Agent-Authored World Modeling (AAWM), a training procedure that constructs world-model supervision based on the policy's own decision needs rather than next-observation prediction, aligning the learning objective with the dynamics required for effective decision-making.
Why We Need World Models for AGI: Where LLMs Fail and How World Models May Outperform
This paper argues that large language models struggle with causal reasoning and long-horizon planning due to a mismatch between sequence prediction and reasoning over latent environment dynamics, and introduces the Latent Dynamics Inference perspective along with the Flux environment to study these limitations.
Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence
Agent-World introduces a self-evolving training framework for general agent intelligence that autonomously discovers real-world environments and tasks via the Model Context Protocol, enabling continuous learning. Agent-World-8B and 14B models outperform strong proprietary models across 23 challenging agent benchmarks.
Do you think World Models will lead to AGI?
A discussion on whether world models, which learn internal environment representations to simulate physics and plan actions, could lead to AGI by overcoming the limitations of reactive predictive text models like LLMs.