Model-Based Online Decision Making via Generative Trajectory Planning

Hugging Face Daily Papers Papers
reinforcement-learning diffusion-models model-based-rl decision-making robotics trajectory-planning

Summary

论文提出 Generative Trajectory Planning (GTP)，一种基于学习到的扩散先验对动作序列进行采样与束搜索精炼的模型在线决策框架，在 MuJoCo 运动与长时程操作任务上超越基线，并开源代码与模型。

Online decision-making faces a tension between efficient inference and long-horizon reasoning. Value-based methods rely on short-horizon bootstrapping that limits the propagation of long-term information, while trajectory optimization methods plan explicitly but must solve a high-dimensional optimization problem at every decision step. We propose Generative Trajectory Planning (GTP), a model-based framework that addresses this tension by learning a generative model as a proposal distribution over action sequences. Instead of optimizing sequences from scratch, GTP samples candidate sequences from a diffusion prior and refines them using learned dynamics, reward, and value models, enabling efficient trajectory-level reasoning. Across continuous control and long-horizon manipulation tasks, GTP matches or exceeds state-of-the-art model-free, diffusion-policy, and model-based baselines, and remains stable in regimes where strong baselines degrade. Our results suggest that generative models are best understood not as policies, but as structured proposal distributions for planning, offering a new perspective on how to integrate generative modeling with model-based decision-making.
Original Article
View Cached Full Text

Cached at: 10/15/26, 02:57 AM

Paper page - Model-Based Online Decision Making via Generative Trajectory Planning

Source: https://huggingface.co/papers/2610.18677 🧭 IntroducingGenerative Trajectory Planning (GTP), a model-based framework for online decision making that plans with alearned diffusion prior over action sequences.

Instead of optimizing action sequences from scratch at every step, GTP samples candidate sequences from the diffusion prior, refines them with beam search, and scores them with learned dynamics, reward and value models, alllearned online from scratchin a self-improving loop.

  • 🏃**Locomotion:**highest return on all five MuJoCo tasks against model-free, diffusion-policy and model-based baselines
  • 🦾**Long-horizon manipulation:**GTP solves PushT from pixels, where every baseline fails, and matches or beats every baseline on the Fetch tasks.
  • 🔍**Structured exploration:**the prior over action sequences steers exploration toward the states that matter for the task, instead of scattering the agent’s visits across the state space.
  • 🔭**Long-horizon training:**by reasoning over whole action sequences, GTP keeps training stably on long-horizon tasks where policy-centric methods, limited by short-horizon bootstrapping, break down.

Code and trained models for every environment are available!

🌐Project page· 💻Code· 🤗Models

We welcome your questions and feedback!

GTP overview

Similar Articles

TrajGenAgent: A Hierarchical LLM Agent for Human Mobility Trajectory Generation

arXiv cs.AI

TrajGenAgent proposes a hierarchical LLM agent framework that decouples macro-level activity planning from micro-level spatiotemporal instantiation for realistic human mobility trajectory generation without fine-tuning. It also introduces an anomaly-detection-based evaluation for behavioral fidelity.

Active Timepoint Selection for Learning Measure-Valued Trajectories

arXiv cs.LG

This paper introduces a framework for active timepoint selection to infer probability paths from sparse snapshots, using linearized optimal transport to map distributions into a tangent space for Gaussian Process modeling, thereby enabling uncertainty-aware acquisition policies.