model-based-rl

The paper introduces density-guided bootstrap boundaries, an adaptive mechanism for controlling imagined-return depth in latent world-model RL, proving contraction of the gated Bellman operator and showing improved sample efficiency within DreamerV3 across continuous-control and Atari tasks.

论文提出 Generative Trajectory Planning (GTP)，一种基于学习到的扩散先验对动作序列进行采样与束搜索精炼的模型在线决策框架，在 MuJoCo 运动与长时程操作任务上超越基线，并开源代码与模型。

This paper introduces QWM, a framework that integrates world models with Q-learning to enhance sample efficiency in reinforcement learning by using imagined trajectories for action selection without compromising training on real data. It demonstrates significant improvements over state-of-the-art methods on manipulation benchmarks.

This paper proposes Dreamer-SAC, a model-based reinforcement learning framework that integrates a recurrent state-space world model with soft actor-critic in latent space for sample-efficient autonomous driving. It outperforms DreamerV3, SAC, and PPO baselines while requiring fewer real environment interactions.

This paper introduces ATLAS, a model-based continual reinforcement learning algorithm that combines Grow When Required networks with Successor Features to achieve high sample efficiency and robust adaptation to non-stationary environments, demonstrating positive backward transfer in spatial navigation tasks.

This paper investigates how much structure a task needs from a world model, showing that the objective's dimensionality determines how many predictive directions the model installs, with the common scalar reward objective being only the rank-one corner of value equivalence.

This paper addresses objective mismatch in model-based RL by proposing offline diagnostics to predict closed-loop performance of latent world models. On LunarLander-v3, the Reward Observability Fraction (ROF) and a Composite score (CROF) enable selecting checkpoints that yield strong MPC and model-based RL policies with far fewer real-environment interactions.

COMET is a model-based reinforcement learning algorithm that combines a frozen object-centric encoder with a transformer-based world model and Monte Carlo Tree Search, using causal attention to focus on task-relevant objects, achieving higher scores on visual RL benchmarks.

This paper proposes a strategic robustness objective for learning simulators in model-based reinforcement learning, formulated as a minimax game between a model player and an adversarial policy player. Theoretical guarantees and a provably convergent algorithm are provided, with experiments showing reduced prediction error and improved real-world policy transfer.

Proposes Model-Based Diffusion Policy Optimization (MBDPO), a framework that unifies search and policy optimization in world models using diffusion policy representations, achieving consistent scaling behavior and superior performance across offline and online reinforcement learning tasks.

GPLD introduces a gradient-penalized latent dynamics regularizer for DreamerV3 to enforce local smoothness in transition learning, improving sample efficiency on continuous control tasks, especially complex locomotion.

Summary of David Silver's Reinforcement Learning Lecture 8 on integrating learning and planning, covering model-based RL and AlphaGo's use of policy and value networks with Monte Carlo Tree Search.

This paper introduces the DR.Q algorithm, which improves model-based representations for Q-learning by maximizing mutual information and using faded prioritized experience replay to reduce bias and overfitting in continuous control tasks.

This paper analyzes the 'training in imagination' paradigm in model-based reinforcement learning, deriving optimal sample allocation strategies and characterizing how dynamics and reward model errors affect policy returns.