offline-rl

提出 TD-Aug 方法，通过从几何 horizon 模型（基于 TD-Flows 训练）采样来想象新的目标状态，作为目标条件行为克隆的数据增强，显著提升了策略在 OGBench 拼接任务中的零样本组合泛化能力。

论文提出 World-Model Policy Arbiter (WMPA)，一个测试时框架，通过在学习到的世界模型中推演多个冻结的目标条件策略并用共享价值函数评分，在 OGBench 上将宏平均成功率从 44% 提升至 58%。

The paper introduces Directed Temporal Representations for Control (DTRC), which learns a directed temporal quasimetric over frozen LeWorldModel features from offline visual trajectories to serve as a goal-relative progress signal for direct goal-conditioned policy learning, achieving strong control performance across ten visual tasks.

The paper introduces Generalized Implicit Temporal Abstraction (GITA), a method for offline goal-conditioned reinforcement learning that trains a single k-conditioned value function across multiple temporal abstraction scales, resolving the trade-off between long-range value signal and local resolution. GITA outperforms offline GCRL baselines like HIQL and OTA on OGBench, raising average success rates by 25 percentage points over HIQL.

This paper introduces certified safety curation for safe offline reinforcement learning, using a filter-then-clone pipeline with distribution-free guarantees to select safe trajectories without needing cost functions on every transition.

This paper investigates offline reinforcement learning for post-training code-generating LLMs, showing that it can improve zero-shot code generation performance using existing datasets without online sampling.

The paper introduces a method for extracting topological necessities as mechanism-invariant subgoals from offline trajectories, enabling cross-embodiment transfer in goal-conditioned reinforcement learning, with empirical improvements on benchmark tasks.

The paper introduces DCRL, a divide-and-conquer approach for offline goal-conditioned reinforcement learning that reduces error accumulation in long-horizon tasks via recursive binary tree decomposition, achieving improved performance on OGBench benchmarks.

DiDrive is a risk-aware hierarchical diffusion framework for safe offline reinforcement learning in autonomous driving that improves performance in complex traffic scenarios through integrated representation learning and distribution correction optimization.

This paper proposes LAC (LightActor, deepCritic), an offline reinforcement learning method that allocates model capacity to the critic for improved inference efficiency, matching performance of complex generative actors with lower latency.

This paper presents the first application of calibration models for offline hyperparameter selection in a real-world industrial setting, using a municipal water treatment plant, and shows they can generate realistic rollouts and recover hyperparameter sensitivity trends.

This paper proposes CSDG, a method for offline reinforcement learning that expresses Bellman backups as in-sample targets plus a convex-hull-neighborhood local correction, controlling OOD action estimation errors and improving value stability.

Sergey Levine announces his talk at the ICML 2026 Workshop on Decision-Making from Offline Datasets, covering foundations of data-driven decision making from model-based optimization to offline RL.

This paper proposes NFTR, a method for offline goal-conditioned reinforcement learning that uses normalizing flows for subgoal policies and a triangle-slack reweighting to address optimistic bias and mode collapse in hierarchical implicit Q-learning.

This paper formalizes the execution harness around LLM agents as a learnable control layer using a Harness MDP trained via offline reinforcement learning, showing improvements in verification behavior and final quality across domains.

AETDICE proposes a unified framework for nonlinear multi-objective reinforcement learning in offline settings, bridging SER and ESR paradigms via density-ratio estimation.

RS-Diffuser 提出了一种风险敏感的离线扩散规划框架，结合扩散轨迹生成和分布值批评器，通过尾部感知目标在推理时灵活调整风险偏好，在安全关键任务中提升了回报和鲁棒性。

This paper introduces LDM-v0, a large decision model trained offline on trajectories from thousands of diverse reinforcement learning environments, demonstrating that a single transformer policy can match the performance of task-specific policies across robotics, autonomous driving, inventory management, cybersecurity, trading, and video games.

This paper investigates whether different offline reinforcement learning losses (RFT, RIFT, DFT, Offline GRPO, DPO) for reasoning distillation produce mechanistically distinct weight updates in a small language model. Using identical math rollouts and a controlled setup with Qwen3-4B and attention-only LoRA, they find that SFT, RFT, and RIFT yield nearly colinear weight deltas, while DPO sits in a near-orthogonal subspace and achieves the highest accuracy.

A new method for off-policy reinforcement learning with diffusion models, using flow reversal to handle off-policy data by reversing the diffusion process on it.