offline-reinforcement-learning

LASER提出了一种基于潜在空间伴随匹配（latent space adjoint matching）的离线强化学习算法，通过熵正则化避免潜在空间策略坍缩和利用critic伪影，在40个OGBench任务上以固定超参数取得SOTA性能。

This position paper argues that offline RL should shift from learning static deployment policies to learning adaptive policy priors that can improve through subsequent online interaction, proposing a framework called Adaptive Offline RL (AORL) and highlighting Bayesian offline RL as a principled approach to preserving epistemic uncertainty.

The paper proposes Q-Target Pretrained Transformers (QTPT) for in-context reinforcement learning, replacing behavior cloning with a Bellman-style Q-target objective to improve robustness to weak or suboptimal data quality.

The paper proposes Predictive Action Chunk Learning (PACL) for robot manipulation, which learns from mixed-quality deployment experiences using a predictive critic and diffusion actor to improve policy performance.

The paper proposes RSIQL, a non-hierarchical method for offline goal-conditioned reinforcement learning that uses selective reward stimulation to improve performance on benchmarks like D4RL.

This paper presents a large-scale empirical study of offline reinforcement and imitation learning, analyzing over 160,000 training runs to understand the effects of hyperparameters and dataset properties, and introduces JumpStart, a resource suite for reliable policy-learning research.

The article presents BFN-RL, a unified generative modeling framework for offline reinforcement learning based on Bayesian Flow Networks, capable of generating effective trajectories across discrete and continuous state spaces.

CoDrift is a compositional framework for offline reinforcement learning that combines behavioral and value objectives into a unified policy field, achieving state-of-the-art performance on benchmarks like OGBench and D4RL.

Proposes decoupling policy extraction from actor training in offline reinforcement learning to address issues like out-of-distribution action amplification and support-value trade-offs, showing improved performance over coupled methods.

Introduces QDOS, a unified pipeline for offline-to-online reinforcement learning that uses advantage-weighted quality-diversity pretraining to extract diverse and high-value skills, significantly improving performance in manipulation and locomotion tasks.

This paper leverages offline reinforcement learning to automatically generate high-quality chess puzzles, enhancing pedagogical value for beginners based on extensive user data from platforms like Chess.com and Lichess.

This paper introduces the Counterfactual Clinical Audit (CCA) framework to evaluate offline reinforcement learning agents for ICU sepsis management, exposing 'toxic mimicry' where agents replicate harmful treatment patterns that standard metrics miss. Using MIMIC-III data, it shows a Medical Decision Transformer fails to escalate vasopressors under rising lactate, while a causal transformer performs safely.

Introduces CODS, an iterative critic-guided data selection method for offline reinforcement learning that retains task performance at low data budgets by selecting high-residual transitions over multiple rounds.

Proposes temporal-distance JEPA (TD-JEPA) which mines directed temporal cost from offline trajectories to improve latent world model predictive control, achieving higher success rates on robotic environments.

This paper proposes a novel offline reinforcement learning framework for active flow control that uses a sensor position-conditioned architecture with Point Attention layers to handle varying sensor configurations, enabling data-driven policy extraction without costly online interactions.

This paper presents an offline reinforcement learning framework for optimizing SLAM throughput control in warehouse fulfillment environments, balancing throughput maximization with downstream stability. The approach is algorithm-agnostic and demonstrates that the CQL policy improves system health by 22.97% and reduces throttling duration by 3.18%.

This paper introduces DRIVE, a unified Transformer-based framework for offline auto-bidding that decouples candidate action generation from decision making, combining distributional action modeling, retrieval-augmented candidate generation, and value-based evaluation to improve bidding performance under budget and cost constraints.

This paper introduces RL4F, an offline reinforcement learning benchmark for plasma control in nuclear fusion, providing closed-loop evaluation environments and baseline comparisons across four profile tracking tasks using real tokamak data from DIII-D. The codebase and datasets are open-sourced to foster further research.

This paper proposes a three-stage diagnostic framework to identify why offline model selectors fail to beat the best single model, applying it to dropout prediction on edX clickstream data. The study finds that the bottleneck is local representational ambiguity rather than learner choice or distribution shift, recommending state redesign or new data collection over further algorithm tuning.

This paper introduces Posterior Hybrid Bayesian Belief (PhyB), a framework that reformulates the expectation in Bayesian RL as a convex combination over dynamics models, enabling efficient regularized offline policy optimization with bounded objective discrepancy and state-of-the-art performance.