q-learning

The paper develops model-free q-learning algorithms for reinforcement learning in continuous-time jump Markov decision processes, applied to network dynamic pricing, showing superior performance over benchmark methods.

This paper introduces QWM, a framework that integrates world models with Q-learning to enhance sample efficiency in reinforcement learning by using imagined trajectories for action selection without compromising training on real data. It demonstrates significant improvements over state-of-the-art methods on manipulation benchmarks.

This paper revisits the overestimation bias in Q-learning under large discrete action spaces, proposing an action intersection strategy that enables semi-decoupling between two Q-functions to balance overestimation and underestimation. Experiments in tabular and deep RL settings show improved performance over several baselines.

This paper studies decentralized multi-player Q-learning in episodic Markov decision processes under three forms of information asymmetry, proposing algorithms that achieve regret bounds matching the single-agent Q-learning rate up to logarithmic factors.

This paper presents RL2C, a Q-learning-based algorithm for optimizing laser cutting parameters (focal length, laser power) for optical films, reducing taper size and wastage. Experiments show it reduces optimization steps by up to 12.5% and processing time by up to 81.8% compared to existing RL methods.

This paper proposes an adaptive training controller for CVaR risk-aware Q-learning, improving finite-budget behavior, reducing Bellman residuals by ~85%, and yielding better risk-adjusted performance in daily Bitcoin trading.

The paper proposes SADQ, a modification to Q-learning that uses one-step rollout predictions from a dynamics model to regularize TD target aggregation, reducing bootstrap-induced overestimation and improving training stability across benchmarks.

Introduces Gated Q-learning, a new Q(λ) framework that smoothly interpolates between Watkins' and Peng's Q-learning to trade off off-policy bias and multistep credit assignment. Provides theoretical guarantees and empirical validation in random-walk environments.

Introduces VRDQ, a decentralized Q-learning algorithm for multi-agent reinforcement learning over static and time-varying networks, with finite-time convergence guarantees that achieve linear speedups in sample complexity with only Õ(1) communication.

This paper introduces DiPS, a Q-learning framework that dynamically selects persuasion strategies for high-stakes scenarios like wildfire evacuations, achieving higher success rates than zero-shot LLM and RAG baselines.

This paper proposes a quantum annealing enhanced Q-learning framework for remaining useful life prediction, using the D-Wave system to solve QUBO formulations for action selection. It outperforms classical and quantum baselines on NASA C-MAPSS and predictive maintenance datasets.

This paper proposes Reversal Q-Learning (RQL), an offline reinforcement learning algorithm that trains a flow policy using an expanded Markov decision process framework and techniques to enable off-policy RL without backpropagation through time. It achieves state-of-the-art performance on challenging simulated robotic tasks.

Moment Matching Q-Learning (MoMa QL) uses maximum mean discrepancy to match all moment statistics for distribution-level convergence in offline RL, achieving computational efficiency and strong performance on D4RL tasks.

This paper proposes FedQHD, a novel federated Q-learning method using hyperdimensional random-feature state encoders with linear readouts to enable closed-form function-space aggregation, addressing the federation gap due to heterogeneous client encoders.

This paper develops a sign-separated finite-time error analysis for constant step-size Q-learning, decomposing the error into negative and positive parts and providing bounds that reveal an asymmetry related to overestimation.

This paper presents a switching-system theory for Q-learning with linear function approximation, using joint spectral radius to analyze convergence stability under deterministic, i.i.d., and Markovian observations.

The paper introduces MemQ, a method that integrates Q-learning into self-evolving memory agents by using eligibility traces over provenance DAGs to solve credit assignment problems in episodic memory retrieval.

This paper introduces the DR.Q algorithm, which improves model-based representations for Q-learning by maximizing mutual information and using faded prioritized experience replay to reduce bias and overfitting in continuous control tasks.

OpenAI presents a novel exploration strategy for deep reinforcement learning using ensembles of Q-functions with upper-confidence bounds (UCB), demonstrating significant performance improvements on the Atari benchmark.

OpenAI researchers demonstrate a precise mathematical equivalence between soft (entropy-regularized) Q-learning and policy gradient methods in reinforcement learning, providing theoretical insight into why Q-learning works despite inaccurate value estimates. They validate this equivalence empirically on the Atari benchmark and show a Q-learning method can closely match A3C's learning dynamics.