UCB exploration via Q-ensembles
Summary
OpenAI presents a novel exploration strategy for deep reinforcement learning using ensembles of Q-functions with upper-confidence bounds (UCB), demonstrating significant performance improvements on the Atari benchmark.
View Cached Full Text
Cached at: 04/20/26, 02:45 PM
Similar Articles
#Exploration: A study of count-based exploration for deep reinforcement learning
OpenAI researchers demonstrate that a simple count-based exploration approach using hash codes can achieve near state-of-the-art performance on high-dimensional deep RL benchmarks, challenging the assumption that count-based methods cannot scale to continuous state spaces.
Revisiting Overestimation Bias Problem of Q-learning: Settling Large Discrete Action Space via Action Intersection
This paper revisits the overestimation bias in Q-learning under large discrete action spaces, proposing an action intersection strategy that enables semi-decoupling between two Q-functions to balance overestimation and underestimation. Experiments in tabular and deep RL settings show improved performance over several baselines.
Benchmarking safe exploration in deep reinforcement learning
OpenAI proposes standardizing constrained RL as the formalism for safe exploration and introduces Safety Gym, a benchmark suite for evaluating safe deep RL algorithms in high-dimensional continuous control tasks with safety constraints.
EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agents
EvoCUA-1.5 introduces an online reinforcement learning framework for multi-turn computer-use agents, achieving a 63.2% success rate on OSWorld-Verified and outperforming comparable open-weight models up to 35B parameters through step-level policy optimization and dynamic curriculum learning.
Learning Montezuma’s Revenge from a single demonstration
OpenAI demonstrates a method for training a reinforcement learning agent to play Montezuma's Revenge from a single human demonstration, addressing the challenge of sparse rewards through curriculum learning and careful hyperparameter tuning. The approach achieves strong performance on the notoriously difficult Atari game while showing generalization limitations on other titles.