eligibility-traces

Introduces Gated Q-learning, a new Q(λ) framework that smoothly interpolates between Watkins' and Peng's Q-learning to trade off off-policy bias and multistep credit assignment. Provides theoretical guarantees and empirical validation in random-walk environments.

This paper revisits the Adam optimizer for streaming reinforcement learning, demonstrating that established methods like DQN and C51 perform well when properly tuned. The authors propose Adaptive Q(lambda), which combines eligibility traces with Adam's variance adaptation to surpass existing streaming RL methods on 55 Atari games.