Tag
This paper proposes a neural ODE-based regularization method that enforces latent embeddings in reinforcement learning agents to follow consistent ODE flows, aligning representation learning with environment dynamics and yielding performance gains on Atari and gridworld benchmarks.
Proposes Collaborative Weighting Actor-Critic (CWAC), a framework that uses distributional critics and a collaborative weighting mechanism to mitigate overestimation bias in off-policy reinforcement learning.
This paper analyzes over 33,000 experiments on actor-critic algorithm design components, finding that common defaults like Gaussian action distributions are unreliable while bounded distributions with adaptive update schedules are robust, offering practical guidance for practitioners.
This paper introduces spectral effective-rank entropy as a metric to measure and control critic complexity in actor-critic reinforcement learning, demonstrating its measurability and controllability in TD3 and PPO experiments.
Introduces EVOM, an agentic meta-evolution framework using an LLM-based design agent to automatically discover high-performance actor-critic architectures for reinforcement learning, outperforming manual baselines and prior methods on continuous control tasks.
This paper develops a statistical theory for offline reinforcement learning from trajectory-level outcome supervision, proposing the OPAC algorithm and characterizing when such supervision enables efficient learning versus when fundamental barriers arise.
This paper presents a theoretical framework for deep reinforcement learning in continuous environments, modeling it as a continuous-time stochastic process using stochastic control theory. The authors characterize an actor-critic algorithm's dynamics in the infinite width limit of two-layer networks, deriving an equation for infinitesimal changes in state distribution under a vanishingly small learning rate.
This paper provides a refined theoretical analysis of actor-critic methods with entropy regularization, showing that an exact critic acts as a strong variance reducer and enables sample complexity comparable to deterministic policy gradient, and that with a sufficiently accurate learned critic the benefits are preserved.
This paper identifies surrogate hacking and temporal uncertainty as failure modes in multi-timescale RL, and proposes a Target Decoupling architecture that removes routing from the actor, using the critic for auxiliary representation learning. The method eliminates policy collapse on the LunarLander-v2 benchmark and stably surpasses the 'Environment Solved' threshold without hyperparameter hacking.
This paper introduces ACSAC, a reinforcement learning method that uses an adaptive chunk size actor-critic algorithm with a causal Transformer Q-network to handle long-horizon, sparse-reward tasks. It demonstrates state-of-the-art performance on manipulation tasks by dynamically adjusting action chunk sizes based on state-dependent needs.
This paper introduces SCALAR, a structured critic-actor loop framework, to evaluate how different interaction patterns between AI agents improve reasoning in theoretical physics problems.
OpenAI proposes an asymmetric actor-critic method for robot learning that leverages full state observability in simulators to train policies that operate on partial observations (RGBD images), enabling effective sim-to-real transfer without real-world training data.
OpenAI releases ACKTR and A2C algorithms as part of its Baselines library, with ACKTR demonstrating improved sample complexity through natural gradient descent while maintaining computational efficiency comparable to first-order methods.