Tag
This paper introduces QWM, a framework that integrates world models with Q-learning to enhance sample efficiency in reinforcement learning by using imagined trajectories for action selection without compromising training on real data. It demonstrates significant improvements over state-of-the-art methods on manipulation benchmarks.
The paper explores using complex-valued states inspired by quantum theory in sequence models, showing it can reduce optimization steps in Mamba and attention-based models with better sample efficiency.
A blog post explores the paradox of reinforcement learning for LLMs achieving rapid sample efficiency despite being information-theoretically inefficient, and highlights the importance of low-bias value functions.
This paper introduces PAIR, a pairwise-aware inclusion reweighting method for adaptive rollout allocation in RLVR, improving sample efficiency and accuracy over pointwise allocators by correcting biases in pairwise gradient estimation.
This paper introduces V-Simba, a visual RL architecture that adds normalization layers and pointwise convolutions to improve sample efficiency and stability. It matches or outperforms state-of-the-art methods across DMC, Adroit, and Meta-World benchmarks while being more computationally efficient than DrQ-v2.
This paper proposes OG-SPR, a model-free visual RL algorithm that combines latent self-prediction with observation prediction to learn dynamics-aware representations, achieving improved sample efficiency on DeepMind Control Suite tasks.
This paper introduces ATLAS, a model-based continual reinforcement learning algorithm that combines Grow When Required networks with Successor Features to achieve high sample efficiency and robust adaptation to non-stationary environments, demonstrating positive backward transfer in spatial navigation tasks.
This paper proposes InK, a neurosymbolic hierarchical reinforcement learning approach that uses incremental knowledge for symbolic planning and reward-shaped low-level neural modules, achieving improved sample efficiency in navigation tasks.
Researchers from Harvard and UIUC discovered a third pretraining axis that improves sample efficiency by 6.2x and speeds up GenAI generation by 250x.
Proposes ERAHBO, an efficient heteroscedastic Bayesian optimization method for risk-aware hyperparameter optimization in reinforcement learning, using adaptive re-sampling to improve sample efficiency over fixed-budget approaches.
This paper proposes FaStR, a method that factorizes the transition kernel in reinforcement learning using CP decomposition into separate state, action, and next-state encoders, improving sample efficiency especially in high-dimensional locomotion tasks.
This paper presents KGRL, a neuro-symbolic algorithm that uses domain knowledge expressed in Datalog to prune actions and constrain parameters in Parametrized Action Markov Decision Processes, improving sample efficiency and episodic return over state-of-the-art baselines.
Introduces Learning from Hindsight (LfH), a method that applies hindsight relabeling to RL post-training of vision-language-action models. By relabeling failed robot rollouts with the tasks they actually achieved, LfH achieves 5x improvement in sample efficiency on out-of-distribution manipulation tasks.
This paper proposes correlation-aware contextual bandit algorithms that leverage surrogate reward signals from machine learning models for LLM routing, achieving improved accuracy-cost trade-offs and sample efficiency compared to standard baselines.
The article argues that training AI on millions of verifiable tasks across diverse RL environments could lead to AGI, and that scaling may overcome current limitations like sample inefficiency. It also examines why progress on computer use has been slower due to lack of grindable environments.
Mesh-RL is a spatial domain-decomposition framework for reinforcement learning that partitions the environment into overlapping subgrids to accelerate temporal-difference learning and long-range credit assignment, improving convergence speed and sample efficiency in sparse-reward environments.
Play2Perfect is a reinforcement learning framework that uses playful interaction with diverse objects to learn general manipulation skills, then fine-tunes for precise assembly tasks, achieving 33x sample efficiency and zero-shot sim-to-real transfer on tight insertions.
This article deeply analyzes the problem that AI's sample efficiency is far lower than that of humans, pointing out that frontier models require massive amounts of domain-specific data, while humans can learn from just a few examples. This data black hole is a core bottleneck in current AI development. Through multiple comparisons (annotation volume, robot manipulation, driving) and refuting common objections, the article demonstrates the severity of this gap and explores its impact on the goals of AI automation.
The article argues that text optimization—modifying prompts, context, memory, and retrieval—should be treated as a legitimate learning mechanism alongside weight optimization, highlighting its sample efficiency and ability to scale via update-time compute.
The author argues that text optimization (prompts, context, memory) is a legitimate and sample-efficient learning mechanism that should be taken more seriously by the ML community, enabling a new scaling axis of update-time compute.