@ickma2311: CMU Advanced NLP: Reinforcement Learning I had been curious about how RL works on top of LLMs, and this CMU lecture mad…
Summary
CMU Advanced NLP lecture clarifies how reinforcement learning optimizes whole-output rewards (correctness, helpfulness, safety) rather than next-token prediction used in pretraining/fine-tuning.
View Cached Full Text
Cached at: 04/22/26, 08:22 AM
CMU Advanced NLP: Reinforcement Learning I had been curious about how RL works on top of LLMs, and this CMU lecture made it much clearer for me: Pretraining/fine tuning focus on the next token; RL focuses on the reward of the whole output: correctness, helpfulness, safety,
Similar Articles
Beyond Reasoning: Reinforcement Learning Unlocks Parametric Knowledge in LLMs
This paper investigates whether reinforcement learning can improve the direct recall of parametric knowledge in LLMs beyond reasoning tasks. It demonstrates that RL with binary rewards yields significant gains in factual QA benchmarks by redistributing probability mass to unlock latent knowledge rather than acquiring new facts.
@ethantsliu: meta-rl induces agentic LLM exploration traditional RL trains LLM agents to use fixed policies that struggle with activ…
A new research paper reframes LLM agent training as a cross-episode Meta-RL problem, using critic-free policy gradients to enable in-context adaptation without gradient updates. The LAMER framework improves test-time performance by 11-19% over standard RL baselines on long-horizon tasks and generalizes better to unseen environments.
@jiqizhixin: Awesome blog! State of RL for reasoning LLMs https://aweers.de/blog/2026/rl-for-llms/…
A comprehensive blog post reviewing the state of reinforcement learning for reasoning LLMs, covering methods from REINFORCE and PPO to GRPO and beyond, with connections to key models like InstructGPT and DeepSeek-R1.
@agarwl_: Good blog, makes you think about the empirical observation that cureent RL methods that work for LLMs are *low bias* - …
A blog post explores the paradox of reinforcement learning for LLMs achieving rapid sample efficiency despite being information-theoretically inefficient, and highlights the importance of low-bias value functions.
@blc_16: MIT just released a new RL method called Pedagogical RL. The main lesson -> correct reasoning traces can still be bad t…
MIT introduces Pedagogical RL, a method that trains a teacher to produce trajectories that are learnable for a student by penalizing surprising steps, improving RL training efficiency.