加州大学开放课程《大语言模型的强化学习》，用“理论 + 实战”的方式，把 AI 训练的关键技术从零到一讲透，帮你系统建立从强化学习到 LLM 训练的完整框架。 课程内容覆盖全面，配套资源齐全：讲座幻灯片、完整视频、实践练习一应俱全，学完就能上手做。 课程地址：http://ernestryu.com/courses/RL-LLM.html… 你将学到： - 深度强化学习核心：MDP、策略梯度、A3C、PPO 等关键算法 - 大语言模型基础：NLP、语言建模、RNN 等入门与脉络 - RLHF 全流程拆解：基于人类反馈的训练方法与落地思路 - 可验证奖励强化学习：面向更安全、更可靠的训练范式 - 动手实践：Jupyter 代码示例 + 课后作业，边学边练 课程由 UCLA 数学系助理教授主讲，YouTube 提供全套视频，内容扎实，适合想把“RL + LLM 训练”真正学明白的人。

Reinforcement Learning of Large Language Models

Source: https://ernestryu.com/courses/RL-LLM.html

Lecture slides

• Chapter 0: Prologue.
• Chapter 1: Deep Reinforcement learning.
• Chapter 2: Large Language Models.
• Chapter 3: Reinforcement Learning of Large Language Models.

Lecture videos

• Chapter 0: Prologue.
• Chapter 1.1: MDP foundations, imitation learning, and value iteration.
• Chapter 1.2: Deep policy evaluation.
• Chapter 1.3: Deep policy gradient methods (A3C).
• Chapter 1.4: Deep policy gradient methods (PPO, GRPO).
• Chapter 1.5: AlphaGo, test-time compute, and expert iteration.
• Chapter 2.1: NLP foundations, language modeling, RNNs.
• Chapter 2.2: Transformers I (BERT, GPT-1).
• Chapter 2.3: Transformers II (modern transformers updates and sampling methods).
• Chapter 2.4: In-context learning and instruction fine-tuning.
• Chapter 3.1: Reinforcement learning from human feedback (PPO, DPO).
• Chapter 3.2: Reinforcement learning with verifiable rewards (RLVR).

Course Information

Instructor

Ernest K. Ryu Assistant Professor of Mathematics, UCLA,

Prerequisites

Students are expected to have basic familiarity with deep learning at the level of image classification. No prior experience with reinforcement learning (RL) or large language models (LLMs) is assumed. For the deep RL lectures, students should be familiar with conditional expectations and the tower property (law of total expectation).