加州大学这课，搞AI的都给我冲！ 理论+实战，把RL和LLM训练从零到一拆成渣。教你MDP、PPO算法、RLHF全流程，还有Jupyter代码实操。UCLA教授主讲，视频+作业都有，学完直接上手。 课程地址：https://ernestryu.com/courses/RL-LLM.html… 别再傻看论文了，这套课能让你真学会RL+LLM训练。不然你连ChatGPT怎么调教出来的都不知道！

Reinforcement Learning of Large Language Models

Source: https://ernestryu.com/courses/RL-LLM.html

Lecture slides

• Chapter 0: Prologue.
• Chapter 1: Deep Reinforcement learning.
• Chapter 2: Large Language Models.
• Chapter 3: Reinforcement Learning of Large Language Models.

Lecture videos

• Chapter 0: Prologue.
• Chapter 1.1: MDP foundations, imitation learning, and value iteration.
• Chapter 1.2: Deep policy evaluation.
• Chapter 1.3: Deep policy gradient methods (A3C).
• Chapter 1.4: Deep policy gradient methods (PPO, GRPO).
• Chapter 1.5: AlphaGo, test-time compute, and expert iteration.
• Chapter 2.1: NLP foundations, language modeling, RNNs.
• Chapter 2.2: Transformers I (BERT, GPT-1).
• Chapter 2.3: Transformers II (modern transformers updates and sampling methods).
• Chapter 2.4: In-context learning and instruction fine-tuning.
• Chapter 3.1: Reinforcement learning from human feedback (PPO, DPO).
• Chapter 3.2: Reinforcement learning with verifiable rewards (RLVR).

Course Information

Instructor

Ernest K. Ryu Assistant Professor of Mathematics, UCLA,

Prerequisites

Students are expected to have basic familiarity with deep learning at the level of image classification. No prior experience with reinforcement learning (RL) or large language models (LLMs) is assumed. For the deep RL lectures, students should be familiar with conditional expectations and the tower property (law of total expectation).