分享一门来自加州大学伯克利分校的进阶课程：Advanced LLM Agents。

这门课聚焦大语言模型 Agent 的最新进展，从推理到规划、从代码到数学证明，系统拆解“能思考、会行动”的代理是怎么做出来的。

课程由 Dawn Song 教授主讲，并邀请 Google、Meta 等公司的研究人员担任客座讲师，内容硬核、案例前沿。

课程地址：https://rdi.berkeley.edu/adv-llm-agents/sp25…

你将学到：

• 推理时技术与后训练：把复杂推理能力真正“训出来、用起来”
• 搜索与规划：让代理更会选、更会走、更会做决策
• 代码生成与验证：打造更可靠的编程助手与自动化工具
• 数学定理证明与自动形式化：探索 AI 进入数学的关键路径
• 多模态自主代理：视觉与文本协同，完成更复杂的任务闭环
• AI 安全与安全性：面向部署的可靠性与风险控制

课程提供完整录像、PPT 与配套资料，适合具备机器学习基础的学生和开发者系统进阶。

CS294/194-280 Advanced Large Language Model Agents

Source: https://rdi.berkeley.edu/adv-llm-agents/sp25

Prospective Students

• Students interested in the course should first try enrolling in the course in CalCentral. The class number for CS194-280 is 33840. The class number for CS294-280 is 33841. Please join the waitlist if the class is full.
• We plan to expand the class size to allow more students to join. Please fill in thepetition formif you are on the waitlist or can’t get added to the waitlist. You will receive an email notification around the beginning of the spring semester if you are allowed in.
• Do not email course staff or TAs. Please useEdstemfor any questions. For private matters, post a private question on Edstem and make sure it is visable to all teaching staff.

Course Staff

Instructor(Guest) Co-instructor(Guest) Co-instructorDawn SongXinyun ChenKaiyu YangProfessor, UC BerkeleyResearch Scientist, Google DeepMindResearch Scientist, Meta FAIRTeaching Staff: Alex Pan, Tara Pande, Ashwin Dara, Jason Yan

Class Time and Location

Lecture: 4-6pm PT Monday at Anthro/Art Building 160

Course Description

Large language model (LLM) agents have been an important frontier in AI, however, they still fall short critical skills, such as complex reasoning and planning, for solving hard problems and enabling end-to-end applications in real-world scenarios. Building on ourprevious course, this course dives deeper into advanced topics in LLM agents, focusing on reasoning, AI for mathematics, code generation, and program verification. We begin by introducing advanced inference and post-training techniques for building LLM agents that can search and plan. Then, we focus on two application domains: mathematics and programming. We study how LLMs can be used to prove mathematical theorems, as well as generate and reason about computer programs. Specifically, we will cover the following topics:

• Inference-time techniques for reasoning
• Post-training methods for reasoning
• Search and planning
• Agentic workflow, tool use, and functional calling
• LLMs for code generation and verification
• LLMs for mathematics: data curation, continual pretraining, and finetuning
• LLM agents for theorem proving and autoformalization

Syllabus

DateGuest Lecture (4:00PM-6:00PM PT)Supplemental ReadingsJan 27thInference-Time Techniques for LLM Reasoning Xinyun Chen, Google DeepMind RecordingIntroSlides-Large Language Models as Optimizers -Large Language Models Cannot Self-Correct Reasoning Yet -Teaching Large Language Models to Self-Debug All readings are optional this week.Feb 3rdLearning to reason with LLMs Jason Weston, Meta RecordingSlides-Direct Preference Optimization: Your Language Model is Secretly a Reward Model -Iterative Reasoning Preference Optimization -Chain-of-Verification Reduces Hallucination in Large Language ModelsFeb 10thOn Reasoning, Memory, and Planning of Language Agents Yu Su, Ohio State University RecordingSlides-Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization -HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models -Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web AgentsFeb 17thNo Class - Presidents’ DayFeb 24thOpen Training Recipes for Reasoning in Language Models Hanna Hajishirzi, University of Washington RecordingSlides-Tulu 3: Pushing Frontiers in Open Language Model Post-Training -Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback -OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMsMar 3rdCoding Agents and AI for Vulnerability Detection Charles Sutton, Google DeepMind RecordingSlides-Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilities -From Naptime to Big Sleep: Using Large Language Models To Catch Vulnerabilities In Real-World CodeMar 10thMultimodal Autonomous AI Agents Ruslan Salakhutdinov, CMU/Meta RecordingSlides-Mind2Web: Towards a Generalist Agent for the Web -WebArena: A Realistic Web Environment for Building Autonomous Agents -VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks -Tree Search for Language Model AgentsMar 17thMultimodal Agents – From Perception to Action Caiming Xiong, Salesforce AI Research RecordingSlides-OSWORLD: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments -AGUVIS: Unified Pure Vision Agents For Autonomous GUI InteractionMar 24thNo Class - Spring RecessMar 31stAlphaProof: when reinforcement learning meets formal mathematics Thomas Hubert, Google DeepMind 10am-noon PT RecordingSlides-AI achieves silver-medal standard solving International Mathematical Olympiad problems -Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm -The Future of Mathematics? -Building the Mathematical Library of the FutureApr 7thLanguage models for autoformalization and theorem proving Kaiyu Yang, Meta FAIR RecordingSlides-LeanDojo: Theorem Proving with Retrieval-Augmented Language Models -Autoformalization with Large Language Models -Autoformalizing Euclidean GeometryApr 14thAdvanced topics in theorem proving Sean Welleck, CMU RecordingSlides-Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs -miniCTX: Neural Theorem Proving with Long-Contexts -Lean-STaR: Learning to Interleave Thinking and Proving -ImProver: Agent-Based Automated Proof OptimizationApr 21stAbstraction and Discovery with Large Language Model Agents Swarat Chaudhuri, UT Austin 10am-noon PT RecordingSlides-An In-Context Learning Agent for Formal Theorem-Proving -Symbolic Regression with a Learned Concept LibraryApr 28thTowards building safe and secure agentic AI Dawn Song, UC Berkeley RecordingSlides-Privtrans: Automatically Partitioning Programs for Privilege Separation -DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks -AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases -Progent: Programmable Privilege Control for LLM Agents## Enrollment and Grading

***Prerequisites:***Students are strongly encouraged to have had experience and basic understanding of Machine Learning and Deep Learning before taking this class, e.g., have taken courses such as CS182, CS188, and CS189.

Please fill out thepetition formif you are on the waitlist or can’t get added to the waitlist.

This is a variable-unit course. All enrolled students are expected to participate in lectures in person and complete weekly reading summaries related to the course content. Students enrolling in one unit are expected to submit an article that summarizes one of the lectures. Students enrolling in more than one unit are expected to submit a lab assignment and a project instead of the article. For students enrolling in 2 units, the project should have a written report, which can be a survey in a certain area related to LLMs. For students enrolling in 3 or 4 units, projects will follow either an applications track or a research track:

• **Applications Track:**Projects in this track focus on applied use cases of LLMs and do not necessarily need to contribute novel research. Students in this track will work in groups of 3-4. The project for 3-unit students should include an implementation (coding) component that programmatically interacts with LLMs, while 4-unit students must complete a more substantial implementation with the potential for real-world impact.
• **Research Track:**Students in this track will conduct novel research under the supervision of postdocs and graduate students, with the goal of publishing in a workshop or conference. Research track projects must be completed in groups of 2-3, and students must apply to participate via a forthcoming Google form. The expectations for implementation and intellectual contributions will align with the project requirements for 3- and 4-unit students.

The grade breakdowns for students enrolled in different units are the following:

1 unit2 units3/4 unitsParticipation40%16%8%Reading Summaries10%4%2%Quizzes10%4%2%Article40%Lab16%8%ProjectProposal10%10%Milestone10%10%Poster Presentation10%10%Presentation Recording10%5%Report20%20%Implementation25%## Lab and Project Timeline

ReleasedDueProject group formation1/272/24Project proposal2/32/24Project milestone2/243/31Lab3/314/28Project final poster presentation4/285/5Project final presentation recording4/285/16Project final report4/285/16## Office Hours

• Alex: 6-7pm on Mondays onZoom