Over the weekend, the star count for Understanding AI Agents in Depth tripled to 7.2k. The community has translated the book into English, Tamil, and Vietnamese — thanks to contributors @nsdevaraj and @toanalien. I also used Kimi K3 to make some refinements on top of the English version. Thanks to several contributors for submitting PRs to fix bugs. At my wife’s suggestion, version V1.2 adds discussions about models and harness, The Bitter Lesson, and some harness techniques (Chapter 5).

bojieli/ai-agent-book

Source: https://github.com/bojieli/ai-agent-book

Understanding AI Agents in Depth: Design Principles and Engineering Practice

English | 中文 | Tiếng Việt | தமிழ்

This repository is the official open-source repository for the book Understanding AI Agents in Depth: Design Principles and Engineering Practice. It contains the full book text and accompanying example code. The full book text, illustrations, and experimental code are all open source — feel free to run the experiments yourself, submit issues, and open PRs.

📖 eBook

The book is available in multiple languages:

• Chinese PDF (Original): book/深入理解-AI-Agent-李博杰-v1.2.pdf
• English PDF (Community-contributed translation, by @nsdevaraj (https://github.com/nsdevaraj)): book-en/AI-Agents-in-Depth-Bojie-Li-v1.2.pdf
• Tamil PDF (Community-contributed translation, by @nsdevaraj (https://github.com/nsdevaraj)): book-ta/AI-Agents-in-Depth-Bojie-Li-v1.2-ta.pdf
• Vietnamese PDF (Community-contributed translation, by @toanalien (https://github.com/toanalien)): book-vi/AI-Agents-in-Depth-Bojie-Li-v1.2-vi.pdf

The Chinese text and compiled PDF are located in the book/ directory. The English, Tamil, and Vietnamese translations are community-contributed and located in book-en/, book-ta/, and book-vi/ respectively; they may lag behind the original Chinese version:

• Source text for book body: book/introduction.md (Introduction), book/chapter1.md ~ book/chapter10.md (Chapters 1–10), book/afterword.md (Afterword)
• Build yourself: Install pandoc, xelatex, the ElegantBook document class, and related fonts, then run bash cd book && bash build_pdf.sh

Figures are generated by book/gen_*_figs.py and stored in book/images/. Typesetting details are in book/preamble.tex and book/*.lua.

📑 Content Overview (Chapters 1–10)

The book unfolds around the core formula Agent = LLM + Context + Tools, with ten chapters as follows:

• Chapter 1 · Agent Basics: Starting from the new paradigm of “Model as Agent,” establishes the core formula Agent = LLM + Context + Tools, and introduces the Harness engineering concept — everything beyond the model itself is the true competitive advantage.
• Chapter 2 · Context Engineering: Context determines the upper bound of an Agent’s capability. Explores LLM API context structures, KV Cache-friendly design, prompt engineering, dynamic prompts and Agent Skills, status bar metadata, and context compression strategies.
• Chapter 3 · User Memory and Knowledge Base: Enables Agents to remember users across sessions and integrate external knowledge. Covers user memory systems, basic RAG pipelines, and knowledge organization and retrieval beyond flat text (structured indexes, knowledge graphs, etc.).
• Chapter 4 · Tools: Tools are the Agent’s hands. Discusses tool classification and general design principles, the MCP protocol and tool selection challenges, three types of tools (perception, execution, collaboration), and event-driven asynchronous Agents.
• Chapter 5 · Coding Agent and Code Generation: Code is “a tool that can create new tools” — the meta-capability of general-purpose Agents. Uses a production-grade Coding Agent as an example to demonstrate the complete implementation of this most powerful general-purpose tool.
• Chapter 6 · Agent Evaluation: Turns Agent performance into comparable signals. Covers evaluation environments, dataset design, metric systems, statistical significance, observability, evaluation-driven model selection, and production-grade internal evaluation with simulation environments.
• Chapter 7 · Model Post-Training: A full panorama of the three stages: pre-training, SFT, and RL. When to choose SFT vs. RL, RLHF, algorithm comparisons, data and environments, and cutting-edge explorations in tool calling and sample efficiency improvement.
• Chapter 8 · Agent Self-Evolution: Learning without modifying weights. Three learning paradigms, learning from experience, active tool discovery, and from “tool user to tool creator” — enabling Agents to move from “intelligent” to “proficient.”
• Chapter 9 · Multimodal and Real-Time Interaction: Extends perception and action from text to speech, GUI, and the physical world. Three speech paradigms (cascade/end-to-end full-modal/full-duplex), streaming speech perception and synthesis, Computer Use, and robotic manipulation.
• Chapter 10 · Multi-Agent Collaboration: Collective intelligence can exceed that of individuals. Multi-Agent classification frameworks, when multi-agent truly outperforms single-agent, collaboration with and without shared context, failure modes, and the emergent “Agent Society.”

💻 Accompanying Code

All projects are organized by chapter, corresponding one-to-one with the ten book chapters, covering a complete learning path from basic concepts to advanced techniques. Directory structure: chapterN/project_name/. Most experiments in Chapters 5, 8, 9, and 10 provide independently runnable demos that have been verified against real LLM APIs.

Project Type Description

Accompanying projects fall into three categories. Check the icon below to see how “ready-to-run” each project is:

• ✅ Independently Runnable: The repository provides full code; just configure an API key (see end of document) to run.
• 📖 Reproduction Guide: The project itself is a detailed reproduction document; it depends on external repositories (training frameworks, evaluation benchmarks, etc.) that need to be git clone’d separately. See “Obtaining External Repositories” below.
• 🚧 Design Document: Currently contains only architecture and implementation design documents; runnable code is still being refined.

The following projects are not ✅ Independently Runnable; please note when cloning this repository:

Obtaining External Repositories (Summary)

Some experiments in Chapters 6, 7, 9, and 10 depend on external repositories (evaluation benchmarks, training frameworks, robotics platforms, etc.) that are not included in this repository due to size or license. To avoid information overload up front, the complete clone commands, upstream addresses, and verified commits used in this book are provided in the appendix “Obtaining External Repositories” at the end of this document. We recommend starting with the independently runnable projects from earlier chapters, and following the appendix instructions when you need to reproduce training/evaluation/robotics experiments.

🚀 Chapter 1 · Agent Basics

learning-from-experience

• Reinforcement Learning vs. LLM Comparison

chapter1/learning-from-experience/

Compares traditional reinforcement learning (Q-learning) with LLM-based in-context learning, reproducing key insights from Shunyu Yao’s blog post “The Second Half.” Demonstrates how LLMs can surpass traditional RL with 250-400x sample efficiency through a treasure hunt game.

Core Concepts: Reinforcement Learning, In-Context Learning, Sample Efficiency, Prior Knowledge

web-search-agent

• Kimi K2 Model as Agent

chapter1/web-search-agent/

Implements an Agent with basic deep search capability, capable of multi-turn search and information integration.

Core Concepts: Web Search, Model-Native Agent

search-codegen

• GPT-5 Native Tool Integration

chapter1/search-codegen/

Builds an Agent with basic deep search and code sandbox capabilities, leveraging tools such as web search and code execution for complex analysis.

Core Concepts: Web Search, Code Generation, Model-Native Agent

context

• Context Ablation Study

chapter1/context/

Demonstrates the importance of each Agent context component through systematic ablation experiments. Supports multiple LLM providers (SiliconFlow Qwen, ByteDance Doubao, Moonshot Kimi) and configures different context modes to observe Agent behavior changes.

Core Concepts: Context Management, Tool Calling, ReAct Loop, Ablation Study

🎯 Chapter 2 · Context Engineering

local_llm_serving

• Local LLM Deployment and Tool Calling

chapter2/local_llm_serving/

A cross-platform local LLM deployment solution that automatically selects the best backend (vLLM or Ollama). Demonstrates that even a 0.6B small model can achieve excellent tool calling capability with good system design. Supports streaming responses for real-time thought process display.

Core Concepts: Model Deployment, Chat Template, Streaming, Tool Calling

attention_visualization

• Attention Mechanism Visualization

chapter2/attention_visualization/

Visualizes the complete input/output token sequence and attention weight distribution of an LLM, providing insights into how the model processes context, performs reasoning, and calls tools.

Core Concepts: Attention Mechanism, Token Analysis, Reasoning Process Visualization

kv-cache

• KV Cache-Friendly Context Design

chapter2/kv-cache/

Explores the impact of different context management patterns on KV Cache, demonstrating common error patterns that break cache efficiency. Experiments show how proper context design can significantly reduce latency and cost.

Core Concepts: KV Cache, Context Optimization, Performance Tuning

context-compression

• Context Compression Strategies

chapter2/context-compression/

Implements and compares multiple context compression strategies, including summarization, key information extraction, and semantic compression. Reduces token usage while maintaining Agent capability.

Core Concepts: Context Compression, Token Optimization, Information Density

prompt-engineering

• Prompt Engineering Ablation Study

chapter2/prompt-engineering/

Extends the Tau-Bench framework to quantify the impact of different prompt engineering factors on Agent performance through systematic ablation experiments. Demonstrates how factors like tone, instruction organization, and tool descriptions affect task completion rates.

Core Concepts: Prompt Engineering, Ablation Study, Performance Benchmarking

system-hint

• System Prompt Optimization

chapter2/system-hint/

Studies the impact of system hints on Agent behavior and explores how to improve performance by optimizing system prompts.

Core Concepts: System Prompt, Behavior Guidance, Prompt Optimization

log-sanitization

• Log Sanitization

chapter2/log-sanitization/

Implements an intelligent log sanitization system that protects sensitive data while preserving debugging information.

Core Concepts: Privacy Protection, Log Processing, Data Security

prompt-injection

• Prompt Injection Attack and Defense Experiments

chapter2/prompt-injection/

Constructs controlled experiments with 3 attack scenarios (direct injection, indirect injection, memory injection) × 4 defense configurations (no defense, prompt hardening, source labeling, combined defense). Uses deterministic rules to calculate attack success rates, intuitively showing how injection success rates drop significantly with layered defenses.

Core Concepts: Prompt Injection, Indirect Injection, Data-Instruction Separation, Runtime Validation

agent-skills-ppt

• Agent Skills Gradual Disclosure for PPT Generation

chapter2/agent-skills-ppt/

Reproduces the “gradual disclosure” idea of Agent Skills: the Agent only sees a thin Skill directory at startup, loads the full flow, detailed documentation, and bundled scripts for the pptx Skill only after recognizing the task requires it, and finally generates a real .pptx file using python-pptx.

Core Concepts: Agent Skills, Gradual Disclosure, On-Demand Loading, Tool Orchestration

📚 Chapter 3 · User Memory and Knowledge Base

user-memory

• User Memory System

chapter3/user-memory/

Builds a long-term user memory system that enables the Agent to remember user preferences and historical interactions, providing personalized service.

Core Concepts: Long-Term Memory, Personalization, User Modeling

mem0 / memobase

• Open-Source Memory Framework Comparison

chapter3/mem0/ and chapter3/memobase/

Implements user memory using two open-source memory frameworks, mem0 and Memobase, as a comparison implementation for Experiment 3-2 “Memory Strategy Comparison,” facilitating horizontal comparison of extraction patterns and answer quality across different memory schemes.

Core Concepts: Memory Framework, mem0, Memobase, Scheme Comparison

user-memory-evaluation

• User Memory Evaluation Framework

chapter3/user-memory-evaluation/

Systematically evaluates the accuracy, relevance, and effectiveness of user memory systems, including multiple test scenarios and evaluation metrics.

Core Concepts: Evaluation Framework, Test Cases, Performance Metrics

dense-embedding

• Dense Embedding Vector Search Service

chapter3/dense-embedding/

Builds a vector similarity search service, comparing two approximate nearest neighbor index algorithms: ANNOY (tree-based) and HNSW (graph-based). Demonstrates the trade-offs in performance, memory usage, and update capability across different index strategies.

Core Concepts: Dense Embedding, Vector Search, ANN Algorithm, Semantic Search

sparse-embedding

• Sparse Search Engine

chapter3/sparse-embedding/

Implements a sparse vector search engine based on the BM25 algorithm from scratch. Provides rich logging and visual interfaces to show the internal workings of the search engine, helping to understand term frequency weight calculation and inverted index principles.

Core Concepts: Sparse Embedding, BM25, TF-IDF, Exact Match

retrieval-pipeline

• Hybrid Retrieval Pipeline

chapter3/retrieval-pipeline/

Builds a complete retrieval pipeline combining dense retrieval, sparse retrieval, and neural re-ranking. Uses carefully designed test cases to systematically demonstrate the complementary advantages of hybrid retrieval in different scenarios.

Core Concepts: Hybrid Retrieval, Neural Re-ranking, Cross-Encoder, Retrieval Fusion

multimodal-agent

• Multimodal Information Extraction

chapter3/multimodal-agent/

Compares three multimodal processing strategies: native multimodal processing, extraction to text, and tool-based analysis. Through ablation studies within a unified framework, reveals the trade-offs in fidelity, cost, and flexibility across different technical paths.

Core Concepts: Multimodal, Visual Understanding, OCR, End-to-End Processing

structured-index

• Structured Index

chapter3/structured-index/

Implements and compares two advanced indexing strategies: RAPTOR (Recursive Abstractive Processing Tree) and GraphRAG (Knowledge Graph). Uses an index technical manual to demonstrate how to build structured indexes that reflect the inherent hierarchy and relationships of knowledge.

Core Concepts: RAPTOR, GraphRAG, Hierarchical Summarization, Knowledge Graph

agentic-rag

• Agentic RAG

chapter3/agentic-rag/

Compares the performance of traditional Non-Agentic RAG and Agentic RAG. Demonstrates how Agents use the ReAct pattern to drive iterative information retrieval, significantly improving answer quality when handling complex legal Q&A.

Core Concepts: Agentic RAG, ReAct Loop, Iterative Retrieval, Active Exploration

agentic-rag-for-user-memory

• Building User Memory with Agentic RAG

chapter3/agentic-rag-for-user-memory/

Applies the Agentic RAG framework to manage user conversation history. Uses multi-turn iterative search capabilities to handle memory retrieval across sessions, enabling basic recall and multi-session retrieval.

Core Concepts: User Memory, Conversation History Indexing, Cross-Session Retrieval

contextual-retrieval

• Context-Aware Retrieval

chapter3/contextual-retrieval/

Implements the context-aware retrieval technique proposed by Anthropic. By generating prefix summaries containing core context for each text chunk, it addresses the context loss problem of traditional chunking methods, reducing retrieval failure rates by 49–67%.

Core Concepts: Context Enrichment, Prefix Generation, Semantic Anchoring, Retrieval Optimization

contextual-retrieval-for-user-memory

• Context-Aware User Memory System

chapter3/contextual-retrieval-for-user-memory/

Applies context-aware retrieval to user memory construction, combining Advanced JSON Cards with context-aware RAG to form a two-layer memory structure, enabling higher-level proactive service capabilities.

Core Concepts: Two-Layer Memory, Structured Facts, Contextual Retrieval, Proactive Service

structured-knowledge-extraction

• Structured Knowledge Extraction

chapter3/structured-knowledge-extraction/

Using legal cases as an example, implements a three-stage pipeline: “bottom-up factor discovery → case clustering into prototypes → conversational advisory Agent.” Instead of predefined rigid fields, the LLM autonomously discovers factors from a large number of cases and summarizes them into a modular schema (core factors + charge-specific expansion factors). Cases are then clustered into prototypes, with factor importance calculated for each prototype. The Agent matches new cases to the most similar prototype, asks for missing information based on importance, and provides evidence-based advice (with legal disclaimers).

Core Concepts: Bottom-Up Knowledge Discovery, Modular Factors, Clustering Prototypes, Explainable Decision Making

🛠️ Chapter 4 · Tools

perception-tools

• Perception Tools MCP Server

chapter4/perception-tools/

Builds a comprehensive set of perception tools, providing capabilities for web search, multimodal understanding, file system operations, and access to public data sources. Most features are based on free open APIs (DuckDuckGo, Open-Meteo, Yahoo Finance, OpenStreetMap, etc.) and require no API keys.

Core Concepts: MCP Protocol, Multimodal Parsing, Public Data Sources, Document Understanding, Geospatial Information Services

execution-tools

• Execution Tools MCP Server

chapter4/execution-tools/

Implements a set of execution tools with security mechanisms, including file operations, code interpreter, virtual terminal, and external system integration. Uses an LLM secondary approval mechanism to prevent dangerous operations, automatically summarizes complex outputs, and performs syntax validation on code.

Core Concepts: MCP Protocol, Execution Safety, LLM Approval, Result Summarization, Automatic Validation

collaboration-tools

• Collaboration Tools MCP Server

chapter4/collaboration-tools/

Provides comprehensive collaboration capabilities, including browser automation (browser-use framework), human-in-the-loop (HITL), multi-channel notifications (Email, Telegram, Slack, Discord), and timer management. Supports admin approval for sensitive operations and scheduled task management.

Core Concepts: MCP Protocol, Browser Automation, HITL Mode, Multi-Channel Notification, Scheduled Tasks

agent-with-event-trigger

• Event-Driven Agent with MCP Integration

chapter4/agent-with-event-trigger/

A modern event-driven Agent built on FastAPI, integrating all tools from the first three MCP servers by default. Uses native asynchronous architecture for clear MCP tool loading, receives events from multiple sources via HTTP API (Web, instant messaging, GitHub, timers, etc.). Provides automatic API documentation (Swagger UI) and background monitoring capabilities.

Core Concepts: FastAPI, Native Asynchronous, MCP Integration, Event-Driven, Automatic API Documentation, Tool Orchestration

active-tool-selection

• Active Tool Selection

chapter4/active-tool-selection/

Implements an intelligent tool selection mechanism that allows the Agent to actively choose the most suitable tool combination based on task requirements, rather than passively accepting a predefined set of tools.

Core Concepts: Tool Selection, Dynamic Tool Loading, Task Analysis

async-agent

• Asynchronous Agent with Parallel Execution and Interruption Capability

chapter4/async-agent/

Implements the core of an event-driven asynchronous Agent framework (Flux) based on a single-threaded asyncio: an inbox event queue dispatches by urgency (interrupt/immediate/queue), supports parallel execution of asynchronous tools, interruption of the current turn during execution, and cancellation and status querying for simulated long-running tasks. Decision-making is performed by a real LLM (function calling).

Core Concepts: Asynchronous Programming, Event Queue, Interruption Mechanism, Parallel Tool Cancellation, Non-Blocking I/O

Additionally, chapter4/docker-compose.yml and chapter4/DOCKER_DEPLOYMENT.md provide a reference solution for containerized deployment of the above MCP tool servers.

💻 Chapter 5 · Coding Agent and Code Generation

coding-agent

• Production-Grade Coding Agent

chapter5/coding-agent/

A production-grade AI coding assistant built on Claude, using pure Python to implement all tools with no command-line dependencies. Contains 17 fully implemented tools covering file operations, search, shell operations, and project management. Notably implements a pure Python Grep tool fully compatible with ripgrep functionality.

Key Features:

• Pure Python implementation, no command-line dependencies, especially suitable for Mac users
• Complete tool suite: file read/write/edit, pure Python regex search, directory listing, shell session management
• System prompt techniques: timestamp, tool call count, TODO list management, detailed error messages
• Persistent shell environment, automatic linting, streaming response support
• Supports multiple LLM providers (Anthropic, OpenAI, OpenRouter)

Core Concepts: Code Generation, File Editing, Pure Python Tools, System Prompt, Linting, Multi-Provider Support

code-for-math

• Using Code to Improve Mathematical Problem Solving

chapter5/code-for-math/

Compares “pure chain-of-thought” and “code-assisted” modes on the same model using the same set of competition math problems. In the latter mode, problems are formalized into Python (sympy/numpy/scipy) and executed in a subprocess sandbox via function calling, replacing error-prone mental calculation with precise computation, resulting in significantly higher accuracy.

Core Concepts: Code Interpreter, Symbolic Computation, Chain-of-Thought Comparison, Tool-Enhanced Reasoning

code-for-logic

• Using Code to Improve Logical Reasoning

chapter5/code-for-logic/

Transforms “Knights and Knaves” logic puzzles into constraint satisfaction problems (CSP). The Agent uses python-constraint to define variables and biconditional constraints, calls the solver, and compares the accuracy of pure natural language reasoning vs. code-assisted modes on a set of K&K puzzles.

Core Concepts: Constraint Solving, CSP Modeling, Formal Reasoning, Code Assistance

small-model-codified-rules

• Codified Rules for Small Models

chapter5/small-model-codified-rules/

Based on controlled experiments in the τ-bench airline customer service scenario: after moving complex business policies (refund rules) from natural language prompts into code/tools, the task success rate and policy compliance of small models improve significantly. In-tool code validation can intercept the model’s incorrect understanding in real-time.

Core Concepts: Codified Business Rules, Policy Enforcement, In-Tool Validation, Small Model Reliability

paper-to-ppt

• Automatic Paper-to-PPT Generation (Proposer-Reviewer)

chapter5/paper-to-ppt/

Reframes “making a PPT” as a code generation problem: the Proposer writes Slidev (Markdown+HTML) code, the Reviewer renders each page into a PNG and uses a Vision LLM to check layout issues, iteratively revising based on structured feedback. The dual-Agent division significantly reduces peak context size.

Core Concepts: Code Generation, Slidev, Proposer-Reviewer, Visual Quality Control

paper-to-video

• Automatic Paper Explanation Video Generation

chapter5/paper-to-video/

Building on “Paper → PPT,” generates conversational explanation scripts for each slide, synthesizes speech using TTS, and uses ffmpeg to combine each slide screenshot with its audio into a narrated explanation video.

Core Concepts: Multimedia Generation, Script Generation, TTS, ffmpeg Audio-Video Synchronization

video-edit

• API-Based Intelligent Video Editing

chapter5/video-edit/

Given a multi-scene video and a natural language request, the Agent uses a two-step Vision localization (coarse-to-fine frame extraction and reading) to determine the time boundaries of the target scene, cuts the clip, and then has the Reviewer extract key frames from the final clip for verification, iterating if unsatisfactory.

Core Concepts: Video Editing, Vision Localization, Coarse-to-Fine, Proposer-Reviewer

adaptive-log-parser

• Adaptive Log Parsing System

chapter5/adaptive-log-parser/

A self-evolving log parsing system: when encountering an unrecognized new format, instead of raising an error, it sends the failed sample and error to a code generation Agent that generates a parse function. After automatic testing, the function is hot-updated and registered into the parsing engine, requiring no human intervention throughout the process.

Core Concepts: Code as System Adapter, Self-Healing Loop, Code Hot Update, Automatic Testing

log-diagnosis

• Production Log Intelligent Diagnosis System

chapter5/log-diagnosis/

The diagnosis Agent reads production trajectory logs, architecture documentation, and PRDs, automatically locates problems and root causes, generates structured reports and regression test cases, executes validation using a replay framework, and (mock) creates GitHub Issues via MCP.

Core Concepts: Trajectory Diagnosis, Root Cause Localization, Regression Test Generation, Replay Validation

dynamic-form

• Dynamic Form for Intent Clarification

chapter5/dynamic-form/

When faced with incomplete requests, the Agent doesn’t ask follow-up questions one by one. Instead, it dynamically generates a self-contained HTML form with cascading logic for the user to fill in all at once. The frontend aggregates the form into JSON and sends it back to the Agent to continue the task.

Core Concepts: Code Generation, Intent Clarification, Dynamic Form, Cascading Logic

erp-agent

• Natural Language ERP Agent (NL → SQL)

chapter5/erp-agent/

Converts Chinese natural language queries into SQL executed against a database, directly presenting the result table. The core is the artifact pattern: the LLM only generates SQL artifacts without manually moving data, saving tokens and avoiding manual calculation errors; even results with tens of thousands of rows can be returned instantly.

Core Concepts: NL2SQL, Artifact Pattern, Database Execution, Cost and Accuracy

conversational-ui

• Conversational UI Customization System

chapter5/conversational-ui/

Users propose UI customization requests (colors/fonts/copy/layout) in natural language. The Agent autonomously locates and modifies the React frontend source code, leveraging Vite’s Hot Module Replacement (HMR) to make changes take effect instantly, supporting multi-turn iterative customization.

Core Concepts: Code Modification, Frontend Customization, Hot Module Replacement, Multi-Turn Iteration

🎯 Chapter 6 · Agent Evaluation

terminal-bench

• Terminal Environment Benchmark

chapter6/terminal-bench/

Terminal-Bench is a benchmark for evaluating AI Agent performance in real terminal environments. From compiling code to training models and setting up servers, it assesses how Agents handle real end-to-end tasks. Includes a dataset of approximately 100 tasks and an execution framework supporting multiple Agent implementations.

Core Concepts: Terminal Automation, Task Evaluation, Docker Sandbox, Benchmarking

SWE-bench

• Software Engineering Benchmark

chapter6/SWE-bench/

SWE-bench is a benchmark for evaluating the ability of large language models to solve real GitHub issues. Given a codebase and issue description, the model must generate a patch that resolves the issue. Includes multiple versions: SWE-bench, SWE-bench Lite, SWE-bench Verified, and SWE-bench Multimodal.

Core Concepts: Code Repair, GitHub Issues, Patch Generation, Docker Evaluation

GAIA

• General AI Assistant Benchmark

chapter6/GAIA/

GAIA aims to evaluate next-generation LLMs (those with tool augmentation, efficient prompting, search access, etc.). Contains 450+ non-trivial questions requiring varying degrees of tool use and autonomy, with clear and unambiguous answers. Divided into 3 difficulty levels.

Core Concepts: Tool Use, Multi-Step Reasoning, Autonomy Evaluation

OSWorld

• Operating System-Level Agent Benchmark

chapter6/OSWorld/

Evaluates an Agent’s ability to perform complex tasks within a full operating system environment, including file management, application operations, and system configuration.

Core Concepts: Operating System Automation, Multi-Application Collaboration, System-Level Tasks

android_world

• Android Environment Benchmark

chapter6/android_world/ (📖 External repository; see “Obtaining External Repositories”)

Evaluates Agent performance in the Android mobile environment, including app navigation, UI interaction, and task completion capabilities.

Core Concepts: Mobile Automation, Android UI, App Interaction

chapter6/android-world/ (hyphenated naming) is not benchmark code, but rather the book’s analysis notes on failure cases of the T3A Agent in android_world (t3a*.md), which can be used as reading material.

tau2-bench

• Tool-Augmented Reasoning Benchmark

chapter6/tau2-bench/

Focuses on evaluating an Agent’s ability to use tools for complex reasoning, including scenarios involving computation, search, and data processing.

Core Concepts: Tool-Augmented Reasoning, Multi-Step Tasks, Tool Composition

elo-leaderboard

• ELO Ranking System

chapter6/elo-leaderboard/

Implements an Agent performance leaderboard based on the ELO rating system, evaluating the relative abilities of different Agents through head-to-head comparisons.

Core Concepts: ELO Rating, Relative Evaluation, Leaderboard System

model-benchmark

• Multi-Dimensional Model Performance Benchmark

chapter6/model-benchmark/

Conducts a horizontal benchmark test across multiple OpenAI-compatible LLM API providers. Uses streaming interfaces to precisely measure time-to-first-token (TTFT), measures end-to-end latency percentiles (p50/p95), throughput, and success rate under concurrency. Generates a multi-dimensional comparison table with a single command, demonstrating that model selection is a multi-faceted trade-off rather than just looking at leaderboards.

Core Concepts: TTFT, Latency Percentiles, Throughput, Concurrent Stress Testing, Model Selection

agent-cost-analysis

• End-to-End Agent Task Cost Analysis

chapter6/agent-cost-analysis/

Performs a full-chain cost breakdown for a typical multi-turn Agent task (customer service refund). Uses a self-built lightweight tracing system to record input/output/cache tokens, latency, and cost for each LLM call, aggregates to find “which step is most expensive,” and uses A/B testing to quantify the actual savings from KV-cache-friendly design plus context compression.

Core Concepts: Observability, Cost Breakdown, Prompt Caching, A/B Comparison

tts-quality-eval

• Fully Automated TTS Quality Evaluation Pipeline

chapter6/tts-quality-eval/

Synthesizes the same set of challenging texts using various TTS configurations (different model/voice/speed), then uses a multimodal LLM-as-a-Judge to score each dimension (clarity, naturalness, etc.) according to a Rubric, producing a reproducible configuration comparison table.

Core Concepts: LLM-as-a-Judge, Rubric Scoring, TTS Evaluation, Multi-Dimensional Comparison

🧠 Chapter 7 · Model Post-Training

This chapter contains multiple model post-training projects, covering various techniques and application scenarios for supervised fine-tuning (SFT) and reinforcement learning (RL).

AdaptThink

• Adaptive Reasoning Depth

chapter7/AdaptThink/ and chapter7/AdaptThink-original/

Teaches reasoning models to adaptively choose between Thinking and NoThinking modes based on problem difficulty. Using constrained optimization and importance sampling, significantly reduces reasoning costs (45-69%) while improving accuracy. Based on the DeepSeek-R1-Distill-Qwen model, trained using the DAPO algorithm.

Core Concepts: Adaptive Reasoning, Reasoning Cost Optimization, Constrained Optimization, Importance Sampling

retool

• Tool-Augmented Mathematical Reasoning

chapter7/retool/

Enhances mathematical reasoning ability of large language models using multi-turn dialogue and code sandbox. Trains the model in two stages (SFT and RL) to learn to use a code execution environment for solving math problems. Based on Qwen2.5-32B-Instruct, trained on the AIME 2024 dataset using the DAPO algorithm and SandboxFusion sandbox.

Core Concepts: Tool Use, Code Execution, Mathematical Reasoning, Multi-Turn Dialogue, DAPO Algorithm

AWorld / AWorld-train

• Embodied Agent Training

chapter7/AWorld/ and chapter7/AWorld-train/

Trains an embodied Agent based on the AWorld framework, enabling the Agent to perform complex tasks in a virtual environment and learn from experience.

Core Concepts: Embodied Intelligence, Environment Interaction, Learning from Experience

SFTvsRL

• SFT vs. RL Comparative Study

chapter7/SFTvsRL/

Systematically compares the effectiveness of supervised fine-tuning (SFT) and reinforcement learning (RL) on different tasks, analyzing the strengths and weaknesses of each method and their applicable scenarios.

Core Concepts: SFT vs. RL, Training Method Comparison, Performance Analysis

verl

• Efficient RL Training Framework

chapter7/verl/

Verl is an efficient reinforcement learning framework specifically designed for LLM RLHF training, supporting multiple algorithms such as PPO, GRPO, and DAPO.

Core Concepts: RLHF, PPO, Distributed Training, Efficient Optimization

Intuitor

• Intuitive Reasoning Training

chapter7/Intuitor/

Trains the model’s intuitive reasoning ability, enabling it to make reasonable judgments quickly without detailed reasoning chains.

Core Concepts: Intuitive Reasoning, Quick Decision-Making, Reasoning Chain Optimization

MultilingualReasoning

• Multilingual Reasoning

chapter7/MultilingualReasoning/

Trains the model’s reasoning ability in multiple language environments, improving performance on cross-lingual tasks.

Core Concepts: Multilingual, Cross-Lingual Reasoning, Language Generalization

SpatialReasoning

• Spatial Reasoning Training

chapter7/SpatialReasoning/

Focuses on training the model’s spatial reasoning ability, handling problems involving spatial relationships such as position, direction, and distance.

Core Concepts: Spatial Reasoning, Geometric Understanding, Positional Relationships

SimpleVLA-RL

• Vision-Language-Action RL

chapter7/SimpleVLA-RL/

Combines vision, language, and action in reinforcement learning training, enabling the model to understand visual input and execute corresponding actions.

Core Concepts: Vision-Language-Action, Multimodal RL, Embodied Intelligence

continued-pretraining

• Continued Pre-training

chapter7/continued-pretraining/

Performs continued pre-training on domain-specific data to improve the model’s performance in the target domain.

Core Concepts: Continued Pre-training, Domain Adaptation, Knowledge Injection

MiniMind-pretrain

• Small Model Pre-training

chapter7/MiniMind-pretrain/

Pre-trains a small language model from scratch, understanding the complete pre-training pipeline and key techniques.

Core Concepts: Pre-training, Small Model, Training Pipeline

sesame

• Sequence Modeling and Evaluation

chapter7/sesame/

Focuses on training and evaluation methods for sequence modeling tasks.

**Core