bojieli/ai-agent-book Source: https://github.com/bojieli/ai-agent-book # Understanding AI Agents: Design Principles and Engineering Practices English | 中文 | Tiếng Việt | தமிழ் This repository is the open-source main repository for the book Understanding AI Agents: Design Principles and Engineering Practices. It contains the full text of the book and supporting example code. The full text, illustrations, and accompanying experimental code are all open source. You are welcome to run the experiments yourself and submit issues and PRs. ## 📖 E-book The full book is available in multiple languages: - Chinese PDF (Original): book/深入理解-AI-Agent-李博杰-v1.2.pdf - English PDF (community-contributed translation, by @nsdevaraj (https://github.com/nsdevaraj)): book-en/AI-Agents-in-Depth-Bojie-Li-v1.2.pdf - Tamil PDF (community-contributed translation, by @nsdevaraj (https://github.com/nsdevaraj)): book-ta/AI-Agents-in-Depth-Bojie-Li-v1.2-ta.pdf - Vietnamese PDF (community-contributed translation, by @toanalien (https://github.com/toanalien)): book-vi/AI-Agents-in-Depth-Bojie-Li-v1.2-vi.pdf The Chinese text and compiled PDF are located in the book/ directory; the English, Tamil, and Vietnamese translations are community contributions located in book-en/, book-ta/, and book-vi/ respectively, and may lag behind the original Chinese version: - Source text: book/introduction.md (Introduction), book/chapter1.md ~ book/chapter10.md (Chapters 1 to 10), book/afterword.md (Afterword) - Self-compilation: Install pandoc, xelatex, the ElegantBook document class, and related fonts, then run bash cd book && bash build_pdf.sh Figures are generated by book/gen_*_figs.py and stored in book/images/. Typesetting details are in book/preamble.tex and book/*.lua. ## 📑 Content Overview (Chapters 1–10) The book revolves around the core formula Agent = LLM + Context + Tools, and the ten chapters are as follows: - Chapter 1 · Agent Fundamentals: Starting from the new paradigm of “Model as Agent,” establish the core formula Agent = LLM + Context + Tools, and introduce the Harness engineering—everything beyond the model is where true competitive advantage lies. - Chapter 2 · Context Engineering: Context determines the upper limit of Agent capabilities. Dive into the context structure of large model APIs, KV Cache-friendly design, prompt engineering, dynamic prompts and Agent Skills, state bar meta-information, and context compression strategies. - Chapter 3 · User Memory and Knowledge Base: Enable Agents to remember users across sessions and connect to external knowledge. Cover user memory systems, basic RAG pipelines, and knowledge organization and retrieval beyond flat text (structured indices, knowledge graphs, etc.). - Chapter 4 · Tools: Tools are the hands of an Agent. Discuss tool classification and general design principles, the MCP protocol and challenges of tool selection, three types of tools (perception, execution, collaboration), and event-driven asynchronous Agents. - Chapter 5 · Coding Agent and Code Generation: Code is “a tool that can create new tools,” a meta-capability of general Agents. Use a production-grade Coding Agent as an example to demonstrate the complete implementation of this most powerful general tool. - Chapter 6 · Evaluation of Agents: Turn Agent performance into comparable signals. Cover evaluation environments, dataset design, metric systems, statistical significance, observability, evaluation-driven selection, and production-level internal evaluation and simulation environments. - Chapter 7 · Post-training of Models: A panoramic view of the three stages: pre-training, SFT, and RL. Discuss when to choose SFT vs. RL, RLHF, algorithm comparisons, data and environment, and cutting-edge explorations in teaching models tool usage and improving sample efficiency. - Chapter 8 · Self-Evolution of Agents: Growth without changing weights. Three learning paradigms: learning from experience, active tool discovery, and “from tool user to tool creator,” moving Agents from “smart” to “skilled.” - Chapter 9 · Multimodal and Real-time Interaction: Extend perception and action from text to speech, GUI, and the physical world. Three speech paradigms (cascaded / end-to-end full-modal / full-duplex), streaming speech perception and synthesis, Computer Use, and robotic manipulation. - Chapter 10 · Multi-Agent Collaboration: Collective intelligence can surpass individual intelligence. Multi-Agent classification framework, when it truly outperforms a single Agent, collaboration with and without shared context, failure modes, and the emergence of an “Agent society.” ## 💻 Supporting Code All projects are organized by chapter, corresponding one-to-one with the ten chapters of the book, covering a complete learning path from basic concepts to advanced techniques. The directory is chapterN/project_name/. Most experiments in Chapters 5, 8, 9, and 10 now provide independently runnable demos that have been verified against real LLM APIs. ### Project Type Description Supporting projects fall into three categories. Check the icons below for the “out-of-the-box” readiness of each project: - ✅ Runnable Standalone: The repository contains complete code; just configure the API Key (see bottom) and run. - 📖 Reproduction Guide: The project itself is a detailed reproduction document, relying on external repositories (training frameworks, evaluation benchmarks, etc.) that need to be git cloned manually. See “Obtaining External Repositories” below. - 🚧 Design Document: Currently only contains design documents for architecture and implementation plans; runnable code is still being improved. The following projects are not ✅ Runnable Standalone; please note when cloning this repository: | Project | Type | Description | | — | — | — | | chapter7/AdaptThink · AWorld-train · MiniMind-pretrain · retool · SpatialReasoning | 📖 Reproduction Guide | Training-type experiments, dependent on external frameworks; follow the README to reproduce | | All benchmarks in Chapter 6 · Most training frameworks in Chapter 7 · browser-use/claude-quickstarts in Chapter 9 · use-computer-while-calling in Chapter 10 | 📖 Reproduction Guide | Dependent on external repositories; see “Obtaining External Repositories” | ### Obtaining External Repositories (Brief) Some experiments in Chapters 6, 7, 9, and 10 depend on external repositories (evaluation benchmarks, training frameworks, robot platforms, etc.) that are not included in this repository due to size or licensing. To avoid information overload upfront, complete clone commands, upstream addresses, and the commits verified in this book are provided in the appendix at the bottom (Appendix · Obtaining External Repositories). It is recommended to start with the independently runnable projects from the earlier chapters, and then follow the guidance at the bottom to quickly obtain the external repositories when you need to reproduce training / evaluation / robot experiments. ## 🚀 Chapter 1 · Agent Fundamentals ### learning-from-experience - RL vs LLM Comparison chapter1/learning-from-experience/ Compares traditional reinforcement learning (Q-learning) with LLM-based in-context learning, reproducing key insights from Shunyu Yao’s “The Second Half” blog post. Demonstrates how LLMs can achieve 250-400x sample efficiency over traditional RL through a treasure hunt game. Core Concepts: Reinforcement Learning, In-context Learning, Sample Efficiency, Prior Knowledge ### web-search-agent - Kimi K2 Model as Agent chapter1/web-search-agent/ Implements an Agent with basic deep search capabilities, capable of multi-round search and information integration. Core Concepts: Web Search, Model-Native Agent ### search-codegen - GPT-5 Native Tool Integration chapter1/search-codegen/ Builds an Agent with basic deep search and code sandbox capabilities, utilizing tools such as web search and code execution for complex analysis. Core Concepts: Web Search, Code Generation, Model-Native Agent ### context - Context Ablation Study chapter1/context/ Demonstrates the importance of various Agent context components through systematic ablation experiments. Supports multiple LLM providers (SiliconFlow Qwen, ByteDance Doubao, Moonshot Kimi), allowing configuration of different context patterns to observe changes in Agent behavior. Core Concepts: Context Management, Tool Calling, ReAct Loop, Ablation Study ## 🎯 Chapter 2 · Context Engineering ### local_llm_serving - Local LLM Deployment and Tool Calling chapter2/local_llm_serving/ Cross-platform local LLM deployment solution, automatically selecting the best backend (vLLM or Ollama). Shows that even a small 0.6B model can achieve excellent tool calling capabilities through good system design. Supports streaming responses, displaying the thought process in real-time. Core Concepts: Model Deployment, Chat Template, Streaming, Tool Calling ### attention_visualization - Attention Mechanism Visualization chapter2/attention_visualization/ Visualizes the complete input-output token sequence and attention weight distribution of LLMs, deeply understanding how models process context, reason, and call tools. Core Concepts: Attention Mechanism, Token Analysis, Reasoning Process Visualization ### kv-cache - KV Cache-Friendly Context Design chapter2/kv-cache/ Explores the impact of different context management patterns on KV Cache, demonstrating how common error patterns can destroy cache efficiency. Experiments show how correct context design can significantly reduce latency and cost. Core Concepts: KV Cache, Context Optimization, Performance Tuning ### context-compression - Context Compression Strategies chapter2/context-compression/ Implements and compares multiple context compression strategies, including summarization, key information extraction, and semantic compression. Reduces token usage while maintaining Agent capabilities. Core Concepts: Context Compression, Token Optimization, Information Density ### prompt-engineering - Prompt Engineering Ablation Study chapter2/prompt-engineering/ Extends the Tau-Bench framework to quantify the impact of different prompt engineering factors on Agent performance through systematic ablation experiments. Shows how factors like tone style, instruction organization, and tool description affect task completion rates. Core Concepts: Prompt Engineering, Ablation Study, Performance Benchmarking ### system-hint - System Prompt Optimization chapter2/system-hint/ Studies the impact of system hints on Agent behavior, exploring how to improve performance by optimizing system prompts. Core Concepts: System Prompt, Behavioral Guidance, Prompt Optimization ### log-sanitization - Log Sanitization chapter2/log-sanitization/ Implements an intelligent log sanitization system that protects sensitive data while preserving debugging information. Core Concepts: Privacy Protection, Log Processing, Data Security ### prompt-injection - Prompt Injection Attack/Defense Experiments chapter2/prompt-injection/ Constructs controlled experiments with 3 attack scenarios (direct injection, indirect injection, memory injection) × 4 defense configurations (no defense, prompt reinforcement, source marking, combined defense). Uses deterministic rules to count attack success rates, visually demonstrating how injection success rates significantly decrease with layered defenses. Core Concepts: Prompt Injection, Indirect Injection, Data-Instruction Separation, Runtime Validation ### agent-skills-ppt - Agent Skills Gradual Disclosure for PPT Generation chapter2/agent-skills-ppt/ Reproduces the “gradual disclosure” concept of Agent Skills: The Agent sees only a thin Skill directory at startup. After identifying that the task requires the pptx Skill, it loads the full process, detailed documentation, and bundled scripts layer by layer, finally generating a real .pptx file using python-pptx. Core Concepts: Agent Skills, Gradual Disclosure, On-Demand Loading, Tool Orchestration ## 📚 Chapter 3 · User Memory and Knowledge Base ### user-memory - User Memory System chapter3/user-memory/ Builds a long-term user memory system, enabling the Agent to remember user preferences and historical interactions for personalized service. Core Concepts: Long-Term Memory, Personalization, User Modeling ### mem0 / memobase - Open-Source Memory Framework Comparison chapter3/mem0/ and chapter3/memobase/ Implements a user memory version using two open-source memory frameworks, mem0 and Memobase, as a side-by-side comparison for Experiment 3-2 “Memory Strategy Comparison,” facilitating horizontal comparison of extraction forms and answer quality across different memory solutions. Core Concepts: Memory Framework, mem0, Memobase, Solution Comparison ### user-memory-evaluation - User Memory Evaluation Framework chapter3/user-memory-evaluation/ Systematically evaluates the accuracy, relevance, and effectiveness of user memory systems, including multiple test scenarios and evaluation metrics. Core Concepts: Evaluation Framework, Test Cases, Performance Metrics ### dense-embedding - Dense Embedding Vector Retrieval Service chapter3/dense-embedding/ Builds a vector similarity search service, comparing two approximate nearest neighbor indexing algorithms: ANNOY (tree-based) and HNSW (graph-based). Shows the trade-offs between different indexing strategies in terms of performance, memory usage, and update capability. Core Concepts: Dense Embedding, Vector Retrieval, ANN Algorithm, Semantic Search ### sparse-embedding - Sparse Search Engine chapter3/sparse-embedding/ Implements a sparse vector search engine based on the BM25 algorithm from scratch. Provides rich logging and visualization interfaces to show the internal workings of the search engine, understanding term frequency weight calculation and inverted index principles. Core Concepts: Sparse Embedding, BM25, TF-IDF, Exact Match ### retrieval-pipeline - Hybrid Retrieval Pipeline chapter3/retrieval-pipeline/ Builds a complete retrieval pipeline combining dense retrieval, sparse retrieval, and neural reranking. Systematically demonstrates the complementary advantages of hybrid retrieval in different scenarios through carefully designed test cases. Core Concepts: Hybrid Retrieval, Neural Reranking, Cross-Encoder, Retrieval Fusion ### multimodal-agent - Multimodal Information Extraction chapter3/multimodal-agent/ Compares three multimodal processing strategies: native multimodal processing, extraction to text, and tool-based analysis. Through ablation studies in a unified framework, reveals the trade-offs in fidelity, cost, and flexibility among different technical paths. Core Concepts: Multimodal, Visual Understanding, OCR, End-to-End Processing ### structured-index - Structured Index chapter3/structured-index/ Implements and compares two advanced indexing strategies: RAPTOR (recursive abstraction tree) and GraphRAG (knowledge graph). Demonstrates how to build structured indexes that reflect the inherent hierarchy and relationships of knowledge using an index of a technical manual. Core Concepts: RAPTOR, GraphRAG, Hierarchical Summarization, Knowledge Graph ### agentic-rag - Agentic RAG chapter3/agentic-rag/ Compares the performance differences between traditional Non-Agentic RAG and Agentic RAG. Shows how Agents use the ReAct pattern to lead iterative information retrieval, significantly improving answer quality when handling complex legal questions. Core Concepts: Agentic RAG, ReAct Loop, Iterative Retrieval, Active Exploration ### agentic-rag-for-user-memory - Building User Memory with Agentic RAG chapter3/agentic-rag-for-user-memory/ Applies the Agentic RAG framework to manage user conversation history. Uses multi-turn iterative search capabilities to handle cross-session memory retrieval, achieving basic recall and multi-session retrieval capabilities. Core Concepts: User Memory, Conversation History Indexing, Cross-Session Retrieval ### contextual-retrieval - Context-Aware Retrieval chapter3/contextual-retrieval/ Implements the context-aware retrieval technique proposed by Anthropic. By generating prefix summaries containing core context for each text chunk, it addresses the context loss problem of traditional chunking methods, reducing retrieval failure rates by 49-67%. Core Concepts: Context Enhancement, Prefix Generation, Semantic Anchoring, Retrieval Optimization ### contextual-retrieval-for-user-memory - Context-Aware User Memory System chapter3/contextual-retrieval-for-user-memory/ Applies context-aware retrieval technology to user memory construction. Combines Advanced JSON Cards with context-aware RAG to form a two-layer memory structure, enabling higher-level proactive service capabilities. Core Concepts: Two-Layer Memory, Structured Facts, Contextual Retrieval, Proactive Service ### structured-knowledge-extraction - Structured Knowledge Extraction chapter3/structured-knowledge-extraction/ Using legal cases as an example, runs a three-stage pipeline: “bottom-up factor discovery → case prototype clustering → conversational advisory Agent.” Does not predefine rigid schemas; instead, uses LLM to autonomously discover factors from a large number of cases and summarize them into a modular schema (core factors + charge-specific expansion factors). Then clusters cases into several prototypes, calculates factor importance for each prototype. The Agent matches new case descriptions to the most similar prototype, asks for missing information based on importance, and provides evidence-based advice (with a legal disclaimer). Core Concepts: Bottom-Up Knowledge Discovery, Modular Factors, Cluster Prototypes, Explainable Decision Making ## 🛠️ Chapter 4 · Tools ### perception-tools - Perception Tools MCP Server chapter4/perception-tools/ Builds a comprehensive set of perception tools, providing capabilities for web search, multimodal understanding, file system operations, and public data source access. Most features are based on free open APIs (DuckDuckGo, Open-Meteo, Yahoo Finance, OpenStreetMap, etc.) and do not require API keys. Core Concepts: MCP Protocol, Multimodal Parsing, Public Data Sources, Document Understanding, Geographic Information Services ### execution-tools - Execution Tools MCP Server chapter4/execution-tools/ Implements a set of execution tools with safety mechanisms, including file operations, code interpreter, virtual terminal, and external system integration. Uses an LLM secondary approval mechanism to prevent dangerous operations, automatically summarizes complex outputs, and performs syntax validation on code. Core Concepts: MCP Protocol, Execution Safety, LLM Approval, Result Summarization, Automatic Validation ### collaboration-tools - Collaboration Tools MCP Server chapter4/collaboration-tools/ Provides comprehensive collaboration capabilities, including browser automation (browser-use framework), Human-in-the-Loop (HITL), multi-channel notifications (Email, Telegram, Slack, Discord), and timer management. Supports admin approval for sensitive operations and scheduled task scheduling. Core Concepts: MCP Protocol, Browser Automation, HITL Mode, Multi-Channel Notifications, Scheduled Tasks ### agent-with-event-trigger - Event-Driven Agent with MCP Integration chapter4/agent-with-event-trigger/ A modern event-driven Agent built on FastAPI, by default integrating all tools from the first three MCP servers. Uses a native asynchronous architecture for clean MCP tool loading, receives multi-source events (Web, instant messaging, GitHub, timers, etc.) via HTTP API. Provides automatic API documentation (Swagger UI) and background monitoring capabilities. Core Concepts: FastAPI, Native Async, MCP Integration, Event-Driven, Automatic API Documentation, Tool Orchestration ### active-tool-selection - Active Tool Selection chapter4/active-tool-selection/ Implements an intelligent tool selection mechanism, allowing the Agent to actively choose the most suitable tool combination based on task requirements, rather than passively accepting a predefined tool set. Core Concepts: Tool Selection, Dynamic Tool Loading, Task Analysis ### async-agent - Asynchronous Agent with Parallel Execution and Interruption chapter4/async-agent/ Implements the core of an event-driven asynchronous Agent framework (Flux) based on single-threaded asyncio: an inbox event queue dispatches by urgency (interrupt/immediate/queue), supports parallel execution of async tools, interrupts within the current turn, and cancellation and status query for simulated long tasks. Decision-making is done by a real LLM (function calling). Core Concepts: Asynchronous Programming, Event Queue, Interrupt Mechanism, Parallel Tool Cancellation, Non-blocking I/O > Additionally, chapter4/docker-compose.yml and chapter4/DOCKER_DEPLOYMENT.md provide a reference solution for containerized deployment of the above MCP tool servers. ## 💻 Chapter 5 · Coding Agent and Code Generation ### coding-agent - Production-Grade Coding Agent chapter5/coding-agent/ A production-grade AI coding assistant built on Claude, using pure Python for all tools without command-line dependencies. Includes 17 fully implemented tools covering file operations, search, shell operations, and project management. Implements a pure Python Grep tool fully compatible with ripgrep functionality. Key Features: - Pure Python implementation, no command-line dependencies, especially suitable for Mac users - Complete tool suite: file read/write/edit, pure Python regex search, directory listing, shell session management - System prompt techniques: timestamps, tool call counters, TODO list management, detailed error messages - Persistent shell environment, automatic lint detection, streaming response support - Supports multiple LLM providers (Anthropic, OpenAI, OpenRouter) Core Concepts: Code Generation, File Editing, Pure Python Tools, System Prompts, Lint Detection, Multi-Provider Support ### code-for-math - Using Code to Enhance Math Problem Solving chapter5/code-for-math/ Compares “pure chain-of-thought” and “code-assisted” modes on the same set of competition math problems using the same model. In the code-assisted mode, the problem is formalized into Python (sympy/numpy/scipy) and executed via function calling in a subprocess sandbox, using precise computation to replace error-prone mental arithmetic, resulting in significantly higher accuracy. Core Concepts: Code Interpreter, Symbolic Computation, Chain-of-Thought Comparison, Tool-Augmented Reasoning ### code-for-logic - Using Code to Enhance Logical Reasoning chapter5/code-for-logic/ Transforms “Knights and Knaves” logic puzzles into Constraint Satisfaction Problems (CSP). The Agent uses python-constraint to define variables and biconditional constraints and calls the solver. Compares the correctness rates of pure natural language reasoning and code-assisted modes on a set of K&K puzzles. Core Concepts: Constraint Solving, CSP Modeling, Formal Reasoning, Code Assistance ### small-model-codified-rules - Codified Rules for Small Models chapter5/small-model-codified-rules/ A controlled experiment based on the τ-bench airline customer service scenario: after moving complex business policies (refund rules) from natural language prompts into code/tools, the task success rate and policy compliance of a small model improved significantly. Code-level validation within tools can intercept incorrect model reasoning in real-time. Core Concepts: Codified Business Rules, Policy Execution, In-Tool Validation, Small Model Reliability ### paper-to-ppt - Automatic Slide Generation from Papers (Proposer-Reviewer) chapter5/paper-to-ppt/ Reframes “making PPTs” as a code generation problem: the Proposer writes Slidev (Markdown+HTML) code, the Reviewer renders each page into a real PNG and uses a Vision LLM to check layout issues, iterating based on structured feedback. The dual-Agent division of labor results in significantly lower peak context size. Core Concepts: Code Generation, Slidev, Proposer-Reviewer, Visual Quality Control ### paper-to-video - Automatic Video Generation from Paper Presentations chapter5/paper-to-video/ Building on “Paper → PPT,” generates colloquial narration scripts for each slide, uses TTS to synthesize speech, and uses ffmpeg to synchronize each page screenshot with its audio into a narrated video segment. Core Concepts: Multimedia Generation, Narration Script Generation, TTS, ffmpeg Audio-Video Sync ### video-edit - API-based Intelligent Video Editing chapter5/video-edit/ Given a multi-scene video and a natural language request, the Agent uses a “two-step Vision localization” (first coarse then fine frame reading) to determine the temporal boundaries of the target scene, clips the segment, and has a Reviewer extract key frames from the result to check for quality, iterating if unsatisfactory. Core Concepts: Video Editing, Vision Localization, Coarse-to-Fine, Proposer-Reviewer ### adaptive-log-parser - Adaptive Log Parsing System chapter5/adaptive-log-parser/ A self-evolving log parsing system: when encountering an unparseable new format, instead of reporting an error, it sends the failed sample and error to a code generation Agent to generate a parse function. After automatic testing passes, the function is hot-updated and registered into the parsing engine, requiring no human intervention throughout the process. Core Concepts: Code as System Adapter, Self-Healing Loop, Code Hot Update, Automatic Testing ### log-diagnosis - Production Log Intelligent Diagnosis System chapter5/log-diagnosis/ A diagnosis Agent reads production trace logs, architecture documents, and PRDs, automatically identifies problems and root causes, generates structured reports and regression test cases, executes validation using a replay framework, and (mocked) creates GitHub Issues via MCP integration. Core Concepts: Trace Diagnosis, Root Cause Localization, Regression Test Generation, Replay Validation ### dynamic-form - Dynamic Form Intent Clarification System chapter5/dynamic-form/ When faced with an incomplete request, the Agent does not ask follow-up questions one by one. Instead, it dynamically generates a self-contained HTML form with cascading logic for the user to fill in all at once. The frontend aggregates the form into JSON and returns it to the Agent to continue the task. Core Concepts: Code Generation, Intent Clarification, Dynamic Forms, Cascading Logic ### erp-agent - Natural Language ERP Agent (NL → SQL) chapter5/erp-agent/ Converts Chinese natural language queries into SQL for database execution and directly presents the result table. The core is an artifact pattern: the LLM only generates the SQL artifact without handling the data itself, saving tokens and avoiding manual calculation errors, with instant return even for tens of thousands of rows. Core Concepts: NL2SQL, Artifact Pattern, Database Execution, Cost and Accuracy ### conversational-ui - Conversational UI Customization System chapter5/conversational-ui/ Users make UI customization requests (colors/fonts/copy/layout) in natural language. The Agent autonomously locates and modifies React frontend source code, using Vite Hot Module Replacement (HMR) to make changes take effect instantly, supporting multi-turn iterative customization. Core Concepts: Code Modification, Frontend Customization, Hot Module Replacement, Multi-Turn Iteration ## 🎯 Chapter 6 · Evaluation of Agents ### terminal-bench - Terminal Environment Benchmark chapter6/terminal-bench/ Terminal-Bench is a benchmark for testing AI Agent performance in a real terminal environment. From compiling code to training models and setting up servers, it evaluates how Agents handle real end-to-end tasks. Contains a dataset of approximately 100 tasks and an execution framework, supporting multiple Agent implementations. Core Concepts: Terminal Automation, Task Evaluation, Docker Sandbox, Benchmarking ### SWE-bench - Software Engineering Benchmark chapter6/SWE-bench/ SWE-bench is a benchmark for evaluating the ability of large language models to solve real GitHub issues. Given a codebase and issue description, the model needs to generate a patch that solves the problem. Includes multiple versions: SWE-bench, SWE-bench Lite, SWE-bench Verified, and SWE-bench Multimodal. Core Concepts: Code Fixing, GitHub Issues, Patch Generation, Docker Evaluation ### GAIA - General AI Assistant Benchmark chapter6/GAIA/ GAIA aims to evaluate next-generation LLMs (those with tool augmentation, efficient prompting, search access, etc.). Contains 450+ non-trivial questions requiring varying degrees of tools and autonomy, with unambiguous answers. Divided into 3 difficulty levels. Core Concepts: Tool Use, Multi-Step Reasoning, Autonomy Evaluation ### OSWorld - OS-Level Agent Benchmark chapter6/OSWorld/ Evaluates an Agent’s ability to perform complex tasks in a complete operating system environment, including file management, application operation, and system configuration. Core Concepts: OS Automation, Multi-Application Collaboration, System-Level Tasks ### android_world - Android Environment Benchmark chapter6/android_world/ (📖 External Repository, see “Obtaining External Repositories”) Evaluates Agent performance in the Android mobile environment, including app navigation, UI interaction, and task completion. Core Concepts: Mobile Automation, Android UI, App Interaction > chapter6/android-world/ (hyphenated name) is not benchmark code, but rather the book’s analysis notes (t3a*.md) on T3A Agent failure cases on android_world, provided as reference reading material. ### tau2-bench - Tool-Augmented Reasoning Benchmark chapter6/tau2-bench/ Focuses on evaluating Agents’ ability to use tools for complex reasoning, including scenarios such as computation, search, and data processing. Core Concepts: Tool-Augmented Reasoning, Multi-Step Tasks, Tool Composition ### elo-leaderboard - ELO Leaderboard System chapter6/elo-leaderboard/ Implements an Agent performance leaderboard based on the ELO rating system, evaluating the relative abilities of different Agents through head-to-head comparisons. Core Concepts: ELO Rating, Relative Evaluation, Leaderboard System ### model-benchmark - Multi-Dimensional Model Performance Benchmark chapter6/model-benchmark/ Conducts horizontal benchmarking of multiple OpenAI-compatible LLM API providers. Uses a streaming interface to precisely measure time-to-first-token (TTFT), measures end-to-end latency percentiles (p50/p95), throughput, and success rate under concurrency. Generates a multi-dimensional comparison table with a single command, illustrating that model selection is a multi-faceted trade-off, not just looking at leaderboards. Core Concepts: TTFT, Latency Percentiles, Throughput, Concurrency Stress Test, Model Selection ### agent-cost-analysis - End-to-End Cost Analysis of Agent Tasks chapter6/agent-cost-analysis/ Performs full-link cost breakdown of a typical multi-turn Agent task (customer service refund). Uses a custom lightweight tracing system to record input/output/cache token counts, latency, and cost for each LLM call, aggregating to identify “which step is most expensive.” Then uses A/B comparison to quantify real savings from KV-cache-friendly design and context compression. Core Concepts: Observability, Cost Breakdown, Prompt Caching, A/B Comparison ### tts-quality-eval - Fully Automated TTS Quality Evaluation Pipeline chapter6/tts-quality-eval/ Synthesizes the same set of challenging texts using various TTS configurations (different model/voice/speed). Then uses a multimodal LLM-as-a-Judge to score each dimension (clarity, naturalness, etc.) according to a Rubric, aggregating the scores into a reproducible configuration comparison table. Core Concepts: LLM-as-a-Judge, Rubric Scoring, TTS Evaluation, Multi-Dimensional Comparison ## 🧠 Chapter 7 · Model Post-Training This chapter contains multiple model post-training projects, covering various techniques and application scenarios for Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). ### AdaptThink - Adaptive Reasoning Depth chapter7/AdaptThink/ and chapter7/AdaptThink-original/ Enables reasoning models to adaptively choose between reasoning modes (Thinking vs. NoThinking) based on problem difficulty. Uses constrained optimization and importance sampling to significantly reduce reasoning costs (45-69%) while improving accuracy. Based on DeepSeek-R1-Distill-Qwen models, trained using the DAPO algorithm. Core Concepts: Adaptive Reasoning, Reasoning Cost Optimization, Constrained Optimization, Importance Sampling ### retool - Tool-Augmented Mathematical Reasoning chapter7/retool/ Uses multi-turn dialogue and code sandboxes to enhance the mathematical reasoning ability of large language models. Trains the model in two stages (SFT and RL) to learn how to use a code execution environment to assist with mathematical problem-solving. Based on Qwen2.5-32B-Instruct, trained on the AIME 2024 dataset, using the DAPO algorithm and SandboxFusion sandbox. Core Concepts: Tool Use, Code Execution, Mathematical Reasoning, Multi-Turn Dialogue, DAPO Algorithm ### AWorld / AWorld-train - Embodied Agent Training chapter7/AWorld/ and chapter7/AWorld-train/ Trains embodied Agents based on the AWorld framework, enabling Agents to perform complex tasks in virtual environments and learn from experience. Core Concepts: Embodied Intelligence, Environment Interaction, Experience Learning ### SFTvsRL - SFT vs RL Comparison Study chapter7/SFTvsRL/ Systematically compares the effects of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) on different tasks, analyzing the strengths, weaknesses, and suitable scenarios of both methods. Core Concepts: SFT vs RL, Training Method Comparison, Performance Analysis ### verl - Efficient RL Training Framework chapter7/verl/ verl is an efficient reinforcement learning framework specifically designed for LLM RLHF training, supporting multiple algorithms like PPO, GRPO, and DAPO. Core Concepts: RLHF, PPO, Distributed Training, Efficient Optimization ### Intuitor - Intuitive Reasoning Training chapter7/Intuitor/ Trains the model’s intuitive reasoning ability, enabling it to make quick, reasonable judgments without detailed chain-of-thought. Core Concepts: Intuitive Reasoning, Quick Decision Making, Chain-of-Thought Optimization ### MultilingualReasoning - Multilingual Reasoning chapter7/MultilingualReasoning/ Trains the model’s reasoning ability in multiple languages, improving performance on cross-lingual tasks. Core Concepts: Multilingual, Cross-Lingual Reasoning, Language Generalization ### SpatialReasoning - Spatial Reasoning Training chapter7/SpatialReasoning/ Focuses on training the model’s spatial reasoning ability to handle problems involving spatial relationships such as position, direction, and distance. Core Concepts: Spatial Reasoning, Geometric Understanding, Positional Relationships ### SimpleVLA-RL - Vision-Language-Action RL chapter7/SimpleVLA-RL/ Combines reinforcement learning with vision, language, and action, enabling the model to understand visual input and perform corresponding actions. Core Concepts: Vision-Language-Action, Multimodal RL, Embodied Intelligence ### continued-pretraining - Continued Pretraining chapter7/continued-pretraining/ Performs continued pretraining on domain-specific data to improve the model’s performance in the target domain. Core Concepts: Continued Pretraining, Domain Adaptation, Knowledge Injection ### MiniMind-pretrain - Small Model Pretraining chapter7/MiniMind-pretrain/ Pretrains a small language model from scratch, understanding the complete pretraining process and key technologies. Core Concepts: Pretraining, Small Model, Training Pipeline ### sesame - Sequence Modeling and Evaluation chapter7/sesame/ Focuses on training and evaluation methods for sequence modeling tasks. Core Concepts: Sequence Modeling, Evaluation Methods, Performance Optimization ### orpheus - Music Generation and Understanding chapter7/orpheus/ Trains the model’s music generation and understanding capabilities. Core Concepts: Music Generation, Audio Understanding, Creative AI ### tinker-cookbook - Collection of Training Tips chapter7/tinker-cookbook/ Collects practical tips and best practices for various model training scenarios. Core Concepts: Training Tips, Best Practices, Optimization Methods ## 🔄 Chapter 8 · Self-Evolution of Agents This chapter focuses on enabling Agents to continuously grow from experience without modifying weights: distilling successful trajectories into reusable experiences, externalizing repetitive operations into tools, and distilling prompts and observations into the model. ### gaia-experience - Learning from Successful Experience chapter8/gaia-experience/ Based on the AWorld framework and the GAIA benchmark,