slow → fast computers

### TL;DR Performance engineers need a deep understanding of metrics such as latency, throughput, and resource utilization, and must be familiar with the latency numbers of common operations (e.g., CPU instructions, disk reads, network access) — because our intuitive sense of "fast" and "slow" is often off by several orders of magnitude from reality. --- ## Chapter 2 of *Systems Performance*: Overview of Key Concepts In the live stream we continued reading *Systems Performance* (about 800 pages, dense but excellent). Chapter 2 provides a high-level preview of the entire book, covering all the critical metrics involved in performance investigations: latency, throughput, IOPS, resource utilization, saturation, and how cache hierarchies affect performance. This chapter lays the foundation for the deeper discussions in subsequent chapters. ## Performance Metrics: Latency vs Throughput vs Resource Utilization vs Saturation When measuring system performance, you need to choose the right metrics. The core questions include: - **Latency**: The time a single operation takes (e.g., request response time). - **Throughput**: The number of operations completed per unit time (e.g., requests per second). - **IOPS**: Input/output operations per second (commonly used for storage systems). - **Resource utilization**: The percentage of a resource (CPU, memory, disk) being used. - **Saturation**: The degree to which a resource cannot keep up with demand (often leading to queue buildup). The book emphasizes that these metrics need to be used together; looking at just one can be misleading. For example, high utilization is not necessarily a problem, but if combined with high saturation, it indicates a need for scaling. ## Cache Hierarchy: CPU Cache, RAM, Disk One of the core ideas in performance optimization is "caching is everywhere." The cache hierarchy summarized in the book includes: - **CPU caches (L1/L2/L3)**: caches for physical RAM. - **RAM**: caches for disk (SSD/HDD). - **Disk caches**: e.g., DRAM cache inside an SSD. - **Network caches**: CDN, DNS caches, etc. Understanding the latency differences between these levels is critical. The book references a classic table (often called "Latency Numbers Every Engineer Should Know") that lists typical times from CPU instructions to cross-continent network transfers. For example: - L1 cache reference: ~0.5 ns - Branch mispredict: ~5 ns - L2 cache reference: ~7 ns - Main memory reference: ~100 ns - SSD random read: ~150,000 ns - Network round trip across the US: ~150,000,000 ns These numbers can differ by orders of magnitude. Memorizing them helps you quickly identify performance bottlenecks. > Visualizing these numbers (e.g., with balls of different sizes) is an effective memory technique. It might seem silly, but it helps build an intuitive sense of latency scale in your mind. ## Latency Number Resources: Jeff Dean and Simon Eskilden In addition to the table in the book, there are two famous resources: - **Jeff Dean's** (Google Fellow) classic article "Latency Numbers Every Engineer Should Know." - **Simon Eskilden's** (TurboPuffer) "Napkin Math" repository, which contains numbers based on actual benchmarks rather than just approximations. These resources are extremely valuable for any engineer, especially those working on databases, distributed systems, or backend engineering. When you know the expected performance of common operations, you can immediately notice if actual performance deviates and start investigating. ## Learning Advice: Hands-On Practice is King For the question "How to get into system design/performance engineering?", the most core advice is: **nothing can replace hands-on practice**. Joining a team that deals with real-world challenges and learning from experienced people is the fastest way to grow. Books and papers are excellent supplements, but you can't just read them. For example: - *Designing Data-Intensive Applications* (DDIA) - *Database Internals* - And *Systems Performance*, which we're reading in this series These resources serve as references when you encounter problems at work, helping you bridge theory and practice. ## High-Frequency Trading: The Ultimate in Performance Sensitivity When discussing low-latency performance, the book mentions that high-frequency trading (HFT) is one of the most performance-sensitive domains in the world. Every millisecond in a trade completion can affect profit or loss. Thus, HFT imposes nearly extreme requirements on latency at every layer: database, network, CPU scheduling, etc. Whether or not you work in this industry, understanding the extreme optimization approaches of HFT helps you grasp the limits of performance engineering. ## Live Stream Series and Resources This live stream series will be recorded and uploaded to a YouTube playlist for easy review. If you like this content, please subscribe and turn on notifications. The next live stream will continue with the remaining part of Chapter 2. Feel free to join and discuss. --- **Source**: YouTube Video: slow → fast computers (https://www.youtube.com/watch?v=5Zb8WwKd-E8)