https://www.youtube.com/watch?v=aE0onltJlOo

TL;DR: This lecture introduces GPU architecture as a flexible evolution of SIMD (vector/array) processors, highlighting how GPUs exploit data parallelism while facing memory bandwidth and serial bottleneck challenges. ## Lecture 20: GPU Architecture Overview We are entering the topic of memory systems, but this lecture first discusses GPU architecture — an extension of the SIMD processors (array and vector processors) covered in Muhammad’s previous lecture. GPUs are essentially more flexible array and vector processors, particularly offering thread mobility in the programming model. ### Core Concepts Review - **Data Parallelism**: Exploiting regular data parallelism, i.e., performing the same operation on a large number of data elements. - **Four Architecture Classifications**: Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD, e.g., systolic arrays), Multiple Instruction Multiple Data (MIMD). GPUs combine these paradigms in a good way. - **Space-Time Duality**: - **Array Processor**: Executes the same instruction at the same time, with different operations in different spaces (functional units). - **Vector Processor**: Executes different operations at different times in the same space (functional unit). - Modern SIMD processing (including GPUs) exploits parallelism in both time and space simultaneously. ### Memory Bank Grouping & Bank Conflicts - **Purpose of Grouping**: Serve multiple accesses simultaneously across different memory banks to tolerate the latency of a single access. For example, in Cray processors, memory access latency is 11 cycles, so at least 11 banks are needed to fetch one word per cycle. - **Bank Conflict**: When two accesses map to the same memory bank, data is delayed, causing the processor to stall. Avoiding such conflicts is an important mathematical problem. Current GPUs use many techniques developed by Bob Rau and others 35 years ago. ### From Vector Processing to GPU Vector instruction example: `vector C = vector A + vector B`, each element operates on a functional unit. Actual hardware typically splits the vector length into multiple parallel lanes (e.g., 4 elements execute in parallel on 4 functional units, subsequent elements are pipelined). Partitioned vector registers: Different elements of an array can only be accessed from a specific functional space, which forces threads to be mapped to particular processing lanes — GPUs have a similar issue. ### Potential and Limits of Vectorization - **Fully Vectorizable Loops**: No dependencies between loop iterations, the compiler can automatically vectorize. For example, applying the same operation to every pixel in an image is highly suitable for SIMD. - **Performance Improvement Limited by Vectorizability**: Amdahl’s Law reminds us that serial bottlenecks are non-negligible. Cray machines had both powerful SIMD and scalar units; GPUs also try to execute scalar code efficiently, sometimes adding CPU-like units internally or using heterogeneous computing with a CPU. ### Serial Bottleneck The serial bottleneck refers to the single-threaded part that cannot be parallelized. Even with unlimited parallel units, this portion of code cannot be accelerated. GPUs are similarly limited. ### History of SIMD Instructions: The MMX Story The x86 ISA, as a sequential von Neumann ISA, added SIMD extensions (starting with MMX in the 1990s) to enable parallel operations. A key example is matrix-vector multiplication — Intel’s MMX paper from 1990. Today (2026), matrix multiplication is again a hot topic for other reasons. The introduction of MMX faced strong internal opposition at Intel: “No workloads use it.” But the designers, Uri Weiser and Alex Peleg, bet on the future and eventually succeeded, making the CPU a accelerator for graphics and many operations for years, even delaying the success of GPUs. This illustrates the uncertainty in architects’ decisions — you never know whether future workloads will use the provided instruction extensions. (The transcription ends here, but the original text notes that these extensions are not as general as a huge vector processor.) --- Source: YouTube video link (https://www.youtube.com/watch?v=aE0onltJlOo)