if u want to study llms end to end be it from each componets in llm from the vaswani till now incliding quantization stuff,pre norm vs post norm,rl for llms,mech interp,safety,etc etc

https://t.co/vqBm1DQvwG

Language AI Handbook

Source: https://mbrenndoerfer.com/books/language-ai-handbook

About This Book

Language AI has transformed from an academic curiosity into the defining technology of our era. But beneath the hype of ChatGPT and Claude lies a rich technical landscape that most practitioners only partially understand. This handbook gives you the complete picture, from classical NLP techniques that still matter to the cutting-edge architectures powering today’s most capable systems.

Begin with the fundamentals that never go out of style: tokenization, embeddings, and the statistical foundations that inform modern approaches. Then dive deep into the transformer architecture. Learn not just how to use it, but how it actually works. Understand self-attention mathematically, grasp why positional encodings matter, and see how architectural choices like layer normalization affect training dynamics.

Track Your Progress

Sign in to mark chapters as complete, track quiz scores, and see your reading journey

Sign in →

Table of Contents

Part I: Text as Data

5chapters1 #### Character Encoding Covers ASCII origins and 7-bit limitations, Unicode code points and planes, UTF-8 variable-width encoding scheme, byte order marks and endianness, encoding detection heuristics, common encoding errors and mojibake, practical encoding/decoding in Python.2 #### Text Normalization Covers Unicode normalization forms (NFC, NFD, NFKC, NFKD), case folding vs lowercasing, accent and diacritic handling, whitespace normalization, ligature expansion, full-width to half-width conversion, implementing a normalization pipeline.3 #### Regular Expressions Covers regex syntax and metacharacters, character classes and quantifiers, grouping and backreferences, lookahead and lookbehind assertions, greedy vs lazy matching, common NLP patterns (emails, URLs, dates), regex performance considerations.4 #### Sentence Segmentation Covers period disambiguation challenges, abbreviation handling, rule-based boundary detection, Punkt sentence tokenizer algorithm, evaluation metrics for segmentation, handling edge cases (quotes, parentheses, lists), multilingual segmentation issues.5 #### Word Tokenization Covers whitespace tokenization limitations, punctuation handling rules, contractions and clitics, language-specific challenges (Chinese, Japanese, German compounds), Penn Treebank tokenization standard, building a rule-based tokenizer, tokenization evaluation.

Part II: Classical Text Representations

9chapters6 #### Bag of Words Covers document-term matrix construction, vocabulary building from corpus, word counting and frequency vectors, sparse matrix representation (CSR/CSC formats), vocabulary pruning (min_df, max_df), binary vs count representations, limitations of word order loss.7 #### N-grams Covers bigram and trigram extraction, n-gram vocabulary explosion, n-gram frequency distributions, Zipf’s law in n-grams, character n-grams for robustness, skip-grams and flexible windows, n-gram indexing for search.8 #### N-gram Language Models Covers Markov assumption and chain rule, maximum likelihood estimation, probability calculation for sequences, handling unseen n-grams, start and end tokens, generating text from n-gram models, model storage and lookup efficiency.9 #### Smoothing Techniques Covers add-one (Laplace) smoothing, add-k smoothing and tuning, Good-Turing smoothing derivation, Kneser-Ney smoothing intuition and formula, interpolation vs backoff, modified Kneser-Ney, comparing smoothing methods empirically.10 #### Perplexity Covers cross-entropy definition and derivation, perplexity as branching factor, relationship to bits-per-character, held-out evaluation methodology, perplexity vs downstream performance, comparing models with perplexity, perplexity limitations and caveats.11 #### Term Frequency Covers raw term frequency, log-scaled term frequency, boolean term frequency, augmented term frequency, L2-normalized frequency vectors, term frequency sparsity patterns, efficient term frequency computation.12 #### Inverse Document Frequency Covers document frequency calculation, IDF formula derivation, IDF intuition (rare words matter more), smoothed IDF variants, IDF across corpus splits, relationship to information theory, implementing IDF efficiently.13 #### TF-IDF Covers TF-IDF formula and variants, TF-IDF vector computation, TF-IDF normalization options, BM25 as TF-IDF extension, document similarity with TF-IDF, TF-IDF for feature extraction, sklearn TfidfVectorizer deep dive.14 #### BM25 Covers BM25 derivation from probabilistic IR, saturation parameter k1, length normalization parameter b, BM25+ and BM25L variants, field-weighted BM25, implementing BM25 scoring, BM25 vs TF-IDF empirically.

Part III: Distributional Semantics

4chapters15 #### The Distributional Hypothesis Covers Firth’s “you shall know a word by the company it keeps,” distributional similarity intuition, context window definitions, paradigmatic vs syntagmatic relations, word similarity from distributions, limitations of distributional semantics.16 #### Co-occurrence Matrices Covers word-word co-occurrence matrices, word-document matrices, context window size effects, weighting by distance, symmetric vs directional contexts, matrix sparsity patterns, efficient construction algorithms.17 #### Pointwise Mutual Information Covers PMI formula derivation, PMI interpretation as association, positive PMI (PPMI), shifted PPMI variants, PMI matrix properties, PMI vs raw counts comparison, PMI for collocation extraction.18 #### Singular Value Decomposition Covers SVD mathematical formulation, truncated SVD for dimensionality reduction, LSA (Latent Semantic Analysis), choosing embedding dimensions, SVD computational complexity, randomized SVD for scale, interpreting SVD dimensions.

Part IV: Word Embeddings

9chapters19 #### Skip-gram Model Covers skip-gram architecture diagram, input/output representations, softmax over vocabulary, skip-gram objective function, training data generation, window size hyperparameter, skip-gram vs CBOW intuition.20 #### CBOW Model Covers CBOW architecture, context word averaging, CBOW objective function, CBOW vs skip-gram training speed, CBOW for frequent words, implementing CBOW forward pass, CBOW gradient derivation.21 #### Negative Sampling Covers softmax computational bottleneck, negative sampling objective derivation, sampling distribution (unigram^0.75), number of negatives hyperparameter, negative sampling gradient computation, NCE vs negative sampling, implementing efficient sampling.22 #### Hierarchical Softmax Covers binary tree construction (Huffman coding), path probability computation, hierarchical softmax objective, gradient computation along paths, tree structure impact on learning, hierarchical softmax vs negative sampling, when to use each approach.23 #### Word2Vec Training Covers data preprocessing pipeline, subsampling frequent words, learning rate scheduling, minibatch vs online training, convergence monitoring, gensim Word2Vec usage, training from scratch in PyTorch.24 #### Word Analogy Covers vector arithmetic for analogies, parallelogram model, analogy evaluation datasets, 3CosAdd vs 3CosMul methods, analogy accuracy metrics, limitations of analogy evaluation, what analogies reveal about embeddings.25 #### GloVe Covers GloVe objective function derivation, weighted least squares formulation, relationship to matrix factorization, weighting function design, bias terms in GloVe, GloVe vs Word2Vec comparison, training GloVe efficiently.26 #### FastText Covers character n-gram representation, word vector as n-gram sum, FastText architecture, handling OOV words, morphological awareness, FastText for morphologically rich languages, training FastText models.27 #### Embedding Evaluation Covers intrinsic vs extrinsic evaluation, word similarity datasets (SimLex, WordSim), analogy accuracy, embedding visualization (t-SNE, UMAP), downstream task evaluation, embedding bias detection, evaluation pitfalls.

Part V: Subword Tokenization

8chapters28 #### The Vocabulary Problem Covers OOV word problem, vocabulary size explosion, rare word representation, morphological productivity, compound words, code and technical text, the case for subword units.29 #### Byte Pair Encoding Covers BPE algorithm step-by-step, merge rules learning, vocabulary size control, BPE encoding procedure, BPE decoding procedure, BPE implementation from scratch, BPE hyperparameters.30 #### WordPiece Covers WordPiece vs BPE differences, likelihood objective for merges, greedy tokenization algorithm, ## prefix notation, WordPiece in BERT, training WordPiece tokenizers, handling unknown characters.31 #### Unigram Language Model Tokenization Covers unigram LM formulation, EM algorithm for training, Viterbi decoding for tokenization, sampling multiple segmentations, subword regularization, unigram vs BPE comparison, SentencePiece unigram mode.32 #### SentencePiece Covers treating text as raw bytes, whitespace handling (▁ prefix), BPE and unigram modes, training from raw text, pretokenization elimination, SentencePiece in production, multilingual tokenization.33 #### Tokenizer Training Covers corpus preparation, vocabulary size selection, special tokens configuration, training with HuggingFace tokenizers, saving and loading tokenizers, tokenizer versioning, domain-specific tokenizers.34 #### Special Tokens Covers [CLS], [SEP], [PAD], [MASK], [UNK] tokens, beginning/end of sequence tokens, custom special tokens, special token embeddings, token type IDs, handling special tokens in generation.35 #### Tokenization Challenges Covers number tokenization issues, code tokenization, multilingual text mixing, emoji and Unicode edge cases, tokenization artifacts, adversarial tokenization, measuring tokenization quality.

Part VI: Sequence Labeling

8chapters36 #### Part-of-Speech Tagging Covers POS tag sets (Penn Treebank, Universal), POS tagging as classification, contextual disambiguation, POS tagging accuracy metrics, POS tagging for downstream tasks, rule-based vs statistical taggers.37 #### Named Entity Recognition Covers entity types (PER, ORG, LOC, etc.), NER as sequence labeling, nested entity challenges, entity boundary detection, NER evaluation (exact vs partial match), NER datasets and benchmarks.38 #### BIO Tagging Covers BIO scheme explanation, BIOES/BILOU variants, converting spans to BIO tags, BIO decoding to spans, handling tagging inconsistencies, BIO for multi-label scenarios, implementing BIO utilities.39 #### Chunking Covers noun phrase chunking, chunk types (NP, VP, PP), IOB tagging for chunks, chunking vs full parsing, chunking evaluation, chunking as preprocessing, regex chunking with NLTK.40 #### Hidden Markov Models Covers HMM components (states, observations, transitions), emission and transition probabilities, HMM assumptions (Markov, independence), HMM for POS tagging, HMM parameter estimation, HMM limitations for NLP.41 #### Viterbi Algorithm Covers optimal path problem formulation, Viterbi recursion derivation, backpointer tracking, Viterbi complexity analysis, log-space computation, implementing Viterbi efficiently, Viterbi for beam search foundation.42 #### Conditional Random Fields Covers CRF vs HMM comparison, CRF feature functions, log-linear formulation, partition function computation, CRF for NER, CRF inference complexity, neural CRF layers.43 #### CRF Training Covers CRF log-likelihood objective, forward-backward algorithm, gradient computation, L-BFGS optimization, feature template design, CRF regularization, CRF training convergence.

Part VII: Neural Network Foundations

13chapters44 #### Linear Classifiers Covers linear decision boundaries, weight vectors and bias, dot product interpretation, multiclass classification (softmax), linear classifier limitations, training with gradient descent.45 #### Activation Functions Covers sigmoid function and saturation, tanh properties, ReLU and dying ReLU, Leaky ReLU and PReLU, ELU and SELU, GELU derivation and properties, Swish and Mish, choosing activation functions.46 #### Multilayer Perceptrons Covers hidden layers and depth, weight matrices between layers, forward pass computation, representational capacity, MLP for classification, MLP for regression, MLP architecture design.47 #### Loss Functions Covers cross-entropy loss derivation, MSE for regression, binary vs multiclass cross-entropy, label smoothing, focal loss for imbalance, loss function numerical stability, custom loss functions.48 #### Backpropagation Covers computational graphs, chain rule review, forward and backward pass, gradient accumulation, backprop complexity analysis, automatic differentiation, implementing backprop from scratch.49 #### Stochastic Gradient Descent Covers batch vs stochastic gradient descent, minibatch gradient descent, learning rate selection, SGD convergence properties, SGD noise as regularization, learning rate schedules basics, SGD implementation.50 #### Momentum Covers momentum intuition (ball rolling), momentum update equations, momentum coefficient selection, dampening oscillations, momentum vs vanilla SGD, Nesterov momentum derivation, implementing momentum.51 #### Adam Optimizer Covers exponential moving averages, first moment (mean) estimation, second moment (variance) estimation, bias correction derivation, Adam update rule, Adam hyperparameters, Adam convergence properties.52 #### AdamW Covers L2 regularization vs weight decay, why they differ with Adam, AdamW formulation, weight decay coefficient selection, AdamW as default optimizer, AdamW vs Adam empirically.53 #### Weight Initialization Covers random initialization importance, Xavier/Glorot initialization derivation, He initialization for ReLU, initialization for different activations, layer-wise initialization, initialization debugging, modern initialization practices.54 #### Batch Normalization Covers internal covariate shift, batch statistics computation, learnable scale and shift, training vs inference mode, batch norm gradient flow, batch norm placement debates, batch norm limitations.55 #### Dropout Covers dropout as ensemble, dropout mask sampling, inverted dropout scaling, dropout rate selection, dropout at inference, spatial dropout for sequences, dropout in modern architectures.56 #### Gradient Clipping Covers gradient explosion detection, clip by value, clip by global norm, gradient clipping implementation, when to use gradient clipping, clipping threshold selection, monitoring gradient norms.

Part VIII: Recurrent Neural Networks

9chapters57 #### RNN Architecture Covers recurrent connection intuition, hidden state as memory, unrolled computation graph, parameter sharing across time, RNN for sequence classification, RNN for sequence generation, RNN equations and dimensions.58 #### Backpropagation Through Time Covers BPTT derivation, gradient flow through time, truncated BPTT, BPTT memory requirements, BPTT implementation, gradient accumulation across timesteps.59 #### Vanishing Gradients Covers gradient product across timesteps, vanishing gradient analysis, long-range dependency failure, gradient visualization, vanishing vs exploding trade-off, architectural solutions overview.60 #### LSTM Architecture Covers cell state as information highway, gate mechanism intuition, LSTM diagram walkthrough, information flow in LSTMs, LSTM for long sequences, LSTM memory capacity.61 #### LSTM Gate Equations Covers forget gate equations, input gate equations, cell state update, output gate equations, hidden state computation, LSTM parameter count, implementing LSTM from scratch.62 #### LSTM Gradient Flow Covers constant error carousel, forget gate gradient highway, gradient flow analysis, LSTM vs vanilla RNN gradients, peephole connections, LSTM gradient clipping needs.63 #### GRU Architecture Covers GRU vs LSTM comparison, reset gate function, update gate function, candidate hidden state, GRU equations, GRU parameter efficiency, when to choose GRU vs LSTM.64 #### Bidirectional RNNs Covers forward and backward passes, hidden state concatenation, bidirectional architectures, bidirectionality for classification, limitations for generation, implementing bidirectional RNNs.65 #### Stacked RNNs Covers multiple RNN layers, residual connections for depth, layer normalization in RNNs, depth vs width trade-offs, gradient flow in deep RNNs, practical depth limits.

Part IX: Sequence-to-Sequence

7chapters66 #### Encoder-Decoder Framework Covers encoder role and design, decoder role and design, context vector as bottleneck, seq2seq for machine translation, seq2seq for summarization, seq2seq training setup.67 #### Teacher Forcing Covers teacher forcing procedure, exposure bias problem, teacher forcing efficiency, scheduled sampling, curriculum learning, teacher forcing vs autoregressive training.68 #### Beam Search Covers greedy decoding limitations, beam search algorithm, beam width selection, length normalization, diverse beam search, beam search implementation, beam search vs sampling.69 #### Attention Intuition Covers attention as soft lookup, attention weight interpretation, attention for variable-length inputs, attention visualization, attention vs pooling, attention computation overview.70 #### Bahdanau Attention Covers alignment model formulation, score function (additive), attention weight computation, context vector as weighted sum, attention in decoder, Bahdanau attention implementation.71 #### Luong Attention Covers dot product attention, general (bilinear) attention, concat attention variant, global vs local attention, Luong vs Bahdanau comparison, attention placement (input vs output).72 #### Copy Mechanism Covers pointer network motivation, copy probability computation, mixing generation and copying, pointer-generator networks, copy mechanism for summarization, OOV handling with copy.

Part X: Self-Attention

6chapters73 #### Self-Attention Concept Covers cross-attention vs self-attention, self-attention motivation, all-pairs interaction, self-attention for representation learning, self-attention computational pattern.74 #### Query, Key, Value Covers QKV intuition (database lookup), projection matrices Wq, Wk, Wv, query-key matching, value retrieval, QKV dimensions and shapes, QKV as learned transformations.75 #### Scaled Dot-Product Attention Covers dot product for similarity, softmax for normalization, scaling factor derivation (1/√dk), attention output computation, attention in matrix form, attention implementation.76 #### Attention Masking Covers padding masks, causal (look-ahead) masks, combining multiple masks, mask shapes and broadcasting, efficient masking implementation, custom attention patterns.77 #### Multi-Head Attention Covers multiple attention heads motivation, head dimension splitting, parallel attention computation, output concatenation and projection, head specialization, multi-head vs single head.78 #### Attention Complexity Covers O(n²) attention complexity, memory requirements, attention bottleneck in long sequences, FLOPs computation, attention vs RNN complexity, practical scaling limits.

Part XI: Positional Encoding

7chapters79 #### Position Problem Covers transformer position blindness, why position matters for language, position information requirements, position encoding vs position embedding, absolute vs relative position.80 #### Sinusoidal Position Encoding Covers sinusoidal formula derivation, wavelength intuition, position encoding visualization, extrapolation properties, sinusoidal encoding implementation, learned vs sinusoidal trade-offs.81 #### Learned Position Embeddings Covers position embedding table, position embedding training, maximum sequence length, learned embedding extrapolation, position embedding analysis, GPT-style position embeddings.82 #### Relative Position Encoding Covers relative position motivation, relative attention formulation, clipping relative positions, relative position in self-attention, Shaw et al. relative positions, relative bias implementation.83 #### Rotary Position Embedding (RoPE) Covers RoPE intuition, rotation matrix formulation, RoPE in complex numbers, relative position through rotation, RoPE implementation, RoPE frequency patterns.84 #### ALiBi Covers ALiBi motivation, linear bias by distance, head-specific slopes, ALiBi extrapolation properties, ALiBi simplicity advantages, ALiBi vs RoPE comparison.85 #### Position Encoding Comparison Covers extrapolation benchmarks, training efficiency comparison, implementation complexity, position encoding for long context, hybrid approaches, current best practices.

Part XII: Transformer Blocks

8chapters86 #### Residual Connections Covers residual connection formulation, gradient highway interpretation, residual scaling, residual connections in transformers, pre-norm vs post-norm residuals.87 #### Layer Normalization Covers layer norm vs batch norm, layer norm formula, learnable affine parameters, layer norm placement, layer norm gradient flow, layer norm implementation.88 #### RMSNorm Covers RMSNorm derivation, removing mean centering, RMSNorm efficiency, RMSNorm vs LayerNorm performance, RMSNorm in modern architectures.89 #### Pre-Norm vs Post-Norm Covers original transformer (post-norm), pre-norm formulation, training stability comparison, gradient flow differences, when to use each, modern consensus.90 #### Feed-Forward Networks Covers FFN architecture, hidden dimension expansion, FFN as two linear layers, position independence, FFN parameter count, FFN computational cost.91 #### FFN Activation Functions Covers ReLU in original transformer, GELU adoption, GELU approximations, SiLU/Swish in modern models, activation function comparison.92 #### Gated Linear Units Covers GLU formulation, gating mechanism, SwiGLU derivation, GeGLU variant, GLU parameter efficiency, GLU in modern architectures.93 #### Transformer Block Assembly Covers standard block structure, component ordering, block implementation, block initialization, forward pass walkthrough, block hyperparameters.

Part XIII: Transformer Architectures

6chapters94 #### Encoder Architecture Covers encoder-only design, bidirectional self-attention, encoder for understanding tasks, encoder output usage, BERT-style encoder, encoder layer stacking.95 #### Decoder Architecture Covers decoder-only design, causal masking requirement, autoregressive generation, decoder for generation tasks, GPT-style decoder, decoder layer stacking.96 #### Encoder-Decoder Architecture Covers encoder-decoder interaction, cross-attention mechanism, encoder-decoder for seq2seq, T5-style architecture, information flow, when to use encoder-decoder.97 #### Cross-Attention Covers cross-attention formulation, KV from encoder, Q from decoder, cross-attention masking, cross-attention placement, cross-attention implementation.98 #### Weight Tying Covers input-output embedding tying, encoder-decoder tying, parameter reduction, weight tying effects on training, when to tie weights.99 #### Architecture Hyperparameters Covers depth vs width trade-offs, number of heads selection, hidden dimension ratios, FFN expansion ratio, total parameter calculation, architecture search.

Part XIV: Efficient Attention

9chapters100 #### Quadratic Attention Bottleneck Covers O(n²) memory analysis, O(n²) compute analysis, attention matrix size, practical sequence limits, bottleneck visualization, motivation for efficiency.101 #### Sparse Attention Patterns Covers local attention windows, strided attention patterns, block-sparse attention, combining sparse patterns, sparse attention implementation.102 #### Sliding Window Attention Covers sliding window formulation, window size selection, dilated sliding windows, sliding window for long sequences, Mistral-style windowed attention.103 #### Global Tokens Covers CLS token global attention, learned global tokens, global-local attention mixing, global token count, implementation strategies.104 #### Longformer Covers Longformer attention pattern, global attention configuration, Longformer complexity, Longformer for documents, Longformer implementation.105 #### BigBird Covers BigBird attention pattern, random attention benefits, BigBird theoretical guarantees, BigBird vs Longformer, BigBird applications.106 #### Linear Attention Covers softmax attention reformulation, kernel feature maps, linear complexity attention, linear attention limitations, Performer and variants.107 #### FlashAttention Algorithm Covers GPU memory hierarchy, tiling for SRAM, online softmax computation, recomputation strategy, FlashAttention complexity, FlashAttention benefits.108 #### FlashAttention Implementation Covers CUDA kernel basics, memory access patterns, FlashAttention-2 improvements, using FlashAttention in PyTorch, FlashAttention limitations.

Part XV: Long Context

7chapters109 #### Context Length Challenges Covers training sequence length limits, attention memory scaling, position encoding extrapolation, long-range dependency learning, evaluation challenges.110 #### Position Interpolation Covers linear position scaling, interpolation vs extrapolation, position interpolation implementation, fine-tuning for longer context, interpolation limitations.111 #### NTK-aware Scaling Covers RoPE frequency analysis, high-frequency preservation, NTK-aware formula, dynamic NTK scaling, NTK vs linear interpolation.112 #### YaRN Covers YaRN motivation, attention scaling factor, YaRN formula, YaRN training requirements, YaRN vs alternatives.113 #### Attention Sinks Covers attention sink phenomenon, StreamingLLM approach, sink token design, streaming inference, infinite context generation.114 #### Memory Augmentation Covers memory network concepts, memory retrieval mechanisms, memory writing and updating, memory-augmented transformers, Memorizing Transformers.115 #### Recurrent Memory Covers Transformer-XL approach, segment-level processing, recurrent state passing, relative position in recurrence, recurrent memory limitations.

Part XVI: Pre-training Objectives

7chapters116 #### Causal Language Modeling Covers CLM objective formulation, autoregressive factorization, CLM loss computation, CLM for generation, CLM training data, CLM scaling properties.117 #### Masked Language Modeling Covers MLM objective formulation, masking strategies (15% rule), [MASK] token usage, MLM for understanding, MLM training dynamics.118 #### Whole Word Masking Covers subword masking problems, whole word masking procedure, WWM implementation, WWM vs random masking, WWM for different tokenizers.119 #### Span Corruption Covers span selection strategies, span length distribution, sentinel tokens, T5-style corruption, span corruption benefits.120 #### Prefix Language Modeling Covers prefix LM formulation, prefix LM attention pattern, prefix LM for generation, prefix LM training, UniLM-style objectives.121 #### Replaced Token Detection Covers generator-discriminator setup, replaced vs original detection, RTD efficiency advantages, ELECTRA training procedure, RTD vs MLM comparison.122 #### Denoising Objectives Covers token deletion, token shuffling, sentence permutation, document rotation, BART-style denoising, combining denoising tasks.

Part XVII: BERT and Variants

8chapters123 #### BERT Architecture Covers BERT model sizes, BERT layer configuration, BERT embedding layers, BERT attention patterns, BERT output representations.124 #### BERT Pre-training Covers pre-training data preparation, MLM implementation, NSP task design, pre-training hyperparameters, pre-training duration.125 #### BERT Fine-tuning Covers classification fine-tuning, sequence labeling fine-tuning, question answering fine-tuning, fine-tuning hyperparameters, catastrophic forgetting.126 #### BERT Representations Covers [CLS] token usage, layer selection strategies, pooling strategies, BERT as feature extractor, frozen vs fine-tuned representations.127 #### RoBERTa Covers dynamic masking, NSP removal, larger batches, more data, RoBERTa training recipe, RoBERTa vs BERT performance.128 #### ALBERT Covers factorized embeddings, cross-layer parameter sharing, sentence order prediction, ALBERT efficiency, ALBERT performance trade-offs.129 #### ELECTRA Covers generator training, discriminator training, RTD objective, ELECTRA sample efficiency, ELECTRA scaling, ELECTRA fine-tuning.130 #### DeBERTa Covers disentangled attention formulation, enhanced mask decoder, DeBERTa position encoding, DeBERTa improvements, DeBERTa-v3 advances.

Part XVIII: GPT Architecture

10chapters131 #### GPT-1 Covers GPT-1 architecture, GPT-1 pre-training, GPT-1 fine-tuning approach, GPT-1 transfer learning, GPT-1 historical significance.132 #### GPT-2 Covers GPT-2 model sizes, GPT-2 architectural changes, zero-shot task performance, GPT-2 training data (WebText), GPT-2 generation quality.133 #### GPT-3 Covers GPT-3 scale (175B), few-shot prompting discovery, in-context learning analysis, GPT-3 capabilities, GPT-3 limitations.134 #### In-Context Learning Covers ICL phenomenon, ICL vs fine-tuning, example selection strategies, ICL scaling behavior, ICL theoretical understanding.135 #### Autoregressive Generation Covers generation procedure, KV caching for efficiency, generation stopping criteria, generation speed optimization, generation code implementation.136 #### Decoding Temperature Covers temperature scaling, temperature effects on distribution, temperature selection guidelines, temperature vs quality trade-off.137 #### Top-k Sampling Covers top-k truncation, k selection strategies, top-k limitations, top-k implementation, combining with temperature.138 #### Nucleus Sampling Covers top-p formulation, cumulative probability threshold, nucleus sampling benefits, p selection guidelines, nucleus vs top-k.139 #### Repetition Penalties Covers repetition in generation, repetition penalty formulation, frequency penalty, presence penalty, n-gram blocking.140 #### Constrained Decoding Covers grammar-guided generation, JSON schema constraints, regex constraints, constrained beam search, constrained sampling.

Part XIX: Modern Decoder Models

7chapters141 #### LLaMA Architecture Covers LLaMA design philosophy, LLaMA architectural choices, LLaMA training data, LLaMA efficiency, LLaMA significance.142 #### LLaMA Components Covers pre-norm with RMSNorm, SwiGLU FFN, RoPE implementation, component interactions, implementation details.143 #### Grouped Query Attention Covers GQA motivation, GQA formulation, KV head grouping, GQA memory savings, GQA vs MHA performance, GQA implementation.144 #### Multi-Query Attention Covers MQA extreme sharing, MQA memory benefits, MQA quality trade-offs, MQA for inference, MQA vs GQA.145 #### Mistral Architecture Covers Mistral design choices, sliding window attention, Mistral efficiency, Mistral performance, Mistral vs LLaMA.146 #### Qwen Architecture Covers Qwen architectural choices, Qwen training approach, Qwen multilingual capabilities, Qwen variants.147 #### Phi Models Covers Phi design philosophy, textbook-quality data, Phi training approach, Phi efficiency, small model capabilities.

Part XX: Encoder-Decoder Models

6chapters148 #### T5 Architecture Covers T5 encoder-decoder design, T5 attention patterns, T5 model sizes, T5 relative positions, T5 implementation.149 #### T5 Pre-training Covers span corruption procedure, sentinel tokens, corruption rate, T5 pre-training data, T5 training scale.150 #### T5 Task Formatting Covers task prefixes, classification as generation, NER as generation, QA as generation, task formatting examples.151 #### BART Architecture Covers BART encoder-decoder, BART attention configuration, BART vs T5 comparison, BART model sizes.152 #### BART Pre-training Covers token masking, token deletion, text infilling, sentence permutation, document rotation, objective combinations.153 #### mT5 Covers mT5 training data, language sampling, cross-lingual transfer, mT5 vs T5 performance, multilingual tokenization.

Part XXI: Scaling Laws

7chapters154 #### Power Laws in Deep Learning Covers power law definition, log-log linear relationships, power law fitting, power law universality, power law intuition.155 #### Kaplan Scaling Laws Covers loss vs parameters, loss vs data, loss vs compute, Kaplan optimal allocation, Kaplan predictions.156 #### Chinchilla Scaling Laws Covers Chinchilla experiments, revised scaling coefficients, optimal tokens per parameter, Chinchilla vs Kaplan, Chinchilla implications.157 #### Compute-Optimal Training Covers compute budget allocation, tokens vs parameters ratio, training efficiency, compute-optimal recipes, practical guidelines.158 #### Data-Constrained Scaling Covers data repetition effects, optimal repetition strategies, data augmentation scaling, synthetic data scaling.159 #### Inference Scaling Covers training vs inference compute, inference-optimal models, over-training for efficiency, deployment cost modeling.160 #### Predicting Model Performance Covers loss extrapolation, capability prediction, scaling law uncertainty, prediction reliability, practical forecasting.

Part XXII: Emergent Capabilities

6chapters161 #### Emergence in Neural Networks Covers emergence definition, phase transitions, emergence examples, emergence mechanisms, emergence debate.162 #### In-Context Learning Emergence Covers ICL emergence curves, ICL vs fine-tuning scaling, ICL mechanism hypotheses, ICL as meta-learning.163 #### Chain-of-Thought Emergence Covers CoT emergence observations, CoT elicitation, CoT scaling behavior, CoT mechanism theories.164 #### Emergence vs Metrics Covers discontinuous metrics, accuracy threshold effects, smooth underlying capabilities, re-examining emergence claims.165 #### Inverse Scaling Covers inverse scaling phenomena, distractor tasks, sycophancy scaling, inverse scaling prize findings.166 #### Grokking Covers grokking phenomenon, grokking in arithmetic, grokking mechanism theories, grokking phase transitions, practical implications.

Part XXIII: Mixture of Experts

10chapters167 #### Sparse Models Covers dense vs sparse trade-offs, conditional computation motivation, sparse model efficiency, sparse model challenges.168 #### Expert Networks Covers expert architecture, expert as FFN, expert capacity, expert count selection, expert placement in transformer.169 #### Gating Networks Covers router architecture, routing score computation, router training, router learned behavior.170 #### Top-K Routing Covers top-1 routing, top-2 routing, k selection trade-offs, routing implementation, combining expert outputs.171 #### Load Balancing Covers expert utilization imbalance, collapse failure mode, load metrics, balanced routing importance.172 #### Auxiliary Balancing Loss Covers load balancing loss formulation, loss coefficient tuning, balancing vs task loss, auxiliary loss implementation.173 #### Router Z-Loss Covers router instability, z-loss formulation, z-loss benefits, z-loss coefficient, combined auxiliary losses.174 #### Expert Parallelism Covers expert placement strategies, all-to-all communication, communication overhead, expert parallelism implementation.175 #### Switch Transformer Covers Switch Transformer design, top-1 routing choice, capacity factor, Switch scaling results.176 #### Mixtral Covers Mixtral architecture, Mixtral expert design, Mixtral performance, Mixtral efficiency, Mixtral vs dense models.

Part XXIV: Fine-tuning Fundamentals

5chapters177 #### Transfer Learning Covers transfer learning paradigm, pre-training/fine-tuning split, what transfers, transfer learning efficiency.178 #### Full Fine-tuning Covers full fine-tuning procedure, fine-tuning hyperparameters, learning rate selection, batch size effects.179 #### Catastrophic Forgetting Covers forgetting phenomenon, forgetting measurement, forgetting mitigation, pre-trained capability preservation.180 #### Fine-tuning Learning Rates Covers discriminative fine-tuning, layer-wise learning rates, warmup for fine-tuning, learning rate decay.181 #### Fine-tuning Data Efficiency Covers few-shot fine-tuning, data augmentation, sample efficiency patterns, small data strategies.

Part XXV: Parameter-Efficient Fine-tuning

12chapters182 #### PEFT Motivation Covers parameter storage costs, multi-task deployment, PEFT efficiency, PEFT quality trade-offs.183 #### LoRA Concept Covers weight update decomposition, low-rank assumption, LoRA efficiency gains, LoRA flexibility.184 #### LoRA Mathematics Covers LoRA formulation W + BA, rank selection, initialization scheme, LoRA gradient computation.185 #### LoRA Implementation Covers LoRA module design, merging weights, LoRA training loop, LoRA in PyTorch, HuggingFace PEFT usage.186 #### LoRA Hyperparameters Covers rank selection guidelines, alpha/rank ratio, which layers to adapt, LoRA dropout.187 #### QLoRA Covers 4-bit quantization for base model, NF4 data type, double quantization, QLoRA memory savings.188 #### AdaLoRA Covers importance-based pruning, SVD-based adaptation, dynamic rank, AdaLoRA training procedure.189 #### IA3 Covers IA3 formulation, learned rescaling vectors, IA3 parameter efficiency, IA3 vs LoRA.190 #### Prefix Tuning Covers prefix tuning formulation, prefix length selection, prefix tuning for generation, prefix vs LoRA.191 #### Prompt Tuning Covers prompt tuning formulation, prompt initialization, prompt tuning scaling, prompt length effects.192 #### Adapter Layers Covers adapter architecture, adapter placement, adapter dimensionality, adapter fusion.193 #### PEFT Comparison Covers performance comparison, parameter efficiency comparison, task suitability, practical recommendations.

Part XXVI: Instruction Tuning

6chapters194 #### Instruction Following Covers instruction tuning motivation, instruction format design, instruction diversity, instruction quality.195 #### Instruction Data Creation Covers human annotation, template-based generation, seed task expansion, quality filtering.196 #### Self-Instruct Covers self-instruct procedure, instruction generation, response generation, filtering strategies.197 #### Instruction Format Covers prompt templates, system messages, multi-turn format, chat templates, role definitions.198 #### Instruction Tuning Training Covers instruction tuning data mixing, training hyperparameters, loss masking, multi-task learning.199 #### Instruction Following Evaluation Covers instruction following benchmarks, human evaluation, automatic evaluation, instruction difficulty.

Part XXVII: Alignment and RLHF

16chapters200 #### Alignment Problem Covers alignment definition, helpfulness vs harmlessness, alignment challenges, alignment approaches overview.201 #### Human Preference Data Covers preference collection UI, comparison design, annotator guidelines, preference data quality.202 #### Bradley-Terry Model Covers pairwise comparison model, preference probability, Bradley-Terry likelihood, preference strength.203 #### Reward Modeling Covers reward model architecture, preference loss function, reward model training, reward model evaluation.204 #### Reward Hacking Covers reward hacking examples, distribution shift, over-optimization, reward hacking mitigation.205 #### Policy Gradient Methods Covers policy definition, REINFORCE algorithm, policy gradient derivation, variance reduction.206 #### PPO Algorithm Covers clipped objective, PPO derivation, trust region intuition, PPO implementation.207 #### PPO for Language Models Covers LLM as policy, action space (tokens), reward assignment, KL penalty importance.208 #### RLHF Pipeline Covers SFT stage, reward model training, PPO fine-tuning, RLHF hyperparameters, RLHF debugging.209 #### KL Divergence Penalty Covers KL penalty motivation, KL coefficient selection, adaptive KL, KL effects on training.210 #### DPO Concept Covers DPO motivation, removing reward model, DPO intuition, DPO benefits.211 #### DPO Derivation Covers DPO from RLHF objective, optimal policy derivation, DPO loss function, DPO as classification.212 #### DPO Implementation Covers DPO data format, DPO loss computation, DPO training procedure, DPO hyperparameters.213 #### DPO Variants Covers IPO formulation, KTO for unpaired feedback, ORPO, cDPO, comparing alignment methods.214 #### RLAIF Covers AI as annotator, constitutional AI principles, AI preference generation, RLAIF scalability.215 #### Iterative Alignment Covers iterative DPO, online preference learning, self-improvement loops, alignment stability.

Part XXVIII: Inference Optimization

14chapters216 #### KV Cache Covers KV cache motivation, cache structure, cache memory requirements, cache management.217 #### KV Cache Memory Covers cache size calculation, batch size effects, sequence length effects, memory bottleneck.218 #### Paged Attention Covers memory fragmentation problem, page-based allocation, vLLM approach, paged attention benefits.219 #### KV Cache Compression Covers cache eviction strategies, attention sink preservation, H2O algorithm, cache quantization.220 #### Weight Quantization Basics Covers quantization fundamentals, per-tensor vs per-channel, symmetric vs asymmetric, calibration.221 #### INT8 Quantization Covers INT8 range mapping, absmax quantization, smooth quantization, INT8 accuracy.222 #### INT4 Quantization Covers 4-bit challenges, group-wise quantization, 4-bit accuracy trade-offs, 4-bit formats.223 #### GPTQ Covers GPTQ algorithm, layer-wise quantization, Hessian approximation, GPTQ implementation.224 #### AWQ Covers salient weight preservation, AWQ algorithm, AWQ vs GPTQ, AWQ benefits.225 #### GGUF Format Covers GGML/GGUF history, quantization types, GGUF file format, llama.cpp integration.226 #### Speculative Decoding: Fast LLM Inference Without Quality Loss Covers speculative decoding concept, draft model selection, verification procedure, acceptance rate.227 #### Speculative Decoding Math: Algorithms & Speedup Limits Covers acceptance criterion, expected speedup, draft quality effects, optimal draft length.228 #### Continuous Batching: Optimizing LLM Inference Throughput Covers static vs continuous batching, iteration-level scheduling, request completion handling, throughput gains.229 #### LLM Inference Serving: Architecture, Routing & Auto-Scaling Master LLM inference serving architecture, token-aware load balancing, and auto-scaling. Optimize time-to-first-token and throughput for production systems.

Part XXIX: Retrieval-Augmented Generation

14chapters230 #### RAG Motivation: Solving Hallucinations & Knowledge Gaps Discover why LLMs need Retrieval-Augmented Generation. Learn how RAG bridges knowledge gaps, reduces hallucinations, and enables non-parametric memory.231 #### RAG Architecture: Components, Timing & Design Patterns Master RAG system design by exploring retriever-generator interactions, timing strategies like iterative retrieval, and architectural variations like RETRO.232 #### Dense Retrieval: Semantic Search & Bi-Encoder Implementation Master dense retrieval for semantic search. Explore bi-encoder architectures, embedding metrics, and contrastive learning to overcome keyword limitations.233 #### Contrastive Learning for Retrieval: InfoNCE & DPR Guide Master contrastive learning for dense retrieval. Learn to train models using InfoNCE loss, in-batch negatives, and hard negative mining strategies effectively.234 #### Document Chunking: Optimizing RAG Retrieval Pipelines Master document chunking for RAG systems. Explore fixed-size, recursive, and semantic strategies to balance retrieval precision with context window limits.235 #### Embedding Models: Architecture, Pooling & Selection Learn how embedding models convert text to vectors for RAG. Covers bi-encoder architecture, pooling strategies, dimensionality trade-offs, and model selection.236 #### Vector Similarity Search: Metrics & Approximate Methods Explore vector similarity search for RAG systems. Compare cosine, dot product, and Euclidean metrics, and implement exact vs. approximate search with FAISS.237 #### HNSW Index: Architecture for Fast Vector Search Master Hierarchical Navigable Small World (HNSW) graphs for vector search. Learn graph architecture, construction, and tuning for high-speed retrieval.238 #### IVF Index: Clustering-Based Vector Search & Partitioning Master IVF indexes for scalable vector search. Learn clustering-based partitioning, nprobe tuning, and IVF-PQ compression for billion-scale retrieval.239 #### Product Quantization: Vector Compression for ANN Search Learn how Product Quantization compresses embeddings up to 100x using learned codebooks and asymmetric distance computation for scalable vector search.240 #### Hybrid Search: BM25 and Dense Retrieval Combined Learn how hybrid search fuses BM25 keyword retrieval with dense vector retrieval using reciprocal rank fusion and weighted score combination to improve recall.241 #### Reranking: Cross-Encoders for Precise Information Retrieval Learn how reranking with cross-encoders solves bi-encoder limitations. Master two-stage retrieval, training strategies, and latency optimization for production search systems.242 #### RAG Prompt Engineering: Context Placement & Citation Strategies Master RAG prompt engineering with strategic context placement, citation formats, and truncation strategies to improve LLM accuracy and reduce hallucinations.243 #### RAG Evaluation: Metrics for Retrieval and Generation Quality Master RAG evaluation with metrics for retrieval quality (Precision@K, NDCG, MRR) and generation faithfulness using the RAGAS framework for AI systems.

Part XXX: Tool Use and Agents

10chapters244 #### Tool Use Motivation: Why LLMs Need External Tools for Accuracy Discover why LLMs require external tools to overcome knowledge cutoffs, computational limits, and hallucinations. Learn about tool-augmented AI systems.245 #### Function Calling: Structured Tool Use for Large Language Models Learn how function calling enables LLMs to invoke external tools and APIs through structured JSON schemas, bridging natural language and executable code.246 #### ReAct Pattern: Interleaving Reasoning and Action for LLM Agents Learn how the ReAct pattern enables LLM agents to interleave reasoning with tool execution. Master thought-action-observation loops for autonomous AI systems.247 #### Tool Selection for LLM Agents: Routing Strategies and Implementation Master LLM tool selection through embedding-based routing, hybrid strategies, and semantic interfaces. Learn to build scalable multi-tool agent systems.248 #### Agent Architectures: Control Loops, State & Planning Master LLM agent architectures including control loops, state management strategies, planning mechanisms, and termination conditions for autonomous AI systems.249 #### Agent Memory Systems: From Context to Persistent Storage Learn how AI agents manage short-term context and long-term memory. Explore vector databases, retrieval algorithms, and memory hierarchies that enable persistent, learning systems.250 #### Agent Evaluation: Metrics, Benchmarks and Safety Standards Learn to evaluate AI agents with task completion metrics, trajectory analysis, and safety testing. Covers WebArena, SWE-bench, GAIA, and OSWorld benchmarks.251 #### Planning Covers task decomposition, goal-directed planning, plan execution, plan revision and recovery.252 #### Multi-Agent Systems Covers agent coordination, communication protocols, role assignment, multi-agent benchmarks.253 #### Agent Safety Covers unsafe action prevention, agent alignment, sandboxing, monitoring and intervention.

Part XXXI: Multimodal Models

12chapters254 #### Vision Transformer Covers image patching, patch embeddings, ViT architecture, ViT pre-training.255 #### CLIP Covers CLIP architecture, CLIP training objective, CLIP zero-shot classification, CLIP embeddings.256 #### Vision Encoders for VLMs Covers ViT variants for VLMs, SigLIP improvements, image resolution handling, encoder selection.257 #### Vision-Language Projection Covers linear projection, MLP projection, Q-Former approach, projection training.258 #### LLaVA Architecture Covers LLaVA design, two-stage training, visual conversation, LLaVA variants.259 #### Flamingo Architecture Covers cross-attention to images, gated cross-attention, few-shot visual learning, Flamingo training.260 #### Multimodal Training Data Covers image-text pairs, interleaved documents, visual instruction data, data quality.261 #### Multimodal Evaluation Covers VQA benchmarks, multimodal understanding benchmarks, multimodal generation evaluation.262 #### Multimodal Foundations Covers multimodal learning principles, cross-modal alignment, joint embedding spaces, multimodal challenges.263 #### Image Understanding Covers visual question answering, image captioning, visual grounding, scene understanding.264 #### Image Generation Covers diffusion models, text-to-image generation, image editing, generation evaluation.265 #### Multimodal Applications Covers document understanding, medical imaging, video understanding, multimodal reasoning tasks.

Part XXXII: Speech and Audio

5chapters266 #### Speech Representations Covers mel spectrograms, mel filterbanks, feature normalization, audio preprocessing.267 #### Whisper Architecture Covers Whisper encoder-decoder, multitask training, language tokens, timestamp prediction.268 #### Whisper Training Covers Whisper training data, weak supervision, multilingual training, Whisper capabilities.269 #### Speech-Language Integration Covers speech encoder + LLM, audio tokens, speech-to-text-to-LLM vs end-to-end, speech LLM architectures.270 #### Text-to-Speech Covers TTS architecture overview, vocoder role, TTS quality metrics, neural TTS approaches.

Part XXXIII: Evaluation Fundamentals

10chapters271 #### Perplexity Evaluation Covers perplexity calculation, perplexity interpretation, perplexity limitations, comparing perplexities.272 #### Cross-Entropy Loss Covers cross-entropy definition, bits-per-character, cross-entropy vs perplexity, loss curves.273 #### BLEU Score Covers n-gram precision, brevity penalty, BLEU formula, BLEU limitations, corpus vs sentence BLEU.274 #### ROUGE Scores Covers ROUGE-N, ROUGE-L, ROUGE-W, ROUGE interpretation, ROUGE limitations.275 #### BERTScore Covers BERTScore computation, token alignment, BERTScore variants, BERTScore vs BLEU.276 #### Exact Match and F1 Covers exact match scoring, token-level F1, normalization for matching, metric selection.277 #### Calibration Covers calibration definition, expected calibration error, calibration plots, calibration methods.278 #### Evaluation Fundamentals Covers evaluation design principles, metric selection, evaluation pitfalls, evaluation frameworks.279 #### Benchmark Design Covers benchmark construction, dataset collection, annotation guidelines, benchmark validity.280 #### Evaluation Challenges Covers benchmark contamination, evaluation brittleness, gaming metrics, evaluation best practices.

Part XXXIV: Benchmark Evaluation

8chapters281 #### MMLU Covers MMLU structure, subject coverage, MMLU evaluation protocol, MMLU limitations.282 #### HellaSwag Covers HellaSwag task design, adversarial filtering, HellaSwag evaluation, HellaSwag saturation.283 #### GSM8K Covers GSM8K problem types, chain-of-thought evaluation, GSM8K accuracy metrics, math reasoning assessment.284 #### HumanEval Covers HumanEval structure, functional correctness, pass@k metric, HumanEval limitations.285 #### MBPP Covers MBPP dataset, MBPP vs HumanEval, code evaluation challenges.286 #### TruthfulQA Covers TruthfulQA design, truthfulness vs informativeness, TruthfulQA evaluation methods.287 #### Benchmark Contamination Covers contamination problem, contamination detection methods, n-gram overlap analysis, contamination mitigation.288 #### Benchmark Saturation Covers ceiling effects, benchmark retirement, dynamic benchmarks, benchmark evolution.

Part XXXV: Human and Model Evaluation

6chapters289 #### Human Evaluation Design Covers evaluation interface design, task instructions, annotator selection, evaluation cost.290 #### Inter-Annotator Agreement Covers Cohen’s kappa, Fleiss’ kappa, Krippendorff’s alpha, handling disagreement.291 #### Preference Evaluation Covers A/B comparison design, Elo rating systems, preference aggregation, statistical significance.292 #### LLM-as-Judge: Scalable AI Evaluation with Language Models Learn how to build LLM-as-Judge evaluation pipelines: prompt design, judge model selection, calibration against human annotations, and bias mitigation.293 #### Position Bias in LLM Judges Covers position bias measurement, bias mitigation (swapping), verbosity bias, sycophancy.294 #### Evaluation Prompt Engineering: Designing Reliable LLM Judges Learn how to design reliable LLM judge prompts using explicit criteria, few-shot examples, and chain-of-thought formatting to maximize evaluation accuracy.

Part XXXVI: Bias and Fairness

5chapters295 #### Bias in Language Models Covers bias sources, bias types (demographic, cultural), bias in training data, bias amplification.296 #### Bias Measurement Covers embedding association tests, generation bias metrics, classification bias metrics, bias benchmarks.297 #### Bias Mitigation: Debiasing, CDA, and Fair Fine-tuning Practical techniques for reducing demographic bias in language models: data balancing, embedding debiasing, adversarial training, and prompt-based interventions.298 #### Fairness Metrics: Demographic Parity, Equalized Odds, and Trade-offs Learn the key mathematical definitions of algorithmic fairness, from demographic parity to equalized odds, and why satisfying all metrics simultaneously is impossible.299 #### Representation Harms Covers stereotyping, erasure, demeaning associations, measuring representation harms.

Part XXXVII: Hallucination and Factuality

6chapters300 #### Hallucination Types in Language Models: A Complete Guide Learn how language models hallucinate: intrinsic and extrinsic hallucination, factual errors, fabrication, and inconsistency with NLI-based detection.301 #### Hallucination Detection: NLI, Self-Consistency & Learned Models Learn four methods for detecting LLM hallucinations: entailment-based scoring, knowledge base verification, self-consistency checks, and learned detection models.302 #### Hallucination Causes Covers training data issues, exposure bias, knowledge gaps, generation pressure.303 #### Hallucination Mitigation: RAG, Decoding, and Training Learn how to reduce LLM hallucination using retrieval augmentation, self-consistency decoding, DPO training, and calibrated uncertainty expression.304 #### Attribution and Citation: Sourcing LLM Outputs Learn how language models link generated claims to source documents, evaluate citation accuracy with NLI, and measure attribution precision and recall.305 #### Uncertainty Quantification Covers confidence calibration, verbalized uncertainty, sampling-based uncertainty, uncertainty communication.

Part XXXVIII: Safety and Security

8chapters306 #### Safety Risks Covers harmful content generation, misuse scenarios, unintended harms, safety threat models.307 #### Red Teaming Learn how red teams systematically probe language models for safety failures, covering attack taxonomies, ASR metrics, RL-based automated attack generation, and real-world findings.308 #### Jailbreaking Covers jailbreak techniques, prompt injection, adversarial suffixes, jailbreak defenses.309 #### Prompt Injection Covers direct prompt injection, indirect prompt injection, injection in RAG, injection defenses.310 #### Content Filtering Covers classification-based filtering, rule-based filtering, filter placement, filter evaluation.311 #### Guardrails Covers input guardrails, output guardrails, guardrail frameworks, guardrail design.312 #### Memorization and Privacy in Language Models How language models memorize training data, methods for measuring extractable memorization, PII risks in web-scale corpora, and practical privacy mitigations.313 #### Differential Privacy Learn how differential privacy protects training data in language models, from the mathematical guarantee to DP-SGD, privacy budgets, and practical LLM fine-tuning.

Part XXXIX: Interpretability

11chapters314 #### Interpretability Goals Covers debugging, trust, safety, scientific understanding, interpretability approaches overview.315 #### Attention Visualization Covers attention weight extraction, attention head visualization, attention interpretation caveats, attention tools.316 #### Attention Analysis Limitations Covers attention vs importance, attention manipulation studies, gradient-based alternatives.317 #### Probing Classifiers Covers linear probing methodology, probing task design, probing interpretation, control tasks.318 #### Probing Layers Covers layer selection, representation evolution, task localization, layer probing patterns.319 #### Activation Patching Covers patching methodology, locating information, patching experiments, causal tracing.320 #### Logit Lens Covers logit lens concept, intermediate vocabulary projection, tuned lens, lens interpretation.321 #### Sparse Autoencoders Covers SAE architecture, sparsity constraints, dictionary learning, SAE for LLMs.322 #### Feature Interpretation Covers feature activation patterns, feature naming, automated interpretation, feature circuits.323 #### Mechanistic Interpretability Reverse-engineer transformer networks into human-understandable algorithms by identifying circuits, induction heads, and mechanistic discoveries.324 #### Activation Steering: Steering Vectors and Representation Engineering Learn how steering vectors and activation addition let you modify language model behavior at inference time by injecting directional nudges into the residual stream.

Part XL: LLM Applications

7chapters325 #### Text Generation Applications Covers creative writing, content generation, text transformation, generation quality control.326 #### Summarization Covers extractive vs abstractive, length control, faithfulness, multi-document summarization.327 #### Question Answering Covers open-domain QA, reading comprehension, knowledge-intensive QA, QA evaluation.328 #### Information Extraction Covers named entity recognition, relation extraction, event extraction, structured output generation.329 #### Classification Applications Covers sentiment analysis, intent detection, topic classification, zero-shot classification.330 #### Conversational AI Covers dialogue management, context tracking, persona consistency, conversation evaluation.331 #### Creative Applications Covers story generation, poetry, code creativity, creative constraints and control.

Part XLI: Data Curation

10chapters332 #### Web Crawling Covers Common Crawl, crawling strategies, robots.txt respect, crawl freshness.333 #### Document Extraction Covers HTML parsing, boilerplate removal, content extraction, trafilatura and similar tools.334 #### Language Identification Covers language ID models, multilingual document handling, code-switching, language filtering.335 #### Deduplication Covers exact deduplication, near-duplicate detection, document vs substring dedup, dedup at scale.336 #### MinHash Covers MinHash algorithm, Jaccard similarity estimation, MinHash LSH, MinHash implementation.337 #### Quality Filtering Covers heuristic filters, perplexity filtering, classifier-based filtering, filter thresholds.338 #### Toxicity Filtering Covers toxicity classifiers, toxicity thresholds, over-filtering risks, toxicity filter evaluation.339 #### PII Removal Covers PII detection methods, PII removal strategies, PII removal evaluation, privacy preservation.340 #### Data Mixing Covers domain proportions, quality weighting, data mixing experiments, optimal mixing.341 #### Synthetic Data Covers synthetic data generation, quality verification, synthetic data diversity, distillation.

Part XLII: Training Infrastructure

11chapters342 #### GPU Architecture Covers GPU memory hierarchy, CUDA cores, tensor cores, GPU specifications.343 #### Memory Management Covers memory breakdown (activations, parameters, gradients, optimizer states), memory estimation, OOM debugging.344 #### Data Parallelism Covers DDP algorithm, gradient synchronization, all-reduce operations, DDP scaling.345 #### Tensor Parallelism Covers column parallelism, row parallelism, communication patterns, Megatron-style parallelism.346 #### Pipeline Parallelism Covers pipeline stages, micro-batching, pipeline bubbles, pipeline schedules (GPipe, 1F1B).347 #### ZeRO Optimization Covers ZeRO stage 1 (optimizer state partitioning), ZeRO stage 2 (gradient partitioning), ZeRO stage 3 (parameter partitioning), ZeRO memory savings.348 #### FSDP Covers FSDP concepts, FSDP vs ZeRO, FSDP sharding strategies, FSDP usage.349 #### Activation Checkpointing Covers checkpointing concept, checkpoint selection, checkpointing overhead, selective checkpointing.350 #### Mixed Precision Training Covers floating point formats, loss scaling, BF16 advantages, mixed precision implementation.351 #### Communication Optimization Covers gradient compression, communication overlap, topology-aware communication, NCCL optimization.352 #### Checkpointing and Recovery Covers checkpoint contents, checkpoint frequency, async checkpointing, fault recovery.

Part XLIII: Training Optimization

8chapters353 #### Learning Rate Warmup Covers warmup motivation, linear warmup, warmup duration, warmup for large batches.354 #### Learning Rate Decay Covers step decay, exponential decay, inverse square root decay, decay scheduling.355 #### Cosine Learning Rate Schedule Covers cosine decay formula, cosine with restarts, cosine schedule parameters, cosine vs linear.356 #### Large Batch Training Covers batch size effects, learning rate scaling, batch size limits, LAMB optimizer.357 #### Weight Decay Covers weight decay formula, decoupled weight decay, weight decay selection, weight decay interaction with Adam.358 #### Gradient Accumulation Covers accumulation procedure, accumulation steps, accumulation for memory, accumulation correctness.359 #### Training Stability Covers loss spikes, gradient norm monitoring, stability techniques, training stability debugging.360 #### Hyperparameter Selection Covers hyperparameter search, hyperparameter transfer, critical vs robust hyperparameters, default recipes.

Part XLIV: Code Generation

6chapters361 #### Code LLM Training Covers code training data, code tokenization, fill-in-the-middle training, code pre-training objectives.362 #### Code Understanding Covers code explanation, bug detection, code review, code search.363 #### Code Completion Covers completion context, completion ranking, completion latency, completion UX.364 #### Code Generation Covers docstring-to-code, test-to-code, code generation strategies, generation quality.365 #### Code Execution Covers sandboxed execution, execution feedback, iterative refinement, execution safety.366 #### Code Evaluation Covers functional correctness, pass@k metric, code benchmarks, beyond correctness.

Part XLV: Production Systems

9chapters367 #### Model Serving Covers serving frameworks, model loading, request handling, serving configuration.368 #### Latency Optimization Covers latency breakdown, batching latency, streaming responses, latency monitoring.369 #### Throughput Optimization Covers batch size tuning, GPU utilization, concurrent requests, throughput measurement.370 #### Auto-scaling Covers scaling metrics, horizontal scaling, scale-up vs scale-out, scaling policies.371 #### Model Routing Covers model selection, A/B testing, model cascades, routing strategies.372 #### Caching Covers prompt caching, semantic caching, cache invalidation, cache hit rates.373 #### Monitoring Covers metrics collection, alerting, logging, dashboards.374 #### Quality Monitoring Covers output quality metrics, drift detection, regression detection, quality alerts.375 #### Cost Management Covers cost modeling, cost optimization, cost allocation, cost monitoring.

Part XLVI: Continual Learning

5chapters376 #### Continual Learning Problem Covers continual learning definition, catastrophic forgetting, continual learning scenarios.377 #### Regularization Methods Covers elastic weight consolidation, synaptic intelligence, parameter importance, regularization trade-offs.378 #### Replay Methods Covers replay buffer design, pseudo-rehearsal, generative replay, replay selection.379 #### Architecture Methods Covers progressive networks, expert expansion, architecture search, modular approaches.380 #### Continual Learning Evaluation Covers forward transfer, backward transfer, evaluation protocols, continual benchmarks.

Part XLVII: Model Compression

6chapters381 #### Knowledge Distillation Covers distillation objective, temperature in distillation, teacher selection, distillation for LLMs.382 #### Distillation Variants Covers feature distillation, attention transfer, progressive distillation, on-policy distillation.383 #### Pruning Basics Covers weight pruning, structured vs unstructured, pruning criteria, pruning schedule.384 #### Structured Pruning Covers head pruning, layer pruning, width pruning, structured pruning implementation.385 #### Model Merging Covers weight averaging, task arithmetic, TIES merging, DARE merging.386 #### Model Merging Applications Covers multi-task merging, style merging, capability composition, merging evaluation.

Part XLVIII: Reasoning

7chapters387 #### Reasoning Foundations Covers reasoning types, reasoning in LLMs, reasoning failure modes, reasoning evaluation.388 #### Chain-of-Thought Covers CoT prompting, zero-shot CoT, CoT fine-tuning, CoT limitations.389 #### Reasoning Strategies Covers self-consistency, tree of thought, least-to-most prompting, decomposition strategies.390 #### Reasoning Verification Covers step verification, process reward models, verification-guided search, self-correction.391 #### Mathematical Reasoning Covers math problem solving, symbolic integration, math benchmarks, math reasoning training.392 #### Reasoning Limitations Covers systematic failures, spurious correlations, reasoning shortcuts, robustness challenges.393 #### Reasoning Frontiers Covers o1-style reasoning, test-time compute scaling, reasoning-capable models, open research questions.

Part XLIX: Advanced Topics

8chapters394 #### Constitutional AI Covers constitutional principles, critique and revision, CAI training, CAI effectiveness.395 #### Process Reward Models Covers outcome vs process reward, PRM training, PRM for math, PRM limitations.396 #### Test-Time Compute Covers multiple sampling, iterative refinement, compute-optimal inference, scaling test-time compute.397 #### Retrieval-Augmented Training Covers RETRO architecture, retrieval during training, retrieved context integration.398 #### Long-Form Generation Covers outline-based generation, hierarchical generation, coherence maintenance, long-form evaluation.399 #### Watermarking Covers watermarking schemes, statistical detection, watermark robustness, watermark evaluation.400 #### Model Cards Covers model card contents, intended use documentation, limitation documentation, model card best practices.401 #### Responsible Deployment Covers release decisions, staged release, access control, deployment monitoring.

Part L: Future Directions

6chapters402 #### Scaling Frontiers Covers scaling limits, beyond power laws, data wall, architectural innovations for scale.403 #### Efficiency Frontiers Covers efficient architectures, hardware co-design, inference efficiency, training efficiency advances.404 #### Capability Frontiers Covers emerging capabilities, world models, planning and agency, multimodal reasoning advances.405 #### Alignment Challenges Covers scalable oversight, alignment tax, goal mis-specification, long-term alignment research.406 #### Societal Implications Covers labor market effects, access and equity, regulation landscape, governance frameworks.407 #### Research Directions Covers open problems, promising research areas, benchmark gaps, community priorities.

In Progress

This comprehensive handbook is currently in development. Each chapter will be published as it’s completed, with practical examples, code implementations, and real-world applications.

Newsletter

Stay up to date.

Get articles, book updates, and news delivered to your inbox.

No spam, unsubscribe anytime.

Join the community.

Sign in to remove popups, track your reading progress, and join the discussion.