Tag
This course note summarizes the architectural evolution from the original Transformer to modern LLMs, focusing on convergent developments such as pre-normalization, RMS normalization, and RoPE, and provides hyperparameter selection recommendations.