@Michaelzsguo: Alisa Liu mentioned the Stanford course CS336: Language Modeling from Scratch while preparing for an OpenAI interview. If you want to systematically learn LLM now, or if you plan to pursue AI research / MTS / ML e…

Alisa Liu mentioned the Stanford course CS336: Language Modeling from Scratch while preparing for an OpenAI interview. If you want to systematically learn LLM now, or later aim for jobs related to AI research / MTS / ML engineer, this course is well worth adding to your study plan. CS336 is called Language Modeling from Scratch, and the focus is indeed on 'from scratch.' It starts from tokenizer and covers transformer, optimizer, PyTorch, GPU, Triton, FlashAttention, parallelism, scaling laws, data pipeline, post-training, and alignment. It ties together a complete pipeline for a language model, from pre-training data processing, model architecture, training efficiency, compute estimation, to post-training and alignment. This is very helpful for those seeking AI jobs. Many interviews may: - Ask you to write attention by hand - Ask you to explain why training is slow - Ask you to determine how to calculate memory and FLOPs - Ask you to debug a training loop - Ask you to discuss trade-offs between data, model size, and compute - Ask you to explain what pretraining, SFT, RLHF / RLVR each solve If you only learn from scattered materials, it's easy to know many terms but not how they connect in a real training pipeline. The advantage of CS336 is that it always stays close to what is actually used in the industry. The 2026 edition already incorporates the latest LLM models like Minimax to cover these topics. More remarkably, Stanford has made the course website, assignments, and YouTube recordings publicly available. Looking at its assignments, they are almost exactly the work you would do on the job at a company. While studying CS336, you might need to supplement some basics. Similarly, both Harvard and Stanford have open courses to help you learn these basics: Harvard CS50 AI: good for building AI foundations and a sense of Python projects Stanford CS229: good for traditional ML and mathematical frameworks Stanford CS224N: good for understanding language model history before NLP / transformers Stanford CS231N: good for deep learning and vision models But if your goal is today's LLM / AI research / AI infra, CS336 is the one I would prioritize recommending.