Hi Reddit, I posted my Build Your Own LLM workshop to Youtube (GPT2 & Qwen3.6 style)

# TL;DR Summary This workshop teaches you how to build your own large language model (LLM) from scratch, with no math or machine learning prerequisites—just familiarity with programming. Through intuitive intuition-building, manual Excel demonstrations, and hands-on Python/PyTorch code, you’ll understand the Transformer architecture, the training pipeline, and finish with a working mini LLM. ## Why Build Your Own Large Language Model? Why spend hours on this deep technical workshop? The core answer is: **Develop taste and intuition for how AI and ML models are built.** You’ll understand: - There’s no magic behind these models—everything is math and backpropagation. - You’ll know the limitations of models, which is critical if you're building applications or agents on top of them. - You’ll be able to write all the machine learning code and LLM architecture code, and know how to handle pre-training and post-training. - You can read the code of open-source models (like nano chat, etc.) and gain transferable skills for working on AI/ML solutions. - When DeepSeek releases a model that moves the stock market, you’ll understand what sparse attention vs. dense attention means. This workshop alone won’t get you hired at a frontier lab, but it provides all the foundations needed for those roles: GPU programming, AI safety, and other subfields all depend on this knowledge. ## Prerequisites and Non-Prerequisites **Required**: Software engineer, comfortable with coding, able to spend 8 hours reading code and learning new concepts. **Not required**: - Math: Linear algebra, calculus, statistics, etc., are not required—we’ll build intuition in one day, not proofs. - Python: Helpful but not necessary—we minimize Python usage, use cross-language idioms, and by 2026 you can use AI to read/write code. - Machine learning: No prior knowledge of perceptrons, neurons needed—all concepts will be covered. ## About the Instructor, Justin Angel 23 years of software engineering experience at Meta, Uber, Amazon, Apple, Microsoft; held IC8 (Meta director-level IC). All code written by AI in 2023-2024. Firm believer that “any software engineer who can’t understand AI at a fundamental level will soon be unemployable.” Recently earned an AI/ML master’s degree with a thesis on LLM psychotherapy. Emphasizes he’s not a 20-year ML engineer, but offers a practical perspective. ## Learning Methodology: Manual AI + Code Practice The workshop is divided into ~23 parts, each with three phases: 1. **Concept Explanation**: Instructor uses slides to explain the problem, how the technique fits into the big picture, and builds intuition. 2. **Manual AI**: In Excel (Google Sheets), manually perform matrix multiplication, move numbers, and build intuition from a mathematical perspective. This method thanks Professor Tom Yey from the University of Colorado Boulder. 3. **Code Review**: Each part comes with a Python/PyTorch notebook based on the slides and Excel content. The instructor walks through each section. **Your responsibilities**: - Fill in the version of the Excel without answers. - Run, tweak, and read the Colab notebooks. - Complete 1–3 exercises per part (about 15–20 minutes) and understand the answers. ## Workshop Schedule ### Introduction: Using an Existing GPT-2 Style Transformer Take an off-the-shelf GPT-2 style Transformer and run it, understand concepts like autoregressive loops, and build the full-day roadmap. ### Machine Learning Basics: Starting with a Single Neuron - What is a neuron? - How to train a network? - What is a loss function? - How to use gradient descent? Deliverable: A trained deep neural network (not a language model), with all concepts 100% transferable to large language models. ### Transformer Architecture: Learn Components One by One and Combine - Multi-Layer Perceptron (MLP) - Dropout - Tokenizer - Embedding Layer - Attention Mechanism Finally, assemble all components into the full Transformer architecture. ### LLM Delivery: Pre-training and Post-training Write post-training and pre-training scripts. By the end of the day, have your own working large language model, branded with your name and version V1. ## Workshop Resource Links All resources follow a simple naming convention: - Slides: `go.justinangel.ai/deck` - Code notebooks: `go.justinangel.ai/code-{part number}` - Excel spreadsheets: `go.justinangel.ai/excel-{part number}` - Master link (includes all links): `go.justinangel.ai/drive` ## Start Training Immediately: Part Zero Because training a large language model takes 8–20 hours, we need to kick off training right at the start of the workshop. Go to `go.justinangel.ai/code-zero`, connect an A100 GPU (requires Google Colab Pro Plus), click “Run All”, and the model will run in the background for 20+ hours. That way, by the end of the workshop, you’ll have your own model! --- **Source**: Hi Reddit, I posted my Build Your Own LLM workshop to Youtube (GPT2 & Qwen3.6 style) (https://www.youtube.com/watch?v=vXiB0UdDhk8&list=PLweJS2YZCfkeXXdfCKGaxAhm2w8p0u1z6&index=1)