I wrote 934 pages on how to build every layer of a large language model from scratch. Many of these concepts were new to me a year back.

Tokenizers, attention, KV cache, MoE, RLHF, quantization, serving. 35 projects. Every chapter has a section where you break the thing you just built.

Everything I use to build TamilLM is in this book. If you have been following the journey and want to understand what is actually happening under the hood, start here. It also ties in to my previous papers as well in the various chapters.

35 copies sold so far.

https://leanpub.com/under-the-hood

Let’s go.

Under The Hood

Source: https://leanpub.com/under-the-hood Build Every Layer of a Large Language Model from Scratch

You pay

Author earns

About

Most LLM books teach you to *use* models. This one teaches you to *build* them — every layer, every optimizer step, every cache, every quantization scheme — and then to deliberately break each piece so you understand why it exists in the first place.

It is a workshop in book form. 35 hands-on projects, ~250,000 words, one tight spiral that takes you from a single autograd scalar all the way to fusing independently trained specialists into a routed system. No black boxes. No “import library, call method.” You write the code, you run it, you break it, you measure what broke.

Each project follows the same disciplined rhythm: Hook → The Concept → Why It Matters → The Build → **BREAK IT** → Optional Homework → Questions To Answer → Go Further → What You Now Know. Reading the book without breaking the code is half the experience. The breaks are where the lessons actually live.

A public code companion lives at github.com/mechramc/Under-the-hood with runnable build.py, tests, and captured outputs for every project.

If you have ever read a transformer paper and felt that the diagram and the code were in two different universes — this book closes that gap.

**Build it. Break it. Measure it.**

Share this book

Categories

Feedback

Installments completed

1/3

Author

Ramchand Kumaresan

Ramchand Kumaresan is a Senior Program Manager at Procore Technologies and the founder of Murai Labs, a one-person AI research lab built on the idea that disciplined engineering matters more than hype.

His published work includes KALAVAI (cooperative LoRA fusion for routing across specialist adapters), UYIR (evolutionary lifecycles for LoRA adapters, currently under review at TMLR), and Orion (the first open end-to-end system for programming Apple’s Neural Engine for LLM inference and training). His current research focuses on heritable sparse mutability masks (MARMAM), grounded in Tamil Siddhamarmam-point therapy.

He wroteUnder the Hoodto teach himself what he didn’t know — and then kept writing as he learned more. The book is the trail he marked along the way.

Launch

Subscribe on YouTube

Clips

• 17 Years Program Management & AI | The Real Story
• 40 Year Old Learns AI | Is It Too Late to Master Language Models
• AI Breaks Silently | The Key To Understanding Language Models
• AI Research | Making the Future Boring, Dependable, and Real
• Break AI Systems | Save Millions on Experiments!

View all clips→

Contents

Under The Hood

1. The LLM Engineering Manual

Copyright

Preface: What This Book Is Actually Asking You To Do

Using the Code Repository

1. How the repo is organized
2. The intended workflow per chapter
3. Getting it running
4. What the repo is not
5. Reporting issues

Preflight: Python, Tensors, and What a Language Model Actually Is

1. What a Language Model Actually Is
2. How a Model Learns: The Three Moves
3. Python You Need to Read This Book
4. Numbers as Tensors
5. The Shape of the Book
6. Quick Reference: What to Do If You Get Stuck
7. What You Now Know

Project 1: The Learning Machine

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 2: Predicting The Next Character

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. Building the neural character model
6. BREAK IT
7. Optional Homework
8. Questions To Answer
9. Go Further
10. What You Now Know
11. Starting Point

Project 3: Building A Tokenizer

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 4: Attention From Scratch

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 5: Your GPT From A Blank File

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 6: From Prototype to nanoGPT

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 7: The Details That Matter

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 8: Flash Attention and Tiled Kernels

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 9: Pretraining On The Real Web

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 10: Data Curation and Contamination

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 11: Training Debugging: Spikes, NaNs, and Profiling

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 12: Distributed Training: FSDP and ZeRO (Single-Box Proxy)

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 13: Fast Inference: The KV Cache

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 14: Speculative Decoding

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 15: Grouped Query Attention

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 16: Long-Context Extension (RoPE, YaRN, NTK-Aware)

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 17: Production Serving: Continuous Batching and PagedAttention

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 18: Mixture Of Experts

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 19: Scaling Laws

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 20: Autonomous Experimentation

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 21: Fine-Tuning And Instruction Tuning

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 22: Evaluation Methodology

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 23: Reward Models And RLHF

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 24: DPO and Preference Optimization

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 25: Test-Time Reasoning (CoT, Self-Consistency, Best-of-N)

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 26: Tool Use and Function Calling

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 27: Quantization and Deployment

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 28: Retrieval-Augmented Generation

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 29: Multimodal: A Tiny Vision-Language Model

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 30: Non-Transformer Architectures (Mamba, RWKV)

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 31: Layer Freezing and Transfer

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 32: Fusing Independently Trained Specialists

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Starting Point

Project 33: The Interface Specification

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Research Anchors
11. Starting Point

Project 34: Incremental Assembly

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. What You Now Know
10. Research Anchors
11. Starting Point

Project 35: Your Architecture

1. Hook
2. The Concept
3. Why It Matters
4. The Build
5. BREAK IT
6. Optional Homework
7. Questions To Answer
8. Go Further
9. Research Anchors
10. What You Now Know
11. Where The Field Is Now
12. What To Sound Like In A Strong Interview
13. Frontier Reading Map
14. What The Book Now Gives You
15. Starting Point

Appendix A: Lecture Companions

1. How to use this appendix
2. Preflight companion: Python and tensor foundations
3. Part I companion: learning mechanics, tokenization, attention
4. Part II companion: building and training a transformer
5. Part III companion: inference, efficiency, and scaling
6. Part IV companion: post-training, alignment, deployment
7. Part V companion: transfer, modularity, interfaces, research
8. Suggested study rhythm

Appendix B: Free Resources

1. Reference architecture by topic
2. Stability rule

Appendix C: Notes, Sources, And Bibliography

1. Project Sources

Glossary

The Leanpub 60 Day 100% Happiness Guarantee

Within60 days of purchaseyou can get a100% refundon any Leanpub purchase, intwo clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay80% royaltieson purchases of**7\.99 or more**, and**80% royalties minus a 50 cent flat fee**on purchases between**0.99 and $7.98**.You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell5000 non-refunded copies of your book for 20**, you'll earn**80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

In fact, authors have earnedover $15 millionwriting, publishing and selling on Leanpub.

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don’t have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub’s ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub