@AYi_AInotes: Fellow AI learners, stop just tuning packages. Writing a large language model from scratch is now easier than mastering PyTorch. Look at these two 19-year-old undergrads—they built a complete ML framework and a large model from scratch in four months. Two 19-year-old Waterloo undergrads, who knew nothing about ML four months ago, now…

X AI KOLs Timeline Tools
from-scratch machine-learning-framework rust typescript llm open-source education

Summary

Two 19-year-old undergraduates from the University of Waterloo built a complete machine learning framework and a 12M-parameter large language model from scratch in four months, using Rust and TypeScript, emphasizing understanding from the bottom up rather than just using packages.

Fellow AI learners, stop obsessing over tuning packages. Writing a large language model from scratch is now easier than mastering PyTorch. Look at these two 19-year-old undergrads. In four months, they built a complete machine learning framework and a large model from scratch. Two 19-year-old undergrads from the University of Waterloo—four months ago they knew nothing about ML. Now they’ve built a complete ML framework from scratch and trained a 12M-parameter language model that runs directly in your browser. It shows that most people learn AI in the wrong order. Everyone first learns how to use packages, then studies model architectures, and only finally dares to touch a little bit of the underlying implementation. These two did the reverse. They started with the calculus of gradient descent, wrote an automatic differentiation engine, then a BPE tokenizer, then built the Transformer layer by layer. In the end, they understand it deeper than most engineers who have been tuning packages for three years. Another point few people notice: The real barrier in AI is no longer the model architecture. The Transformer paper came out in 2017. The math behind each layer is public; any college student can recite it. The hard part is what papers never mention: how to write Flash Attention to reduce memory usage by an order of magnitude, how to combine bias, activation, and normalization into a single kernel, how to dump the entire dataset into GPU memory at once so training has zero data transfer. These are the key factors that determine whether you can actually run the model. Big companies are also making simple things complicated. PyTorch piles on countless layers of abstraction to handle all cases, and in the end you don't even know what your one line of code is actually doing on the GPU ಠ_ಠ Meanwhile, open source is making complicated things simple. These two wrote the backend in Rust and the frontend in TypeScript. The underlying part runs blazingly fast, while the top-level usage is as simple as PyTorch. The entire codebase is under 10,000 lines and cleaner than any mainstream framework. Can you believe it? LOL This is the best AI education: not reading a hundred papers or binge-watching ten online courses, but starting from first principles and building your own wheel. Once you’ve written automatic differentiation by hand, you’ll never see AI as some mysterious black magic again. It’s just a bunch of carefully optimized matrix operations, plus some smart data flow management. That’s it. The most astonishing part of this video isn’t how talented those two students are, but that it proves something only big corporate labs could do two years ago is now a hobby project for ordinary people. Lastly, I want to say: We’re in the best era, the era of tech equality. The democratization of AI is not big companies handing out API keys; any of us, if willing to spend four months, can build our own large language model on our personal computer. If you want to try it, go straight to their GitHub repo, run `npm install`, and it will work. I’ll put the browser demo link in the comments.
Original Article
View Cached Full Text

Cached at: 05/16/26, 07:12 AM

Brothers learning AI, stop obsessing over using packages. Writing a large language model from scratch is now easier than mastering PyTorch.

Take a look at these two 19-year-old undergrads. In four months, they built a complete machine learning framework and a large language model from scratch.

Two 19-year-old undergraduate students from the University of Waterloo knew nothing about machine learning four months ago. Now they have built a complete ML framework from scratch and trained a 12-million-parameter LLM that runs directly in your browser.

This shows most people have the order of learning AI completely backwards. Everyone learns to use packages first, then model architecture, and only then dare to touch the fundamentals.

These two did the opposite. They started with the calculus of gradient descent, wrote an automatic differentiation engine, then a BPE tokenizer, then built the Transformer layer by layer. In the end, they have a far deeper understanding than most engineers who have been tuning packages for three years.

There’s another point few people notice: the real barrier to AI is no longer model architecture.

The Transformer paper was published in 2017. The math for every layer is public. Any college student can recite it.

What’s truly hard are the things papers never mention: how to write Flash Attention to reduce memory usage by an order of magnitude, how to fuse bias + activation + normalization into a single kernel, how to feed the entire dataset into the GPU at once so training never has to transfer data. These are the things that determine whether you can actually run a model.

Big companies are also making simple things complicated. To maintain compatibility with every possible scenario, PyTorch piles on layers of abstraction until you have no idea what that one line of code is actually doing on the GPU ಠ_ಠ

Meanwhile, open source is making complicated things simple. These two wrote the backend in Rust and the frontend in TypeScript. The low level runs fast, the high level is as easy as PyTorch, and the entire codebase is under 10,000 lines — cleaner than any mainstream framework. Can you handle that? 😂

This is the best AI education: not reading a hundred papers or taking ten online courses, but building a wheel from first principles.

Once you’ve written an automatic differentiation engine by hand, you’ll never think AI is some mysterious black magic again.

It’s just a bunch of carefully optimized matrix operations, plus a bit of clever dataflow management. That’s all.

I think the most terrifying part of this story isn’t how impressive those two students are. It’s that it proves something that only big company labs could do two years ago is now a hobby project for ordinary people.

Finally, I want to say: we live in the best era — an era of technology democratization. The democratization of AI isn’t big companies handing out API keys. Any of us, if willing to spend four months, can build our own large language model on our own computers.

If you want to give it a try, go to their GitHub repository and run npm install. I’ll put the link to the browser demo in the comments.

Aadi Kulshrestha (@MankyDankyBanky):
I trained a 12M parameter LLM on my own ML framework using a Rust backend and CUDA kernels for flash attention, AdamW, and more.
Wrote the full transformer architecture, and BPE tokenizer from scratch.
The framework features:

  • Custom CUDA kernels (Flash Attention, fused

Similar Articles

@NFTCPS: Brothers, doing AI without large models is like doing nothing! Today I have to recommend an open-source masterpiece 'Foundations of LLMs' to you. Don't wait, just read it! This book doesn't beat around the bush—it goes deep from the start! From getting started with large language models to architectural evolution, and then it breaks down Prompt engineering, parameter-efficient fine-tuning, model editing, RAG (Retrieval-Augmented Generation) and other hardcore techniques in one go—a one-stop service.

X AI KOLs Timeline

This article promotes the open-source book 'Foundations of LLMs', which systematically explains knowledge about large language models, and introduces the multi-agent development framework Agent-Kernel.

@Xx15573208: I've read many articles about Transformers and understand the theory, but when I actually sit down to write code, I have no idea where to start. LLMs-from-scratch is specifically designed to solve this problem: it accompanies the book "Build a Large Language Model" and guides you through implementing GPT from scratch using PyTorch…

X AI KOLs Timeline

LLMs-from-scratch is a GitHub repository that accompanies the book "Build a Large Language Model," providing complete code to implement GPT from scratch with PyTorch, covering the full pipeline including pretraining, fine-tuning, and RLHF. It has gained 93K+ stars and is ideal for developers who want to deeply understand the principles behind large language models.

@NFTCPS: You keep talking about AI, but can't even explain what a Transformer is? There's a repo that goes all out — builds a GPT from scratch without using any high-level libraries. It lays out exactly how Attention, Multi-Head, Feed-Forward, Embedding, Residual connections, and Layer Norm are pieced together. And it's not just the model; the entire pipeline is covered…

X AI KOLs Timeline

A GitHub open-source project that implements the complete GPT training pipeline from scratch, including data preprocessing, pretraining, SFT, and RLHF post-training, all based on native PyTorch. Ideal for developers who want to deeply understand the Transformer architecture.

@tanzhengmc97: https://x.com/tanzhengmc97/status/2066531753762656730

X AI KOLs Timeline

Explained the operating principles of large models in easy-to-understand language, including word vectors, Transformer attention mechanism, next-word prediction training, and emergent abilities, suitable for beginners to understand basic AI concepts.

@XAMTO_AI: Stop bookmarking those flashy but useless AI tutorials. This 'Hands-On Large Models' is what you really need—open source, free, and code that runs. The book covers 12 chapters, guiding you step by step through the complete workflow of deploying large models: ① Language Model Basics ② Prompt Engineering ③ Semantic Search ④ Model Fine-Tuning ⑤ Multimodal…

X AI KOLs Timeline

Recommend an open-source free tutorial 'Hands-On Large Models', covering 12 chapters including language model basics, prompt engineering, semantic search, model fine-tuning, multimodal applications, etc. All code can be run directly in Colab.