@maxxfuu: Day 13/90 of Inference Engineering I built a MNIST classifier in C. Not proud of it, man enough to say it. It took me 5…

Day 13/90 of Inference Engineering I built a MNIST classifier in C. Not proud of it, man enough to say it. It took me 5 days to fully understand and get the intuition behind a matmul with 1D vectors. It’s just 3 nested for-loops, and somehow, it just didn’t click. It was not intuitive at all. Here are some things I wish I knew before starting. Practically speaking, I think sometimes drawing things out, creating different visualizations, and talking to language models to build that understanding can help out a lot, if done correctly. I spent around 6 hours just tracing the code, drawing the data flow, predicting the shapes, and understanding the math. None of which required any coding. Once I fully understood the flow of MNIST, writing the code became blazingly fast. Implementing MNIST in C took about 1 hour, understanding everything took 6 hours. Mindset-wise, I think learning about Deep Learning truly is a different game. Coming from a fullstack engineering perspective, there's almost a playbook on building robust backends and building tasteful frontends. I’ve found that diving straight into things and figuring it out along the way teaches you more than reading the documentation. However, learning Inference, CUDA, and building a simple project like MNIST just isn’t the same as fullstack engineering. There's this big hurdle of understanding why something works and building an intuition of things. Moving forward, I’m definitely spending my tokens on understanding the fundamentals instead of coding. On the technical side, here's the intuition that made the triple for-loops implementation of the matmul_forward() click! Since the matrices are in the form of a 1D vector, selecting the right cell is just figuring out which of the three indexing variables corresponds to the step, stride, and pin. This isn't formal terminology, but it just makes sense. The step is the axis you're moving on. It could span row-wise or column-wise. Which way you span is determined by the pin. The stride is how big of a jump you make on the 1D array with each step. Take a look at B[l*k + j] that's in the animation. The pin is j which pins you to one column, so the only free variable is the step, l. The iteration flows top to bottom down that column. Since B lives in a 1D array, "down one row" isn't one cell over, it's a full stride or row width of k. While which variable plays the stride, pin, and step depends entirely on how the for-loops are structured, I've been able to generalize variations of matmuls using this framework. Starting tomorrow, I'll be able to rewrite this C code in CUDA!