Tag
A discussion questioning the common interpretation of premature optimization in software development, referencing insights from performance expert Casey Muratori.
The article highlights how LLMs make it easy to game performance benchmarks, leading to deceptive software optimization claims, as shown with a regex engine that overfits to benchmarks.
The article argues that the rapid decrease in AI inference costs is driven by software optimizations rather than hardware improvements, and that open-weight models running on consumer GPUs are becoming increasingly competitive with frontier models.
Luce Megakernel demonstrates that NVIDIA's efficiency gap is a software problem, achieving 1.8x throughput on an RTX 3090 compared to llama.cpp and matching Apple M5 Max efficiency at lower cost.