MNIS Fasion : 17.6 sec for 92.32% test accuracy on the official 10k set (trained on the full 60k) on an AMD Ryzen 7 PRO 8700G (8C/16T)
Summary
A developer reports achieving 92.32% accuracy on the MNIST dataset in 17.6 seconds using a binary XNOR ensemble model on an AMD Ryzen CPU, demonstrating significant speed improvements over traditional floating-point CNNs.
Similar Articles
Spent two weeks on a kernel that benchmarked 29x faster. End to end it's maybe 6-10%, and it's not even wired in yet.
The author optimized a matmul kernel for BitNet's ternary models on CPU, achieving 29x speedup in isolation, but found that the model is memory-bound, resulting in only 6-10% end-to-end gain. The inference engine is available as open-source.
Building a Fast Multilingual OCR Model with Synthetic Data
NVIDIA introduces Nemotron OCR v2, a fast multilingual OCR model built using synthetic data generation. The model achieves 34.7 pages/second on a single A100 GPU by using a unified FOTS-based architecture with feature reuse across detection, recognition, and relational components.
@rohanpaul_ai: I had to test it myself to believe this unreal inference speed. 3,000 tokens/s for 1 user on standard datacenter GPUs. …
Kog AI achieves 3,000 tokens/s inference speed on 8× AMD MI300X GPUs and 2,100 on 8× NVIDIA H200, leveraging a hidden efficiency gap in GPU token generation.
Vendor-agnostic ML inference on production edge devices [R]
Describes using ncnn's Vulkan backend for vendor-agnostic ML inference on production edge devices, achieving 10x speedup over CPU ONNX for face detection and embedding models.
40% speedup of MoE training with faster megakernel, by cursor, of all people (for B200s)
Cursor open-sources Mixture-of-Kittens (MoK), a deterministic MoE training megakernel for NVIDIA Blackwell GPUs that fuses computation and communication, delivering up to 2.37x speedup over baseline implementations.