Tag
This paper introduces MOSAIC, a framework that jointly optimizes sparse Mixture-of-Experts model architecture and hardware systems for large-scale pretraining, showing that compute-optimal sparsity is not necessarily cluster-optimal when MFU, communication costs, and parallel layouts are considered.
This paper systematically evaluates component-wise quantization of small vision-language models on Jetson edge devices, finding that model architecture (MoE vs dense) significantly affects quantization sensitivity and that quantization errors are largely additive except along modality-alignment paths.
Hawk is a training-free framework that uses hardware-aware knowledge to improve NPU kernel generation via LLMs, raising generation accuracy from 49.4% to 80.0% and achieving up to 2.2× execution speedup over state-of-the-art baselines.
This blog post presents Gram Newton-Schulz, a hardware-aware optimization of the Newton-Schulz orthogonalization procedure used in the Muon optimizer, achieving significant speedups for training large language models while preserving model quality.
AutoMCU is a multi-agent system leveraging LLMs to automate neural network design for microcontroller units, significantly reducing customization time while ensuring feasibility under hardware constraints.
Andrew Ng has launched a new course on LLM production deployment. The free version provides access to all videos and base code. The course dives deep into LLM internals, inference optimization (such as quantization, KV Cache, Flash Attention, speculative decoding), and hardware-aware optimization. Taught by AMD's VP of Engineering, it aims to help developers transform Transformer from an academic concept into a debuggable, optimizable engineering tool.