Tag
This paper proposes a hardware-software co-design framework for N:M sparse vision Transformer inference, achieving over 2.2× latency speedup on GPUs while maintaining accuracy through a novel CUDA kernel (MD-SpMM) and a deployment-aware sparsity search.
详细介绍了针对语音克隆模型的W4A4 CUDA内核优化，通过INT4量化和融合LoRA，实现了比FP16快2.6倍的推理速度。
MIT HAN Lab proposes a method to automatically design and optimize CUDA kernels using an AI agent workflow. Through a process of task contracts, agent loops, and small-step verification, the agent can autonomously iterate and optimize within a specialized toolchain, replacing manual tuning.
面壁智能 has open-sourced ForgeTrain, a pretraining framework autonomously written by an AI Agent. It achieves 44% MFU on H100, about 10% higher than the Megatron-LM baseline, marking an iteration of AI self-evolution.
Luce Megakernel demonstrates that NVIDIA's efficiency gap is a software problem, achieving 1.8x throughput on an RTX 3090 compared to llama.cpp and matching Apple M5 Max efficiency at lower cost.
BEAM introduces binary expert activation masking for dynamic routing in Mixture-of-Experts LLMs, achieving up to 85% FLOPs reduction with minimal performance loss and up to 2.5× faster decoding.