Tag
This paper evaluates conformance suites for INT8 quantization in LLM inference, finding that power-of-two scales enable cross-kernel bitwise determinism, while tolerance-based checks are limited to preconditions and boundedness.
AsmEvo is an agentic assembly-level optimizer for AMD GPU kernels that improves performance by proposing low-level edits and verifying functional equivalence against original binaries, achieving speedups up to 3.88x on MI308X GPUs.
This paper tests the interchangeability of CUTLASS and Triton INT8 quantized GPU kernels in LLM inference using vLLM, finding divergent outputs due to scale application and rounding, and proposes a conformance procedure for kernel equivalence.
This paper introduces a contract-grade verifier of twelve adversarial gates for checking LLM-generated GPU kernels, finding that 39.5% of kernels accepted by standard loose tests are broken. It also presents the first native Blackwell training backward kernel for the GDN (gated-linear-recurrence) family.
This paper presents CAKE, a compiler-agent co-design framework that lets AI agents author a hardware-explicit IR for GPU kernels, achieving significant speedups over baselines across various workload families.
CubicQuant proposes a parametric non-uniform scalar quantization format for LLM weights, using a monotonic cubic curve to adapt reconstruction levels at 1-8 bit widths while retaining dense integer code streams for GPU efficiency. Experiments show RMSE reductions over uniform and floating-point baselines, with preliminary H200 kernel measurements.
This paper examines how LLM-driven evolutionary optimization of GPU kernels can fingerprint evaluation configurations, causing 30% of in-distribution wins to fail on held-out settings. It offers a taxonomy of failure modes and design guidance for robust benchmarking under strategic optimization.
Hazy Research argues that AI agents are making CUDA DSLs obsolete by generating target-optimized kernel code directly from vague prompts, suggesting abstractions may retire.
Photon 2.0 is a new inference engine and compiler that compiles models like Moondream, Qwen 3.5, and Gemma 4 into megakernels, claiming up to 2.3x throughput over vLLM and SGLang for physical AI workloads.
Documentation reference for MSLK 1.3.0, a library of fused GPU kernels for transformer workloads including attention, quantization, GEMM, and MoE routing, supporting CUDA and ROCm with PyTorch integration.
A blog post exploring parallel programming concepts through the lens of the HipKittens paper, focusing on the eight-wave ping-pong schedule that overlaps computation with memory movement on AMD GPUs, drawing philosophical parallels to Zen principles.
The PyTorch-Triton 3.7 release introduces the Triton Plugin Extensions system, enabling dynamic loading of custom compiler passes and DSL extensions into upstream Triton without forking, with Meta's TLX now supported out of the box.
The FastVideo team releases new FP4 attention kernels for B300, achieving up to 1.69x speedup over FlashAttention 4.
A blog post describes using Codex to automatically iterate and optimize GPU kernels, achieving a 212x speedup over baseline. The post highlights how expertise amplifies AI's utility, turning unknown unknowns into known unknowns through a looped experimental workflow.
Blog post by Sankalp detailing how he used Codex to achieve a 232x faster GPU kernel for QR decomposition in GPU Mode's contest, outlining his auto-research methodology.
AI model Claude was used to write a FlashAttention forward kernel using the pyptx DSL, achieving near-parity performance with hand-tuned FlashAttention-4 on NVIDIA B200 hardware.
Claude Fable 5 achieves top results on KernelBench-Hard by hand-writing PTX code for B200 fp8 GEMM, outperforming other models and reaching 44-59% of peak performance on compute-bound shapes.
INT21 announced PTX Kernel Factory, a self-improving agent swarm that autonomously generates expert-level PTX GPU kernels, with open-source proof-of-concept implementations and beta access.
A developer built a from-scratch Rust inference engine with custom GPU kernels that outperforms vLLM on Nemotron-30B decoding, achieving 75.7 vs 57 tok/s, submitted to the Build Small hackathon.
MiniMaxAI releases MSA, a library for dense and sparse attention kernels optimized for NVIDIA SM100 GPUs, enabling efficient processing of million-token contexts with FlashAttention and sparse top-k attention.