kernel-optimization

This paper presents CAKE, a compiler-agent co-design framework that lets AI agents author a hardware-explicit IR for GPU kernels, achieving significant speedups over baselines across various workload families.

Kernel Forge is an open-source agent harness that uses LLMs and Monte Carlo Tree Search to automatically generate and optimize CUDA kernels for any unmodified PyTorch model, achieving up to 2.83× speedup on softmax in Gemma 4 E2B.

The author optimized a matmul kernel for BitNet's ternary models on CPU, achieving 29x speedup in isolation, but found that the model is memory-bound, resulting in only 6-10% end-to-end gain. The inference engine is available as open-source.

SonicSampler presents a unified suite of tile-aware Triton kernels that vertically fuse the entire LLM sampling pipeline, supporting dynamic per-request behaviors and speculative verification, achieving up to 16x speedup over state-of-the-art baselines.

JAXBench is a new benchmark suite of 50 JAX workloads for evaluating AI-generated kernel optimization on Google Cloud TPUs, with hand-tuned baselines and an agent evaluation harness. The paper finds that conditioning on curated TPU documentation significantly improves correctness and speedup, with Autocomp beam-search achieving up to 1.6x geomean speedup over XLA on hand-tuned kernels.

The author, as part of a 100-part GPU learning series, reviews a GTC 2020 lecture by Andrew Kerr on developing high-performance CUDA kernels for Tensor Cores on NVIDIA A100, discussing techniques and the trade-off between raw CUDA and CUTLASS.

Sharon Zhou proposes a vendor-agnostic, kernel-level GPU performance benchmark to help AI agents optimize compute efficiency for frontier models, and highlights AMD's AgentKernelArena as a starting point.

The user shares plans to use Sol for optimizing MTP support in llama.cpp, standardizing chat templates, reviving an abandoned whisper project, finishing a personal finance app, and kernel optimization.

Xenova used Fable 5 to write optimized kernels achieving 255 tokens per second for Gemma 4 on WebGPU with M4, demonstrating agentic kernel optimization for on-device inference.

A PyTorch core engineer at Meta demonstrated a fast CUDA kernel optimization loop that outperforms expensive bootcamps, with the winning code merged into PyTorch via the KernelBot competition.

Linux 7.2 kernel merges a performance optimization for anonymous/unnamed pipes, improving throughput by 6-48% and reducing latency by 17-33% by pre-allocating pages outside of mutex lock to avoid contention.

INT21 announced PTX Kernel Factory, a self-improving agent swarm that autonomously generates expert-level PTX GPU kernels, with open-source proof-of-concept implementations and beta access.

A tweet discussing two agentic GPU kernel optimization systems: Auto GPU Kernel by @dogacel0 and Kernel Design Agents from @songhan_mit's lab, both winners at the MLSys Sparse Attention FlashInfer competition. The thread highlights different approaches using subagents and Claude skills for GPU programming.

Explains that inference kernels differ from training, with Flash Attention 4 focusing on changing parallelism across KV and supporting small irregular loads.

A blog post details contributions to FlashAttention-4 to improve its performance for large language model inference, especially for decode-heavy workloads, by adjusting parallelism strategies and supporting irregular memory accesses.

This paper proposes using language models as selective surrogates to optimize GPU kernel runtime, demonstrating a novel approach to performance forecasting.

Alibaba's Qwen3.7-Max model autonomously optimized a production kernel on unfamiliar T-Head PPU hardware over 35 hours, making 1,158 tool calls and achieving a 10x speedup, demonstrating sustained autonomous agentic behavior without human guidance.

A lecture on LLM deployment techniques covering AWQ, vLLM, FlashAttention, quantization, and activation smoothing for efficient serving.

Metal-Sci introduces a 10-task benchmark for optimizing scientific computing kernels on Apple Silicon, paired with an evolutionary search framework driven by large language models. The study evaluates models like Claude Opus 4.7, Gemini 3.1 Pro, and GPT 5.5, demonstrating significant speedups while using out-of-distribution testing to catch silent performance regressions.

Opus 4.7 auto-generated a custom WebGPU kernel that accelerates Qwen3.5 inference up to 13× via fused LinearAttention, now shipping in Transformers.js v4.2.0.