kernels

A tweet explains the core components of inference engineering: loadable encodings, backend operations, and native hardware arithmetic, with a discussion on kernels and GPU efficiency.

This article provides an introduction to GPU kernels in the context of AI inference and performance engineering, explaining their definition, how they are used, and the advantages of custom kernels for optimization.

Hugging Face releases 207 WebGPU kernels for local AI inference in browsers, along with a JavaScript loader library and a benchmarking tool called Fleet.

Introduces v100-skinny, a custom kernel library enabling fast NVFP4 inference on V100 (sm70) GPUs, achieving 366 t/s for Qwen3.6 27B in best-case extraction, with lower speeds for structured generation and code.

Y Combinator hosted a Paper Club where researchers presented innovations in multi-GPU kernel optimization, including ParallelKittens, a CUDA framework that simplifies development of overlapped multi-GPU kernels and achieves significant speedups across workloads.

QuixiAI rebrands ThunderKittens and ThunderMittens into QuixiCore-CUDA and QuixiCore-Metal, creating a unified family of cross-platform kernels for AI workloads.

A highly recommended 4.5-hour GPU programming lesson on CUDA and ThunderKittens by Ben Spector, offering an in-depth, behind-the-scenes look at kernel optimization.

QuixiAI ported ThunderKittens to Metal, enabling kernel support on MPS and MLX for training models on Mac.

TokenSpeed-Kernel is a portable, high-performance kernel system for LLM inference that enables zero vendor-specific model code and supports multiple GPU architectures, achieving up to 3.6x higher throughput on AMD MI355X.

A blog post describing a tiny compiler that demonstrates how to lower data-parallel kernels by converting for loops into vectorized loops with lanes and masks, implemented in ~180 lines of Python.

Downloaded and compiled 16,459 CUDA kernels from SakanaAI's open-source dataset, benchmarking them for performance.

Ahmad Osman announces four upcoming articles covering LLM decoding/prefilling, LLM kernels, and hardware comparisons (CPUs, GPUs, Tenstorrent, Apple Silicon), building on his existing 'Local LLMs From Zero to Hero' series.

The tweet explains that running AI models is really about running optimized kernels, and that inference engines and their kernel implementations are critical for performance, not just the model or hardware.

Author describes building FlashRT, a CUDA-first inference runtime that rewrites model inference paths with C++/CUDA kernels to address bottlenecks beyond GEMM for small-batch/realtime workloads, achieving significant latency improvements on Jetson Thor and RTX 5090. The article discusses lessons on precision (FP8 helpful, FP4 mixed) and the need to bypass generic runtimes for realtime inference.

Hugging Face's kernels project is expanding and seeking contributors for agentic kernel development to provide real optimization value to models.

Deepseek open-sourced DeepEP V2 and TileKernels, new GPU kernel libraries aimed at accelerating AI workloads.