Tag
OpenBMB's ForgeKernel agent autonomously optimized a GPU kernel against NVIDIA's FlashMLA black-box baseline over 22 hours, achieving a 9.8% speedup on B200, and is open-sourcing 27 optimized operators along with the ForgeKernel harness.
AMD is pouring substantial engineering and compute resources into vLLM to bypass NVIDIA’s CUDA moat, drive the standardization of inference software, and break hardware lock-in—benefiting the entire ecosystem while yielding asymmetric returns for the two companies.
Berth is a lightweight Go-based GPU resource manager that lets CUDA workloads like llama.cpp and ComfyUI share VRAM via leases, priorities, and a queue, with model parking, CUDA checkpointing, and an OpenAI-compatible gateway. It is currently in alpha (v0.4.1) with Linux NVIDIA support and experimental AMD/Windows support.
qwenGPU is an experimental Qwen3.5-2B variant that embeds a SIMT GPU within its weights, using Gated DeltaNet layers as registers/memory and running real CUDA kernels in its forward passes.
A free, open-source structured digital notebook called 'Before GPU Engineering' with 56 chapters that teaches GPU architecture and performance engineering from the ground up, covering warps, SIMT, memory hierarchies, kernel optimization, and profiling for AI/ML engineers.
作者发布了 Betelgeuse（bel-cli / bel-server），一个 llama.cpp 分叉，通过 VRAM 槽位池、CUDA 门铃内核和 io_uring 分页器，在 RTX 3060 12GB + 16GB DDR4 上以 100% 位精确方式将 Qwen 3.8 Flash Next（68GB MoE）的推理速度从 1.47 tok/s 提升到 22 tok/s（峰值 48 tok/s）。
NInfer v0.3.0, an inference engine for running Qwen3.6/8-27B on a modded 22 GB RTX 2080 Ti, adds native Windows support and prebuilt releases, cutting binaries to 0.28 GB while delivering up to 47.5 tok/s decode with MTP speculative decoding.
OpenJet 的新版本在单张 RTX 3090 上通过自定义 CUDA megakernel 引擎并行服务最多 4 个编码代理，总吞吐达约 290 tok/s，是 llama-server --parallel 的 3-4 倍。核心思路是每步将所有活跃代理的工作堆叠成一个批处理，权重只读一次并用张量核同时处理所有行。
Practical notes from Session 1 of a GPU kernel engineering series, covering CUDA 13's GPU compute-capability cutoff, the driver/toolkit/PyTorch-runtime split, async GPU timing pitfalls, and fixing Nsight Compute's ERR_NVGPUCTRPERM profiler counter permissions error.
A Linux kernel module hack reportedly unlocks 1.9x faster BF16 tensor math on NVIDIA RTX 3090 cards, boosting BF16 TFLOPS from 82 to 155 and cuBLAS GEMM performance, since NVIDIA ships the cards with fp32-accumulate tensor math at half rate.
Arch Linux package maintainer Josie documents getting Arch Linux ARM working on the ASUS ProArt P16 RTX Spark laptop (128 GB RAM, NVIDIA N1X GPU) that only officially supports Windows, releasing build scripts, ACPI patches, and kernel fixes as a bootable USB setup on GitHub.
A user reports GPU hangs with NVIDIA Xid 8 on an RTX 5090 Laptop GPU under Linux when running local LLMs, pointing to a llama.cpp issue where disabling CUDA graphs (GGML_CUDA_DISABLE_GRAPHS=1) serves as a performance-sapping workaround with no fix yet.
作者发布了一个免费、代码优先的 GPU 内核工程系列课程（共 44 节）的介绍与路线图视频，从朴素 GEMM 讲到 FlashAttention，覆盖 Hopper 与 Blackwell 架构，并以 roofline 模型和数据移动为核心教学理念。
A free "GPU Kernel Engineering" course series of 44 lessons has been released, starting from CPU parallel computing and CUDA basics, and progressively diving into memory optimization, GEMM, Tensor Cores, Flash Attention, and all the way to cutting-edge hardware like H100/Blackwell and AI-written kernels.
Stealthium 介绍 Rank-One Model Editing (ROME) 攻击——恶意 Python 模块可在 GPU 内存中改写模型权重而不触碰磁盘，并展示其运行时传感器如何通过追踪 CUDA 内核调用与 Python 调用栈来检测此类篡改。
斯坦福 CS336（2026 春）第 6 讲的笔记，讲解如何通过减少 GPU 内存往返来优化内核：对比 softmax 多次遍历与融合行内核的内存流量，介绍 Triton 的块级编程模型，并强调先用 CUDA event 测量基线、再做内核融合与分块优化。
PyTorch 联合主席 Chris Hoge（NVIDIA）推荐了 PyTorchCon 北美 2026 的五场重点演讲，涵盖 PyTorch 生态系统 Landscape、MoE 专家并行通信、动态形状 CUDA 图、Symmetric Memory 融合内核以及数据加载器扩展等内容。
llama.cpp の CUDA バックエンドにおいて、低ビット型（low-bit types）向けの MMVQ（マトリックス・ベクトル量子化積）演算を高速化する GitHub プルリクエスト (#30517) が公開された。量子化されたモデルの GPU 推論性能の改善を目指すもの。
介绍 SentryWard：一个开源的 GPU kernel 形式化验证工具，可证明同步安全、无数据竞争与资源生命周期正确性，并通过系统性移除冗余同步使 NVIDIA B200 上的 kernel 最快提速 1.4 倍。
Drama3 模型现已支持在 audio.cpp 中运行，在 RTX 5090 上全速运行时最快可达实时速度的 5 倍，或在低于 8 GB 显存下以约 2 倍实时速度运行，仍需进一步优化与修正。