Tag
OpenAI and Anthropic claim their AI systems have cracked a 27-year-old open problem in computer science, marking a major milestone for AI-driven mathematical discovery.
斯坦福 CS336(2026 春)第 6 讲的笔记,讲解如何通过减少 GPU 内存往返来优化内核:对比 softmax 多次遍历与融合行内核的内存流量,介绍 Triton 的块级编程模型,并强调先用 CUDA event 测量基线、再做内核融合与分块优化。
本文提出代数整数量化(AIQ)作为大语言模型推理的有损压缩格式,利用代数整数可实现无误差乘加运算的特性构建矩阵乘法引擎,在等比特预算下达到与业界标准 MX 格式竞争的精度。
An internal Anthropic Claude model discovered the first truly subquadratic 3SUM algorithm (O(n^1.9992)) and truly subcubic APSP algorithm (O(n^2.9995)), refuting the long-standing 3SUM and APSP hypotheses in fine-grained complexity theory. The result, formalized in a paper by Josh Alman and Virginia Vassilevska Williams, stems from a new thin matrix multiplication algorithm and was verified using Lean 4.
The paper investigates replacing matrix multiplication in Transformer layers with an associative algebra product to reduce arithmetic cost while retaining parameters, demonstrating feasibility with improved throughput but some performance trade-offs.
This update to llama.cpp introduces a Vulkan-based implementation of int8 cooperative matrix multiplication for AMD RDNA3 and RDNA4 GPUs, enhancing inference performance on these hardware platforms.
The article describes experiments on split-K matrix multiplications, revealing that answer variability depends on block layouts and split counts, with tests on a B200 GPU showing output changes under different conditions.
The paper proposes improvements to the optimization problem in combination loss analysis using modern techniques and AlphaEvolve, yielding an improved upper bound on the matrix multiplication exponent.
This article provides a step-by-step guide to calculating the Discrete Fourier Transform (DFT) by hand, demonstrating that it involves matrix multiplications similar to those in deep neural networks.
A walkthrough of backpropagation by hand through a 3-layer network using matrix multiplication, showing all 11 steps from gradients to weight updates.
A systematic exploration of FP32 matrix multiplication optimization on AMD Zen 3, achieving 85.30 GFLOPS (63.5% of theoretical peak) using AVX2/FMA intrinsics, surpassing naive implementation by 56.5x and matching optimized libraries.
This article explains how systolic arrays handle over 95% of AI chip compute, detailing their design, modes of operation, and why they are efficient for matrix multiplication.
The article analyzes the new ACE specification from the x86 Ecosystem Advisory Group, which extends Intel's AMX for AI matrix multiplication with fixed tile sizes and outer product instructions, comparing it to Arm's SME.
Google's TPU uses the systolic array architecture from 1978 to accelerate matrix multiplication with less memory movement. The post shares links to the original paper and TPU design, and suggests building a small-scale version on an FPGA.
The x86 Ecosystem Advisory Group has published the AI Compute Extensions (ACE) specification, defining new x86 instructions and register state for accelerating matrix multiplication and reduced precision data formats in machine learning workloads.
The author details the process of optimizing custom matrix multiplication kernels in Swift to train a Large Language Model on Apple Silicon, aiming to outperform C implementations by leveraging CPU, SIMD, AMX, and GPU capabilities.
A technical deep-dive into achieving peak TOPS performance on the AMD Ryzen AI 7 350 NPU, comparing it to Xilinx AIE-ML v2 AI engines and explaining the hardware architecture for matrix multiplication workloads.