npu

Reverse-engineered the RK3588 NPU and built an open compiler and runtime to run GPT-2 and SigLIP from PyTorch, ONNX, and JAX without vendor SDK.

Introduces AgenticCANN, a knowledge-augmented agentic evolution framework for automated Ascend C operator generation on Huawei NPUs, achieving significant speedups and improved feasibility across multiple operator patterns.

Microsoft announces general availability of Video Super Resolution for NPU and CPU, using on-device AI to enhance lower-resolution video quality.

This paper presents SLAI T-Rex, a full-parameter post-training optimization framework for trillion-parameter MoE models on Ascend NPU SuperPOD, achieving 34.22% MFU and outperforming GPT-5.4-Mini on Operations Research tasks by 3.98 percentage points.

This literature review analyzes performance efficiency trade-offs for LLM inference on mobile devices, NPUs, and GPUs under sustained load, focusing on benchmarking LLMs on phones.

Hawk is a training-free framework that uses hardware-aware knowledge to improve NPU kernel generation via LLMs, raising generation accuracy from 49.4% to 80.0% and achieving up to 2.2× execution speedup over state-of-the-art baselines.

The NPU on AMD Strix Halo devices is now usable for AI inference, enabling hybrid mode that combines NPU and iGPU for faster prompt processing. Tools like Lemonade and AMD's ROCm software make this possible.

Reverse engineering the Qualcomm NPU compiler reveals undocumented VTCM memory management, MILP-based placement, automatic precision alteration, and a hidden analytical simulator (Hextimate) for edge deployment optimization.

Gemma 4 E2B achieves 1.3x faster prefill and 2.8x better performance-per-watt on Intel AI PCs using OpenVINO with LiteRT NPU support, enabling efficient background LLM tasks.

xdna-top is a terminal monitor that shows both NPU and iGPU activity on Ryzen AI Max/Strix Halo systems, providing an honest view of NPU counter deltas instead of fake utilization percentages.

This paper presents the first end-to-end RAG pipeline running entirely on a mobile NPU (Qualcomm Hexagon on Snapdragon X Elite), achieving up to 18x faster LLM prefilling and 4x lower energy vs. CPU, with no quality regression.

Sipeed's new K3 RISC-V single-board computers feature 32GB LPDDR5 and a 60 TOPS NPU, enabling local inference of large language models at up to 15 tokens per second.

A technical deep-dive into achieving peak TOPS performance on the AMD Ryzen AI 7 350 NPU, comparing it to Xilinx AIE-ML v2 AI engines and explaining the hardware architecture for matrix multiplication workloads.

A tweet highlights how coding agents can clarify complex ideas, using GPU vs NPU memory competition on devices as an example demonstrated through code.