webgpu

Bonsai 27B is a 1-bit dense large language model that can run locally in a browser using custom WebGPU kernels, enabling efficient on-device inference.

Bonsai 27B achieves 93% size reduction via 1-bit quantization while retaining 90% intelligence, enabling local browser inference with custom WebGPU kernels.

Google announces LiteRT.js, a high-performance JavaScript runtime for running AI models directly in the browser using WebAssembly and hardware acceleration, as an evolution from TensorFlow.js.

A web version of Rome Total War was created using Fable and Codex Rust, compiled to WASM and rendered with WebGPU, supporting ~20k units at decent FPS.

A novel 4D Gaussian splat format called .splat4d with tunable error bounds, offering 16-58x compression over raw splats and native HTTP Range streaming for dynamic scenes, with code and demo available.

Frontier models have become so proficient at generating web apps that the author now benchmarks them on building a WebGPU water renderer from scratch, comparing Opus-4.8 and Fable-5 orchestrator with GPT-5.5 implementer.

Xenova used Fable 5 to write optimized kernels achieving 255 tokens per second for Gemma 4 on WebGPU with M4, demonstrating agentic kernel optimization for on-device inference.

A modular, high-performance Rust implementation of GraphRAG (Graph-based Retrieval Augmented Generation) with support for WebGPU acceleration and three deployment architectures: server-only, WASM-only (client-side), and hybrid.

Open source AI inference reaches 300 tok/s on mobile, with a WebGPU framework pushing Liquid AI's LFM2.5 230M to 1,400 tok/s in browser.

Kuma is a compiler/runtime that compiles exported PyTorch models into self-contained WebGPU executables, enabling direct browser inference without Python or server dependencies.

LFM2.5 230M model achieves 1,400 tokens per second in-browser using custom WebGPU kernels, demonstrating efficient local inference.

Eyal Toledano built an LLM inference engine using pure WebGPU/WGSL, running on-device in browser and Node without API keys, and is seeking peer review.

Stable Diffusion XL (SDXL) can now run locally in the browser using WebGPU, enabling high-quality AI image generation directly on-device with open-source code.

This guest post explores the proposed Cross-Origin Storage API to improve caching of AI model resources in Transformers.js, enabling efficient reuse across origins while maintaining privacy and integrity for in-browser inference.

Simon Willison ported the Moebius 0.2B image inpainting model to run in the browser using WebGPU and ONNX Runtime, assisted by Claude Code. The resulting demo allows users to upload images and remove objects via inpainting.

The team at Vercel has significantly optimized the performance of their homepage, using techniques like WebGPU shaders and scrutinizing every frame, and they plan to share the lessons learned.

The article introduces Xenova's open-sourcing of 27 custom WebGPU kernels, enabling Gemma 4 to run fully offline and locally in the browser at 255 tok/s, and discusses advantages like privacy and offline use. It also mentions FLUX.2's 3D generation capability.

Gemma 4 is demonstrated running in-browser via WebGPU at 255 tokens per second, using kernels generated by Fable 5, showcasing efficient on-device inference.

A developer built a self-contained browser-use agent that runs entirely in WASM/WebGPU at zero server cost, enabling full webpage control via natural language prompts.

Introduces llm.istanbul, a WebGPU LLM workbench that lets you train small models, train tokenizers, and generate text entirely in the browser, no server required, fully local.