@xenovacom: We've just made a breakthrough: the first true omni model running 100% locally in a browser... fully end-to-end, with n…
Summary
Xenova 宣布突破：首个 100% 本地在浏览器中运行的真正全模态模型 LFM3-Omni-2B，借助自定义 WebGPU kernels 实现约 280 tok/s，支持视觉、音频和文本输入并生成交错的文本与音频输出。
View Cached Full Text
Cached at: 11/01/26, 01:40 AM
We’ve just made a breakthrough: the first true omni model running 100% locally in a browser… fully end-to-end, with no separate ASR or TTS models needed.
With custom WebGPU kernels, we’re able to run LFM3-Omni-2B at ~280 tok/s on my M4 Max, accepting visual, audio, or text and producing interleaved text and audio tokens. Just 90 ms to the first token and 140 ms to the first audio.
The future of on-device multimodal AI agents is going to be wild.
Similar Articles
LFM3-Omni-2B running locally in-browser on WebGPU
LiquidAI 的 LFM3-Omni-2B 成为首个可在浏览器中通过 WebGPU 100% 本地运行的全模态模型，无需独立 ASR/TTS，在 M4 Max 上实现约 280 token/s、90ms 首 token 延迟。
@hank_aibtc: Amazing! Running Gemma 4 in the browser, on par with ChatGPT?! Completely zero server, zero data upload, offline, pure WebGPU local inference! Xenova has open-sourced all 27 custom WebGPU kernels written by Fable 5: - Gemma 4 E2B (2.3B parameters...)
The article introduces Xenova's open-sourcing of 27 custom WebGPU kernels, enabling Gemma 4 to run fully offline and locally in the browser at 255 tok/s, and discusses advantages like privacy and offline use. It also mentions FLUX.2's 3D generation capability.
LFM2.5 230M running in-browser at 1,400 tok/s using custom WebGPU kernels
LFM2.5 230M model achieves 1,400 tokens per second in-browser using custom WebGPU kernels, demonstrating efficient local inference.
@loktar00: Can you believe this is NOT an image model, this is running locally in my house on my 3090s, GLM 5.3 Flash. Absolutely …
开发者 loktar00 展示 GLM 5.3 Flash 在本地 RTX 3090 显卡上通过 JavaScript 直接生成图像（而非专门的图像模型），展示了开源模型在消费级硬件上的强大本地推理能力。
An 80B MoE (Qwen3-Next, 48 GB GGUF) answering in a Chrome tab: llama.cpp's WebGPU backend, experts paged in from disk
作者将上周的 MoE 专家按需从磁盘流式加载的方案接入 llama.cpp 的 WebGPU 后端，使 80B MoE 模型（Qwen3-Next，48GB GGUF）也能在 Chrome 标签页中运行，无需为每个模型手写引擎。该方案新增了 ggml-webgpu 的缓冲类型、slot 映射的 MUL_MAT_ID 着色器和驻留逻辑，实测在 M4 Pro MacBook 上以 2.9 tok/s 解码 80B 模型，较小模型可达 22 tok/s。