How fast can I get a voice assistant to respond without a GPU? Qwen3-ASR and Kokoro-TTS ONNX on CPU.
Summary
Explores the performance of running a voice assistant with Qwen3-ASR and Kokoro-TTS ONNX models on CPU, measuring response times without a GPU.
Similar Articles
Local, CPU-Friendly, High-Quality TTS (Text-to-Speech) with Kokoro
This article introduces Kokoro, a lightweight 82M-parameter text-to-speech model that runs locally on CPU, providing high-quality speech synthesis across multiple languages while preserving privacy. It explains how to set up Kokoro via a Docker container with an OpenAI-compatible API for easy integration.
CPU TTS benchmark with UTMOS MOS scoring: Kokoro, Supertonic, Inflect-Nano, and Kyutai's new Pocket TTS [P]
A CPU TTS benchmark compares Kokoro, Supertonic, Inflect-Nano, and Kyutai's Pocket TTS using UTMOS MOS scores, revealing interesting findings about RTF scaling, UTMOS limitations with small vocoders, and undocumented output caps. Pocket TTS offers unique zero-shot voice cloning on CPU.
@ngxson: Qwen3.6-27B running 100% on WebGPU. Not the best speed but still
A developer demonstrates running the Qwen3.6-27B AI model entirely on WebGPU in a browser, though speed is not optimal.
@FeitengLi: A 99M parameter TTS runs on CPU, faster than a 2B model on A100. Supertone's newly open-sourced supertonic-3 with ONNX Runtime, fully local, can run in browser, on phone, and even on Raspberry Pi.
Supertone released Supertonic 3, an open-source TTS model with 99M parameters that runs faster on CPU than a 2B model on A100, supporting 31 languages and ONNX Runtime for fully local inference.
@HuggingPapers: NVIDIA just released an optimized version of the Kokoro TTS model on Hugging Face A lightweight 82M parameter speech sy…
NVIDIA released an optimized ONNX version of the Kokoro TTS model on Hugging Face. The 82M parameter model is lightweight, fast, and ready for commercial use.