VibeVoice Technical Report
Summary
VibeVoice is a new model from Microsoft that synthesizes long-form multi-speaker speech using next-token diffusion and a highly efficient continuous speech tokenizer. It achieves superior fidelity and compression, supporting up to 90 minutes of audio with multiple speakers.
View Cached Full Text
Cached at: 05/08/26, 08:34 AM
Paper page - VibeVoice Technical Report
Source: https://huggingface.co/papers/2508.19205 Published on Aug 26, 2025
·
Submitted byhttps://huggingface.co/unilm
Li Dongon Aug 27, 2025
#1 Paper of the day Authors:
,
,
,
,
,
,
,
,
,
Abstract
VibeVoice synthesizes long-form multi-speaker speech using next-token diffusion and a highly efficient continuous speech tokenizer, achieving superior performance and fidelity.
This report presents VibeVoice, a novel model designed to synthesizelong-form speechwith multiple speakers by employingnext-token diffusion, which is a unified method for modeling continuous data by autoregressively generating latent vectors via diffusion. To enable this, we introduce a novelcontinuous speech tokenizerthat, when compared to the popularEncodecmodel, improves data compression by 80 times while maintaining comparable performance. The tokenizer effectively preservesaudio fidelitywhile significantly boostingcomputational efficiencyfor processing long sequences. Thus, VibeVoice can synthesizelong-form speechfor up to 90 minutes (in a 64K context window length) with a maximum of 4 speakers, capturing the authentic conversational ``vibe’’ and surpassing open-source and proprietarydialogue models.
View arXiv pageView PDFProject pageGitHub46.8kAdd to collection
Get this paper in your agent:
hf papers read 2508\.19205
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper66
#### microsoft/VibeVoice-1.5B Text-to-Speech• 3B• UpdatedJan 22 • 259k • 2.36k
#### microsoft/VibeVoice-Realtime-0.5B Text-to-Speech• 1B• UpdatedDec 12, 2025 • 949k • 1.22k
#### aoi-ot/VibeVoice-Large Text-to-Speech• 9B• UpdatedSep 25, 2025 • 8.38k • 230
No dataset linking this paper
Cite arxiv.org/abs/2508.19205 in a dataset README.md to link it from this page.
Spaces citing this paper123
Collections including this paper22
Similar Articles
microsoft/VibeVoice
Microsoft open-sourced VibeVoice, a unified speech-to-text model handling 60-minute audio in a single pass with speaker/timestamp/content transcription, and a real-time text-to-speech model supporting streaming input and multilingual voices. The project includes an edge CPU inference engine for ASR via quantization.
[audio.cpp] VibeVoice 1.5B released — 90-min podcast in 22.95 min, 4.08x real-time, 2.86x faster than Python without quantization. Native C++/ggml
VibeVoice 1.5B, a long-form multi-speaker TTS model, is now supported in audio.cpp, a native C++/ggml runtime, achieving 4.08x real-time speed on RTX 5090, 2.86x faster than Python baseline without quantization.
@oliviscusAI: microsoft just released a tool that transcribes a full hour of audio at once, tracking who spoke and when. it's called …
Microsoft released vibevoice, a 7B model that transcribes up to an hour of audio in one shot with built-in speaker diarization and timestamps, supporting 50+ languages and running locally without API costs.
microsoft/VibeVoice-ASR-BitNet
Microsoft releases VibeVoice-ASR-BitNet, a compressed multilingual ASR model for real-time CPU inference. It achieves 1.6-2.3x faster inference than Whisper.cpp with real-time capability on as few as 3 CPU threads.
@uniswap12: Microsoft open-sourced a voice AI that can transcribe 60 minutes of long audio in one go, handling 4 people speaking simultaneously. VibeVoice, open-sourced by Microsoft, 24.8k stars, I only found out about it today. For converting recordings to text, I've been using Whisper, but it often times out on long meeting recordings and struggles with multi-speaker recognition...
Microsoft open-sourced the VibeVoice speech AI framework, which supports one-shot transcription of 60-minute long audio, multi-speaker diarization and timestamp labeling, and also provides multi-role TTS synthesis capabilities. It is based on Qwen2.5 and comes with a 0.5B lightweight real-time version. It has received 24.8k stars on GitHub.