Tacit-TTS: From Autoregressive Decoding to Masked Prediction for Efficient Transcript-Free Voice Cloning
Summary
Tacit-TTS 将 IndexTTS2 的自回归文本到语义解码替换为掩码非自回归生成,并结合免训练声学长度估计与 ReFlow 蒸馏,实现无需参考文本的零样本语音克隆,在超过 5 秒的语音上比 IndexTTS2 快 10 倍以上,同时支持跨语言及非语音(如婴儿咿呀学语、合成乱码)的参考音频克隆。
View Cached Full Text
Cached at: 10/01/26, 08:24 PM
Paper page - Tacit-TTS: From Autoregressive Decoding to Masked Prediction for Efficient Transcript-Free Voice Cloning
Source: https://huggingface.co/papers/2609.38658
Abstract
TTSsystemswithautoregressivesemanticmodelinghavedemonstratedstrongzero-shotvoicecloningperformanceandrichexpressivevariation,buttheirsequentialdecodingincurssubstantiallatency.Non-autoregressivealternativesoffermuchfastergeneration,yetoftenrelyonmorerestrictivereferenceconditioning,suchasrequiringtranscriptsofthereferencespeechduringinference.WepresentTacit-TTS,anefficienttranscript-freezero-shotvoicecloningsystemdistilledfromIndexTTS2.Ourmodelreplacesautoregressivetext-to-semanticdecodingwithmaskednon-autoregressivegeneration,introducestraining-freeacousticlengthestimation,andacceleratestheflow-matchingrendererthroughReFlowdistillation.AcrosstwoEnglishandtwoMandarindatasets,Tacit-TTSachievescompetitivezero-shotqualitywhilegeneratingspeechover10xfasterthanIndexTTS2forutteranceslongerthan5seconds.Itstranscript-freeconditioningfurthersupportscross-lingualandnon-lexicalreferences.Wevalidatethiscapabilityusingreferencesfromeightotherlanguages,infantbabble,andsyntheticgibberish,wheretranscript-dependentsystemsoftendegradeorfailduetounreliableASRtranscripts.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2609\.38658
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2609.38658 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2609.38658 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.38658 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech System
IndexTTS is an enhanced text-to-speech system that combines XTTS and Tortoise models with hybrid character-pinyin modeling and optimized vector quantization, achieving improved naturalness, controllable pronunciation, and faster inference than existing open-source TTS systems.
Kitten-TTS 2: New Open-source SOTA 1.58 bit TTS (CPU-only with voice cloning, multilingual, emotion control)
KittenML 开源发布了 Kitten-TTS 2,这是全球首个 1.58-bit(三值化权重)TTS 模型,1.7B 参数,可在纯 CPU 上实时推理,支持少于 5 秒参考音频的零样本语音克隆、20+ 语言多语种支持以及情感与强调控制。
Kokoro can now clone any voice with just ~600 MB of extra VRAM!
为 audio.cpp 新增 any-tts 配方,将 Kokoro TTS 与 OpenVoice 的 ToneColorConverter 结合,仅需约 600 MB 额外 VRAM 即可实现免训练的语音克隆能力。
Open TTS Leaderboard: Scalable Evaluation for Multilingual Text-to-Speech and Voice Cloning
Hugging Face introduces the Open TTS Leaderboard, an objective, scalable evaluation framework for multilingual TTS and voice cloning using ASR-based intelligibility (WER/CER), speed (RTFx/TTFA), and WavLM speaker-similarity metrics, reducing model evaluation time from weeks to hours.
seshat-tts: A local real-time narrator for games that supports voice cloning
seshat-tts is an open-source tool that enables real-time game narration with voice cloning, using OCR or an LLM for text extraction and local synthesis with pocket-tts. Voice cloning takes ~10 seconds on an RTX 2070 Super and runs on CPU after caching.