Tag
TontaubeAI released TontaubeV1-Verbalizer, an Apache 2.0 open-source LLM-based verbalization model fine-tuned from a Qwen3 1.7B backbone that converts written English text into spoken-form text for small TTS models, processing a 20,000-character document in under 2 seconds on an RTX 3090.
Podcast episode featuring Pete Warden, co-creator of Moonshine Voice, discussing why AI is shifting to on-device execution, progress in open-weight and quantized models, low-latency voice interfaces, and hardware alternatives to GPU-centric infrastructure.
The next breakthrough in voice AI may come from rethinking how we evaluate models rather than crowning a new top performer, with Voice Arena's Turing TTS benchmark measuring how often human listeners mistake AI voices for real ones.
Cartesia launched Managed Agents, an end-to-end enterprise voice agent platform that topped Artificial Analysis' Speech Agent Arena and ServiceNow's EVA-Bench in blind human preference rankings by co-designing its cascaded voice models and agent harness as one system.
SVPino highlights Voice Arena AI's Joy, a 100,000-hour dual-channel conversational voice dataset with separate per-speaker tracks, overlapping speech, interruptions, pauses, and backchannels, designed to teach voice models conversational timing.
提出 Full-Duplex Long-Horizon Benchmark(FD-LHB),发现现有全双工语音模型在长时间对话中会因上下文耗尽或重复、静音、乱码而崩溃,并通过 Initialization-Anchor Pinning 和 Duplex Repetition Penalty 两种免重训练修复方法显著缓解对话退化。
这篇论文提出「说话人激活后门」(SAB)设置,攻击者在微调阶段仅用目标说话人约两分钟的自然语音即可植入后门,使该说话人的声音本身成为触发器,在Whisper-medium和Parakeet上分别达到90.2%和97.0%的后门成功率,且触发在转写层不可见,还能向GPT-4o和Claude等下游LLM代理注入恶意指令。
蚂蚁集团(inclusionAI/Ant Ling)发布 Ming-3.0 系列语音与音频模型,涵盖 TTS、ASR 及音频模型,已在 Hugging Face 和 ModelScope 上线,参数规模从 0.5B 到 128B 不等。
PEDBench is the first phonetic error detection benchmark spanning natural, acted, and synthetic speech, evaluating 13 speech models and finding they fail to reliably detect phonetic errors. Fine-tuned speech-language models on synthetic data outperform baselines on word-level detection, though a sim-to-real gap remains.
该论文提出 PeopleMakingConversation 基准,通过 176 段非脚本对话中的 427 个停顿点和 482 个话轮结束点,评估六个实时语音系统在何时开口、延迟、回复长度与对话记忆等方面的自然对话表现,并公开数据与评测框架。
The paper identifies 'phonological interference' as a systematic failure mode in multilingual phoneme-level speech models, where models impose the phonology of a single assumed language and override phoneme decisions on code-switched or unseen-language input. It proposes windowed language estimation (WLE), an inference-time fix that reduces 34%–69% of interference without harming monolingual performance.
A position paper introducing 'Model as a Library' (MaaL), a software architecture that packages small, community-enrolled speech models as versioned on-device dependencies for offline, hallucination-free data collection in low-resource African languages. It proposes keyword spotting to turn closed-vocabulary digital forms into voice-based, on-device data collection for low-literacy populations, and transpiling existing form tools into MaaL schemas.
The paper introduces Duplex-MPE, a benchmark for evaluating AI assistants in multi-party, full-duplex dialogues with 2,000 scenarios, assessing models on response initiation, accuracy, and silence preservation.
The paper proposes a frontend-backend architecture for enabling tool calls in full-duplex speech models, where a speech frontend delegates to a text-based LLM backend for tool invocation, maintaining low-latency and natural interaction.
This paper proposes Self-Listening, a method for full-duplex spoken language models that feeds realized speech back as input to improve interruption recovery and consistency with actually spoken responses.
AirCaps Audio Research Lab launches with streaming speech and audio models designed for complex real-world environments, claiming superior performance on edge hardware compared to leading cloud models.
The research finds that emotion representation is mostly universal across seven languages in speech models, with a measurable 'accent' that mirrors human cross-cultural studies and affects cross-lingual transfer.
A study from OrukLabs shows that speech models trained solely on transcription or masked audio tasks spontaneously learn to represent emotions in their deeper layers, as revealed by mapping with real voice clips.
This paper introduces RedVox, a multilingual safety and fairness benchmark for speech models. Evaluating eight state-of-the-art models across five languages, it finds persistent vulnerabilities that worsen in non-English settings and with spoken input.
SpeechEQ introduces a benchmark and dataset for evaluating emotional intelligence in speech-language models, covering 15 EQ subscales across 2,265 dialogues. Experiments reveal current models struggle with paralinguistic cues, exhibiting text-reliant shortcuts and other limitations.