Ant releases an open audio model that can discuss a recording and answer back in speech

Reddit r/ArtificialInteligence Models
audio-model open-source speech-to-speech mixture-of-experts multimodal mit-license

Summary

Ant 发布了开源音频模型家族 Ming-3.0 Audio，采用 MIT 许可并开放权重与代码，支持语音与文本混合输入并可输出文字或语音回复，提供 126B 总参数的 Flash 和 9.3B 总参数的 tiny 两个 MoE 检查点。

A person says they are nervous about tomorrow’s presentation. Ming-3.0 Audio responds with a spoken, reassuring conversation. In another section of the same demo, it listens to a riddle and answers “keyboard.” Ant has released this audio model family under MIT, including weights and code. It accepts audio together with text instructions and can produce a written or spoken response. The complete video also shows it analyzing an earnings call, identifying sounds and discussing a song. There are two MoE checkpoints: Flash, with 126B total and 6.5B active parameters, and tiny, with 9.3B total and 2.7B active parameters. The release gives developers an open model for conversations grounded in what the user says or plays to it. The input can be a spoken question, a longer recording or non-speech audio, and the response can come back as speech.
Original Article

Similar Articles

inclusionAI/Ming-3.0-audio-flash · Hugging Face

Reddit r/LocalLLaMA

inclusionAI 发布 Ming-3.0-audio-flash，一个 126B 总参数（每步激活 6.5B）的原生音频-语言 MoE 模型，支持中英文语音识别、语音问答、长音频理解、音频推理以及可选的语音合成输出，采用 MIT 许可证。

Qwen-Audio-3.1-ASR Technical Report

Hugging Face Daily Papers

Qwen-Audio-3.1-ASR 技术报告介绍了一系列音频语言模型，在语音识别、说话人分离转录和通用音频理解方面较 Qwen-Audio-3.0-ASR 有显著提升，支持 30 种语言和 16 种汉语方言，并提供流式与非流式等多种部署变体。

@Ryrenz: So smooth—Ming-3.0-tts can clone a voice directly from a reference audio clip, and you can even tell it how to speak with a single sentence. It was just open-sourced yesterday by Ant Group’s inclusionAI. On Artificial Analysis’s October speech synthesis leaderboard, Ming-3.0-tts’s…

X AI KOLs Timeline

Ant Group’s inclusionAI has open-sourced the Ming-3.0-tts speech synthesis model, which ranked first overall in pronunciation robustness with a score of 93.2% on Artificial Analysis’s October leaderboard and first among open-source models for voice controllability. It supports voice cloning, control through natural-language instructions, 14 paralinguistic tags, and speech synthesis across multiple dialects and languages, including Sichuanese and Cantonese.

@Chenzeze777: Found an open-source voice synthesis model that I just had to share. 2 billion parameters, trained on 2 million hours of data, supports 30 languages plus 9 Chinese dialects—just input text and it synthesizes speech, including Sichuanese, Cantonese, and Northeastern dialects. The craziest part? Use natural language to describe a voice—like "young female, gentle and sweet"—and it creates a brand-new voice from scratch without needing any reference audio.

X AI KOLs Timeline

Introducing an open-source voice synthesis model with 2 billion parameters and 2 million hours of training. It supports 30 languages and 9 Chinese dialects, allows voice description via natural language, can clone voices from a 3-second recording, delivers 48kHz studio-quality audio, and is free for commercial use under the Apache-2.0 license.