Fish Audio Drama 3 Technical Report

Hugging Face Daily Papers Models
text-to-speech speech-generation voice-cloning open-source mixture-of-experts audio-generation streaming

Summary

Fish Audio 发布开源语音生成模型 Fish Audio Drama 3 的技术报告：基于 35B 参数混合专家骨干（每 token 激活 3B），支持多参考语音克隆、自然语言导演指令、多轮修订、多人场景配音、编辑式填充、长音频与流式合成，单张 H200 推理速度达 271 帧/秒。

We introduce Fish Audio Drama 3, an open-source speech generation model that turns text into directed performances, built on a 35B-parameter mixture-of-experts backbone that activates 3B parameters per token. It clones voices from multiple references, follows natural-language direction, revises a take when the next turn gives a note, voices multi-character scenes, edits existing speech by infilling, and synthesizes long-form and streaming audio. Multi-turn revision, scene dubbing, editing by infilling, and streaming text input are new in Drama 3, all in one model. Drama 3 pairs the hybrid linear-attention backbone with a new codebook multi-token-prediction audio decoder that predicts the acoustic codebooks of each frame in dedicated layers that attend over the entire sequence. The model is trained with a three-stage pre-training recipe, instruction fine-tuning, and large-batch reinforcement learning whose only reward is a single learned model of human preference. Its inference engine decodes a single stream at 271 frames per second on one H200, against 177 for Fish Audio S2, although Drama 3 has more than seven times as many parameters. The model weights are available on GitHub (https://github.com/fishaudio/fish-speech) and Hugging Face (https://huggingface.co/fishaudio/drama-3), and inference is available through SGLang-Omni (https://github.com/sgl-project/sglang-omni). We highly encourage readers to visit https://fish.audio to direct their own performances.
Original Article
View Cached Full Text

Cached at: 10/14/26, 06:51 AM

Paper page - Fish Audio Drama 3 Technical Report

Source: https://huggingface.co/papers/2610.18010 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

WeintroduceFishAudioDrama3,anopen-sourcespeechgenerationmodelthatturnstextintodirectedperformances,builtona35B-parametermixture-of-expertsbackbonethatactivates3Bparameterspertoken.Itclonesvoicesfrommultiplereferences,followsnatural-languagedirection,revisesatakewhenthenextturngivesanote,voicesmulti-characterscenes,editsexistingspeechbyinfilling,andsynthesizeslong-formandstreamingaudio.Multi-turnrevision,scenedubbing,editingbyinfilling,andstreamingtextinputarenewinDrama3,allinonemodel.Drama3pairsthehybridlinear-attentionbackbonewithanewcodebookmulti-token-predictionaudiodecoderthatpredictstheacousticcodebooksofeachframeindedicatedlayersthatattendovertheentiresequence.Themodelistrainedwithathree-stagepre-trainingrecipe,instructionfine-tuning,andlarge-batchreinforcementlearningwhoseonlyrewardisasinglelearnedmodelofhumanpreference.Itsinferenceenginedecodesasinglestreamat271framespersecondononeH200,against177forFishAudioS2,althoughDrama3hasmorethanseventimesasmanyparameters.ThemodelweightsareavailableonGitHub(https://github.com/fishaudio/fish-speech)andHuggingFace(https://huggingface.co/fishaudio/drama-3),andinferenceisavailablethroughSGLang-Omni(https://github.com/sgl-project/sglang-omni).Wehighlyencouragereaderstovisithttps://fish.audiotodirecttheirownperformances.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2610\.18010

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### fishaudio/drama-3 Text-to-Speech• 36B• Updated37 minutes ago • 23 • 159

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.18010 in a dataset README.md to link it from this page.

Spaces citing this paper5

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Fish Audio Drama 3

Product Hunt

Fish Audio 推出其最新 AI 语音模型 Drama 3，支持通过自然语言控制情感、节奏、耳语、笑声和停顿，可生成多角色场景、有声书与游戏对话，覆盖 83 种语言，并以社区许可证开放权重供自托管。

fishaudio/drama-3

Hugging Face Models Trending

Fish Audio 发布新一代文本转语音模型 Drama 3，采用 35B 参数 MoE 架构（每 token 激活 3B），支持最多 16 个说话人的多角色长文本生成、语音克隆与自然语言控制，并可在 SGLang-Omni 上流式部署。

Fish Audio S2 Technical Report

Papers with Code Trending

Fish Audio S2 is an open-source text-to-speech system featuring multi-speaker capabilities, multi-turn generation, and instruction-following control, backed by a production-ready inference engine with low latency.

Drama 3 (Website)

TLDR AI

Fish Audio 发布了 Drama 3 预览版文本转语音模型，支持用自然语言、括号内标签、音效与局部标记来精确指导每一句台词的表演方式，并可通过 API 调用。