Fish Audio S2 Technical Report

Papers with Code Trending Models
text-to-speech open-source multi-speaker fish-audio audio-generation

Summary

Fish Audio S2 is an open-source text-to-speech system featuring multi-speaker capabilities, multi-turn generation, and instruction-following control, backed by a production-ready inference engine with low latency.

We introduce Fish Audio S2, an open-sourced text-to-speech system featuring multi-speaker, multi-turn generation, and, most importantly, instruction-following control via natural-language descriptions. To scale training, we develop a multi-stage training recipe together with a staged data pipeline covering video captioning and speech captioning, voice-quality assessment, and reward modeling. To push the frontier of open-source TTS, we release our model weights, fine-tuning code, and an SGLang-based inference engine. The inference engine is production-ready for streaming, achieving an RTF of 0.195 and a time-to-first-audio below 100 ms.Our code and weights are available on GitHub (https://github.com/fishaudio/fish-speech) and Hugging Face (https://huggingface.co/fishaudio/s2-pro). We highly encourage readers to visit https://fish.audio to try custom voices.
Original Article
View Cached Full Text

Cached at: 05/10/26, 06:36 PM

Paper page - Fish Audio S2 Technical Report

Source: https://huggingface.co/papers/2603.08823 Authors:

,

,

,

,

,

,

,

,

,

Abstract

Fish Audio S2 is an open-source text-to-speech system with multi-speaker capabilities, multi-turn generation, and instruction-following control through natural-language descriptions, utilizing a multi-stage training approach and production-ready inference engine.

We introduce Fish Audio S2, an open-sourcedtext-to-speechsystem featuringmulti-speaker,multi-turn generation, and, most importantly,instruction-following controlvianatural-language descriptions. To scale training, we develop amulti-stage trainingrecipe together with astaged data pipelinecoveringvideo captioningandspeech captioning,voice-quality assessment, andreward modeling. To push the frontier of open-source TTS, we release our model weights, fine-tuning code, and anSGLang-based inference engine. The inference engine is production-ready for streaming, achieving anRTFof 0.195 and atime-to-first-audiobelow 100 ms.Our code and weights are available on GitHub (https://github.com/fishaudio/fish-speech) and Hugging Face (https://huggingface.co/fishaudio/s2-pro). We highly encourage readers to visit https://fish.audio to try custom voices.

View arXiv pageView PDFProject pageGitHub30.2kAdd to collection

Community

Upload images, audio, and videos by dragging in the text input, pasting, orclicking here.

Tap or paste here to upload images

Get this paper in your agent:

hf papers read 2603\.08823

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper37

#### fishaudio/s2-pro Text-to-Speech• 5B• UpdatedMar 11 • 104k • 936 #### drbaph/s2-pro-fp8 Text-to-Speech• UpdatedMar 12 • 1.84k • 18 #### mlx-community/fish-audio-s2-pro-bf16 Text-to-Speech• 5B• UpdatedMar 20 • 1.47k • 17 #### mlx-community/fish-audio-s2-pro-8bit Text-to-Speech• 1B• UpdatedMar 20 • 856 • 10 Browse 37 models citing this paper## Datasets citing this paper1

#### Izzyzlin/CFSDD Viewer• UpdatedApr 7 • 395k • 264

Spaces citing this paper22

Collections including this paper6

Browse 6 collections that include this paper

Similar Articles

StepAudio 2.5 Technical Report

Hugging Face Daily Papers

StepAudio 2.5 is a unified audio-language model that achieves state-of-the-art results across ASR, TTS, and real-time spoken interaction by leveraging task-tailored reinforcement learning from human feedback to optimize shared representations.

@YichiZ03: https://x.com/YichiZ03/status/2078588932191895976

X AI KOLs Timeline

MOSS-TD, a speaker-aware ASR system, is optimized within the SGLang-Omni serving stack, enabling 38-minute multi-speaker audio to be transcribed in about 49 seconds on a single H100, with concurrent processing of 16 meetings.

Sora 2 System Card

OpenAI Blog

OpenAI has released Sora 2, a state-of-the-art video and audio generation model featuring improved physics simulation, synchronized audio, enhanced steerability, and broader stylistic range. The system card outlines capabilities, safety measures, and a phased rollout via sora.com, a new iOS app, and future API access.

Zyphra/ZONOS2

Hugging Face Models Trending

ZONOS2 is a new text-to-speech model from Zyphra trained on over 6 million hours of multilingual speech, offering high-quality voice cloning and low latency using a mixture-of-experts architecture. It supports 30+ languages and includes a high-performance inference server.