Fish Audio S2 Technical Report

Papers with Code Trending 03/09/26, 06:34 PM Models

Summary

Fish Audio S2 is an open-source text-to-speech system featuring multi-speaker capabilities, multi-turn generation, and instruction-following control, backed by a production-ready inference engine with low latency.

We introduce Fish Audio S2, an open-sourced text-to-speech system featuring multi-speaker, multi-turn generation, and, most importantly, instruction-following control via natural-language descriptions. To scale training, we develop a multi-stage training recipe together with a staged data pipeline covering video captioning and speech captioning, voice-quality assessment, and reward modeling. To push the frontier of open-source TTS, we release our model weights, fine-tuning code, and an SGLang-based inference engine. The inference engine is production-ready for streaming, achieving an RTF of 0.195 and a time-to-first-audio below 100 ms.Our code and weights are available on GitHub (https://github.com/fishaudio/fish-speech) and Hugging Face (https://huggingface.co/fishaudio/s2-pro). We highly encourage readers to visit https://fish.audio to try custom voices.

Original Article

View Cached Full Text

Cached at: 05/10/26, 06:36 PM

Paper page - Fish Audio S2 Technical Report

Source: https://huggingface.co/papers/2603.08823 Authors:

Abstract

Fish Audio S2 is an open-source text-to-speech system with multi-speaker capabilities, multi-turn generation, and instruction-following control through natural-language descriptions, utilizing a multi-stage training approach and production-ready inference engine.

We introduce Fish Audio S2, an open-sourcedtext-to-speechsystem featuringmulti-speaker,multi-turn generation, and, most importantly,instruction-following controlvianatural-language descriptions. To scale training, we develop amulti-stage trainingrecipe together with astaged data pipelinecoveringvideo captioningandspeech captioning,voice-quality assessment, andreward modeling. To push the frontier of open-source TTS, we release our model weights, fine-tuning code, and anSGLang-based inference engine. The inference engine is production-ready for streaming, achieving anRTFof 0.195 and atime-to-first-audiobelow 100 ms.Our code and weights are available on GitHub (https://github.com/fishaudio/fish-speech) and Hugging Face (https://huggingface.co/fishaudio/s2-pro). We highly encourage readers to visit https://fish.audio to try custom voices.