Zero-shot voice cloning for 30 languages

https://t.co/H3gRfI6z3U https://t.co/6XoJ4zTjTy

sunnyxrxrx/X-Voice

Source: https://github.com/sunnyxrxrx/X-Voice

X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning

X-Voice is a flow-matching-based multilingual text-to-speech system that enables one speaker to speak 30 languages.

News

• 2026/04/30: X-Voice codebase, model, demo, Hugging Face Space, dataset, and benchmark are released.
• 2026/05/08: X-Voice paper publicly released on arXiv.

Installation

Create a separate environment if needed

# Create a conda env with python_version>=3.10
conda create -n x-voice python=3.11
conda activate x-voice

# Install FFmpeg if you haven't yet
conda install ffmpeg

Install PyTorch with matched device

Install X-Voice

git clone https://github.com/sunnyxrxrx/X-Voice.git
cd X-Voice
pip install -e .

Check your ESpeak-ng installation:

espeak-ng --version

If not found, run bash src/x_voice/prepare_ipa.sh first.

Inference

• In order to achieve desired performance, take a moment to read detailed guidance.

1. Gradio App

x-voice_infer-gradio --host 0.0.0.0 --port 7860

2. CLI Inference

# X-Voice Stage1
python -m x_voice.infer.infer_cli_stage1 -c src/x_voice/infer/examples/basic/basic_stage1.toml

# X-Voice Stage2
python -m x_voice.infer.infer_cli_stage2 -c src/x_voice/infer/examples/basic/basic_stage2.toml

Training

TTS Model Training

Refer to training guidance for best practice.

Speaking Rate Predictor Training

Refer to speaking rate predictor guidance for the multilingual speaking rate predictor used in X-Voice.

Evaluation

Refer to evaluation guidance for benchmark and metric scripts.

Repo Structure

X-Voice/
├── ckpts/                  # checkpoints
├── data/                   # datasets and processed data
├── src/
│   ├── rate_pred/          # speaking rate predictor
│   ├── third_party/
│   │   └── BigVGAN/        # BigVGAN submodule
│   └── x_voice/            # main X-Voice package
└── pyproject.toml          # package definition and dependencies

Development

Use pre-commit to ensure code quality:

pip install pre-commit
pre-commit install
pre-commit run --all-files

Acknowledgements

• F5-TTS brilliant work and the foundation of this codebase
• Cross-Lingual F5-TTS 2 for its supervised fine-tuning strategy with synthetic audio prompts
• Cross-Lingual F5-TTS for its speaking rate predictor
• NLLB for translation in the Gradio demo
• torchdiffeq as ODE solver, Vocos and BigVGAN as vocoder
• FunASR, faster-whisper, UniSpeech, SpeechMOS for evaluation tools
• MAVL for Japanese syllable counting

Citations

If you find our work useful, please cite as:

@article{xu2026xvoiceenablingspeak30,
      title={X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning}, 
      author={Rixi Xu and Qingyu Liu and Haitao Li and Yushen Chen and Zhikang Niu and Yunting Yang and Jian Zhao and Ke Li and Berrak Sisman and Qinyuan Cheng and Xipeng Qiu and Kai Yu and Xie Chen},
      journal={arXiv preprint arXiv:2605.05611},
      year={2026},
}

License

Our code is released under MIT License. The pre-trained models are licensed under the CC-BY-NC license due to the training data X-Voice Dataset.