Edge0/Audio8-TTS-Realistic-0.6b · Hugging Face

Source: https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b

Audio8-TTS-Realistic-0.6Bbuilds onAudio8-TTS-Preview-0.6b, deliveringsubstantially improved speech realismwhile retaining its compact0.6Barchitecture and zero-shot voice cloning capabilities.

This release supports23 languages, adding12 languages compared with Preview. It uses the same Transformers inference interface as Preview. This repository includes the complete model checkpoint, its 44.1 kHz neural audio codec, tokenizer, processor, and custom Transformers code.

Try speech generation and voice cloning in theinteractive Playground. Hosted on Hugging Face ZeroGPU; availability depends on the queue and your daily GPU allowance.

Audio8 is also participating inTTS Arena, where it has received encouraging ratings from listeners. We are grateful to everyone who has listened and voted.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#highlightsHighlights

• **Improved realism:**more natural prosody and expressive delivery, building on Preview.
• **Broader multilingual coverage:**23 languages, including 12 additions to Preview’s 11 supported languages.
• **Compact model:**601 million main-model parameters, with the audio codec bundled separately.
• **Flexible generation:**zero-shot voice cloning from a reference recording and transcript, or speech generation without a reference.
• **GPU serving:**an SGLang Omni adapter with paged attention, dynamic batching, and an OpenAI-compatible speech API.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#supported-languagesSupported Languages

CoverageLanguagesSupported in Preview (11)Cantonese, Chinese, Dutch, English, French, German, Italian, Japanese, Korean, Polish, SpanishAdded in Realistic (12)Arabic, Finnish, Hindi, Indonesian, Malay, Norwegian, Portuguese, Russian, Swedish, Thai, Turkish, Vietnamese

**Note:**Language quality varies; see the per-language evaluation below. Benchmark coverage includes some additional languages and is distinct from the supported-language list.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#model-detailsModel Details

Audio8 TTS uses a DualAR architecture inspired byFish Audio S2 Pro. The slow AR transformer predicts one semantic token for each audio frame. The fast AR transformer predicts the frame’s codec codebooks, conditioned on the slow hidden state and preceding codebooks.

ComponentConfigurationMain model601,159,424 parameters, excluding the codecSlow AR24 layers, width 896, 14 attention heads, 2 KV headsFast AR4 layers, width 896, 14 attention heads, 2 KV headsAcoustic tokens10 codebooks, 4,096 entries per codebookCodec44.1 kHz, 2,048 samples per model frame (~21.5 frames/s)ContextUp to 2,048 packed text/audio positions The bundled codec handles both reference-audio encoding and waveform decoding, so no additional codec checkpoint is required. The0.6Bsize refers to the main DualAR model; the codec is excluded from that parameter count.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#installationInstallation

Python 3.10 or newer and a CUDA-capable GPU are recommended.

pip install "torch>=2.5.0" "torchaudio>=2.5.0" \
  "transformers>=4.57.0,<5" "soundfile>=0.12" "safetensors>=0.4"

The Transformers release package was smoke-tested with PyTorch 2.8.0 and Transformers 4.57.3 on an NVIDIA H20 using BF16. Checks covered model loading, English and Chinese generation with and without reference audio, and generation in Arabic, Cantonese, Portuguese, and Russian. These are runtime checks, separate from the benchmark results below.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#usageUsage

The model uses custom Transformers code. Review the files in this repository, then load it withtrust\_remote\_code=True.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#zero-shot-voice-cloningZero-shot voice cloning

Use a clean reference recording. Its transcript must match the spoken content exactly.

import soundfile as sf
import torch
from transformers import AutoModel, AutoProcessor

model_id = "Edge0/Audio8-TTS-Realistic-0.6b"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32

processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=dtype,
).eval().to(device)

inputs = processor(
    text=["Welcome to Audio8 TTS. It is wonderful to hear from you."],
    reference_audio=["reference.wav"],
    reference_text=["The exact transcript of the reference recording."],
    return_tensors="pt",
)
inputs = {name: value.to(device) for name, value in inputs.items()}

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=1024,
        temperature=0.8,
        top_p=0.95,
        top_k=50,
        do_sample=True,
        return_dict_in_generate=True,
    )
    waveforms, waveform_lengths = model.decode_audio(output.codes)

audio = waveforms[0, : int(waveform_lengths[0])].float().cpu().numpy()
sf.write("output.wav", audio, model.config.codec_sample_rate)

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#generation-without-a-referenceGeneration without a reference

Replace the processor call above with:

inputs = processor(
    text=["This utterance does not use a reference voice."],
    return_tensors="pt",
)

Then move the inputs to the model device and run the same generation and decoding steps. For command-line inference, batching, and supervised fine-tuning, see theAudio8 TTS repository, usingEdge0/Audio8\-TTS\-Realistic\-0\.6bas the model path.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#high-throughput-serving-sglang-omniHigh-throughput serving: SGLang Omni

TheSGLang Omni adapterprovides GPU serving for Audio8 TTS. It is installed as an independent model plugin and does not overwrite SGLang Omni core files.

CapabilitySupportAttention and batchingSGLang paged attention and dynamic batchingDualAR executionSlow AR serving with a fixed KV cache for the Fast AR codebook decoderVoice cloningReference-audio encoding and waveform decodingAPIOpenAI-compatible/v1/audio/speechservice Use the0.6Bconfiguration,sglang\_omni/configs/audio8\_tts\_0\_6b\.yaml, and set the model path toEdge0/Audio8\-TTS\-Realistic\-0\.6bor its downloaded snapshot. The adapter supports generation without a reference and zero-shot voice cloning. See thedeployment guidefor compatible dependency versions, installation, server configuration, and API examples.

**Validation note:**Audio8-TTS-Realistic was tested on an NVIDIA H20 in BF16 for generation with and without a reference through the non-streaming speech API, with Torch compilation and CUDA graphs disabled. These checks validate runtime compatibility; throughput and streaming were not benchmarked.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#evaluationEvaluation

Error rates are percentages (lower is better). Speaker similarity (SIM) is on a0–1 scale(higher is better). These metrics measure intelligibility and speaker similarity; they do not directly measure perceived speech realism.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#benchmark-summaryBenchmark summary

Audio8-TTS-PreviewandAudio8-TTS-Realistic, both0.6B:

BenchmarkCoveragePreview error % ↓Realistic error % ↓Preview SIM ↑Realistic SIM ↑Seed-TTSEnglish / Chinese1.231.250.6820.704CV38 shared languages4.804.56—0.663CV39 core languages—4.86—0.661MiniMax24 evaluated languages—5.89—0.754

**Note:**Values are unweighted language macro averages. Preview results are calculated from itspublished model card, with SIM converted from percentages to a 0–1 scale. The two releases were evaluated separately. A dash means the corresponding result was not reported on Preview’s card. Seed-TTS averages English WER and Chinese CER, excluding hard Chinese. The shared CV3 subset is Chinese, English, Japanese, Korean, German, French, Spanish, and Italian; the 9-language set also includes Russian. CV3 averages exclude hard subsets and the 13 additional languages. MiniMax averages all 24 evaluated languages, including languages outside the supported list.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#reference-comparisonReference comparison

The following error rates come from the sources used in our technical report. Different evaluation pipelines and text normalization make this a reference comparison rather than a strictly matched ranking. A dash means that a result is unavailable in the comparison, not that the model lacks support.

ModelParameters (B)Seed-TTS EN/ZH error ↓CV3 (9) error ↓MiniMax (24) error ↓Audio8-TTS-Realistic0.61.254.865.89Qwen3-TTS-12Hz-0.6B0.61.12——Qwen3-TTS-1.7B1.71.017.7327.41Higgs Audio v341.114.412.74Higgs Audio v24.71.174.49—Fish S2 Pro4.61.324.494.40MOSS-TTS-v1.58.51.526.113.78CosyVoice3-1.5B1.51.676.89—VoxCPM22.31.415.35—

**Sources:**Qwen Seed-TTS results are from the Qwen3-TTS report, linked from theofficial Qwen3-TTS repository; Qwen’s CV3/MiniMax results, all Higgs Audio v3 results, and MOSS-TTS-v1.5’s CV3/MiniMax results are fromBoson AI’s evaluation. Fish S2 Pro’s MiniMax result is from the Fish Audio S2 report, linked from theofficial Fish Speech repository; its Seed-TTS/CV3 results and Higgs Audio v2’s results are our reevaluations. MOSS-TTS-v1.5’s Seed-TTS result and the CosyVoice3/VoxCPM2 results are compiled in theVoxCPM repository. Cross-project SIM values are omitted because speaker encoders and evaluation conventions differ.

Parameter conventions:Audio8’s 0.6B excludes its separate codec; Qwen and CosyVoice use release labels; Higgs Audio v3 uses its nominal4Brelease label. The remaining sizes are rounded checkpoint counts used in our report (VoxCPM2 excludes its separate AudioVAE). These counts use different conventions across systems.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#seed-tts-detailSeed-TTS detail

SplitMetricPreview error % ↓Realistic error % ↓Preview SIM ↑Realistic SIM ↑EnglishWER1.5061.4640.6320.663ChineseCER0.9501.0290.7310.745Chinese, hardCER11.51012.7970.6870.702

**Note:**Preview values are from itspublished Seed-TTS table, with SIM converted to a 0–1 scale.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#multilingual-detailMultilingual detail

CV3: 22 languages and hard subsetsLanguageMetricError % ↓SIM ↑ChineseCER1.8140.6898EnglishWER2.9070.6295JapaneseCER7.5110.6659KoreanCER4.8010.6962GermanWER3.2000.6669FrenchWER7.9380.6355SpanishWER3.3890.6680ItalianWER4.9330.6498RussianWER7.2580.6451ArabicWER21.0980.6122FinnishWER32.2790.7028HindiWER21.7140.7245IndonesianWER13.7430.6682MalayWER17.7980.6710DutchWER10.8730.6918NorwegianWER17.9610.6905PolishWER19.2330.6738PortugueseWER6.8710.6772SwedishWER19.2520.7104ThaiWER48.5740.6691TurkishWER12.5670.6496VietnameseWER28.1110.6790 Hard Chinese CER is10.665%and hard English WER is4.736%. These hard subsets are excluded from the 9-language summary. Each of the 22 regular language rows contains 500 evaluation items. Chinese, Japanese, and Korean use CER; the other regular language rows use WER.

MiniMax: all 24 evaluated languagesLanguageMetricError % ↓SIM ↑ArabicCER2.4030.7040CantoneseCER31.3490.7710ChineseCER1.1120.7950Czech †WER8.6280.7330DutchWER1.7150.7050EnglishWER2.1610.7740FinnishWER5.0720.7910FrenchWER3.1980.6660GermanWER0.8980.7510Greek †CER4.1810.7910HindiCER9.2090.8170IndonesianWER2.1280.7270ItalianWER1.9580.7370JapaneseCER5.1420.7890KoreanCER3.7310.7850PolishWER1.9570.7890PortugueseWER2.2340.7860Romanian †WER17.4250.7330RussianWER3.0710.7490SpanishWER1.3500.7450ThaiCER6.6740.7760TurkishWER1.5600.7650Ukrainian †WER16.3430.6940VietnameseCER7.9660.7120 Each language has 100 evaluation items. Rows marked † are evaluated for completeness and are outside this release’s supported-language list. The metric column preserves the evaluation suite’s language-specific WER/CER choice. Cantonese is included in the 24-language average.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#compared-with-previewCompared with Preview

Realistic expands Preview’s11-languagecoverage to23 languages, addingArabic, Finnish, Hindi, Indonesian, Malay, Norwegian, Portuguese, Russian, Swedish, Thai, Turkish, and Vietnamese.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#limitations-and-responsible-useLimitations and Responsible Use

• Quality varies by language and evaluation set. The detailed results include difficult cases, including Cantonese and some newly supported languages; the support list is not a claim of equal quality across languages.
• Noisy reference audio or mismatched transcripts can reduce stability and speaker similarity. The reference audio, transcript, and target text share a finite context budget; split long passages into manageable utterances.
• Speech may contain pronunciation errors, repetitions, omissions, or early stopping. Listen to generated audio before using it in a finished product.
• Obtain permission to clone a voice and disclose synthetic audio where appropriate.

https://huggingface.co/Edge0/Audio8-TTS-Realistic-0.6b#license-and-acknowledgementsLicense and Acknowledgements

The code and model weights are released under theApache License 2.0. SeeNOTICEfor attribution details.

We thank the Fish Audio team for publishing the DualAR architecture used in Fish Audio S2 Pro.