🚀 AuK is officially here. Nano banana🍌 for audio

An open-source foundation model for unified speech generation and editing. Natural-language instructions + reference audio. One interface. Zero-shot TTS. Instruction-controlled generation. Content editing. Whisper-conversion. De-accent. Timbre/style/emotion edit. Speed/Pitch control. Enhancement, denoising, multi-speaker and music separation. Also releasing AuK-Flash: 4-step inference. ~4.5× faster under matched conditions. Code, weights, and demo are live. Try it and share your feedback.

🤗 Paper & upvote: https://huggingface.co/papers/2609.08936… ⭐ GitHub & star: https://github.com/Tencent-Hunyuan/AuK…

Paper page - AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing

Source: https://huggingface.co/papers/2609.08936 Published on Sep 8

#2 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

AuK is an open-source foundational model that unifies speech generation and editing via natural-language instructions and audio context, using a multimodal language model, joint VAE, hybrid rectified-flow Transformer, and efficient distillation for fast inference.

We introduce AuK, an open-source foundational model that unifies speech generation and editing through a common interface of natural-language instructions and audio context. To support this broad capability set, we construct approximately 3.03 billion instruction--audio instances and 1.95 million hours of effective supervision across five task families: speech generation, content editing, enhancement and separation, paralinguistic editing, and acoustic editing. AuK combines amultimodal large language modelfor semantic conditioning, anVAEjointly trained on speech, general audio, and music for acoustic conditioning, and a hybridrectified-flow Transformerthat performs dual-streamMMDiTblocks followed by unified single-streamDiTblocks for generation. Training begins with generation-only warm-up and proceeds to joint generation--editing pre-training. We then apply complementary post-training strategies: human-feedbackpreference optimizationfor open-ended editing and reward-basedreinforcement learningfor speech generation. To reduce inference cost, we further distill the model withconsistency initializationand task-routedDecoupled DMD. The resulting AuK-Flash performs 4-step inference withoutclassifier-free guidanceand achieves a 4.5 wall-clock speedup over the full model under matched conditions. Experiments demonstrate leading performance on zero-shot and instruction-controlled speech generation and general instruction-guided editing, while remaining competitive on signal-level restoration tasks. We release both the source code and model weights to support reproducibility and further research.

View arXiv pageView PDFProject pageGitHub395Add to collection

Get this paper in your agent:

hf papers read 2609\.08936

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper4

#### tencent/AuK Text-to-Speech• Updated1 day ago • 179 • 77 #### tencent/AuK-Flash Text-to-Speech• Updated2 days ago • 129 • 42 #### drbaph/AuK-comfyui Text-to-Speech• Updatedabout 17 hours ago • 3 #### AEmotionStudio/auk-models Text-to-Speech• Updatedabout 16 hours ago

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.08936 in a dataset README.md to link it from this page.

Spaces citing this paper2

Collections including this paper1