Unified Audio Intelligence Without Regressing on Text Intelligence
Summary
This paper introduces Audex, a unified audio-text LLM from NVIDIA that achieves state-of-the-art performance across multiple audio and speech tasks while preserving strong text reasoning capabilities without regression.
View Cached Full Text
Cached at: 07/07/26, 02:44 PM
Paper page - Unified Audio Intelligence Without Regressing on Text Intelligence
Source: https://huggingface.co/papers/2607.05196 Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
A unified audio-text large language model is presented that integrates audio and text processing through a shared transformer decoder, achieving superior performance across multiple audio and speech tasks while maintaining strong text reasoning capabilities.
Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unifiedaudio-text LLMbuilt on Nemotron-Cascade-2-30B-A3B, a strong text-only MoE LLM. Audex adopts a simple unified design with a singleTransformer decoder: audio inputs are encoded and projected into the text embedding space, while text tokens and quantized audio output tokens are treated uniformly during generation. This architecture enables strong audio-text fusion, seamlessmultimodal generation, and compatibility with standard LLM training and inference infrastructure. For training, we meticulously curate audio-text datasets comprising 157.4B audio tokens and 320.5B text tokens. We apply multi-stagesupervised trainingon these datasets, followed by text-onlyCascade RLand multi-domainon-policy distillation. Audex delivers state-of-the-artaudio understanding,speech recognitionand translation,text-to-speech,audio generation, andspeech-to-speech generation, while preserving very compelling reasoning, alignment, knowledge, long-context, and agentic capabilities of its text-only LLM backbone with marginal or no regression. We release the model checkpoints to facilitate open research.
View arXiv pageView PDFAdd to collection
Models citing this paper2
#### nvidia/Nemotron-Labs-Audex-30B-A3B Text Generation• Updatedabout 11 hours ago • 27
#### nvidia/Nemotron-Labs-Audex-2B Text Generation• Updatedabout 11 hours ago • 14
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.05196 in a dataset README.md to link it from this page.
Spaces citing this paper1
Collections including this paper1
Similar Articles
nvidia/Nemotron-Labs-Audex-30B-A3B · Hugging Face
NVIDIA released Nemotron-Labs-Audex-30B-A3B, a unified audio-text LLM built on a 30B MoE backbone with 3B activated parameters, offering strong performance on audio understanding, speech recognition/translation, and generation while preserving text reasoning and alignment capabilities.
LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning
LatentOmni proposes a unified latent space for audio-visual reasoning, avoiding the information loss of text-based chain-of-thought. It achieves state-of-the-art performance among open-source models on audio-visual reasoning benchmarks.
Continuous Audio Thinking for Large Audio Language Models
The paper introduces Continuous Audio Thinking (CoAT), a framework that equips large audio language models with a continuous latent workspace to organize acoustic information before generating textual responses, improving performance on audio reasoning, understanding, and transcription tasks without additional decoding cost.
Researchers let AI Agents Optimize LLM Reasoning and Cut Tokens by 70%
Researchers developed AutoTTS, a framework where AI agents automatically design control policies to optimize LLM inference, cutting token consumption by approximately 70% while maintaining high reasoning accuracy.
@kwindla: https://x.com/kwindla/status/2062544580105359686
NVIDIA released Nemotron 3.5 ASR, an open-source multilingual speech-to-text model with the lowest latency tested, available in multilingual and English-only variants, ideal for voice agents and self-hosted deployments.