inclusionAI/Ming-3.0-audio-flash · Hugging Face

Source: https://huggingface.co/inclusionAI/Ming-3.0-audio-flash

Hear clearly. Understand deeply. Act intelligently.

Ming-3.0-audio-flash is a native audio-language model from inclusionAI. It accepts audio together with text instructions and supports speech recognition, spoken question answering, audio understanding, audio reasoning, and optional synthesized-speech responses.

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#model-summaryModel Summary

PropertyDescriptionDeveloperinclusionAIModel typeNative audio-language Mixture-of-Experts modelParameters126B total parameters, 6.5B activated per inference stepInputsAudio and text instructionsOutputsText, or text with optional synthesized speechPrimary languagesChinese and English, including varied accents and dialectsLicenseMIT

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#highlightsHighlights

• **Complex speech recognition:**Handles Chinese and English speech, dialects, accents, noisy environments, and multi-speaker recordings.
• **Long-audio understanding:**Comprehends and analyzes long-form recordings.
• **Multidimensional perception:**Understands speech, non-speech sounds, acoustic scenes, and music.
• **Audio reasoning:**Answers questions and performs reasoning over audio content.
• **Audio interaction:**Produces text responses and can optionally synthesize speech with built-in voices or a reference voice.

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#architectureArchitecture

Ming-3.0-audio-flash combines an audio encoder and projector with a Mixture-of-Experts language backbone. An optional Talker generates speech responses from the model’s output.

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#quick-start-with-transformersQuick Start with Transformers

The model uses custom Transformers code. Review the repository code before enablingtrust\_remote\_code=True.

git clone https://github.com/inclusionAI/Ming-3.0-audio-flash.git
cd Ming-3.0-audio-flash

python -m venv .venv-transformers
source .venv-transformers/bin/activate
pip install -r requirements-transformers.txt

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#audio-understandingAudio Understanding

python examples/transformers_audio.py examples/audio/introduce_ant_group.wav \
  --model inclusionAI/Ming-3.0-audio-flash \
  --prompt "Please answer the question in the audio."

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#speech-recognitionSpeech Recognition

python examples/transformers_audio.py examples/audio/introduce_ant_group.wav \
  --model inclusionAI/Ming-3.0-audio-flash \
  --task asr \
  --max-new-tokens 256 \
  --no-repeat-ngram-size 10

The validated ASR prompt asks the model to identify the spoken language and place a tab between the language label and transcription.

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#direct-python-usageDirect Python Usage

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model_id = "inclusionAI/Ming-3.0-audio-flash"
audio_path = "examples/audio/introduce_ant_group.wav"

processor = AutoProcessor.from_pretrained(
    model_id,
    trust_remote_code=True,
    use_fast=False,
)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
).eval()

messages = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": audio_path},
            {"type": "text", "text": "Please answer the question in the audio."},
        ],
    }
]

inputs = processor(messages).to(model.device)
inputs["audio_feats"] = inputs["audio_feats"].to(model.dtype)

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
    )

generated_ids = output_ids[0, inputs["input_ids"].shape[1] :]
response = processor.decode(generated_ids, skip_special_tokens=True).strip()
print(response)

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#speech-outputSpeech Output

The Talker example supports built-in voices and reference-audio voice prompting:

python examples/transformers_thinker_talker.py \
  --model inclusionAI/Ming-3.0-audio-flash \
  --audio examples/audio/introduce_ant_group.wav \
  --ip-name US_Stella \
  --output output.wav

See thevoice guidefor available voice names and reference-audio usage.

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#servingServing

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#vllm-omnivLLM-Omni

export MODEL_PATH=inclusionAI/Ming-3.0-audio-flash
MING_OUTPUT_MODE=text ./scripts/serve_vllm_omni.sh "$MODEL_PATH"

The reference 128K serving profile uses tensor parallelism across eight 80 GB H800 GPUs. Select the deployment YAML that matches text-only or Talker output. See thedeployment guidefor details.

For speech output, vLLM-Omni uses the built-in IP voice灵小柔by default. The model directory also provides灵楷andUS\_Jamesas preset reference voices undervoice\_presets/voice\_audio/, with their matching transcripts invoice\_presets/voice\_prompts\.json. Both names are also available as built-in IP voices.

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#sglangSGLang

export MODEL_PATH=inclusionAI/Ming-3.0-audio-flash
./scripts/serve_sglang.sh "$MODEL_PATH"

The validated SGLang profile uses four GPUs and provides text output. See theSGLang examplefor client usage.

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#evaluationEvaluation

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#speech-recognition-1Speech Recognition

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#audio-intelligenceAudio Intelligence

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#text-benchmarksText Benchmarks

All benchmark values are percentages. Higher is better unless the metric is marked with a downward arrow. Results marked with an asterisk are self-evaluated. Refer to theGitHub READMEfor the latest notes and evaluation details.

https://huggingface.co/inclusionAI/Ming-3.0-audio-flash#licenseLicense

This project is released under theMIT License. SeeLEGAL.mdfor the legal disclaimer.