ai-sage/GigaChat3.1-Audio-10B-A1.8B · Hugging Face

Source: https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B

https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B#gigachat-audio-10b-a18bGigaChat Audio 10B (A1.8B)

GigaChat Audio 10Bis an audio-native LLM built on top of theGigaChat 3.1 Lightningtext model. A Conformer speech encoder and a modality adapter feed audio embeddings directly into a Mixture-of-Experts decoder, so the model keeps the text quality of its base while adding speech understanding.

Capabilities: audio question answering and classification, temporal grounding (localization in long audio, timestamped event descriptions, audio summarization with timestamps), tool-use, and text-only tasks.

The temporal grounding skills are trained onTimeGround-1M— a purpose-built dataset of long-form audio paired with time-aligned annotations.

https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B#evaluationEvaluation

https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B#1-core-audio-tasks-vs-open-models1. Core audio tasks vs open models

TaskSetMetricGigaChat Audio (10B-A1.8B)Voxtral (3B)Phi-4 (4B)Qwen3-Omni (30B-A3B)Audio QAMMAUacc ↑62.259.868.374.7Audio QAMMLU-speechacc ↑50.338.835.172.2Audio mathMQAacc ↑72.535.342.086.7Audio QA (ru)RuBQacc ↑60.023.42.343.7Temporal Localization≤10mmIoU ↑40.33.40.212.9Temporal Localization20–60mmIoU ↑48.30.10.20.1EmotionDusha crowdacc ↑90.043.911.477.2EmotionDusha podcastacc ↑92.479.67.280.7ASR (ru)Golos crowdWER ↓14.725.9180.013.1ASR (ru)Golos farfieldWER ↓9.730.3188.718.4ASR (ru)FLEURS ruWER ↓4.47.8208.53.3ASR (en)FLEURS enWER ↓6.54.04.25.0TranslationFLEURS ru→enBLEU ↑33.434.00.133.8TranslationFLEURS en→ruBLEU ↑26.021.419.929.3

https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B#2-timing-tasks-detailed2. Timing tasks (detailed)

Full timing metrics across length bucketsTL — temporal localization: find when something is discussed (mIoU vs the reference span). TD — timestamped descriptions of audio events (overall grade 0–5). SUM — long-audio summarization with timestamps; single score = mean of factual accuracy, timing structure and audio coverage with timestamps.

MetricBucketGigaChat Audio (10B-A1.8B)Voxtral (3B)Phi-4 (4B)Qwen3-Omni (30B-A3B)TL mIoU ↑≤10m40.33.40.212.9TL mIoU ↑10–20m46.80.00.20.0TL mIoU ↑20–60m48.30.10.20.1TL mIoU ↑60–120m48.90.93.74.6TL mIoU ↑AMI meeting30.30.00.00.0TD overall ↑ (0–5)≤10m3.452.892.623.23TD overall ↑ (0–5)10–20m3.212.332.072.49TD overall ↑ (0–5)20–60m3.272.151.952.17TD overall ↑ (0–5)60–120m3.261.451.211.84SUM overall ↑≤10m71.664.126.265.7SUM overall ↑10–20m71.458.923.054.8SUM overall ↑20–60m67.950.221.443.8SUM overall ↑60–120m55.440.19.817.3

https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B#3-text-quality-vs-the-text-base-model3. Text quality vs the text base model

Adding the audio modality shifts text quality: some benchmarks regress (MMLU-Pro, IFEval-ru, BBH), others improve (RuBQ, GPQA Diamond).

BenchmarkText 10bAudio 10bΔMMLU_PRO_EN62.0452.86−9.18RUBQ (ru)67.4668.91+1.45IFEVAL (ru)66.2262.35−3.87BBH75.7268.46−7.26GPQA Diamond39.7340.91+1.18

https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B#quickstart-transformersQuickstart (Transformers)

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model_name = "ai-sage/GigaChat3.1-Audio-10B-A1.8B"
processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name, trust_remote_code=True, dtype=torch.bfloat16, device_map="cuda:0",
)

messages = [{"role": "user", "content": [
    {"type": "audio", "path": "90min_lecture.wav"},
    {"type": "text", "text": "When does the speaker mention Star Wars midi-chlorians?"},
]}]
inputs = processor.prepare_for_inference(messages, device=model.device)
output_ids = model.generate(**inputs, max_new_tokens=256)
answer_ids = output_ids[0, inputs["input_ids"].shape[1]:]
print(processor.decode(answer_ids))
# > ... in the interval 01:27:46 to 01:27:53. In this segment they explain ...

https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B#vllm-single-gpu-native-multimodalvLLM (single GPU, native multimodal)

Both encoder and decoder run inside vLLM.compilation\_configdisablestorch\.compile(needed for the Conformer tower);max\_num\_batched\_tokenssizes the audio encoder cache (~90 min is 33k tokens):

import librosa
import os
from transformers import AutoProcessor
from vllm import LLM, SamplingParams

os.environ.setdefault("VLLM_WORKER_MULTIPROC_METHOD", "spawn")

model_name = "ai-sage/GigaChat3.1-Audio-10B-A1.8B"

def main():
    processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
    llm = LLM(
        model=model_name,
        trust_remote_code=True,
        max_model_len=65536,
        max_num_batched_tokens=65536,
        limit_mm_per_prompt={"audio": 1},
        compilation_config={"mode": 0, "cudagraph_mode": "FULL"},
    )

    messages = [{"role": "user", "content": [
        {"type": "audio", "path": "90min_lecture.wav"},
        {"type": "text", "text": "When does the speaker mention Star Wars midi-chlorians?"},
    ]}]
    text, audio_paths = processor.render_prompt(messages)
    wav, _ = librosa.load(audio_paths[0], sr=16000, mono=True)
    out = llm.generate(
        {"prompt": text, "multi_modal_data": {"audio": [(wav, 16000)]}},
        SamplingParams(temperature=0.0, max_tokens=256),
    )
    print(processor.decode(out[0].outputs[0].token_ids))
    # > ... in the interval 01:27:46 to 01:27:53. In this segment they explain ...

if __name__ == "__main__":
    main()

https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B#throughputThroughput

Single-request decode speed on one H100 (vLLM 0.18.0, bf16, greedy), by amount of audio held in context:

In-context audioDecode throughput (tok/s)≤ 10 min24210–20 min23320–60 min22560–120 min211

https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B#audio-features-encoder–adapterAudio features (encoder + adapter)

model\.encode\_audioreturns the LLM-space audio embeddings:

spec = processor.feature_extractor.process("audio.wav")   # (frames, 64) log-mel
specs = spec.unsqueeze(0).to(model.device, torch.bfloat16)
lengths = torch.tensor([spec.shape[0]], device=model.device)
feats, feat_lengths = model.encode_audio(specs, lengths)   # (1, tokens, hidden)

https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B#recommended-versionsRecommended versions

• torch 2.10.0, torchaudio 2.10.0
• transformers 4.57.6
• vllm 0.18.0
• flash-attn 2.8.3

https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B#citationCitation

If you use GigaChat Audio in your research, please cite:

@misc{kutsakov2026_gigachataudio,
  title         = {{GigaChat Audio}: Time-aware Large Audio Language Model},
  author        = {Kutsakov, Aleksandr and
                   Sadovina, Mariia and
                   Gospodinov, Georgii and
                   Maximenko, Alexandr and
                   Kutuzov, Oleg and
                   Bogomolov, Pavel and
                   Minkin, Fyodor},
  year          = {2026},
  eprint        = {2607.10387},
  archivePrefix = {arXiv},
  primaryClass  = {eess.AS},
  url           = {https://arxiv.org/abs/2607.10387},
  note          = {Accepted to Interspeech 2026}
}