MOSS-Transcribe-Diarize-0.9B is now open source on @huggingface.

Built with an end-to-end audio-to-structured-transcript paradigm:

0.9B open-source ASR model Apache license 2.0 128k long-context transcription Up to ~90-min audio input Speaker labels + timestamps in one generation Multi-speaker diarization for meetings, interruptions, and overlapping voices Hotword biasing for names, terms, and domain-specific vocabulary ~100 token/s on NVIDIA RTX 4090, RTF ~0.017

Thank you @sgl_project @vllm_project @Prince_Canuma @lllucas for day-0 support!

Github: https://github.com/OpenMOSS/MOSS-Transcribe-Diarize… Huggingface: https://huggingface.co/spaces/OpenMOSS-Team/MOSS-transcribe-diarize… API: https://shorturl.at/DWwe3 Live demo: https://shorturl.at/wRZ3j Technical Report：http://arxiv.org/abs/2601.01554 HF Space: https://huggingface.co/spaces/OpenMOSS-Team/MOSS-transcribe-diarize… AtomGit：https://ai.atomgit.com/OpenMOSS/MOSS-Transcribe-Diarize… SGLang-Omni: https://github.com/sgl-project/sglang-omni… vLLM: https://github.com/vllm-project/vllm… MLX-audio: https://github.com/Blaizzy/mlx-audio… Discord:https://discord.gg/SmVQHGffZU

OpenMOSS/MOSS-Transcribe-Diarize

Source: https://github.com/OpenMOSS/MOSS-Transcribe-Diarize

MOSS-Transcribe-Diarize

MOSS-Transcribe-Diarize 0.9B is an open-source SOTA end-to-end audio understanding model for long-form multi-speaker transcription, diarization, timestamps, and acoustic event awareness. MOSS-Transcribe-Diarize Pro is a stronger model with higher overall performance and will be available through API access soon.

News

• 2026-07-09: Open-sourced MOSS-Transcribe-Diarize 0.9B.

Contents

• Introduction
• Model Architecture
• Evaluation
  • Objective Evaluation
• Quickstart
  • Environment Setup
  • Python Usage
  • Custom Prompt and Hotwords
  • Serve with vLLM
  • Serve with SGLang Omni
  • Subtitle Web App
• Citation
• Star History

Introduction

MOSS-Transcribe-Diarize is our flagship SOTA model family for turning real-world long-form audio into structured, speaker-aware transcripts in one pass. Instead of stitching together separate ASR and diarization systems, these models jointly perform speech transcription and speaker diarization, producing time-aligned text with precise timestamps and consistent speaker labels such as [S01], [S02], and beyond.

Built for meetings, calls, podcasts, interviews, lectures, and video content, MOSS-Transcribe-Diarize is designed to handle long, messy, multi-speaker recordings where reliability matters. It can also emit optional acoustic event annotations, giving downstream systems a richer understanding of what happened, who spoke, and when.

The model accepts raw audio and emits a compact timestamped transcript. The canonical output format is:

[start_time][Sxx]transcribed speech[end_time]

Timestamps are expressed in seconds, and adjacent segments are concatenated into a single stream, for example:

[0.48][S01]Welcome everyone[1.66][12.26][S02]The new transcription pipeline is ready for evaluation[13.81][14.36][S01]Great, include the diarization results in the report[18.76]

Model Architecture

Evaluation

Objective Evaluation

We evaluate MOSS-Transcribe-Diarize using three objective metrics: Character Error Rate (CER), concatenated minimum-permutation Character Error Rate (cpCER), and Δcp. Lower is better for all metrics. Best results are bolded, second-best results are underlined. A dash (-) indicates that the result is unavailable.

Quickstart

Environment Setup

Use a clean Python environment. The project is tested with Python 3.12 and Transformers 5.x.

git clone https://github.com/OpenMOSS/MOSS-Transcribe-Diarize.git
cd MOSS-Transcribe-Diarize
uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e ".[torch-runtime]" --torch-backend=auto

Python Usage

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

from moss_transcribe_diarize import parse_transcript
from moss_transcribe_diarize.inference_utils import (
    build_transcription_messages,
    generate_transcription,
    resolve_device,
)

model_id = "OpenMOSS-Team/MOSS-Transcribe-Diarize"
audio_path = "audio.wav"

device = resolve_device("auto")
dtype = torch.bfloat16 if device.type == "cuda" else torch.float32

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype="auto",
).to(dtype=dtype).to(device).eval()
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)

messages = build_transcription_messages(audio_path)
result = generate_transcription(
    model,
    processor,
    messages,
    max_new_tokens=2048,
    do_sample=False,
    device=device,
    dtype=dtype,
)

print(result["text"])

for segment in parse_transcript(result["text"]):
    print(segment.start, segment.end, segment.speaker, segment.text)

The message flow follows the common Qwen multimodal pattern. The chat template is loaded from the model by AutoProcessor:

1. processor.apply_chat_template(messages, tokenize=False) renders text with audio placeholders.
2. process_audio_info(messages, sampling_rate) loads audio waveforms from the same messages.
3. processor(text=text, audio=audios) computes Whisper input features and expands audio placeholders.
4. model.generate(...) produces timestamped transcription and diarization text.

Custom Prompt and Hotwords

The default prompt is optimized for timestamped transcription and speaker diarization:

请将音频转写为文本，每一段需以起始时间戳和说话人编号（[S01]、[S02]、[S03]…）开头，正文为对应的语音内容，并在段末标注结束时间戳，以清晰标明该段语音范围。

To add hotwords, append a short hint to the default prompt:

请将音频转写为文本，每一段需以起始时间戳和说话人编号（[S01]、[S02]、[S03]…）开头，正文为对应的语音内容，并在段末标注结束时间戳，以清晰标明该段语音范围。热词提示：热词1, 热词2, 热词3

More prompt recipes are available in examples/prompts.md. The same prompt can be passed to build_transcription_messages, mtd-subtitle, and mtd-subtitle-web.

Serve with vLLM

MOSS-Transcribe-Diarize supports vLLM serving through the OpenAI-compatible transcription API. Use a pinned vLLM nightly build that includes the MOSS-Transcribe-Diarize model registration. Choose one of the following commands: for CUDA 12 environments, use cu129; for CUDA 13 environments, use cu130.

uv pip install -U vllm \
  --torch-backend=auto \
  --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129

or:

uv pip install -U vllm \
  --torch-backend=auto \
  --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu130

vllm serve OpenMOSS-Team/MOSS-Transcribe-Diarize --trust-remote-code

curl http://localhost:8000/v1/audio/transcriptions \
  -F model="OpenMOSS-Team/MOSS-Transcribe-Diarize" \
  -F file=@"audio.wav" \
  -F response_format="json" \
  -F temperature="0"

Serve with SGLang Omni

The recommended way to serve MOSS-Transcribe-Diarize is SGLang Omni through the OpenAI-compatible /v1/audio/transcriptions endpoint. Install sglang-omni by following the installation guide, then download the model:

hf download OpenMOSS-Team/MOSS-Transcribe-Diarize

Serve the model:

sgl-omni serve \
  --model-path OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --port 8000 \
  --max-running-requests 16 \
  --cuda-graph-max-bs 16 \
  --mem-fraction-static 0.80

Use response_format=verbose_json when you need parsed speaker segments. json returns the raw transcript text only.

curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F model=OpenMOSS-Team/MOSS-Transcribe-Diarize \
  -F [email protected] \
  -F response_format=verbose_json

import requests

with open("audio.wav", "rb") as f:
    resp = requests.post(
        "http://localhost:8000/v1/audio/transcriptions",
        data={
            "model": "OpenMOSS-Team/MOSS-Transcribe-Diarize",
            "response_format": "verbose_json",
        },
        files={"file": ("audio.wav", f, "audio/wav")},
        timeout=300,
    )

resp.raise_for_status()
payload = resp.json()
print(payload["text"])
for segment in payload.get("segments", []):
    print(f"[{segment['start']:.2f}-{segment['end']:.2f}] {segment['text']}")

For longer multi-speaker audio, raise max_new_tokens so the decoder can finish the full diarized transcript:

curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F model=OpenMOSS-Team/MOSS-Transcribe-Diarize \
  -F [email protected] \
  -F response_format=verbose_json \
  -F max_new_tokens=65536

For benchmarking, performance numbers, and implementation details, see the SGLang Omni cookbook. The following single-H100 results are reported for short- and long-sequence multi-speaker ASR tasks.

movies short-sequence ASR:

aishell4_long long-sequence ASR:

Subtitle Web App

The package also includes a local subtitle workflow for upload, review, subtitle export, and optional FFmpeg burn-in:

mtd-subtitle-web \
  --model OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --host 127.0.0.1 \
  --port 7860

Open http://127.0.0.1:7860, upload an audio/video file, review the parsed subtitle segments, then download JSON/SRT/ASS or burn an MP4 if ffmpeg and ffprobe are available on PATH.

For batch processing:

mtd-subtitle /path/to/input.mp4 \
  --model OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --out-dir runs/example \
  --render

Citation

If you use MOSS-Transcribe-Diarize, please cite the technical report:

@misc{moss_transcribe_diarize_2026,
  title={MOSS Transcribe Diarize Technical Report},
  author={{MOSI.AI}},
  year={2026},
  eprint={2601.01554},
  archivePrefix={arXiv},
  primaryClass={cs.SD},
  url={https://arxiv.org/abs/2601.01554}
}

Star History