MOSS-VL Technical Report

Hugging Face Daily Papers Papers
vision-language-model real-time-interaction open-source streaming-benchmark gated-cross-attention video-understanding

Summary

MOSS-VL is an open vision-language model family designed for real-time interaction, using gated cross-attention to process vision during generation and achieving top performance in streaming benchmarks among open-source models.

We present MOSS-VL, an open vision-language model family that treats real-time interaction -- perceiving while it speaks -- as a first-class capability. It is co-designed across the stack: the language decoder attends to vision only through gated cross-attention, so the model can naturally see incoming frames while generating; a synthesized interaction corpus supervises when to speak, when to stay silent, and when to revise; and a staged curriculum concentrates all real-time-specific training in one light final stage over a strong offline foundation. Offline, MOSS-VL-Instruct is competitive at comparable scale and leads temporal-reasoning video sets. Across four streaming benchmarks, MOSS-VL-Realtime posts the best average on three (second on the fourth) among open-source streaming models, sweeping the three subsets that squarely test proactive behavior -- 66.0 vs. 37.5 for the best baseline on OmniMMI Proactive Alerting. With 11.3B parameters but visual tokens outside the decoded sequence, MOSS-VL widens its time-to-first-token advantage over same-backbone Qwen3-VL-8B from 2.8x to 5.1x as visual context grows. We release all five checkpoints, the training curriculum, and the real-time inference code at https://github.com/OpenMOSS/MOSS-VL.
Original Article
View Cached Full Text

Cached at: 08/18/26, 11:51 AM

Paper page - MOSS-VL Technical Report

Source: https://huggingface.co/papers/2608.15045 Published on Aug 15

#3 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

MOSS-VL is an open vision-language model family enabling real-time interaction by attending to vision via gated cross-attention during generation, using a synthesized interaction corpus and staged curriculum to achieve strong streaming performance with reduced time-to-first-token latency.

We present MOSS-VL, an openvision-language modelfamily that treatsreal-time interaction-- perceiving while it speaks -- as a first-class capability. It is co-designed across the stack: the language decoder attends to vision only throughgated cross-attention, so the model can naturally see incoming frames while generating; a synthesizedinteraction corpussupervises when to speak, when to stay silent, and when to revise; and astaged curriculumconcentrates all real-time-specific training in one light final stage over a strong offline foundation. Offline, MOSS-VL-Instruct is competitive at comparable scale and leads temporal-reasoning video sets. Across fourstreaming benchmarks, MOSS-VL-Realtime posts the best average on three (second on the fourth) among open-source streaming models, sweeping the three subsets that squarely test proactive behavior -- 66.0 vs. 37.5 for the best baseline on OmniMMI Proactive Alerting. With 11.3B parameters but visual tokens outside the decoded sequence, MOSS-VL widens itstime-to-first-tokenadvantage over same-backbone Qwen3-VL-8B from 2.8x to 5.1x as visual context grows. We release all five checkpoints, the training curriculum, and the real-time inference code at https://github.com/OpenMOSS/MOSS-VL.

View arXiv pageView PDFProject pageGitHub430Add to collection

Get this paper in your agent:

hf papers read 2608\.15045

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper7

#### OpenMOSS-Team/MOSS-VL-Realtime Video-Text-to-Text• 11B• Updatedabout 4 hours ago • 1.81k • 104 #### OpenMOSS-Team/MOSS-VL-Instruct-0708 Video-Text-to-Text• 11B• Updatedabout 4 hours ago • 844 • 29 #### OpenMOSS-Team/MOSS-VL-Base-0708 Video-Text-to-Text• 11B• Updatedabout 4 hours ago • 45 • 18 #### OpenMOSS-Team/MOSS-VL-Instruct-0708-FP8 Video-Text-to-Text• 11B• Updatedabout 4 hours ago • 63 • 17 Browse 7 models citing this paper## Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.15045 in a dataset README.md to link it from this page.

Spaces citing this paper5

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

OpenMOSS-Team/MOSS-VL-Realtime

Hugging Face Models Trending

MOSS-VL-Realtime is a realtime streaming vision-language model that processes continuous video frames, supports interruptible interaction, proactive silence, and dynamic correction, with timestamp-aware encoding and a 256K context window.

MAOAM: Unified Object and Material Selection with Vision-Language Models

Hugging Face Daily Papers

This paper presents MAOAM, a unified vision-language model framework that enables precise object and material selection through text or click interactions for interactive image editing. It introduces a scalable data generation pipeline and shows emergent improvement when combining text and clicks at inference.

Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model

Hugging Face Daily Papers

Mage-VL is an efficient codec-native streaming multimodal foundation model that reduces visual token consumption by over 75% using a custom tokenizer, achieving up to 3.5x inference speedup while matching or outperforming existing models on static and video tasks.

CaVe-VLM-CoT: An Interpretable Vision-Language Model Framework

arXiv cs.AI

CaVe-VLM-CoT is a modular reflection-based agentic-RAG framework for vision-language models that enforces evidence-grounded reasoning through a five-stage pipeline, achieving 87.1% accuracy on ScienceQA and proposing a suite of 23 metrics for evaluation.