Tag
WeMM-Video introduces a family of Chinese-English bilingual omni-modal foundation models (2B/4B/9B/35B-A3B) that unify text, image, audio, and video for joint audio-visual reasoning, achieving a new open-source state of the art on omni-modal video benchmarks and outperforming proprietary models like Qwen3.5-Omni-Flash and Seed2.0-Mini-0428.
LiquidAI 的 LFM3-Omni-2B 成为首个可在浏览器中通过 WebGPU 100% 本地运行的全模态模型,无需独立 ASR/TTS,在 M4 Max 上实现约 280 token/s、90ms 首 token 延迟。
Youtu-Parsing-Omni 是一个统一的多模态解析器,用单一编码器和 OmniSchema 将文档、图像、音频与视频编码为时空结构化记录,并通过自蒸馏后训练(OSAD)在 OmniDocBench v1.6 等基准上取得最先进成绩,5B 检查点已开源。
Liquid AI 发布 LFM3-Omni-2B 的 GGUF 版本,支持在 CPU/GPU 上本地推理。该模型可理解文本、图像、视频和音频,并以文本和语音输出,专为设备端使用打造。
本文研究多模态大模型中的检索头机制如何随输入模态扩展,发现跨模态共享的检索头虽不足2%,却对文档理解、空间感知等至关重要,并通过排序目标优化这些注意力头,将Qwen3-VL-8B在MMDocIR上的页面Recall@1提升了6.03个点。
The paper introduces MOSAIC, an omni-modal benchmark of 923 questions over persona-grounded multi-session histories spanning text, image, video, and audio, designed to diagnose where conversational agents lose evidence across write, retrieve, deliver, and read stages rather than just producing a single leaderboard score.
Liquid AI 发布 LFM3-Omni-2B,其 LFM3 家族首款全模态模型,可理解文本、图像、视频和音频,并以文本和语音作答,专为手机、笔记本等边缘设备设计,2.2B 参数即可替代独立的 ASR、VLM、LLM 和 TTS 模型。
The paper introduces FocusGUI, a lightweight adapter that dynamically selects relevant multimodal evidence for GUI agents to prevent compounding errors from irrelevant context, improving action accuracy without fine-tuning the backbone.
OmniCompass 提出了一个面向全模态视频定位与理解的整体基准,涵盖视觉-语言与音视频两条评测轨道,包含时间定位、空间定位与问答三类任务,并对50个多模态大模型进行了系统评估。研究发现开源模型在音视频理解上仍落后于闭源模型,精确的时间与空间定位对领先模型而言仍具挑战性。
该论文发现全模态大模型(Omni-LLMs)存在'知道却说不出'(knows-but-cannot-say)的现象,即能感知非文本模态信息却难以在输出中表达。作者提出Token级模态对齐(TMA)指标和OmniCatalyst方法,通过自生成文本补充模态输入,在七个Omni-LLM上平均提升7.29%性能并平衡模态偏好。
Tencent 发布 Youtu-Parsing-Omni,一个 5B 的全能模态解析模型,统一处理文档、图像、图表、音频和视频输入并输出结构化 JSON,在 OmniDocBench v1.6 上取得 96.96 的最佳成绩,并开源模型权重与 vLLM 插件。
该报告介绍了MiMo-V2.6系列全模态模型,通过扩大强化学习计算规模(大批量异步训练、多样化环境、群组智能体评分)推动模型智能走向自我改进,并开源了训练动态、RL环境与框架。
OmniConfess proposes a training-free inference-time method that re-scores fixed candidate responses at token resolution under channel-wise evidence interventions to expose and mitigate omni-modal hallucinations in OmniLLMs. The paper also introduces OmniHalluBench, a 3,540-example benchmark spanning text, image, audio, and video tasks.
A research paper proposing a text-centric post-training paradigm for Omni-LLMs, where text-only reasoning training drives most gains (boosting Qwen2.5-Omni-7B's reasoning scores by 25.83% with 56.6% fewer GPU-hours) and reduced-data native audio-visual RL restores perception, retaining 93.5% of reasoning gains at ~90% fewer input tokens.
OmniReasoning introduces OmniReasoningBench, a data engine (OmniQA) producing SFT and RL training sets, and a Modality-Factored Self-Distillation method, yielding a 30B-A3B model that outperforms Qwen3-Omni-30B-A3B-Thinking by double-digit margins on audio-visual joint reasoning benchmarks.
LEAP introduces a block-wise evidence retrieval framework for hour-scale audio-visual question answering that decouples localization from reasoning, keeping context independent of recording duration and improving over Qwen3-Omni-30B-A3B by 4.5-16.8% while transferring to MiniCPM-o4.5.
Omni-Decision introduces an evidence-ledger planning approach for omni-modal agents to handle noisy multimodal observations, achieving state-of-the-art accuracy on OmniGAIA and WorldSense benchmarks.
The paper introduces Ovis-Embedding, a state-of-the-art omni-modal embedding model that uses a shared backbone to encode text, image, video, and audio in a common representation space, achieving top performance on benchmarks like MMEB-v3 and MVEB.
OmniEcho introduces a spatially aware omni-modal model and a new benchmark for spatial audio-visual perception and navigation in embodied agents, achieving state-of-the-art performance.
This paper evaluates MiniMax-H3, an omni-modal generative model, by introducing a comprehensive framework to assess its reasoning about the physical world through multimodal inputs. The evaluation reveals that video-based decision reasoning performs best, while audio-based disambiguation reasoning is the weakest.