multimodal-llm

Tag

Cards List
#multimodal-llm

Incongruence-Aware Evaluation of Multimodal Speech Emotion Recognition in Audio Large Language Models

arXiv cs.AI ↗ · 6d ago Cached

The paper introduces an incongruence-aware evaluation framework measuring disagreement between textual and acoustic emotional cues in audio LLMs, finding that cross-modal incongruence consistently increases emotion recognition errors across three speech corpora while supervised classifiers show the opposite trend.

0 favorites 0 likes
#multimodal-llm

Aligning MLLMs with the Latent Structure of Human Cognition via Behavior-Derived Semantic Dimensions

arXiv cs.CL ↗ · 6d ago Cached

该论文提出一个通过行为推导的语义维度将多模态大语言模型(MLLM)与人类相似性判断的认知结构对齐的框架,并借助 fMRI 数据的搜索light RSA 验证了表征层面的人机对齐提升。

0 favorites 0 likes
#multimodal-llm

AskStream: When and What to Ask Proactively under Uncertainty in Streaming Video Understanding

Hugging Face Daily Papers ↗ · 2026-10-28 Cached

论文提出 AskStream,首个覆盖视频与用户不确定性的流式视频理解多轮基准,重新定义主动交互为等待、提问、回答三类动作,并指出即使最强模型在 Ask-F1 指标上也仅有 52.18%,凸显主动信息获取的不足。

0 favorites 0 likes
#multimodal-llm

Meta's Interactive Simulator for Wearable AI (GitHub Repo)

TLDR AI ↗ · 2026-10-28 Cached

Meta 开源了 InteractiveWorld——一个面向可穿戴 AI 助手的交互式仿真环境,配合 InteractiveBench 基准(120 个多轮任务),用于构建和评估能在第一人称视角下观察、沟通并操作 3D 世界的多模态大模型助手。

0 favorites 0 likes
#multimodal-llm

@_akhaliq: InteractiveWorld A Verifiable Human-Centric Environment for Building Better Interactive MLLM Assistants paper: https://…

X AI KOLs Timeline ↗ · 2026-10-27 Cached

InteractiveWorld introduces the first verifiable simulation environment and InteractiveBench benchmark for evaluating whether MLLMs can reliably guide humans through physical tasks, finding that even frontier models like Claude Fable 5.1 achieve at most 20-25% on long-horizon guidance and planning tasks.

0 favorites 0 likes
#multimodal-llm

CodeCap-Bench: Can Code Serve as a Good Video Caption?

Hugging Face Daily Papers ↗ · 2026-10-27 Cached

CodeCap-Bench 提出用可执行代码作为视频字幕的新范式,构建了包含 323 个视频的基准来评估多模态大模型生成代码字幕的能力,结果显示代码字幕在推理和重建任务上优于自然语言,但现有模型在捕捉复杂时间动态方面仍存在不足。

0 favorites 0 likes
#multimodal-llm

DriveMind: Mind-Evolving Belief Tracking for Closed-Loop Autonomous Driving

arXiv cs.AI ↗ · 2026-10-26

DriveMind 是一个基于多模态大模型的自动驾驶框架,借鉴粒子滤波思想维护并更新多个场景假设,在 Bench2Drive 上将驾驶分数从 68.50 提升至 81.37,在无信号交叉口等歧义场景中提升尤为显著。

0 favorites 0 likes
#multimodal-llm

InteractiveWorld: A Verifiable Human-Centric Environment for Building Better Interactive MLLM Assistants

Papers with Code Trending ↗ · 2026-10-25 Cached

InteractiveWorld 提出了首个用于人机闭环交互的可验证模拟环境,并配套发布 InteractiveBench 基准(含六大类 120 个多轮任务),用于评估 MLLM 引导人类完成物理任务的能力;实验显示 12 个前沿 MLLM 仍远未可靠,最强模型在长程任务上得分仅约 20-25%。

0 favorites 0 likes
#multimodal-llm

Humans in Space: Can MLLMs Construct Human-Centric Spatial Understanding via Active Exocentric Exploration?

Hugging Face Daily Papers ↗ · 2026-10-25 Cached

论文提出SATE-Bench交互式基准,包含316个真实场景与2525个多轮QA对,用于评估多模态大模型的视觉自我感知与主动全景视角探索能力。结果显示SOTA MLLM往往在证据不足时过早停止探索且自信度过高,而GRPO微调虽提升探索行为却因过度探索和多视角融合失败而降低整体准确率。

0 favorites 0 likes
#multimodal-llm

AV-FABLE: Audio-Visual Manipulation Forensics in Long-Form Videos

Hugging Face Daily Papers ↗ · 2026-10-24 Cached

AV-FABLE introduces a long-form audio-visual benchmark for sparse manipulation detection, showing that frozen AV-LLM perception features contain forensic signals best recovered by separate temporal decoders (NFSR) rather than language outputs.

0 favorites 0 likes
#multimodal-llm

A Broader Look into MLLM's Retrieval Heads

Hugging Face Daily Papers ↗ · 2026-10-23 Cached

该论文研究多模态大模型(MLLM)中的稀疏检索头在新增音频-视觉模态下的分布与共享机制,并通过排序目标优化这些注意力头,将 Qwen3-VL-8B 在 MMDocIR 上的页面 Recall@1 提升 6.03 个点。

0 favorites 0 likes
#multimodal-llm

OSynth: A Longitudinal, Context-Sensitive Evaluation of Meeting Understanding

arXiv cs.CL ↗ · 2026-10-22 Cached

Otter.ai researchers present OSynth, a large-scale synthetic meeting dataset of 570 hours with latent ground-truth state, showing that state-of-the-art MLLMs gain significant quality on entity-sensitive dimensions when given broader workspace context. Human raters distinguished OSynth meetings from real transcripts with only 56% accuracy.

0 favorites 0 likes
#multimodal-llm

EDMA: Entropy-Driven Multimodal Answering

arXiv cs.AI ↗ · 2026-10-21 Cached

EDMA提出了一种基于逻辑熵的多模态问答方法,将答题过程形式化为熵减迭代,并量化各模态对答案的贡献,在多个MMQA基准上显著优于Chain-of-Thought等基线。

0 favorites 0 likes
#multimodal-llm

CausGround: Causal Interaction Grounding for Adaptive Embodied Planning

Hugging Face Daily Papers ↗ · 2026-10-21 Cached

CausGround 提出一种非参数化的因果交互 grounding 框架,从交互经验中提炼可复用的动作前置条件与效果,无需重训模型即可提升 MLLM 在 VirtualHome、RoboCasa 等基准上的具身规划成功率(如 24.9% 提升至 53.3%)。

0 favorites 0 likes
#multimodal-llm

A Plausible Explanation Is Enough to Mislead Multimodal Judges

arXiv cs.CL ↗ · 2026-10-19 Cached

该论文提出了一个名为JUPE的视觉问答基准,发现普通的看似合理的解释就足以误导14个多模态大模型评判者,使错误答案的拒绝率下降约12.7-13.8个百分点,研究者将此现象称为“plausibility capture”。

0 favorites 0 likes
#multimodal-llm

Brain-Aligned Multimodal LLMs for Social and Emotional Understanding of Cartoon Movies

arXiv cs.AI ↗ · 2026-10-19 Cached

该论文比较了扩散式与自回归多模态大语言模型生成的标注与人类在观看卡通短片时的Theory of Mind和情感效价评分的时间一致性,并检验其对fMRI脑反应的预测能力,发现MLLM标注可作为脑预测性社会情感内容标注的可扩展工具,但与人类评分存在系统性差异且对齐效果因模型和影片而异。

0 favorites 0 likes
#multimodal-llm

ECHOES: Seeing Beyond the Current Image via Recursive History Compression and Privileged On-Policy Distillation

arXiv cs.AI ↗ · 2026-10-19 Cached

ECHOES 提出一种通过递归历史压缩与特权在线蒸馏来增强长时程 GUI 智能体状态保持能力的方法,无需在部署时保留历史截图即可取得有竞争力的性能,并发布了细粒度诊断基准 ECHOESBMK。

0 favorites 0 likes
#multimodal-llm

HumanSkill-4D: Training-Free Language Querying of Animatable Gaussian Avatars in Space and Time

Hugging Face Daily Papers ↗ · 2026-10-17 Cached

HumanSkill-4D 提出一个免训练框架,通过共享多模态大语言模型与几何算子,实现对可动画高斯人体头像在空间与时间上的语言查询,支持部件分割与 3D/4D 指代分割,并引入三个标注基准以超越现有基线。

0 favorites 0 likes
#multimodal-llm

Xiaomi-Lens-0 Technical Report

Hugging Face Daily Papers ↗ · 2026-10-16 Cached

Xiaomi-Lens-0 is a 4B multimodal model built on Qwen3.5 that predicts masks from its internal visual representations to improve fine-grained region reasoning, gaining 5.6 points on average over Qwen3.5 across RefCOCO-family referring comprehension benchmarks.

0 favorites 0 likes
#multimodal-llm

Decoupling Knowledge Forgetting and Refusal Learning in MLLM Unlearning

arXiv cs.AI ↗ · 2026-10-15 Cached

该论文提出 Forget-Aware Self-Distillation (FASD) 方法,将多模态大模型遗忘解耦为行为通道与知识通道,以实现「诚实无知」——既真正擦除目标知识,又让模型能承认自身知识边界,避免幻觉与信息泄露。

0 favorites 0 likes
Next →
← Back to home

Submit Feedback