Tag
The paper proposes MOSAIC, a modality-separated architecture with inter-modal cross-attention and modality-partitioned gradient routing, which improves average macro-F1 by 7.4 points across four tasks on the CEJC corpus while reducing backbone parameters by 72% versus fusion-based baselines.
This paper investigates how different pretrained visual representations organize Indian foods into similarity neighborhoods, finding that food similarity depends on representation, modality, and abstraction level rather than being an intrinsic property, using an expanded Khana dataset and seven visual models.
该论文提出了一种接触式多模态筛查框架，利用与 Amazon Alexa 交互过程中采集的视频、音频和文本信号来检测轻度认知障碍（MCI），实验表明多模态融合模型在 Command-Reading 和 Question-Answering 任务上均达到 0.76 的准确率，优于单一模态模型。
论文提出 CogDrills-4D（CogDrift-4D），一种层级式 4D Transformer 多模态框架，通过 18 个月内七次非脚本化交互的行为数据建模时间性行为衰退，用于轻度认知障碍（MCI）早期检测，分类准确率达 0.733±0.13。
该论文通过受控消融实验研究原生多模态预训练中视觉信号对扩展行为的影响，发现视觉内容作为可扩展锚点不仅能提升多模态理解，其收益还能延伸到纯文本空间推理任务（如定位、导航和符号规划），同时不损害整体语言性能。
EVA-Cap 提出一种事件中心的音视频视频描述框架，将描述分解为原子事件并通过 SFT 与 GRPO 优化对齐，在 ChronusAV 上平均提升 9.0%，在 WorldSense 上提升 3.3%。
Lluna improves multimodal predictions when some input modalities are missing by combining private unimodal predictions with a graph-based residual collaboration branch and progressive consistency training, achieving a 4.26% average improvement in Mean-Missing over baselines across eight datasets.
PXtal 提出一种在模态间信息不对称条件下对齐粉末X射线衍射（PXRD）与晶体结构表示的框架，利用非平衡最优传输（UOT）和广义Kullback-Leibler散度监督传输计划，在PXRD到晶体候选检索及下游材料任务上优于基线模型。
该论文提出一种泄漏可控的多模态多任务学习框架，结合SFCN MRI编码器、因果临床Transformer和ODE-GRU动力学，用于阿尔茨海默病诊断与病程进展预测，在ADNI、OASIS-2、MIRIAD数据集上取得内部诊断AUROC 0.935、混合AUROC 0.951，并在OASIS-3上完成外部验证。
TouchScale 引入了一个 500 小时的人类视觉-触觉交互数据集，使用统一的可穿戴设备同步采集 egocentric RGB-D 视频、手腕视频与全手双触觉测量，用于视觉-触觉预训练和机器人操作策略学习，在零样本触觉预测和真实机器人操作成功率上均取得显著提升。
TasteBench introduces a multimodal benchmark and privacy-preserving competition for sensory prediction, spanning food-level ranking (21K+ human evaluations across 215 plant-based foods) and molecular-level taste classification (15K flavor molecules), with baselines reaching pairwise accuracy competitive with median human panelists. It aims to provide computational proxies for sustainable protein discovery, analogous to molecular docking in drug design.
MERID 是一个多模态抑郁症分析框架，通过经验驱动的递归自我改进智能体自动探索和优化检测流水线，在多个抑郁症基准上取得领先结果。
The paper introduces CALIBRA, a calibration-first multimodal temporal framework for cross-cohort asthma risk forecasting that uses recurrent encoders, gradient-reversal domain adaptation, hierarchical logistic calibration, and conformal prediction to remain reliable under distribution shift and incomplete modalities, validated on a semi-synthetic three-cohort benchmark.
The paper proposes Tex-Zero, a native 3D texture generation framework trained entirely on constructed 2D images rather than real 3D assets, by converting high-quality 2D images into 3D training samples via plane representations and patch-wise random rotations, showing high-fidelity texture synthesis is possible without 3D asset data.
PixelJev is introduced as a native-image decision interface using small open multimodal models to map images, instructions, and candidate sets to structured choices. The study demonstrates adaptation improves accuracy on benchmarks like Pets and highlights challenges in calibration and generalization.
The paper proposes AV-GRPO, a modality-anchored reinforcement learning framework for joint audio-video generation that improves generation quality, semantic alignment, and cross-modal synchronization over existing methods.
This paper systematically evaluates input representations (images, text, or both) for multimodal document QA, finding that images improve accuracy but increase latency, and proposes a TF-IDF router to optimize performance.
The paper introduces RecCAR, a regularization method to address the reciprocal correspondence gap in joint multimodal diffusion transformers, improving performance in video generation tasks.
This paper introduces a generalized multimodal foundation model capable of handling arbitrary modality combinations and prediction tasks, achieving competitive performance through training on large-scale synthetic datasets with diverse causal structures.
The study identifies 'ECG Mirage' in vision-language models for clinical prediction, where models underutilize ECG data despite apparent multimodal capability, and proposes visual prompt tuning as an efficient mitigation strategy.