multimodal-agents

Tag

Cards List
#multimodal-agents

Continual Tool-Space Extension Learning for Multimodal Agents

arXiv cs.AI ↗ · yesterday Cached

The paper proposes COTEX, a continual tool-space extension learning method that adds new specialist tools to trained multimodal agents via orthogonally constrained low-rank adapters, preserving existing tool capabilities while achieving gains across nine vision-language benchmarks.

0 favorites 0 likes
#multimodal-agents

Vision-as-a-Tool: Visual Harness for Multimodal Agentic Coding

arXiv cs.AI ↗ · 5d ago Cached

The paper introduces the Vision-as-a-Tool (VaaT) paradigm, reframing multimodal large language models as callable external utilities so text-only coding agents can offload visual reasoning, instantiated in the VHarness framework evaluated on Image-to-HTML/SVG/3D tasks with improved visual fidelity and geometric accuracy.

0 favorites 0 likes
#multimodal-agents

WorldDraft: If Code Builds the World, Can Video Models Just Render It?

Hugging Face Daily Papers ↗ · 2026-11-12 Cached

The paper introduces WorldDraft, a benchmark of 432 paired test cases across 48 executable 4D world families, evaluating whether video models can act as generative renderers of code-built worlds. It finds a stark tension between world preservation and visual realization across 12 models, and proposes WorldDraft-Primer, a plug-and-play visual prompting strategy that improves both objectives.

0 favorites 0 likes
#multimodal-agents

A Cross-Scenario Survey and Analysis of Self-Evolving Multimodal Agents

Hugging Face Daily Papers ↗ · 2026-11-10 Cached

A survey analyzing why self-improvement mechanisms for multimodal agents transfer unevenly across reasoning, generation, navigation, and manipulation scenarios, framing them as trade-offs over scarce factors and task-structural resources, and identifying portable, conditional, blocked, or redundant mechanisms across transfer directions.

0 favorites 0 likes
#multimodal-agents

HiVe-Mem: Interleaving Horizontal Memory Expansion and Vertical Evidence Composition for Multimodal Agents

Hugging Face Daily Papers ↗ · 2026-11-09 Cached

HiVe-Mem 提出一种多模态记忆框架,通过横向记忆扩展(关系图扩展语义种子)与纵向证据组合(查询自适应选择异构证据形式)交替进行,提升多模态智能体的长期交互记忆能力,并在三个基准上以约 90% 的 token 消耗减少持续改进任务性能。

0 favorites 0 likes
#multimodal-agents

CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments

arXiv cs.AI ↗ · 2026-10-28

CogArena は社会科学・行動科学の10の実験パラダイムをブラウザ経由でエージェントに実行させるベンチマークを提案し、マルチモーダルモデル4種を評価した結果、実験の完了と行動特性の再現が別物であることを示した。

0 favorites 0 likes
#multimodal-agents

OmniBrain-0: A Unified Agentic Model for Embodied Intelligence

arXiv cs.CL ↗ · 2026-10-28

OmniBrain-0 is a lightweight unified embodied agent that coordinates manipulation and navigation through a shared high-level decision model, trained via multi-teacher on-policy distillation and a rubric-guided recursive self-improvement data flywheel, achieving strong cross-domain performance with much lower overhead than its closed-source teacher.

0 favorites 0 likes
#multimodal-agents

Seeing the Options: Typed Decisions over Visual Candidates with Reusable Option Memory

arXiv cs.AI ↗ · 2026-10-27 Cached

This paper introduces Iris, a multimodal System One architecture that extends typed decisions over text candidates to image candidates via independent per-candidate encoding, preserving order invariance and exact cache reuse while achieving strong accuracy on a visual-candidate benchmark built from Multimodal-Mind2Web, plus a 37×–191× latency reduction from candidate reuse and a fix for a Qwen3.5 vision tower Conv3d bottleneck under bf16 cuDNN.

0 favorites 0 likes
#multimodal-agents

VibeVideoBench: Evaluating Multimodal Agents on Code-Driven Video Generation

Hugging Face Daily Papers ↗ · 2026-10-27 Cached

VibeVideoBench introduces a benchmark of 239 tasks evaluating multimodal agents that generate videos by writing rendering code, finding that GPT-6 Astra and Claude Opus 5.5 lead but that time-management failures cause most lost videos.

0 favorites 0 likes
#multimodal-agents

Adaptive Control of Scientific Instruments with Nonstationary Responses through Multimodal Agents

arXiv cs.AI ↗ · 2026-10-26

本文提出一个物理接地的多模态智能体框架,用于磁约束聚变装置中性束注入离子源的非平稳自适应调控,在EXL-50U平台455次放电的时序回放中验证了符号化物理量优于波形图像,并在监督运行下实现86.80%的束提取率。

0 favorites 0 likes
#multimodal-agents

The Trinity:Capability-Driven Recursive Self-Improvement for Multimodal Agents

Hugging Face Daily Papers ↗ · 2026-10-26 Cached

论文提出 The Trinity 框架,通过 Creator、Explorer、Evolver 三个角色将多模态智能体的递归自我改进组织成能力树与 Harness 树,在 28 个训练基准上平均提升 34 分,18 个未见基准上提升 26 分。

0 favorites 0 likes
#multimodal-agents

MMedAgent-Plus: Scaling Multimodal Medical Agents across Modalities, Tasks and Workflows

arXiv cs.CL ↗ · 2026-10-21 Cached

MMedAgent-Plus is a multimodal medical tool-use agent that pairs a vision-language planner with 18 specialist models across 12 medical tasks and 8 modalities, orchestrating multi-step clinical workflows. It achieves 90.6% correct specialist invocation and 0.98 final-answer accuracy, far exceeding the strongest baseline (0.28).

0 favorites 0 likes
#multimodal-agents

Multimodal Embodied Agents: A Unified Capability-centric Perspective from Computer-Use to Robot-Use

Hugging Face Daily Papers ↗ · 2026-10-21 Cached

PAPAV 提出了一个以能力为核心的统一框架(Perceive、Anticipate、Plan、Act、Verify),用于对比多模态具身智能体、计算机使用智能体与机器人系统,并基于 64 个基准揭示了 Anticipate 与 Verify 能力评估的显著缺口。

0 favorites 0 likes
#multimodal-agents

OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents

Hugging Face Daily Papers ↗ · 2026-10-15 Cached

OS-Helix is an end-to-end trained multimodal agent that unifies GUI actions and code execution in a single action space, using hybrid SFT plus online RL with a Step-balanced GSPO objective to reach 66.44% success on OS-World-Verified while showing cross-platform transfer to WindowsAgentArena.

0 favorites 0 likes
#multimodal-agents

Seeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memory

Hugging Face Daily Papers ↗ · 2026-10-14 Cached

该论文提出了一个面向多模态智能体记忆的双轴分析框架:轴一关注多模态信息的形成与表示方式,轴二将记忆管理划分为静态记忆、动态记忆和自进化记忆三个层级,并总结了相关设计模式与未来研究方向。

0 favorites 0 likes
#multimodal-agents

VIEWCRAFT: Evolving What Multimodal Agents See and How They Read It

Hugging Face Daily Papers ↗ · 2026-10-11 Cached

VIEWCRAFT 通过联合进化多模态代理的证据界面与读取流程,在八个基准上平均提升 6.1 分任务得分,并以更少的输入 token 实现跨执行器迁移。

0 favorites 0 likes
#multimodal-agents

RoboQuest: Generalist Physical Agents that Search, Inspect and Test

Hugging Face Daily Papers ↗ · 2026-10-07 Cached

RoboQuest introduces a benchmark for goal-directed embodied exploration in mobile manipulation, testing frontier multimodal agents on search, inspection, and interactive testing tasks. Results show the best agent succeeds in only 23% of episodes, with failures mainly attributed to premature stopping of exploration rather than execution skills.

0 favorites 0 likes
#multimodal-agents

RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments

Hugging Face Daily Papers ↗ · 2026-10-07 Cached

RobotWorld 提出了一个涵盖 84 项任务的机器人智能体仿真基准,横跨操作、移动操作、运动、驾驶与空中控制,用于评估多模态智能体将数字任务能力迁移到物理世界的表现,并揭示了模型间的能力差异与关键短板。

0 favorites 0 likes
#multimodal-agents

VisAudit: Evaluating Multimodal Agents for Visual Diagnosis and Repair

arXiv cs.LG ↗ · 2026-10-05 Cached

VisAudit is a new benchmark for evaluating multimodal agents on autonomously diagnosing, repairing, and verifying flawed data visualizations, spanning 1,900 flawed charts across 21 chart types and 10 flaw categories. Experiments show the strongest multimodal model fully recovers only 47.4% of flawed charts in the autonomous-repair setting, revealing a substantial gap toward human-like visualization review.

0 favorites 0 likes
#multimodal-agents

Learning to Route in Visual Space via Multi-Step Embedding Retrieval

arXiv cs.AI ↗ · 2026-10-01 Cached

This paper introduces VHop, a data generation framework and benchmark for multi-step visual retrieval, and VHop-Router, an end-to-end trained autoregressive multi-step retriever that operates in visual latent space, boosting retrieval from under 5% to 76.3% and improving agentic search success by 52.7% while drastically cutting tokens and API payload.

0 favorites 0 likes
Next →
← Back to home

Submit Feedback