一颗心意味着递归：探索视觉语言模型中的视觉意义与交流

arXiv cs.AI 论文
vision-language-models visual-communication multimodal interpretability emergent-behavior embodied-communication

摘要

一篇研究论文探讨了视觉语言模型（VLM）如何使用和解读视觉符号，发现像红心这样的简单视觉图案可以隐式引导模型行为（例如，使其倾向于采用递归编程方案），而且模型能够发展出不同于人类惯例的创造性视觉交流策略。

arXiv:2610.36449v1 Announce Type: new Abstract: Do vision-language models (VLMs) share our ability to communicate visually? We share preliminary findings, starting from a surprising observation: simple visual symbols like a red heart can strongly influence model behavior on seemingly unrelated tasks, such as steering models towards recursive solutions on simple coding problems. Notably, this steering effect is largely implicit, as models' direct interpretations and thinking traces about visual symbols diverge substantially from their observed behavior. In visual communication between models, this leads to a generation--interpretation gap: while models tend to send shape primitives that they (and humans) find intuitive, these visual cues often fail to steer other models. In further communication games involving abstract concepts and stories, models can creatively use visuals to transmit meaning to one another, although some communications remain non-intuitive to humans. These findings reveal that VLMs might be capable of rich, creative, and visually-grounded communication, but their interpretation and production of visual meaning diverges from human conventions in surprising ways. Together this work highlights both the promise and challenges of visual communication between humans and VLMs.
查看原文
查看缓存全文

缓存时间: 2026/10/28 07:56 

# A Heart Means Recursion: Exploring Visual Meaning and Communication in VLMs
Source: [https://arxiv.org/abs/2610.36449](https://arxiv.org/abs/2610.36449)
[View PDF](https://arxiv.org/pdf/2610.36449)[HTML \(experimental\)](https://arxiv.org/html/2610.36449v1)

> Abstract:Do vision\-language models \(VLMs\) share our ability to communicate visually? We share preliminary findings, starting from a surprising observation: simple visual symbols like a red heart can strongly influence model behavior on seemingly unrelated tasks, such as steering models towards recursive solutions on simple coding problems\. Notably, this steering effect is largely implicit, as models' direct interpretations and thinking traces about visual symbols diverge substantially from their observed behavior\. In visual communication between models, this leads to a generation\-\-interpretation gap: while models tend to send shape primitives that they \(and humans\) find intuitive, these visual cues often fail to steer other models\. In further communication games involving abstract concepts and stories, models can creatively use visuals to transmit meaning to one another, although some communications remain non\-intuitive to humans\. These findings reveal that VLMs might be capable of rich, creative, and visually\-grounded communication, but their interpretation and production of visual meaning diverges from human conventions in surprising ways\. Together this work highlights both the promise and challenges of visual communication between humans and VLMs\.

## Submission history

From: Alexa Tartaglini \[[view email](https://arxiv.org/show-email/04c2d37b/2610.36449)\] **\[v1\]**Tue, 27 Oct 2026 00:51:17 UTC \(14,761 KB\)

相似文章

LoopVL：循环视觉智能

Hugging Face Daily Papers

LoopVL 将 Loop Transformer 扩展到视觉-语言模型中，通过共享模块（Module-Loop 和 Model-Loop）迭代更新统一的多模态状态，在理解与推理基准上超越了同等规模的非循环模型，并在注意力中呈现出引人注目的'视觉顿悟时刻'。

揭示VLM可解释的故障模式

arXiv cs.AI

本文介绍了Revelio，这是一个通过搜索离散概念组合来系统性地发现视觉语言模型（VLM）中可解释故障模式的框架。应用于自动驾驶和室内机器人领域，它揭示了此前未报道的、可能导致碰撞或安全危险的漏洞。

大型视觉-语言模型在注意力机制中迷失

arXiv cs.AI

这篇研究论文利用信息论分析了大型视觉-语言模型（LVLM）的内部机制，揭示了注意力机制可能存在冗余，而前馈网络才是推动语义创新的关键。作者证明，将学习到的注意力权重替换为随机值仍可获得相当的性能，这表明当前模型“在注意力中迷失”。