How can embedding models bind concepts?
Summary
This paper investigates why CLIP struggles with concept binding, showing that while CLIP's binding function is high-complexity, controlled transformer models can learn low-complexity binding functions through multiplicative interactions that generalize better.
View Cached Full Text
Cached at: 06/01/26, 03:20 PM
Paper page - How can embedding models bind concepts?
Source: https://huggingface.co/papers/2605.31503
Abstract
Vision-language models like CLIP struggle with concept binding despite recognizing individual concepts, but controlled transformer models can learn low-complexity binding functions that generalize better through multiplicative interactions.
Humans easily determine which color belongs to which shape in multi-object scenes, an ability known asconcept binding.Vision-language embedding modelssuch asCLIPstruggle with binding: they recognize individual concepts but fail to represent which concepts form which objects. AlthoughCLIPbehaves like a bag-of-concepts model incross-modal retrieval, object information is recoverable from its image and text embeddings separately. We study this tension through thebinding function, which maps concepts toscene embeddings. We find thatscene embeddingsdecompose additively intoobject representations, explaining why uni-modal probes can recover object information. However,CLIP’sbinding functionis high-complexity, which likely prevents the image and text encoders from learning a shared binding mechanism that generalizes to unseen concept combinations. We then ask whether this limitation is fundamental. We show that it is not. In controlledtransformer modelstrained from scratch, bindinggeneralizationemerges with sufficient data coverage. These models learn low-complexitybinding functions characterized bymultiplicative interactionsbetween concepts, enabling systematicgeneralization. Code is publicly available at https://github.com/oshapio/binding-concepts-complexity.
View arXiv pageView PDFGitHub0Add to collection
Get this paper in your agent:
hf papers read 2605\.31503
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.31503 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.31503 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.31503 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
@kdariina: Why do embedding models struggle with binding, a core requirement for multi-object understanding? We find that objects …
This thread discusses a finding that embedding models struggle with binding (a core requirement for multi-object understanding) and that solving it requires going beyond the linear representation hypothesis.
CLIF: Concept-Level Influence Functions for Transparent Bottleneck Models
This paper proposes CLIF, a method using influence functions to interpret NLP models at both sample and concept levels within Concept Bottleneck Models, enabling transparent debugging and concept-level analysis.
Models Can Model, But Can't Bind: Structured Grounding in Text-to-Optimization
This paper introduces Text2Opt-Bench, a scalable benchmark for text-to-optimization, and identifies that LLMs struggle with 'binding' (grounding problem data) rather than 'modeling' (choosing optimization structure). The authors propose BIND, a simple inference-time method that externalizes numeric data, significantly improving accuracy across models.
Cell-Based Representation of Relational Binding in Language Models
Study reveals that LLMs encode discourse-level relational binding through Cell-based Binding Representation (CBR), a low-dimensional linear subspace where each cell maps to entity-relation pairs, providing causal evidence for how models track entities and relations.
CLIP: Connecting text and images
CLIP is OpenAI's vision-language model that learns from text-image pairs from the internet, enabling zero-shot visual classification without task-specific training data. It addresses major limitations in traditional computer vision by reducing dependence on expensive labeled datasets and improving real-world generalization.