In-Context Multiple Instance Learning
Summary
This paper proposes a Perceiver-style architecture pretrained on synthetic bag-structured data to enable efficient, task-adaptive classification from few labeled examples in multiple instance learning, outperforming supervised baselines across twelve benchmarks.
View Cached Full Text
Cached at: 06/10/26, 05:46 PM
Paper page - In-Context Multiple Instance Learning
Source: https://huggingface.co/papers/2606.06458
Abstract
Pretraining a Perceiver-style architecture on synthetic bag-structured data enables efficient, task-adaptive classification from few labeled examples in multiple instance learning scenarios.
Multiple Instance Learning(MIL) addresses problems where supervision is available at the level of bags of instances and has been successfully applied in fields ranging from computational pathology to satellite imagery. Nevertheless, existing algorithms struggle in the low-label regime that characterizes many real-world applications. Flexible models overfit and rigid ones fail to adapt to the task at hand. We show thatpretrainingan in-context learner with aPerceiver-style architectureonsynthetic datayields a model that can solve new tasks from a handful of labeled bags. At inference time, classification happens in a single forward pass and requires no gradient updates. We propose and investigate differentsynthetic datagenerators forbag-structured dataand find that they capture complementary inductive biases. A model pretrained on a mixture of these generators inherits their per-task strengths and achieves the best average performance across twelve MIL benchmarks, outperforming supervised baselines that require task-specific training.
View arXiv pageView PDFGitHub1Add to collection
Get this paper in your agent:
hf papers read 2606\.06458
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.06458 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.06458 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.06458 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
MAG: MAnifold Guided Semi-Supervised Multi-modal In-Context Learning
This paper introduces MAG, a manifold-guided framework for semi-supervised multi-modal in-context demonstration selection, leveraging unlabeled data to improve few-shot ICL for MLLMs. Experiments on eight benchmarks show consistent gains in label-scarce regimes.
In-Context Learning Operates as Concept Subspace Learning
This paper proposes that in-context learning in LLMs operates through low-dimensional concept subspaces, where task-relevant information concentrates in a small fraction of the representation space, supported by experiments on Llama-3-8B and Qwen2.5-7B.
Learning from Lost Provenance: Multiple Instance Learning for Cancer Registry Tumor Group Classification
This paper presents an Attention-Based Multiple Instance Learning (ABMIL) framework that leverages patient-level labels from cancer registries to train deep learning classifiers for tumor group classification without requiring per-report annotations, achieving a macro F1 of 0.83 on tasks at the BC Cancer Registry.
InstanceControl: Controllable Complex Image Generation without Instance Labeling
InstanceControl enables multi-instance controllable image generation without manual instance labeling by leveraging a vision-language model to establish instance-level correspondences between text prompts and visual conditions, with adaptive mask refinement for improved accuracy.
LC-ICL: Label-Guided Contrastive In-Context Learning for Robust Information Extraction
This paper proposes LC-ICL, a novel few-shot technique that uses both correct and incorrect examples with error-cause labels to improve large language models' performance on information extraction tasks like named entity recognition and relation extraction.