Token-Based Dual-view Fusion and Adaptation of Large Vision Models for Breast Cancer Classification
Summary
This paper proposes a token-centric dual-view learning framework that unifies prompt-based adaptation and cross-view fusion within a frozen vision transformer to improve breast cancer classification from mammography images, achieving consistent improvements on VinDr-Mammo and CMMD datasets.
View Cached Full Text
Cached at: 07/09/26, 07:41 PM
Paper page - Token-Based Dual-view Fusion and Adaptation of Large Vision Models for Breast Cancer Classification
Source: https://huggingface.co/papers/2607.06309
Abstract
A token-centric dual-view learning framework unifies prompt-based adaptation and cross-view fusion in a frozen vision transformer to improve breast cancer classification from mammography images.
Accurate breast cancer classification from mammography requires effective integration of complementary information from craniocaudal (CC) and mediolateral oblique (MLO) views, which provide a more complete characterization of breast abnormalities. However, existing multi-view learning approaches typically rely on feature-level aggregation or single-stagecross-attention, which can entangle view-specific and shared representations and restrict interaction to limited network depths. To address these limitations, we propose atoken-centric dual-view learningframework that unifiesprompt-based adaptationandcross-view fusionwithin a frozenvision transformerbackbone. The framework reformulates inter-view interaction as structured token-level communication, where dedicatedfusion tokensexplicitly encode bidirectional information exchange between CC and MLO views viacross-attention, serving as intermediate carriers of cross-view dependencies rather than relying on direct feature fusion. Unlike conventional methods that apply fusion at a single layer, fusion modules are inserted at multiple transformer depths, enabling progressive and repeated interaction across the encoder hierarchy.Fusion tokensare reintegrated into the token sequence and refined by subsequenttransformer layers, facilitatinghierarchical propagationof complementary information while preserving view-specific structure. Experiments on VinDr-Mammo and CMMD datasets demonstrate consistent improvements over linear probing, prompt-only adaptation, and conventional fusion baselines. On the VinDr-Mammo BI-RADS classification task, the framework achieves 50.40% F1-score and 0.8090 AUC, including a 0.10 AUC improvement over a dual-view fusion baseline in the binary setting. Ablation studies further validate the effectiveness of token-based fusion andmulti-depth interactiondesign.
View arXiv pageView PDFGitHub0Add to collection
Get this paper in your agent:
hf papers read 2607\.06309
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.06309 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.06309 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.06309 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Late-Layer Fusion is Enough: Dual-Path Vision Token Routing for Multimodal Large Language Models under Visual Saturation
This paper proposes DPVR-LF, a modality-asymmetric routing framework for MLLMs that routes vision tokens at their saturation point into a lightweight side branch and performs late fusion, reducing visual computation while maintaining competitive performance.
From 2D Grids to 1D Tokens: Reforming Shared Representations for Multimodal Image Fusion
This paper introduces a multimodal image fusion method that uses a 1D token interface from a pretrained image tokenizer to enhance global appearance coherence while preserving local details through selective token editing (STE). Experiments on four benchmarks show state-of-the-art performance in both global coherence and local fidelity.
Multi-Modal Machine Learning for Breast Cancer Recurrence Prediction
This paper examines the integration of multi-modal clinical data, including treatment records, pathology reports, and clinician notes, using rule-based extraction and machine learning to improve breast cancer recurrence prediction compared to single-modal approaches.
Document Classification Pattern Recognition via Information Fusion: A Systematic Review of Multimodal and Multiview Representation Approaches
This systematic review of 139 studies proposes a unified framework and meta-analysis for document classification via multimodal and multiview information fusion, finding that fusion improves accuracy (mean gain of +5.28 percentage points) but highlights reproducibility challenges.
Probing, Fusion, and Trustworthiness: A Systematic Evaluation of Foundation Model Representations for Multimodal Cancer Analysis
This paper systematically evaluates foundation model representations for multimodal cancer analysis, benchmarking unimodal and multimodal fusion strategies on real-world cohorts, and assessing trustworthiness via conformal prediction.