Meta-CLIP:用于零样本疾病推断的元数据条件化对比学习

arXiv cs.AI 论文

摘要

Meta-CLIP 引入了一种以元数据为条件的对比学习框架,利用 DICOM 采集字段对跨模态负样本重新加权,在 CT-RATE 和多个胸部 X 光(CXR)基准测试中的零样本疾病分类表现优于 CT-CLIP。

arXiv:2610.33674v1 Announce Type: new Abstract: Medical vision--language pretraining learns transferable representations from paired images and radiology reports, yet differences in scanners and acquisition protocols can impair transfer. Existing approaches primarily enrich clinical semantics, leaving structured acquisition information in DICOM headers underused. We introduce Meta-CLIP, a metadata-conditioned contrastive framework that uses acquisition similarity to reweight cross-modal negatives during pretraining. A kernel over numerical and categorical DICOM fields defines a bounded pairwise margin. This margin increases the contribution of acquisition-similar negatives, emphasizing distinctions among studies that share imaging conditions. The kernel remains fixed during image--text optimization, and inference follows the standard dual-encoder architecture without requiring metadata. We evaluate Meta-CLIP on chest radiography and volumetric chest CT through zero-shot classification, appearance perturbations, grounding, and representation analyses. On CT-RATE, Meta-CLIP improves macro-AUROC over CT-CLIP by 1.41 points, with a 95\% paired bootstrap confidence interval of 0.96--1.85 points. It also achieves the highest macro-AUROC on four of five CXR benchmarks. An exploratory BrgSA evaluation extends the analysis to another CT backbone. These findings support acquisition metadata as a practical training-time prior for medical vision--language learning. Code will be made publicly available upon publication.
查看原文
查看缓存全文

缓存时间: 2026/10/28 02:04

# Meta-CLIP: Metadata-Conditioned Contrastive Learning for Zero-Shot Disease Inference
Source: [https://arxiv.org/html/2610.33674](https://arxiv.org/html/2610.33674)
Nirosh SivanesanShelley Zixin ShuFrancesco NittiMauricio Reyes††thanks:Haozhe Luo is with the University of Bern, Bern, Switzerland \(e\-mail: haozhe\.luo@unibe\.ch\)\.††thanks:Nirosh Sivanesan, Shelley Zixin Shu, Francesco Nitti, and Mauricio Reyes are with the ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland \(e\-mail: Sivanirosh@gmail\.com; zixin\.shu@unibe\.ch; francesco\.nitti@unibe\.ch; mauricio\.reyes@unibe\.ch\)\.

###### Abstract

Medical vision–language pretraining learns transferable representations from paired images and radiology reports, yet differences in scanners and acquisition protocols can impair transfer\. Existing approaches primarily enrich clinical semantics, leaving structured acquisition information in DICOM headers underused\. We introduce Meta\-CLIP, a metadata\-conditioned contrastive framework that uses acquisition similarity to reweight cross\-modal negatives during pretraining\. A kernel over numerical and categorical DICOM fields defines a bounded pairwise margin\. This margin increases the contribution of acquisition\-similar negatives, emphasizing distinctions among studies that share imaging conditions\. The kernel remains fixed during image–text optimization, and inference follows the standard dual\-encoder architecture without requiring metadata\. We evaluate Meta\-CLIP on chest radiography and volumetric chest CT through zero\-shot classification, appearance perturbations, grounding, and representation analyses\. On CT\-RATE, Meta\-CLIP improves macro\-AUROC over CT\-CLIP by 1\.41 points, with a 95% paired bootstrap confidence interval of 0\.96–1\.85 points\. It also achieves the highest macro\-AUROC on four of five CXR benchmarks\. An exploratory BrgSA evaluation extends the analysis to another CT backbone\. These findings support acquisition metadata as a practical training\-time prior for medical vision–language learning\. Code will be made publicly available upon publication\.

###### Index Terms:

Chest computed tomography, chest radiography, contrastive learning, DICOM metadata, medical vision–language models, robustness\.

## IIntroduction

Vision–language pretraining \(VLP\) offers a scalable way to learn medical image representations from routinely collected radiology reports\. Aligning images and text in a shared embedding space enables zero\-shot disease classification and cross\-modal retrieval, reducing the need for task\-specific annotations\[[1](https://arxiv.org/html/2610.33674#bib.bib10),[2](https://arxiv.org/html/2610.33674#bib.bib14),[3](https://arxiv.org/html/2610.33674#bib.bib15),[4](https://arxiv.org/html/2610.33674#bib.bib16)\]\. Progress in this area has strengthened clinical supervision through biomedical text representations, medical knowledge, and large\-scale volumetric pretraining\[[5](https://arxiv.org/html/2610.33674#bib.bib2),[6](https://arxiv.org/html/2610.33674#bib.bib20),[7](https://arxiv.org/html/2610.33674#bib.bib5)\]\.

Transfer across acquisition settings presents a complementary challenge\. Scanner vendors, exposure settings, and reconstruction protocols can change image appearance without changing the underlying pathology\. Medical vision–language models remain sensitive to imaging artifacts and distribution shifts\[[8](https://arxiv.org/html/2610.33674#bib.bib4)\]\. In standard contrastive learning, unmatched image–report pairs contribute according to their embedding similarity\. The objective has no explicit acquisition prior to guide which negative comparisons it emphasizes\.

DICOM headers make this context available through structured acquisition fields\. Metadata has supported representation learning in ultrasound, retinal imaging, and longitudinal OCT\[[9](https://arxiv.org/html/2610.33674#bib.bib6),[10](https://arxiv.org/html/2610.33674#bib.bib7),[11](https://arxiv.org/html/2610.33674#bib.bib9)\]\. For image–report pretraining, our insight is that acquisition cues shared by two studies are less informative for determining which report corresponds to which image\. Emphasizing comparisons between acquisition\-similar studies encourages alignment to rely on other image–report evidence\. We use acquisition similarity as a prior for weighting these contrastive negatives\.

We introduce*Meta\-CLIP*, which implements this prior through metadata\-conditioned margins for chest radiography and volumetric chest CT \(Fig\.[1](https://arxiv.org/html/2610.33674#S1.F1)\)\. A kernel combines numerical acquisition fields and categorical scanner attributes into a pairwise similarity\. This similarity defines bounded margins that raise negative logits in both contrastive directions, giving acquisition\-similar negatives greater weight while leaving positive logits unchanged\. The kernel stays fixed during image–text optimization\. Inference retains the standard dual\-encoder architecture and requires no metadata\.

Experiments cover CXR and CT classification, external transfer, and analyses of robustness, grounding, and representation structure\. Meta\-CLIP improves CT\-RATE macro\-AUROC over CT\-CLIP by 1\.41 points and achieves the highest macro\-AUROC on four of five CXR benchmarks\.

The contributions of this work are threefold:

- •We formulate acquisition similarity as a prior over cross\-modal negatives and construct this relation from numerical and categorical DICOM fields\.
- •We develop a symmetric contrastive objective that emphasizes acquisition\-similar negatives through bounded pairwise margins, preserving the underlying encoders and their metadata\-free inference procedure\.
- •We evaluate CXR and CT transfer, characterize the selected metadata, and examine robustness, grounding, representation structure, and exploratory transfer to another CT backbone\.

![Refer to caption](https://arxiv.org/html/2610.33674v1/main_chart_neutral.png)Fig\. 1:Motivation and overview of Meta\-CLIP\. Left: \(a\) Acquisition protocols and scanner vendors influence image appearance\. \(b\) Standard image–report pretraining leaves structured acquisition metadata unused\. \(c\) Acquisition variation can shape the learned representation\. Right: Meta\-CLIP constructs a pairwise acquisition kernel from numerical and categorical DICOM fields and uses it to modulate negative logits in the contrastive objective\. Metadata is used during pretraining and is unnecessary at inference\.
## IIRelated Work

### II\-AMedical Vision–Language Pretraining

Contrastive pretraining established that natural\-language supervision can produce transferable visual representations\[[1](https://arxiv.org/html/2610.33674#bib.bib10)\]\. Medical adaptations use radiology reports as scalable supervision, progressing from global image–text alignment in ConVIRT\[[2](https://arxiv.org/html/2610.33674#bib.bib14)\]to combined global and local alignment in GLoRIA\[[3](https://arxiv.org/html/2610.33674#bib.bib15)\]\. BioViL strengthens biomedical text representations\[[4](https://arxiv.org/html/2610.33674#bib.bib16)\]\. Further developments introduce soft prompts\[[12](https://arxiv.org/html/2610.33674#bib.bib1)\], granular masked alignment\[[13](https://arxiv.org/html/2610.33674#bib.bib3)\], or anatomical consistency\[[14](https://arxiv.org/html/2610.33674#bib.bib18)\]to improve transfer\.

A complementary line of work enriches report supervision with structured medical knowledge\. KAD and MedKLIP associate visual features with disease entities and knowledge graphs\[[5](https://arxiv.org/html/2610.33674#bib.bib2),[6](https://arxiv.org/html/2610.33674#bib.bib20)\], while DeViDe decomposes disease descriptions into multiple semantic facets\[[15](https://arxiv.org/html/2610.33674#bib.bib8)\]\. CT\-CLIP extends image–text pretraining to volumetric chest CT at scale\[[7](https://arxiv.org/html/2610.33674#bib.bib5)\]\. These methods strengthen clinical supervision and image–text alignment\. Meta\-CLIP complements this direction by using acquisition context to determine the contribution of cross\-modal negatives during pretraining\.

### II\-BRobustness and Vision–Language Adaptation

Distribution shifts remain a major obstacle for medical VLP\. Controlled evaluation has shown that common medical image artifacts can substantially change VLM predictions\[[8](https://arxiv.org/html/2610.33674#bib.bib4)\]\. Adaptation methods seek to improve transfer through learned prompts\[[16](https://arxiv.org/html/2610.33674#bib.bib13)\], training\-free feature adapters\[[17](https://arxiv.org/html/2610.33674#bib.bib12)\], or disease\-informed adaptation for medical images\[[18](https://arxiv.org/html/2610.33674#bib.bib11)\]\. These approaches modify prompts, features, or model parameters for the target task\. They address a different stage of learning from the construction of contrastive relations during source pretraining\.

Meta\-CLIP uses acquisition information accompanying each source image to modify the pretraining objective before target\-domain inference\. The resulting model can be evaluated directly in zero\-shot settings or combined with later adaptation when target data are available\.

### II\-CMetadata\-Aware Medical Representation Learning

Metadata can supply supervision beyond the image itself\. DICOM fields provide weak supervision for ultrasound pretraining\[[9](https://arxiv.org/html/2610.33674#bib.bib6)\], while patient identity, laterality, and temporal information define relations for contrastive learning from longitudinal retinal OCT\[[10](https://arxiv.org/html/2610.33674#bib.bib7)\]\. PRETI combines patient metadata with learnable metadata embeddings and retinal\-aware masking for fundus representation learning\[[11](https://arxiv.org/html/2610.33674#bib.bib9)\]\. Together, these studies show that routinely available context can improve medical image representations through weak targets, patient relations, or sample\-pair definitions in unimodal pretraining\.

Our setting differs in both the metadata and its role\. We use acquisition parameters from DICOM headers to characterize protocol and scanner similarity in paired image–report pretraining\. Rather than treating metadata as an input, prediction target, or rule for constructing positive pairs, Meta\-CLIP applies the acquisition relation as a continuous margin on cross\-modal negatives\. This preserves the standard CLIP encoders and removes the need for metadata at inference\.

![Refer to caption](https://arxiv.org/html/2610.33674v1/images/CT_radar_fra2_cleanlegend.png)Fig\. 2:\(a\) Disease\-wise AUC comparison on RAD\-ChestCT and CT\-RATE\. \(b\) Volume\-to\-volume retrieval measured by label similarity at different top\-kklevels\. \(c\) Robustness under synthetic perturbations; the light bars show clean inputs and the dark bars show perturbed inputs\.![Refer to caption](https://arxiv.org/html/2610.33674v1/distribution_chart_neutral.png)Fig\. 3:Embedding structure\.UMAP visualizations and silhouette scores for \(a\) CT and \(b\) CXR\. Meta\-CLIP shows stronger disease\-related clustering and, on CT, lower separation by dataset than CT\-CLIP\.

## IIIMethod

Meta\-CLIP introduces structured acquisition metadata into contrastive pretraining through a pairwise margin on cross\-modal negatives\. As illustrated in Fig\.[1](https://arxiv.org/html/2610.33674#S1.F1), a kernel measures similarity between acquisition conditions, and the resulting relation modulates logits in the contrastive objective\. The image and text encoders retain their original architecture\.

### III\-AProblem Formulation

Let𝒟=\{\(xi,ri,mi\)\}i=1M\\mathcal\{D\}=\\\{\(x\_\{i\},r\_\{i\},m\_\{i\}\)\\\}\_\{i=1\}^\{M\}denote a pretraining dataset, wherexix\_\{i\}is a CXR image or CT volume,rir\_\{i\}is its paired report, andmim\_\{i\}contains structured DICOM metadata\. An image encoderfθf\_\{\\theta\}and a text encodergϕg\_\{\\phi\}produceℓ2\\ell\_\{2\}\-normalized representations

ziI=fθ​\(xi\)‖fθ​\(xi\)‖2,ziT=gϕ​\(ri\)‖gϕ​\(ri\)‖2\.z\_\{i\}^\{\\mathrm\{I\}\}=\\frac\{f\_\{\\theta\}\(x\_\{i\}\)\}\{\\\|f\_\{\\theta\}\(x\_\{i\}\)\\\|\_\{2\}\},\\qquad z\_\{i\}^\{\\mathrm\{T\}\}=\\frac\{g\_\{\\phi\}\(r\_\{i\}\)\}\{\\\|g\_\{\\phi\}\(r\_\{i\}\)\\\|\_\{2\}\}\.\(1\)The metadatamim\_\{i\}describes acquisition conditions such as exposure, reconstruction, and scanner manufacturer\. We use these conditions to define a prior over unmatched pairs within each minibatch\. This prior controls their contribution to the contrastive normalizer, while matched image–report pairs provide the positive supervision\.

### III\-BAcquisition Relation Modeling

For a minibatchℬ=\{\(xi,ri,mi\)\}i=1N\\mathcal\{B\}=\\\{\(x\_\{i\},r\_\{i\},m\_\{i\}\)\\\}\_\{i=1\}^\{N\}, we construct a pairwise acquisition kernel from numerical and categorical DICOM fields\. The two field types capture complementary aspects of image formation and scanner identity\.

#### III\-B1Numerical metadata

Let𝒟num\\mathcal\{D\}\_\{\\mathrm\{num\}\}index the numerical DICOM fields and let each field be standardized independently\. For a pair\(i,j\)\(i,j\),𝒟i​j⊆𝒟num\\mathcal\{D\}\_\{ij\}\\subseteq\\mathcal\{D\}\_\{\\mathrm\{num\}\}contains the fields observed for both samples, so missing values are excluded from that pairwise comparison\. We measure numerical similarity using an RBF kernel,

Ki​jnum=exp\(−∑d∈𝒟i​j\(mi,d−mj,d\)22​σd2\),K^\{\\mathrm\{num\}\}\_\{ij\}=\\exp\\\!\\left\(\-\\sum\_\{d\\in\\mathcal\{D\}\_\{ij\}\}\\frac\{\(m\_\{i,d\}\-m\_\{j,d\}\)^\{2\}\}\{2\\sigma\_\{d\}^\{2\}\}\\right\),\(2\)whereσd\>0\\sigma\_\{d\}\>0controls sensitivity to differences in fielddd\. Standardization places fields with different units on comparable scales, while the bandwidth determines how quickly similarity decays\.

#### III\-B2Categorical metadata

Let𝒟cat\\mathcal\{D\}\_\{\\mathrm\{cat\}\}index categorical fields such as scanner manufacturer\. Each fieldcchas a fixed lookup embeddingEcE\_\{c\}\. WithE^c​\(v\)=Ec​\(v\)/‖Ec​\(v\)‖2\\widehat\{E\}\_\{c\}\(v\)=E\_\{c\}\(v\)/\\\|E\_\{c\}\(v\)\\\|\_\{2\}, categorical similarity is

Ki​jcat=1\|𝒟cat\|​∑c∈𝒟catmax⁡\(0,⟨E^c​\(mi,c\),E^c​\(mj,c\)⟩\)\.K^\{\\mathrm\{cat\}\}\_\{ij\}=\\frac\{1\}\{\|\\mathcal\{D\}\_\{\\mathrm\{cat\}\}\|\}\\sum\_\{c\\in\\mathcal\{D\}\_\{\\mathrm\{cat\}\}\}\\max\\\!\\left\(0,\\left\\langle\\widehat\{E\}\_\{c\}\(m\_\{i,c\}\),\\widehat\{E\}\_\{c\}\(m\_\{j,c\}\)\\right\\rangle\\right\)\.\(3\)Numerical and categorical similarities are combined through a normalized aggregation operator:

Ki​jmeta=Agg⁡\(Ki​jnum,Ki​jcat\)\.K^\{\\mathrm\{meta\}\}\_\{ij\}=\\operatorname\{Agg\}\\\!\\left\(K^\{\\mathrm\{num\}\}\_\{ij\},K^\{\\mathrm\{cat\}\}\_\{ij\}\\right\)\.\(4\)For CXR, the Gaussian relation uses a single normalized bandwidth; for CT, per\-field similarities are combined by normalized field weights\. The resulting acquisition kernel lies in\[0,1\]\[0,1\], with larger values denoting more similar imaging conditions\.

#### III\-B3Meta\-CLIP relation

We convert the acquisition kernel into a negative correction for every off\-diagonal pair and leave the matched image–report pair unchanged:

Ki​jtotal=\{−α​Ki​jmeta,i≠j,0,i=j\.K^\{\\mathrm\{total\}\}\_\{ij\}=\\begin\{cases\}\-\\alpha K^\{\\mathrm\{meta\}\}\_\{ij\},&i\\neq j,\\\\ 0,&i=j\.\\end\{cases\}\(5\)Hereα∈\[0,1\]\\alpha\\in\[0,1\]controls the correction strength\. Thus,Ki​jtotal∈\[−α,0\]K^\{\\mathrm\{total\}\}\_\{ij\}\\in\[\-\\alpha,0\], and acquisition\-similar negative pairs receive corrections of larger magnitude\. Both modalities use this relation\.

### III\-CMetadata\-Conditioned Contrastive Learning

For image\-to\-text retrieval, the standard CLIP similarity between imageiiand reportjjis

Si​jI2T=⟨ziI,zjT⟩\.S^\{\\mathrm\{I2T\}\}\_\{ij\}=\\left\\langle z\_\{i\}^\{\\mathrm\{I\}\},z\_\{j\}^\{\\mathrm\{T\}\}\\right\\rangle\.\(6\)The text\-to\-image matrix is defined analogously asSi​jT2I=⟨ziT,zjI⟩S^\{\\mathrm\{T2I\}\}\_\{ij\}=\\langle z\_\{i\}^\{\\mathrm\{T\}\},z\_\{j\}^\{\\mathrm\{I\}\}\\rangle\. We inject the relation into each off\-diagonal logit using an additive margin:

S~i​jq=Sqi​j−μt1\[i≠j\]Ktotali​jτ,q∈\{I2T,T2I\},\\widetilde\{S\}^\{q\}\_\{ij\}=\\frac\{S^\{q\}\_\{ij\}\-\\mu\_\{t\}\\,\\mathbf\{1\}\[i\\neq j\]K^\{\\mathrm\{total\}\}\_\{ij\}\}\{\\tau\},\\qquad q\\in\\\{\\mathrm\{I2T\},\\mathrm\{T2I\}\\\},\(7\)whereμt=μmax​s​\(t\)\\mu\_\{t\}=\\mu\_\{\\max\}s\(t\)controls the margin magnitude at training steptt,s⁡\(t\)∈\[0,1\]s\(t\)\\in\[0,1\]is a prescribed schedule, andτ\>0\\tau\>0is the temperature\. CXR uses a static margin withs⁡\(t\)=1s\(t\)=1, while CT gradually increases the margin according to the schedule specified in the experimental setup\. The indicator restricts the correction to unmatched pairs\.

For either retrieval directionqq, the resulting contrastive loss is

ℒq=−1N∑i=1Nlogexp⁡\(S~i​iq\)∑j=1Nexp⁡\(S~i​jq\)\.\\mathcal\{L\}\_\{q\}=\-\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\log\\frac\{\\exp\(\\widetilde\{S\}^\{q\}\_\{ii\}\)\}\{\\sum\_\{j=1\}^\{N\}\\exp\(\\widetilde\{S\}^\{q\}\_\{ij\}\)\}\.\(8\)The metadata\-conditioned contrastive term averages both directions:

ℒMeta=12​\(ℒI2T\+ℒT2I\)\.\\mathcal\{L\}\_\{\\mathrm\{Meta\}\}=\\frac\{1\}\{2\}\\left\(\\mathcal\{L\}\_\{\\mathrm\{I2T\}\}\+\\mathcal\{L\}\_\{\\mathrm\{T2I\}\}\\right\)\.\(9\)
Substituting Eq\. \([5](https://arxiv.org/html/2610.33674#S3.E5)\) into Eq\. \([7](https://arxiv.org/html/2610.33674#S3.E7)\) shows that each negative logit increases byμt​α​Ki​jmeta/τ\\mu\_\{t\}\\alpha K^\{\\mathrm\{meta\}\}\_\{ij\}/\\tau\. Acquisition\-similar pairs therefore contribute more strongly to the softmax normalizer\. This relation\-aware hard\-negative reweighting emphasizes distinctions among studies with similar imaging conditions, while the positive logits retain their standard form\.

### III\-DTraining and Inference

The acquisition relation is computed during pretraining and detached before logit modulation\. Categorical embeddings, bandwidths, field weights, andα\\alpharemain fixed during image–text optimization\. The contrastive loss therefore updates the encoders against a prescribed acquisition prior\. Inference uses the trained image and text encoders without DICOM metadata\.

For CXR, Eq\. \([9](https://arxiv.org/html/2610.33674#S3.E9)\) is incorporated into the KAD training objective\. For CT, we combine it with the symmetric InfoNCE base objective used by CT\-CLIP:

ℒCT=ℒInfoNCE\+γ​ℒMeta,\\mathcal\{L\}\_\{\\mathrm\{CT\}\}=\\mathcal\{L\}\_\{\\mathrm\{InfoNCE\}\}\+\\gamma\\mathcal\{L\}\_\{\\mathrm\{Meta\}\},\(10\)whereγ\\gammacontrols the weight of the metadata\-conditioned term\. The acquisition relation is shared across modalities, while each backbone retains its base training objective\.

## IVExperimental Setup

We evaluate Meta\-CLIP on 2D chest radiography and 3D chest CT, combining zero\-shot classification with analyses of representation structure, grounding, and robustness to appearance perturbations\.

### IV\-ABackbones and Evaluation Protocol

For CT, Meta\-CLIP uses CT\-ViT\[[19](https://arxiv.org/html/2610.33674#bib.bib23)\]and CXR\-BERT\[[4](https://arxiv.org/html/2610.33674#bib.bib16)\], with both encoders optimized following CT\-CLIP\. Pretraining uses CT\-RATE, whose official validation split provides internal evaluation, while RAD\-ChestCT provides external evaluation\. For CXR, we follow KAD\[[5](https://arxiv.org/html/2610.33674#bib.bib2)\], training an M3AE\-initialized ViT\-B/16 while keeping the pretrained BioClinicalMPBERT text encoder111[https://huggingface\.co/Laihaoran/BioClinicalMPBERT](https://huggingface.co/Laihaoran/BioClinicalMPBERT)frozen\. The CXR benchmark comprises ChestXray14, CheXpert, PadChest\-seen, RSNA Pneumonia, and SIIM\-ACR\. Each task uses its designated evaluation split\.

CXR pretraining uses 224×\\times224 inputs, batch size 32, 100 epochs, AdamW with learning rate5×10−55\\times 10^\{\-5\}and weight decay 0\.02, cosine decay to10−610^\{\-6\}, and 20 warm\-up epochs\. Its base contrastive temperature is 0\.07; the metadata margin usesμmax=0\.5\\mu\_\{\\max\}=0\.5, margin temperatureτ=0\.1\\tau=0\.1, a static schedule, Gaussian bandwidthσ=1\.0\\sigma=1\.0, andα=0\.5\\alpha=0\.5\.

The CT configuration uses batch size 8, learning rate5×10−55\\times 10^\{\-5\}, cosine decay with a1\.25×10−61\.25\\times 10^\{\-6\}minimum learning rate, and 1,000 warm\-up steps\. The metadata term usesγ=1\.0\\gamma=1\.0,μmax=0\.5\\mu\_\{\\max\}=0\.5, andτ=0\.1\\tau=0\.1\. The acquisition kernel usesσd=1\.0\\sigma\_\{d\}=1\.0, uniform field weights, andα=0\.1\\alpha=0\.1\. The margin scales⁡\(t\)s\(t\)takes values0\.00\.0,0\.10\.1,0\.40\.4,0\.60\.6, and0\.80\.8over epochs00–55,55–1010,1010–1515,1515–2020, and after epoch2020, respectively\. Kernel parameters remain fixed in both modalities\. CXR incorporates the margin within the KAD objective and uses no separateγ\\gammacoefficient\.

We explore backbone portability by adding the metadata\-conditioned loss \(weight0\.10\.1\) to BrgSA\[[20](https://arxiv.org/html/2610.33674#bib.bib22)\]with a five\-field Gaussian kernel\. Both variants are trained on CT\-RATE and selected by pretraining validation loss, with evaluation on CT\-RATE, RAD\-ChestCT, and INSPECT\. INSPECT volumes use canonical LPS orientation\. The metadata variant uses a different batch size and omits BrgSA’s local\-alignment term, making this an exploratory comparison of training configurations\.

Zero\-shot evaluation uses the pretrained model without target\-dataset parameter updates\. Target evaluation labels and metadata are excluded from adaptation\.

### IV\-BMetadata Selection and Characterization

For CT\-RATE, we use seven DICOM attributes describing scanner identity or image formation:XRayTubeCurrent,Exposure,ExposureTime,DistanceSourceToDetector,CTDIvol,ReconstructionDiameter, andManufacturer\. This compact set focuses on acquisition conditions\. Demographic and unrestricted DICOM fields can also encode disease prevalence or institutional label distributions\. We characterize the selected fields on the CT\-RATE training set \(n=47,149n=47\{,\}149\) using class\-balanced logistic probes with an 80/20 stratified split\. Acquisition\-domain signal is measured by balanced\-accuracy lift over chance when predicting scanner manufacturer, scanner model, and convolution kernel\. Disease association is measured by AUROC across the 18 CT\-RATE findings\. These probes characterize the metadata independently of model training\.

CXR pretraining uses MIMIC\-CXR\-JPG image–report pairs and acquisition metadata\. Its kernel combines five numerical acquisition fields \(KVP,ExposureIndex,ExposureTimeInuS,XRayTubeCurrent, andRelativeXRayExposure\) with the categorical fieldManufacturer\. Training metadata comes exclusively from the pretraining source\. CheXpert, PadChest, ChestXray14, RSNA, and SIIM\-ACR supply no training metadata\.

The CT field\-set ablation compares the default set with three alternatives\. Physics\-5 removesExposureTimeandManufacturer\. Alternative\-5 usesZSpacing,ExposureTime,NumberofSlices,RotationDirection, andManufacturer\. The final variant addsXYSpacingto the default set\. All variants share the CT\-CLIP backbone, the objective in Eq\. \([10](https://arxiv.org/html/2610.33674#S3.E10)\), a fixed acquisition relation, a frozen text encoder, a cosine learning\-rate schedule, and model selection by validation AUROC\. Freezing the text encoder isolates the field\-set comparison from the joint encoder fine\-tuning used in the main CT model\.

Table[I](https://arxiv.org/html/2610.33674#S4.T1)summarizes acquisition fields exposed by MIMIC\-CXR\-JPG and four public CXR resources\. Availability refers to dataset schemas; completeness can vary across individual studies\. This variation motivates selecting training fields according to source coverage, while metadata\-free inference accommodates evaluation datasets with different schemas\.

TABLE I:Availability of representative DICOM fields across public CXR datasets\.For CT robustness evaluation, we independently apply three MONAI intensity transforms:RandHistogramShiftwith 10 control points,RandAdjustContrastwithγ∈\[0\.85,1\.15\]\\gamma\\in\[0\.85,1\.15\], andRandKSpaceSpikeNoisewith default intensity and channel\-wise injection\. CXR evaluation tests metal artifacts, brightness shifts, and noise\. For retrieval, uniform sampling from the database provides a random reference\.

### IV\-CAblations and Statistical Analysis

The focused CXR comparison summarizes KAD and Meta\-CLIP performance on ChestXray14 and CheXpert \(Table[II](https://arxiv.org/html/2610.33674#S4.T2)\), using the corresponding results from the five\-dataset benchmark in Table[VI](https://arxiv.org/html/2610.33674#S5.T6)\.

For acquisition\-stratified analysis, CT\-CLIP and Meta\-CLIP predictions are matched to the same cases and 18 target findings on CT\-RATE\. Continuous attributes define quartile\-based subgroups where ties permit\. Within each subgroup, AUROC is macro\-averaged over findings containing both positive and negative cases\. We estimate 95% confidence intervals using 10,000 case\-bootstrap resamples, with paired resampling for model comparisons\. These intervals describe case\-level uncertainty for the trained models and exclude variability from retraining\.

TABLE II:Zero\-shot CXR classification compared with KAD\. Values are macro\-AUROC \(%\)\.

## VResults

\(i\)Zero\-shot disease classification\.Meta\-CLIP improves CT disease classification on both internal and external evaluation cohorts\. On CT\-RATE, macro\-AUROC increases from 73\.08% to 74\.49%, a gain of 1\.41 points \(95% paired bootstrap CI, 0\.96–1\.85\)\. Figure[2](https://arxiv.org/html/2610.33674#S2.F2)a shows improvements in 13 of 18 CT\-RATE findings and all seven displayed RAD\-ChestCT findings\. Nodule AUROC increases by 3\.67 points on CT\-RATE and 7\.82 points on RAD\-ChestCT, while lung opacity improves by 9\.38 and 4\.73 points, respectively\. The external results extend the classification gains beyond the pretraining dataset\.

On CXR, Meta\-CLIP achieves the highest macro\-AUROC on ChestXray14, PadChest\-seen, RSNA Pneumonia, and SIIM\-ACR among the methods in Table[VI](https://arxiv.org/html/2610.33674#S5.T6)\. The focused comparison with KAD in Table[II](https://arxiv.org/html/2610.33674#S4.T2)shows macro\-AUROC gains of 4\.16 points on ChestXray14 \(76\.49% to 80\.65%\) and 2\.08 points on CheXpert \(89\.23% to 91\.31%\)\. These values summarize the overall model comparison on the two benchmarks\.

The exploratory BrgSA comparison in Fig\.[4](https://arxiv.org/html/2610.33674#S5.F4)shows macro\-AUROC gains of 1\.27 points on CT\-RATE, 2\.40 points on RAD\-ChestCT, and 0\.98 points on INSPECT\. These results suggest that acquisition\-aware training can also benefit another volumetric backbone\. As a complementary retrieval analysis, Fig\.[2](https://arxiv.org/html/2610.33674#S2.F2)b reports top\-1 label similarities of 0\.958 on RAD\-ChestCT and 0\.886 on CT\-RATE, using random retrieval as a reference\.

Fig\. 4:CT backbone portability\.Zero\-shot macro\-AUROC of BrgSA and Meta\-CLIP \+ BrgSA on CT\-RATE, RAD\-ChestCT, and INSPECT\.\(ii\)Representation structure\.Figure[3](https://arxiv.org/html/2610.33674#S2.F3)examines how the classification gains relate to embedding organization\. On CT, disease\-cluster silhouette scores increase from 0\.0437 to 0\.1085 for lymphadenopathy and from 0\.0243 to 0\.0721 for lung opacity\. The dataset\-distribution silhouette decreases from 0\.1533 to 0\.1206, indicating less separation by dataset\. On CXR, Meta\-CLIP also yields a higher disease\-cluster silhouette than KAD\. Together, these analyses suggest that the learned geometry is more closely aligned with disease categories\.

\(iii\)Disease grounding\.The grounding evaluation tests whether improved image–text alignment also corresponds to better localization\. On ChestXray14\-Det, Fig\.[5](https://arxiv.org/html/2610.33674#S5.F5)b shows gains over KAD of 12\.80 points in Pointing Game accuracy, 5\.48 points in mean IoU, and 6\.54 points in mean Dice\. These results indicate stronger spatial correspondence between textual findings and image evidence\.

\(iv\)Robustness to appearance perturbations\.Figure[2](https://arxiv.org/html/2610.33674#S2.F2)c compares CT performance under synthetic intensity perturbations\. Under histogram shift, macro\-AUROC decreases by 0\.41 points for Meta\-CLIP and 1\.04 points for CT\-CLIP\. On CXR, Fig\.[5](https://arxiv.org/html/2610.33674#S5.F5)a shows smaller degradation than KAD under metal artifacts, brightness shifts, and noise across ChestXray14, RSNA, and SIIM\-ACR\. These comparisons use the clean and perturbed inputs within each robustness protocol; the five\-dataset classification benchmark is reported separately in Table[VI](https://arxiv.org/html/2610.33674#S5.T6)\.

\(v\)Acquisition information in the selected metadata\.Table[III](https://arxiv.org/html/2610.33674#S5.T3)characterizes the information carried by the selected CT fields\. Coverage is nearly complete except for source\-to\-detector distance, which is available for 47\.9% of scans\. The numerical kernel excludes unobserved fields from pairwise comparisons\. Individual fields predict acquisition\-domain proxies while showing limited disease association, with mean disease AUROC ranging from 0\.502 to 0\.542\. Jointly, the seven fields yield a mean balanced accuracy of 0\.774 across the three acquisition\-domain proxies and a mean disease AUROC of 0\.593\. Because manufacturer is included among the inputs, the manufacturer probe describes information retained by the field set rather than independent domain prediction\. The unrestricted pool of 43 attributes yields higher domain prediction \(0\.995\) but also higher disease AUROC \(0\.724\)\. This comparison supports focusing on acquisition fields with lower direct disease association\.

TABLE III:Characterization of selected CT metadata\. Values report coverage, acquisition\-domain lift, and disease AUROC\.\(vi\)Effect of metadata selection\.Table[IV](https://arxiv.org/html/2610.33674#S5.T4)tests whether additional or alternative fields improve transfer\. The default seven\-field set gives the highest macro\-AUROC point estimates on CT\-RATE and RAD\-ChestCT\. AddingXYSpacingdecreases performance by 0\.63 and 0\.58 points, respectively\. Alternative\-5 retains similar external performance but loses 1\.18 points on CT\-RATE, while Physics\-5 also scores below the default on both datasets\. The confidence intervals overlap, particularly on RAD\-ChestCT, so these comparisons support the compact default as a practical configuration within the evaluated field sets\.

TABLE IV:Effect of metadata selection on CT zero\-shot classification\. Values are macro\-AUROC \(%\) with 95% bootstrap confidence intervals\.\(vii\)Pairwise acquisition similarity\.We examine the acquisition kernel on 999,921 non\-self pairs sampled from 12,000 CT\-RATE training volumes\. Aggregate similarity is0\.657±0\.2160\.657\\pm 0\.216\(median0\.6880\.688\), with an average of 5\.23 jointly observed numerical fields per pair \(range: 4–6\)\. Source\-to\-detector distance is jointly observed in 23\.5% of pairs\. The fields contribute different similarity distributions: excludingExposureTime, their means range from 0\.597 to 0\.684 and medians from 0\.678 to 0\.821\. In contrast,ExposureTimeis concentrated near one \(median0\.9990\.999, IQR0\.9990\.999–1\.0001\.000\), contributing little pairwise variation at the chosen bandwidth\. The other numerical fields provide a broader range of similarities\. This diagnostic covers numerical fields;Manufacturerenters the categorical kernel\.

\(viii\)Performance across acquisition strata\.Table[V](https://arxiv.org/html/2610.33674#S5.T5)examines CT\-RATE performance within manufacturer and acquisition\-parameter subgroups\. Meta\-CLIP improves macro\-AUROC for all three manufacturer groups, including gains of 2\.02 points for Siemens and 2\.37 points for the smaller PNMS group\. Gains also remain in the highest CTDIvol, tube\-current, exposure, and reconstruction\-diameter strata, where baseline performance is below its overall average\. Across the 22 evaluated strata, 21 improve\. The longest exposure\-time stratum \(n=179n=179\) decreases by 0\.54 points\. These descriptive results indicate that the overall gain extends across multiple acquisition settings, with heterogeneity in the size and direction of subgroup effects\.

TABLE V:Acquisition\-stratified CT\-RATE validation results\. Values are macro\-AUROC \(%\) over evaluable findings\.TABLE VI:Zero\-shot disease classification across five CXR datasets for findings represented during pretraining\. Values are point estimates \(%\)\. Best results are bolded and second\-best results are underlined\.![Refer to caption](https://arxiv.org/html/2610.33674v1/Xray_chart.png)Fig\. 5:Robustness and grounding study on CXR\.\(a\) Zero\-shot AUC under metal artifacts, brightness shifts, and noise on ChestXray14, RSNA, and SIIM\-ACR \(light: clean; dark: perturbed\)\. \(b\) ChestXray14\-Det grounding: cross\-attention maps for five findings \(blue: ground\-truth boxes\) and metrics \(Pointing Game, mIoU, and Dice\)\.
## VIDiscussion

### VI\-AAcquisition Metadata as a Training\-Time Prior

Acquisition metadata provides context for contrastive comparisons beyond the image and report embeddings\. Meta\-CLIP uses this context to emphasize differences among studies acquired under similar conditions\. The CXR benchmark shows improvements over KAD, while the CT evaluation demonstrates gains on internal and external cohorts\. The BrgSA comparison provides exploratory evidence for application to another volumetric backbone\. Together, these findings support acquisition similarity as a useful prior for weighting negative pairs\. The mechanism acts on negative comparisons and does not explicitly align same\-disease studies across acquisition conditions; conditional acquisition invariance remains a separate property to evaluate\.

### VI\-BPractical Implications

Meta\-CLIP retains the inference computation, memory footprint, and input requirements of its underlying dual encoder\. A deployment site therefore requires images and text prompts, with no additional DICOM input\. This separation is useful when metadata availability differs across institutions, as illustrated by the CXR field inventory\. During pretraining, field selection should consider coverage and information content together\. The compact CT set carries less direct disease signal than the unrestricted pool, while addingXYSpacingfails to improve the evaluated transfer results\. These observations favor source\-specific selection of complementary acquisition fields over indiscriminate inclusion of available metadata\.

### VI\-CLimitations

The study uses retrospective public chest CT and CXR cohorts\. Prospective, multi\-institutional evaluation is needed to assess transfer in clinical workflows\. The synthetic perturbations test selected appearance changes and cover only part of the variation introduced by real scanners and workflows\. In addition, DICOM fields can be missing, inconsistently encoded, or correlated with institution and patient population\. Evaluation under unseen acquisition protocols and heterogeneous metadata schemas will further characterize the applicability of the acquisition prior\.

## VIIConclusion

We introduced Meta\-CLIP, which incorporates acquisition context into medical vision–language pretraining through pairwise contrastive margins\. Its kernel combines numerical and categorical DICOM fields to emphasize negative comparisons between acquisition\-similar studies\. This fixed prior shapes optimization while preserving the image and text encoders and their standard inference procedure\. Metadata is used during pretraining, with no additional inputs at deployment\.

Experiments on chest radiography and volumetric CT show improved zero\-shot classification, smaller performance losses under appearance perturbations, and stronger disease grounding\. The CXR benchmark demonstrates gains over KAD, while external CT evaluation extends the transfer evidence beyond the pretraining cohort\. Representation analyses and metadata characterization provide complementary views of the learned embeddings and their acquisition prior\. Together, these findings support acquisition context as a practical complement to image–report supervision for learning transferable medical representations\.

## References

- \[1\]A\. Radford, J\. W\. Kim, C\. Hallacy, A\. Ramesh, G\. Goh, S\. Agarwal, G\. Sastry, A\. Askell, P\. Mishkin, J\. Clark,et al\.\(2021\)Learning transferable visual models from natural language supervision\.InInternational conference on machine learning,pp\. 8748–8763\.Cited by:[§I](https://arxiv.org/html/2610.33674#S1.p1.1),[§II\-A](https://arxiv.org/html/2610.33674#S2.SS1.p1.1)\.
- \[2\]Y\. Zhang, H\. Jiang, Y\. Miura, C\. D\. Manning, and C\. P\. Langlotz\(2022\)Contrastive learning of medical visual representations from paired images and text\.InMachine Learning for Healthcare Conference,pp\. 2–25\.Cited by:[§I](https://arxiv.org/html/2610.33674#S1.p1.1),[§II\-A](https://arxiv.org/html/2610.33674#S2.SS1.p1.1),[TABLE VI](https://arxiv.org/html/2610.33674#S5.T6.2.1.3.1)\.
- \[3\]S\. Huang, L\. Shen, M\. P\. Lungren, and S\. Yeung\(2021\)Gloria: a multimodal global\-local representation learning framework for label\-efficient medical image recognition\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 3942–3951\.Cited by:[§I](https://arxiv.org/html/2610.33674#S1.p1.1),[§II\-A](https://arxiv.org/html/2610.33674#S2.SS1.p1.1),[TABLE VI](https://arxiv.org/html/2610.33674#S5.T6.2.1.4.1)\.
- \[4\]B\. Boecking, N\. Usuyama, S\. Bannur, D\. C\. Castro, A\. Schwaighofer, S\. Hyland, M\. Wetscherek, T\. Naumann, A\. Nori, J\. Alvarez\-Valle,et al\.\(2022\)Making the most of text semantics to improve biomedical vision–language processing\.InEuropean conference on computer vision,pp\. 1–21\.Cited by:[§I](https://arxiv.org/html/2610.33674#S1.p1.1),[§II\-A](https://arxiv.org/html/2610.33674#S2.SS1.p1.1),[§IV\-A](https://arxiv.org/html/2610.33674#S4.SS1.p1.1),[TABLE VI](https://arxiv.org/html/2610.33674#S5.T6.2.1.5.1),[TABLE VI](https://arxiv.org/html/2610.33674#S5.T6.2.1.6.1)\.
- \[5\]X\. Zhang, C\. Wu, Y\. Zhang, W\. Xie, and Y\. Wang\(2023\)Knowledge\-enhanced visual\-language pre\-training on chest radiology images\.Nature Communications14\(1\),pp\. 4542\.Cited by:[§I](https://arxiv.org/html/2610.33674#S1.p1.1),[§II\-A](https://arxiv.org/html/2610.33674#S2.SS1.p2.1),[§IV\-A](https://arxiv.org/html/2610.33674#S4.SS1.p1.1),[TABLE VI](https://arxiv.org/html/2610.33674#S5.T6.2.1.9.1)\.
- \[6\]C\. Wu, X\. Zhang, Y\. Zhang, Y\. Wang, and W\. Xie\(2023\)Medklip: medical knowledge enhanced language\-image pre\-training for x\-ray diagnosis\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 21372–21383\.Cited by:[§I](https://arxiv.org/html/2610.33674#S1.p1.1),[§II\-A](https://arxiv.org/html/2610.33674#S2.SS1.p2.1),[TABLE VI](https://arxiv.org/html/2610.33674#S5.T6.2.1.8.1)\.
- \[7\]I\. E\. Hamamci, S\. Er, C\. Wang, F\. Almas, A\. G\. Simsek, S\. N\. Esirgun, I\. Dogan, O\. F\. Durugol, B\. Hou, S\. Shit,et al\.\(2026\)Generalist foundation models from a multimodal dataset for 3d computed tomography\.Nature Biomedical Engineering,pp\. 1–19\.Cited by:[§I](https://arxiv.org/html/2610.33674#S1.p1.1),[§II\-A](https://arxiv.org/html/2610.33674#S2.SS1.p2.1)\.
- \[8\]Z\. Cheng, A\. Y\. Ong, S\. K\. Wagner, D\. A\. Merle, L\. Ju, H\. Zhang, R\. Chen, L\. Pang, B\. Li, T\. He,et al\.\(2025\)Understanding the robustness of vision\-language models to medical image artefacts\.NPJ Digital Medicine8\(1\),pp\. 727\.Cited by:[§I](https://arxiv.org/html/2610.33674#S1.p2.1),[§II\-B](https://arxiv.org/html/2610.33674#S2.SS2.p1.1)\.
- \[9\]S\. Hu, S\. Wang, W\. Weng, J\. Wang, X\. Wang, A\. Ozturk, Q\. Li, V\. Kumar, and A\. E\. Samir\(2020\)Self\-supervised pretraining with dicom metadata in ultrasound imaging\.InMachine learning for healthcare conference,pp\. 732–749\.Cited by:[§I](https://arxiv.org/html/2610.33674#S1.p3.1),[§II\-C](https://arxiv.org/html/2610.33674#S2.SS3.p1.1)\.
- \[10\]R\. Holland, O\. Leingang, H\. Bogunović, S\. Riedl, L\. Fritsche, T\. Prevost, H\. P\. N\. Scholl, U\. Schmidt\-Erfurth, S\. Sivaprasad, A\. J\. Lotery, D\. Rueckert, and M\. J\. Menten\(2024\)Metadata\-enhanced contrastive learning from retinal optical coherence tomography images\.Medical Image Analysis97,pp\. 103296\.External Links:[Document](https://dx.doi.org/10.1016/j.media.2024.103296)Cited by:[§I](https://arxiv.org/html/2610.33674#S1.p3.1),[§II\-C](https://arxiv.org/html/2610.33674#S2.SS3.p1.1)\.
- \[11\]Y\. Lee, W\. Han, Y\. Jun, H\. Kim, J\. Cho, and S\. J\. Hwang\(2025\)PRETI: patient\-aware retinal foundation model via metadata\-guided representation learning\.InInternational Conference on Medical Image Computing and Computer\-Assisted Intervention,pp\. 523–533\.Cited by:[§I](https://arxiv.org/html/2610.33674#S1.p3.1),[§II\-C](https://arxiv.org/html/2610.33674#S2.SS3.p1.1)\.
- \[12\]Z\. Chen, S\. Diao, B\. Wang, G\. Li, and X\. Wan\(2023\)Towards unifying medical vision\-and\-language pre\-training via soft prompts\.arXiv preprint arXiv:2302\.08958\.Cited by:[§II\-A](https://arxiv.org/html/2610.33674#S2.SS1.p1.1)\.
- \[13\]W\. Huang, H\. Zhou, C\. Li, H\. Yang, J\. Liu, and S\. Wang\(2023\)Enhancing representation in radiography\-reports foundation model: a granular alignment algorithm using masked contrastive learning\.arXiv preprint arXiv:2309\.05904\.Cited by:[§II\-A](https://arxiv.org/html/2610.33674#S2.SS1.p1.1)\.
- \[14\]Z\. Zhou, H\. Luo, J\. Pang, X\. Ding, M\. Gotway, and J\. Liang\(2023\)Learning anatomically consistent embedding for chest radiography\.arXiv preprint arXiv:2312\.00335\.Cited by:[§II\-A](https://arxiv.org/html/2610.33674#S2.SS1.p1.1)\.
- \[15\]H\. Luo, Z\. Zhou, M\. Hou, C\. Royer, M\. Reyes, A\. Sekuboyina, and B\. Menze\(2025\)DeViDe: faceted medical knowledge to enhance vision foundation model pretraining for radiology\.In2025 IEEE International Conference on Bioinformatics and Biomedicine \(BIBM\),pp\. 1779–1782\.Cited by:[§II\-A](https://arxiv.org/html/2610.33674#S2.SS1.p2.1),[TABLE VI](https://arxiv.org/html/2610.33674#S5.T6.2.1.12.1)\.
- \[16\]K\. Zhou, J\. Yang, C\. C\. Loy, and Z\. Liu\(2022\)Learning to prompt for vision\-language models\.International journal of computer vision130\(9\),pp\. 2337–2348\.Cited by:[§II\-B](https://arxiv.org/html/2610.33674#S2.SS2.p1.1)\.
- \[17\]R\. Zhang, R\. Fang, W\. Zhang, P\. Gao, K\. Li, J\. Dai, Y\. Qiao, and H\. Li\(2021\)Tip\-adapter: training\-free clip\-adapter for better vision\-language modeling\.arXiv preprint arXiv:2111\.03930\.Cited by:[§II\-B](https://arxiv.org/html/2610.33674#S2.SS2.p1.1)\.
- \[18\]J\. Zhang, G\. Wang, M\. K\. Kalra, and P\. Yan\(2024\)Disease\-informed adaptation of vision\-language models\.IEEE Transactions on Medical Imaging\.Cited by:[§II\-B](https://arxiv.org/html/2610.33674#S2.SS2.p1.1)\.
- \[19\]I\. E\. Hamamci, S\. Er, A\. Sekuboyina, E\. Simsar, A\. Tezcan, A\. G\. Simsek, S\. N\. Esirgun, F\. Almas, I\. Doğan, M\. F\. Dasdelen,et al\.\(2024\)GenerateCT: text\-conditional generation of 3d chest ct volumes\.InEuropean Conference on Computer Vision,pp\. 126–143\.Cited by:[§IV\-A](https://arxiv.org/html/2610.33674#S4.SS1.p1.1)\.
- \[20\]H\. Lai, Z\. Jiang, Q\. Yao, R\. Wang, Z\. He, X\. Tao, W\. Lv, W\. Wei, and S\. K\. Zhou\(2025\)Bridged semantic alignment for zero\-shot 3d medical image diagnosis\.IEEE Journal of Biomedical and Health Informatics\.External Links:[Document](https://dx.doi.org/10.1109/JBHI.2025.3629096)Cited by:[§IV\-A](https://arxiv.org/html/2610.33674#S4.SS1.p4.1)\.
- \[21\]E\. Tiu, E\. Talius, P\. Patel, C\. P\. Langlotz, A\. Y\. Ng, and P\. Rajpurkar\(2022\)Expert\-level detection of pathologies from unannotated chest x\-ray images via self\-supervised learning\.Nature Biomedical Engineering6\(12\),pp\. 1399–1406\.Cited by:[TABLE VI](https://arxiv.org/html/2610.33674#S5.T6.2.1.7.1)\.
- \[22\]V\. M\. H\. Phan, Y\. Xie, Y\. Qi, L\. Liu, L\. Liu, B\. Zhang, Z\. Liao, Q\. Wu, M\. To, and J\. W\. Verjans\(2024\)Decomposing disease descriptions for enhanced pathology detection: a multi\-aspect vision\-language pre\-training framework\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 11492–11501\.Cited by:[TABLE VI](https://arxiv.org/html/2610.33674#S5.T6.2.1.10.1)\.
- \[23\]H\. Lai, Q\. Yao, Z\. Jiang, R\. Wang, Z\. He, X\. Tao, and S\. K\. Zhou\(2024\)Carzero: cross\-attention alignment for radiology zero\-shot classification\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 11137–11146\.Cited by:[TABLE VI](https://arxiv.org/html/2610.33674#S5.T6.2.1.11.1)\.

相似文章

ClinLens:面向纵向多模态临床数据科学的长期编码智能体

arXiv cs.AI

ClinLens是一个新的基准,包含200个可执行的临床数据科学任务,涵盖五个相互关联的MIMIC资源,评估长期编码智能体在纵向多模态数据上的表现。结果显示代码执行能力强,但临床分析正确性较差,凸显了可运行提交与有效分析之间的差距。

CLIP:连接文本与图像

OpenAI Blog

CLIP 是 OpenAI 的视觉语言模型,从互联网上的文本-图像对中学习,实现零样本视觉分类,无需任务特定的训练数据。它通过减少对昂贵标注数据集的依赖并提高现实世界泛化能力,解决了传统计算机视觉的主要局限性。