EGiME: Entropy-Guided Model Evaluation in Few-Shot Learning

# EGiME, Entropy-Guided Model Evaluation; ViT, Vision Transformer; KNN, 𝐾-nearest neighbor; SVM, support vector machine; LEEP, Log Expected Empirical Prediction; LogME, Logarithm of Maximum Evidence; PACTran, PAC-Bayesian Transferability.
Source: [https://arxiv.org/html/2610.27029](https://arxiv.org/html/2610.27029)
Journal:Pattern RecognitionDequan JinEmail:[dqjin@gxu\.edu\.cn](mailto:[email protected])Affiliation:Department of Mathematics and Applied Mathematics, School of Mathematics and Information Science, Guangxi University, Nanning, Guangxi, 530004, ChinaTao LiEmail:[2406301025@st\.gxu\.edu\.cn](mailto:[email protected])Affiliation:Department of Mathematics and Applied Mathematics, School of Mathematics and Information Science, Guangxi University, Nanning, Guangxi, 530004, ChinaLingfeng XingEmail:[2406301052@st\.gxu\.edu\.cn](mailto:[email protected])Affiliation:Department of Mathematics and Applied Mathematics, School of Mathematics and Information Science, Guangxi University, Nanning, Guangxi, 530004, ChinaShihui YingEmail:[shying@shu\.edu\.cn](mailto:[email protected])Affiliation:Department of Mathematics, School of Science, Shanghai University, Shanghai, 200444, ChinaHui ZhaoEmail:[zhaohui\_2020@gxu\.edu\.cn](mailto:[email protected])Affiliation:School of Economics, Guangxi University, Nanning, Guangxi, 530004, ChinaCorresponding author:Corresponding author

###### Abstract

The feature\-extraction quality of pre\-trained models sets the ceiling of downstream transfer learning, yet few\-shot model selection remains difficult: supervised evaluation on scarce labels is unstable and yields fluctuating rankings\. We propose Entropy\-Guided Model Evaluation \(EGiME\), a semi\-supervised criterion that assesses a candidate model from the clustering structure of its feature space without any classifier training\. EGiME performsKK\-means clustering over all labeled and unlabeled samples, then combines two entropy terms: an inter\-cluster entropy measuring global separability, and an intra\-cluster entropy measuring the compactness of labeled anchors, fused through an inverse\-frequency weight that emphasizes small clusters\. On five few\-shot benchmarks and nine pre\-trained models, EGiME achieves a correlation score of 0\.966 with the full\-pool ground\-truth accuracy obtained by averaging the accuracies of K\-means, KNN, and SVM\. At labeling ratios of 4%–20%, it produces markedly more stable model rankings than cross\-validated supervised baselines, establishing it as a practical alternative to conventional supervised evaluation for pre\-trained model selection under data scarcity\.

###### Keywords:

Few\-shot learning , information entropy , model selection , representation evaluation , semi\-supervised learning , transfer learning

## 1Introduction

The powerful feature extraction capability of deep neural networks is fundamental in image recognition tasks\. It enables the mapping of raw pixel data into high\-dimensional representations with distinct semantic discriminability, determines the generalization, transferability, and robustness of network models\[[16](https://arxiv.org/html/2610.27029#bib.bib17),[12](https://arxiv.org/html/2610.27029#bib.bib30)\], and serves as a core prerequisite for stable task performance in challenging scenarios such as few\-shot learning and domain adaptation\. In practical industrial and academic scenarios including industrial defect detection, medical image analysis, and fine\-grained species identification, annotated sample scarcity is a ubiquitous and restrictive constraint\. Accordingly, the capability validation and quantitative evaluation of deep models in few\-shot settings have become a key bottleneck limiting the practical deployment of deep learning models\. Therefore, exploring stable, interpretable quantitative evaluation methods for feature extraction capabilities in few\-shot scenarios is of significant theoretical research value and practical engineering significance\.

Existing evaluation paradigms for the feature extraction capacity of deep neural networks fall into two mainstream categories\. The first way corresponds to indirect evaluation grounded on downstream task performance, which adopts task metrics such as classification accuracy and Top\-k precision to quantify feature quality, serving as a standard protocol across academia and industry\. Nevertheless, such methods heavily rely on large\-scale annotated test sets, and their outcomes are susceptible to irrelevant confounding factors including classifier design and training strategies\. In few\-shot learning tasks, these evaluation methods suffer severe fluctuations due to prominent sampling bias, failing to eliminate interferences introduced by task layers and reflect the intrinsic model performance in feature extraction\. The second way to directly evaluate model performance is based on geometric properties of the feature space\. It characterizes the distribution property of “intra\-class compactness and inter\-class separability” via metrics such as intra\-cluster and inter\-class distances and silhouette coefficients, delivering superior interpretability\. However, these approaches compute metrics using labeled samples and yield poor statistical stability in few\-shot learning tasks\. Furthermore, they fail to exploit global distribution constraints embedded in massive unlabeled samples, and thus cannot comprehensively measure the overall quality of the feature space\.

To address these issues, cross\-validation acts as a widely adopted statistical augmentation technique\. It iteratively partitions and cross\-reuses limited labeled samples\. Its representative implementations include k\-fold cross\-validation and leave\-one\-out cross\-validation\. Cross\-validation reduces evaluation variance induced by single random splits, improving the confidence and stability of generalization performance\. Within the N\-way K\-shot evaluation protocol in few\-shot learning, the standard pipeline that repeatedly samples support and query sets and averages the corresponding accuracy metrics essentially constitutes a task\-specific variant derived from the cross\-validation paradigm\. Nevertheless, cross\-validation merely reuses available annotated samples without lifting the limitation of limited annotation volume, nor can it incorporate global distribution information from unlabeled samples to fundamentally mitigate systematic biases inherent to few\-shot evaluation\. Furthermore, cross\-validation only outputs statistical distributions of task metrics and lacks interpretability at the feature space level, making it incapable of revealing intrinsic structural discrepancies in the feature extraction capacity of different models\.

Semi\-supervised representation learning offers data\-oriented solutions to address the issues of feature evaluation with very few samples\. Advances in semi\-supervised learning and self\-supervised contrastive learning have fully demonstrated that abundant unlabeled samples can impose global distribution constraints on the semantic structure of feature spaces\. Consistency regularization and contrastive loss guide models to learn more generalizable feature representations\. Their effectiveness has been extensively validated in tasks such as few\-shot classification and cross\-domain transfer learning\. Nevertheless, most existing studies focus on “how to leverage unlabeled samples to boost feature extraction performance”, while rarely reversing this logic to adopt distribution information of unlabeled samples for “evaluating feature extraction capacity”\. It prevents the full exploitation of available samples to evaluate model performance on feature extraction in few\-shot learning scenarios\.

From the perspective of measurement tools, information entropy, a classical metric for characterizing the disorder degree of data distributions, has been employed in uncertainty estimation, representation quality assessment, and feature selection for deep models\. Several studies quantify representation discriminability by calculating entropy values of feature dimensions or category distributions\. Nonetheless, most existing entropy\-based evaluation metrics still rely on labeled samples for measurement construction, excluding the global distribution entropy derived from unlabeled samples in evaluation frameworks\. Meanwhile, a unified entropy measurement framework oriented toward the core criterion of “intra\-class compactness and inter\-class separability” remains absent\. This makes it infeasible to comprehensively quantify feature extraction capacity\. In summary, current literature has not established a unified few\-shot feature evaluation paradigm integrating semi\-supervised data utilization and information entropy measurement, and the distribution value of unlabeled samples during evaluation procedures has not been systematically excavated\.

According to the above discussion, we can see that when labeled samples have become extremely scarce, any evaluation paradigm that relies on training a classifier with labeled data is inherently unsustainable\. To address this issue, distribution information from unlabeled samples must be incorporated to reliably evaluate the representation quality of models\. On the other hand, if the distribution of extracted samples in the feature space satisfies the core feature quality criterion of intra\-class compactness and inter\-class separability and exhibits a clear and well\-defined clustering structure, the model can be regarded as possessing strong feature extraction capability\. Motivated by this insight, this paper proposes a novel Entropy\-Guided Model Evaluation \(EGiME\) method\. EGiME first extracts feature representations of all available samples \(both labeled and unlabeled\) using each candidate pre\-trained model, and then performsKK\-means clustering in the resulting feature space to reveal their latent geometric topology\. Subsequently, entropy\-guided metrics are formulated to quantitatively characterize the quality of the clustering configuration\. Specifically, an inter\-cluster entropy is defined to measure the uniformity of the sample distribution across different clusters, capturing global separability; concurrently, an intra\-cluster entropy is introduced to evaluate the concentration of labeled samples within each individual cluster, reflecting local compactness\. Finally, these two complementary indicators are fused into a unified metric that serves as a comprehensive measure of model performance\. The primary contributions of this work are summarized as follows:

1. 1\.We propose a novel semi\-supervised evaluation paradigm for model assessment in few\-shot scenarios\. It breaks the purely supervised constraint of traditional pre\-trained model evaluation and systematically integrates the global distribution of massive unlabeled samples into the feature extraction capability evaluation workflow, fundamentally alleviating the core issues of unstable model ranking caused by labeled data scarcity in few\-shot scenarios\.
2. 2\.We provide a unified two\-dimensional evaluation metric constructed on information entropy\. Taking information entropy as the core theoretical instrument, we design the inter\-cluster entropy and intra\-cluster entropy to characterize the global separability of semantic clusters and the local compactness of homogeneous samples within each cluster and fuse them into a comprehensive metric, realizing accurate and interpretable quantification of the core feature quality principle of intra\-class compactness and inter\-class separability\.
3. 3\.We present a lightweight evaluation pipeline\. The proposed method employsKK\-means clustering as the unsupervised structure discovery module\. The entire evaluation procedure requires no auxiliary classifier training or fine\-tuning of pre\-trained models, which technically circumvents evaluation distortion induced by classifier overfitting under few\-shot settings\. With only the true class number of the target dataset as the prior parameter, this method features high computational efficiency and strong cross\-domain adaptability, enabling efficient model selection for large\-scale pre\-trained model zoos\.

The remainder of this paper is organized as follows\. Section 2 reviews related work on transferable model selection, feature selection, and entropy\-based clustering analysis\. Section 3 presents the methodology\. Section 4 reports the experimental setup and results\. Section 5 concludes the paper and outlines future research directions\.

## 2Related Work

### 2\.1Model Selection in Transfer Learning

Selecting an appropriate feature extractor for a downstream task is a critical step that determines ultimate performance\. With the proliferation of pre\-trained models such as AlexNet\[[24](https://arxiv.org/html/2610.27029#bib.bib7)\], VGG\[[32](https://arxiv.org/html/2610.27029#bib.bib8)\], GoogLeNet\[[34](https://arxiv.org/html/2610.27029#bib.bib9)\], ResNet\[[19](https://arxiv.org/html/2610.27029#bib.bib10)\], and Vision Transformers \(ViT\)\[[12](https://arxiv.org/html/2610.27029#bib.bib30)\]in public model zoos, efficiently choosing the best model for a target task has become a fundamental challenge, formally defined as the model selection problem\[[42](https://arxiv.org/html/2610.27029#bib.bib14)\]\.

The importance of model selection lies in the observation that different backbone architectures exhibit significantly varying representation capabilities on the same target dataset, and selecting an unsuitable backbone leads to degraded downstream performance\. To avoid the costly full fine\-tuning of every candidate model, researchers have proposed the research direction of transferability estimation\[[41](https://arxiv.org/html/2610.27029#bib.bib29)\], which aims to design a metric that can predict the performance of a pre\-trained model on a target dataset without exhaustive fine\-tuning\. Representative works include the LogME method based on Bayesian evidence maximization\[[41](https://arxiv.org/html/2610.27029#bib.bib29)\], the LEEP method based on empirical conditional probability\[[26](https://arxiv.org/html/2610.27029#bib.bib19)\], the H\-score method from information theory\[[4](https://arxiv.org/html/2610.27029#bib.bib18)\], the ETran method based on energy models\[[14](https://arxiv.org/html/2610.27029#bib.bib38)\], the PACTran method based on PAC\-Bayes theory\[[11](https://arxiv.org/html/2610.27029#bib.bib41)\], and the NCTI method based on neural collapse distance\[[40](https://arxiv.org/html/2610.27029#bib.bib39),[28](https://arxiv.org/html/2610.27029#bib.bib24)\], which evaluates transferability by measuring the deviation of the feature space from the ideal simplex structure predicted by the neural collapse phenomenon\. Furthermore, systematic evaluations of multiple transferability metrics across different benchmark datasets have been conducted\[[2](https://arxiv.org/html/2610.27029#bib.bib35),[20](https://arxiv.org/html/2610.27029#bib.bib36)\], revealing the applicability differences of various methods in different scenarios\.

The mainstream approach to evaluating candidate models iskk\-fold cross\-validation\[[23](https://arxiv.org/html/2610.27029#bib.bib3)\]\. It divides the limited labeled samples intokksubsets, alternately trains auxiliary classifiers such as linear SVM or KNN, and averages the accuracy\. This approach provides asymptotically unbiased estimates when labeled samples are sufficient\.

Applied directly to few\-shot scenarios, these methods become unreliable\[[29](https://arxiv.org/html/2610.27029#bib.bib31),[36](https://arxiv.org/html/2610.27029#bib.bib16)\]\. The compressed per\-fold training set cannot support a stable auxiliary classifier, inflating the variance of accuracy estimates and increasing sensitivity to individual support samples\. Moreover, as a purely supervised paradigm, cross\-validation discards the structural information in unlabeled data, which constitutes the vast majority in practice\. These factors cause ranking fluctuations across sampling rounds and amplify overfitting when the sample count is far below the feature dimensionality\.

The core of model selection is to discriminate the quality of features extracted by candidate models, so the problem above essentially reduces to feature\-quality evaluation under few\-shot conditions—which motivates us to examine how the feature selection community has addressed similar dilemmas\.

### 2\.2Feature Selection Methods in Few\-Shot Scenarios

Feature selection identifies the most discriminative subset or transformation from a high\-dimensional feature space\[[15](https://arxiv.org/html/2610.27029#bib.bib20)\], a challenge especially acute in few\-shot learning\[[38](https://arxiv.org/html/2610.27029#bib.bib11),[33](https://arxiv.org/html/2610.27029#bib.bib13),[13](https://arxiv.org/html/2610.27029#bib.bib12),[39](https://arxiv.org/html/2610.27029#bib.bib23)\]\. Traditional methods fall into three categories: filter methods evaluate features independently through statistical indicators such as mutual information and correlation coefficients, offering efficiency but ignoring inter\-feature interactions; wrapper methods embed selection into classifier training through recursive elimination or heuristic search, capturing interactions at high computational cost and overfitting risk; and embedded methods treat selection as a regularization constraint during training, such as L1 regularization in Lasso\.

In few\-shot scenarios these methods face thep≫np\\gg ndilemma: feature\-label correlation measures are highly susceptible to sampling noise, and wrapper and embedded methods overfit with insufficient samples, yielding selected features that generalize poorly\. Recent few\-shot\-oriented variants design automatic selection modules that weight features from different backbones, or combine mid\-level transferability with high\-level discriminability for cross\-domain tasks; however, most remain embedded in the training pipeline of a specific model and cannot be directly transferred to cross\-model comparison\.

Clustering structures\[[5](https://arxiv.org/html/2610.27029#bib.bib15)\]and information entropy\[[21](https://arxiv.org/html/2610.27029#bib.bib32)\]have long served as effective tools for measuring feature quality: information compression indices evaluate feature importance through clustering changes after feature removal, mutual information measures feature\-label correlation, and clustering purity and silhouette coefficients reflect inter\-class separation\. The core insight is that the clustering structure reflects feature discriminability, and entropy provides a natural tool for quantifying it\. This motivates our study: taking the clustering structure as the analysis object and information entropy as the quantification tool, we construct a feature\-quality criterion free of classifier training that addresses both scarce labels and cross\-model comparison\.

### 2\.3Feature Space Distribution Analysis Methods

Few\-shot feature\-quality evaluation requires a structure\-discovery tool that exploits unlabeled data without supervision\. Self\-supervised and vision\-language pre\-training have shown that unlabeled data alone can yield highly transferable representations\[[9](https://arxiv.org/html/2610.27029#bib.bib25),[17](https://arxiv.org/html/2610.27029#bib.bib26),[18](https://arxiv.org/html/2610.27029#bib.bib37),[30](https://arxiv.org/html/2610.27029#bib.bib33),[27](https://arxiv.org/html/2610.27029#bib.bib40)\], while supervised contrastive learning further improves the learned embedding when labels are available\[[22](https://arxiv.org/html/2610.27029#bib.bib27)\]\. Earlier semi\-supervised learning established the same value of unlabeled structure\[[8](https://arxiv.org/html/2610.27029#bib.bib5)\], with a frozen embedding already competitive in few\-shot classification\[[35](https://arxiv.org/html/2610.27029#bib.bib22)\]\. Cluster analysis provides precisely such a tool\[[25](https://arxiv.org/html/2610.27029#bib.bib2)\], and variants such as nonlinear manifold clustering\[[1](https://arxiv.org/html/2610.27029#bib.bib34)\]and self\-supervised clustering\[[6](https://arxiv.org/html/2610.27029#bib.bib28)\]enrich the toolbox\.

Cluster analysis partitions data so that intra\-cluster similarity is maximized and inter\-cluster similarity is minimized\. Requiring no labels, it naturally suits few\-shot scenarios by using all samples, including the unlabeled majority, to build a global geometric description of the feature space\. Since the true number of target categories is known, the cluster number is set toK=sK=swithout additional tuning\. We adopt classicKK\-means\[[25](https://arxiv.org/html/2610.27029#bib.bib2)\], whose fixed\-centered variant stabilizes centers against noisy initializations\[[3](https://arxiv.org/html/2610.27029#bib.bib42)\]\.KK\-means is widely adopted for its intuitive principle, simple implementation, and computational efficiency\. In EGiME, we avoid relying on random restarts: for each labeled fold, the class\-mean feature vector of every class is used as a centroid, andKK\-means is run once from this initialization \(ninit=1n\_\{\\mathrm\{init\}\}=1\) over the complete feature set\. This class\-mean initialization anchors the clusters to the available semantic information and reduces the variability that would arise from a single random initialization\. Its workflow takes all sample features as input, searches forKKcenters, assigns each sample to the nearest center, and outputs cluster assignments reflecting the intrinsic topology of the feature space\.

Cluster assignments are converted into model\-quality signals through information entropy\[[31](https://arxiv.org/html/2610.27029#bib.bib1),[10](https://arxiv.org/html/2610.27029#bib.bib4)\]: inter\-cluster entropy measures the uniformity of sample distribution across clusters to reflect global separability, while intra\-cluster entropy measures the concentration of labeled samples within each cluster to reflect local compactness\. Combining the two with an inverse frequency weighting yields the unified weighted entropy metric, detailed in Section 3\.

## 3Methodology

### 3\.1Motivation

If a model can extract sample features effectively, the clusters formed by samples in the feature space should be consistent with their semantic categories\. That is, samples of the same category form a single cluster, and clusters corresponding to different categories are well separated\. Nevertheless, practical models often fail to satisfy this ideal property\. It is possible that the extracted features of two samples lie close to each other in the feature space although they belong to distinct categories\. Meanwhile, due to considerable intra\-class variance, samples from the same category may scatter across multiple clusters\.

This phenomenon can be intuitively verified from the visualization results in Fig\.[1](https://arxiv.org/html/2610.27029#S3.F1), which shows thett\-SNE\[[37](https://arxiv.org/html/2610.27029#bib.bib6)\]projection of feature distributions of AlexNet \(left\) and ViT\-B/16 \(right\) on the Mini\-ImageNet dataset\. It can be clearly observed that for the high\-performance ViT\-B/16 model, samples of the same class are tightly clustered in the feature space, with clear boundaries between different classes, and the clustering structure exhibits a distribution pattern highly consistent with true semantic categories\. For AlexNet, the feature distribution is relatively chaotic, with samples of the same class dispersed and severe overlap across classes, making them difficult to distinguish\. This observation indicates that the degree of consistency between the clustering structure and the true categories is an effective indicator of feature extractor quality: the higher the consistency, the stronger the model’s representational capability\. Therefore, if we have a way to measure this consistency, we can evaluate the model’s representational capability\.

However, under few\-shot settings, the scarcity of labeled samples impedes accurate quantification of the consistency between sample clustering structures and ground\-truth categories\. To circumvent this obstacle, an alternative evaluation strategy is required\. As illustrated in Fig\.[1](https://arxiv.org/html/2610.27029#S3.F1), compared with AlexNet, ViT with superior feature extraction performance generates more uniformly distributed clusters within the feature space\. Meanwhile, each cluster almost exclusively contains samples belonging to a single category\. This observation inspires us to adopt these two properties as surrogate criteria for measuring the consistency between clustering structures and true categories\. Specifically: when all samples \(including unlabeled ones\) form more evenly distributed clusters in the feature space, and the labeled samples within each cluster are purer, the corresponding model possesses stronger feature extraction capability\. Given the above definition, the subsequent question becomes how to measure the distribution uniformity of features within the feature space\.

\(a\) AlexNet

\(b\) ViT\-B/16

Figure 1:The feature\-extracted sample distributions derived by AlexNet \(left\) and ViT\-B/16 \(right\) from the Mini\-ImageNet dataset and visualized bytt\-SNE\. Different colors represent different categories\.In information theory, entropy is a widely adopted metric for quantifying information volume\. For a discrete distribution overKKgroups, letpj=nj/Np\_\{j\}=n\_\{j\}/N, wherenjn\_\{j\}is the number of samples in thejj\-th group andNNis the total number of samples\. The Shannon entropy and its normalized form areH=−∑j=1KpjlogpjH=\-\\sum\_\{j=1\}^\{K\}p\_\{j\}\\log p\_\{j\}andE=H/log⁡KE=H/\\log K, respectively\. Thus,E∈\[0,1\]E\\in\[0,1\], withE=0E=0when all samples fall into one group andE=1E=1when they are uniformly distributed across theKKgroups\. Entropy decreases when samples are distributed more compactly and increases as the distribution becomes more uniform, so it can measure the uniformity of sample distribution\.

To calculate information entropy, it is necessary to acquire the distribution frequency of samples in the feature space\. To this end, the feature space first needs to be discretized into multiple grid cells, and the frequency is then calculated based on these grids\. Nevertheless, as illustrated in Figure 1, the grid\-based frequency computation breaks the integrity of sample clusters, where a single cluster may be split across multiple grids\. In addition, the choice of grid size exerts a substantial impact on the estimated sample frequency distribution\. Under few\-shot settings, it is difficult to determine an appropriate grid partition relying on sample labels\. A more natural alternative is to compute distribution frequencies according to the sample clusters formed in the feature space, which requires clustering algorithms to capture the spatial distribution patterns of samples\. As an unsupervised approach, clustering does not depend on sample labels\. Hence, it can effectively characterize the spatial aggregation of samples even under few\-shot constraints and eliminate disturbances to frequency calculation arising from different grid partition strategies\. On this basis, we need to construct two entropy metrics to characterize the properties of sample distribution: one measures the macroscopic distribution uniformity of sample clusters, and the other quantifies the purity of labeled samples within each cluster\.

Motivated by the above discussion, we propose an Entropy\-Guided Model Evaluation \(EGiME\) method to infer the candidate models’ capability on feature extraction by clustering all samples in the corresponding feature spaces and examining their consistency between sample distributions and semantic categories with limited labels\. As shown in Fig\.[2](https://arxiv.org/html/2610.27029#S3.F2), the EGiME pipeline consists of three main stages\. First, candidate pre\-trained models map all input images to high\-dimensional feature vectors, completing feature embedding\. Second, K\-means clustering is performed in the feature space, with the number of clusters naturally set to the number of target categories\. Third, based on the clustering results, the labeled samples serve as semantic references, and entropy\-based metrics simultaneously quantify the inter\-class separability and intra\-class compactness of the feature space, ultimately providing a comprehensive evaluation score of model performance\. The following sections will elaborate on the implementation details of these three stages\.

![Refer to caption](https://arxiv.org/html/2610.27029v1/framework.png)Figure 2:Overall framework of the proposed EGiME methodology\. The pipeline consists of three main stages: Feature embedding, structure discovery, and entropy\-based evaluation\.
### 3\.2Problem Definition and Notation

LetD=\{y1,y2,…,yn\}D=\\\{y\_\{1\},y\_\{2\},\\dots,y\_\{n\}\\\}be an dataset, whereyiy\_\{i\}represents the raw image\. Denote the set ofqqpre\-trained modelsℳ=\{M1,M2,…,Mq\}\\mathcal\{M\}=\\\{M\_\{1\},M\_\{2\},\\dots,M\_\{q\}\\\}\. For any candidate modelMr∈ℳM\_\{r\}\\in\\mathcal\{M\}, it is regarded as a feature extraction functionfr:𝒴→ℝmf\_\{r\}:\\mathcal\{Y\}\\rightarrow\\mathbb\{R\}^\{m\}, which maps a raw image into anmm\-dimensional feature vector\. Consequently, we obtain the corresponding feature setDf\(r\)D\_\{f\}^\{\(r\)\}, where each sample corresponds to a feature vector expressed asxi\(r\)=fr​\(yi\)x\_\{i\}^\{\(r\)\}=f\_\{r\}\(y\_\{i\}\)\.

Among the totalnnsamples, onlyllsamples are provided with category labels, constituting the label setDlD\_\{l\}\. These labels belong to the category setℒ\\mathcal\{L\}with cardinality\|ℒ\|=nl\|\\mathcal\{L\}\|=n\_\{l\}\. The total number of true categories in the entire dataset is denoted asss\(typicallys≥nls\\geq n\_\{l\}\), which is assumed to be known as prior information\. In the following discussion, we supposes=nls=n\_\{l\}\.

Our objective is to construct a quantitative metricErE\_\{r\}based on the feature setDf\(r\)D\_\{f\}^\{\(r\)\}and the labeled setDlD\_\{l\}, which evaluates the feature extraction capability of the modelMrM\_\{r\}on the target domain\. Furthermore, this metric must maintain high stability and discriminability even in few\-shot scenarios where labeled samples are extremely scarce\.

### 3\.3Feature Space Clustering and Inverse Frequency

To evaluate the structural characteristics of the feature space, K\-means clustering is performed on the feature spaceD\(r\)D^\{\(r\)\}, where the number of clusters is set toK=sK=s\. This yields the cluster assignment vector𝐜\(r\)\\mathbf\{c\}^\{\(r\)\}, in whichci\(r\)c\_\{i\}^\{\(r\)\}denotes the cluster to which theii\-th sample belongs\.

The cluster count vector is defined as𝐂𝐫=\(c​r1,c​r2,…,c​rK\)T\\mathbf\{Cr\}=\(cr\_\{1\},cr\_\{2\},\\dots,cr\_\{K\}\)^\{\\mathrm\{T\}\}, wherec​rjcr\_\{j\}is the number of samples in thejj\-th cluster\.

Subsequently, the raw cluster frequency vector is defined as𝐟r=\(fr​1,fr​2,…,fr​K\)T\\mathbf\{f\}\_\{r\}=\(f\_\{r1\},f\_\{r2\},\\dots,f\_\{rK\}\)^\{\\mathrm\{T\}\}withfr​j=c​rj/nf\_\{rj\}=cr\_\{j\}/n\.

To enhance the sensitivity to small clusters, an inverse frequency weight is proposed:f~r​j=1/fr​j∑k=1K\(1/fr​k\)\\tilde\{f\}\_\{rj\}=\\frac\{1/f\_\{rj\}\}\{\\sum\_\{k=1\}^\{K\}\(1/f\_\{rk\}\)\}forj=1,…,Kj=1,\\dots,K\. This weighting strategy explicitly assigns higher significance to low\-frequency clusters \(i\.e\., small clusters\), thereby highlighting the model’s discriminative capability for hard samples and rare categories during the evaluation process\.

### 3\.4Inter\-cluster Entropy and Intra\-cluster Entropy

The inter\-cluster entropy measures the uniformity of the sample distribution across theKKclusters:

Er\(inter\)=−1log⁡K∑j=1Kf~r​jlogf~r​jE\_\{r\}^\{\(\\mathrm\{inter\}\)\}=\-\\frac\{1\}\{\\log K\}\\sum\_\{j=1\}^\{K\}\\tilde\{f\}\_\{rj\}\\log\\tilde\{f\}\_\{rj\}\(1\)A larger value indicates that the samples are uniformly dispersed across the clusters, reflecting superior global separation capability\.

For labeled samples, we consider the distribution of each category within the clusters\. Letℐi=\{j∈\{1,…,l\}:lj=i\}\\mathcal\{I\}\_\{i\}=\\\{j\\in\\\{1,\\dots,l\\\}:l\_\{j\}=i\\\}denote the index set of labeled samples for categoryiiwithNi=\|ℐi\|N\_\{i\}=\|\\mathcal\{I\}\_\{i\}\|, and let the class\-cluster count vector be𝐧r​i=\(nr​i​1,nr​i​2,…,nr​i​K\)T\\mathbf\{n\}\_\{ri\}=\(n\_\{ri1\},n\_\{ri2\},\\dots,n\_\{riK\}\)^\{T\}withnr​i​j=∑k∈ℐi𝕀\[ck\(r\)=j\]n\_\{rij\}=\\sum\_\{k\\in\\mathcal\{I\}\_\{i\}\}\\mathbb\{I\}\[c\_\{k\}^\{\(r\)\}=j\]\.

Since the labeled samples are evenly distributed among classes in most few\-shot scenarios, we form the frequency vector𝐟r​i=\(fr​i​1,fr​i​2,…,fr​i​K\)T\\mathbf\{f\}\_\{ri\}=\(f\_\{ri1\},f\_\{ri2\},\\dots,f\_\{riK\}\)^\{T\}without inverse weighting\. The intra\-cluster entropy isEr​i\(intra\)=−1log⁡K∑j=1Kfr​i​jlogfr​i​jE\_\{ri\}^\{\(\\mathrm\{intra\}\)\}=\-\\frac\{1\}\{\\log K\}\\sum\_\{j=1\}^\{K\}f\_\{rij\}\\log f\_\{rij\}, and the average intra\-cluster entropy isEr\(intra\)=1nl​∑i=1nlEr​i\(intra\)E\_\{r\}^\{\(\\mathrm\{intra\}\)\}=\\frac\{1\}\{n\_\{l\}\}\\sum\_\{i=1\}^\{n\_\{l\}\}E\_\{ri\}^\{\(\\mathrm\{intra\}\)\}\. A smallerEr\(intra\)E\_\{r\}^\{\(\\mathrm\{intra\}\)\}indicates that samples of the same class are highly concentrated within clusters, demonstrating better local compactness\.

### 3\.5Entropy\-guided Metric for Model Evaluation

In the above discussion, a largerEr\(inter\)E\_\{r\}^\{\(\\mathrm\{inter\}\)\}indicates superior global separation while a smallerEr\(intra\)E\_\{r\}^\{\(\\mathrm\{intra\}\)\}demonstrates better local compactness\. An excellent feature extractor should simultaneously maximize the former and minimize the latter, so we evaluate model performance by combining them as follows:

Er=Er\(inter\)1\+Er\(intra\)E\_\{r\}=\\frac\{E\_\{r\}^\{\(\\mathrm\{inter\}\)\}\}\{1\+E\_\{r\}^\{\(\\mathrm\{intra\}\)\}\}\(2\)
This metric reflects the trade\-off between local compactness and global separability, and a largerErE\_\{r\}indicates superior model performance\. The constant 1 in the denominator prevents division by zero\. To reduce the dependence of the score on a single partition of the scarce labeled samples, we adopt a fold\-averaged estimation procedure\. Specifically, the labeled samples are divided intoFFstratified folds, whereF=min⁡\(10,⌊nshot/2⌋\)F=\\min\(10,\\lfloor n\_\{\\mathrm\{shot\}\}/2\\rfloor\); thus,FFranges from 2 to 10 fornshot∈\[4,20\]n\_\{\\mathrm\{shot\}\}\\in\[4,20\]\. For each fold, the class\-mean feature vector of every class in the current labeled fold is used as the correspondingKK\-means centroid, andKK\-means is run once from this initialization \(ninit=1n\_\{\\mathrm\{init\}\}=1\) on the complete evaluation pool containing both labeled and unlabeled samples\. The resulting assignments are then used to compute the inter\- and intra\-cluster entropies, with all labeled samples serving as semantic references\. The final score is the average of the fold\-level weighted entropies,E¯r=F−1​∑j=1FEr​j\\bar\{E\}\_\{r\}=F^\{\-1\}\\sum\_\{j=1\}^\{F\}E\_\{rj\}, whereEr​jE\_\{rj\}denotes the weighted entropy obtained in foldjj\. Algorithm 1 summarizes this fold\-averaged procedure\.

Table 1:EGiME: Fold\-Averaged Entropy\-Guided Model Evaluation

## 4Experiments

### 4\.1Experimental Setup

Datasets:We evaluate our proposed method on five widely\-used few\-shot learning benchmark datasets: the validation set of Mini\-ImageNet \(Mini\-val, 16 classes\), STL\-10 \(10 classes,96×9696\\times 96pixels\), CIFAR\-10 \(10 classes,32×3232\\times 32pixels\), EuroSAT \(10 classes,64×6464\\times 64pixels\), and CIFAR\-100 \(100 classes,32×3232\\times 32pixels\)\.

Evaluation Pool:To construct a unified and controllable evaluation benchmark, we draw samples from the labeled data source used for each dataset\. Specifically, 100 images are randomly selected from each class to form the evaluation pool\. All selected samples participate in the subsequent feature\-based clustering and evaluation\.

Experimental Setting:nshotn\_\{\\mathrm\{shot\}\}samples are randomly selected per class as labeled samples from the evaluation pool, wherenshot∈\{4,6,8,10,12,14,16,18,20\}n\_\{\\mathrm\{shot\}\}\\in\\\{4,6,8,10,12,14,16,18,20\\\}, simulating different levels of few\-shot labeling\. The remaining100−nshot100\-n\_\{\\mathrm\{shot\}\}samples serve as unlabeled data\. Such sampling ensures that the proportion of labeled samples ranges from 4% to 20%, effectively simulating real\-world scenarios with extremely scarce labeled samples\. Cross\-validation \(CV\) is adopted for comparison: standard 10\-fold cross\-validation whennshot≥10n\_\{\\mathrm\{shot\}\}\\geq 10, andnshotn\_\{\\mathrm\{shot\}\}\-fold cross\-validation \(i\.e\., leave\-one\-out\) whennshot<10n\_\{\\mathrm\{shot\}\}<10, ensuring that each fold contains at least one representative sample from each class, avoiding evaluation bias caused by the absence of certain classes in the validation set due to insufficient samples\. For eachnshotn\_\{\\mathrm\{shot\}\}, the random sampling process is repeated 100 times to eliminate statistical variance\.

Models:We evaluate nine backbone architectures pre\-trained on the ImageNet dataset, covering traditional convolutional neural networks \(AlexNet, VGG11, VGG13, GoogLeNet, ResNet18, ResNet34\) and modern Vision Transformers \(ViT\-S, ViT\-B/16, ViT\-L\)\. The detailed specifications of these models, including their architecture types and extracted feature dimensions, are summarized in Table[2](https://arxiv.org/html/2610.27029#S4.T2)\. Since the original classification heads of different network models yield inconsistent impacts on the network performance, we discard them and retain only the feature extraction backbone of each network\. The resulting features are used to compute KNN, SVM, and K\-means accuracies solely for constructing the full\-label reference ranking described below; EGiME itself does not train a classifier\. For CNN\-based models, feature vectors are extracted directly after the global average pooling layer; for ViT models, the output vector corresponding to the\[CLS\]token is extracted as the image representation\.

Table 2:Specifications of the Pre\-trained Models Evaluated in the ExperimentsGround\-truth Ranking and Evaluation Metrics:At present, there are no criteria for quantitatively evaluating the feature extraction performance of different deep neural network models\. We therefore define the ground\-truth score of a model as the equal\-weight mean of the accuracies of three complementary predictors evaluated over the full evaluation pool: KNN with one neighbor, a linear SVM, and K\-means initialized with class means so that the cluster indices align with the class labels\. For each model, the mean accuracy of each predictor is computed independently by 10\-fold cross\-validation over the fully labeled evaluation pool\. These three mean accuracies are then averaged with equal weights to obtain the ground\-truth score, and the nine models are ranked by this score \(rank 1 denotes the highest accuracy; ties receive the average rank\)\. In other words, the average is taken over accuracies, not over model rankings\. KNN and SVM capture local neighborhood consistency and supervised decision\-boundary quality, whereas K\-means supplements them with the global geometric organization of the feature space, rewarding intra\-class compactness and clear inter\-class separation\. The resulting ranking is reported in Table[3](https://arxiv.org/html/2610.27029#S4.T3)\.

For each pre\-trained model and eachnshotn\_\{\\mathrm\{shot\}\}configuration, we compute three evaluation metrics: the proposed EGiME score, cross\-validation classification accuracy with KNN \(CV\(KNN\)\), and cross\-validation classification accuracy with SVM \(CV\(SVM\)\)\.

To investigate the stability of model evaluation approaches, we employ a quantitative metric known as ranking error \(R​ERE\)\. LetR=\{ri​j\}qR=\\\{r\_\{ij\}\\\}\_\{q\}be the ranking matrix whose diagonal elements are00\. Its elementri​j=1r\_\{ij\}=1when theiith model is superior to thejjth one, and otherwiseri​j=−1r\_\{ij\}=\-1\. LetRgR\_\{g\}be the ground\-truth ranking matrix andReR\_\{e\}be the one obtained by an evaluation method\. The ranking error is then defined asR​E=‖Re−Rg‖0/2RE=\\\|R\_\{e\}\-R\_\{g\}\\\|\_\{0\}/2, where∥⋅∥0\\\|\\cdot\\\|\_\{0\}denotes the 0\-norm, i\.e\., the number of nonzero entries of the difference matrix divided by two\.

### 4\.2Agreement with the Ground\-Truth Accuracy

Figure 3:EGiME as a low\-label proxy for the three\-predictor ground\-truth accuracy\. The overall regression score ofR2=0\.966R^\{2\}=0\.966indicates that EGiME preserves the accuracy trend and can serve as a practical pre\-screening alternative to fully supervised model selection\.The practical role of Fig\.[3](https://arxiv.org/html/2610.27029#S4.F3)is to test whether EGiME can act as a low\-label proxy for the fully supervised reference rather than merely to report a statistical coefficient\. In a model\-selection workflow, EGiME can first be computed with limited labels, weak backbones can be removed before expensive full\-label evaluation, and only the remaining models need the three\-predictor reference\. The fitted trend in Fig\.[3](https://arxiv.org/html/2610.27029#S4.F3)reachesR2=0\.966R^\{2\}=0\.966, indicating strong agreement between the two evaluation procedures\. Thus, on this benchmark, EGiME preserves both the accuracy trend and the relative ordering of models while requiring labels for only a small subset, making it a practical alternative to conventional supervised evaluation when full annotation is unavailable\.

The plot also separates models according to representation quality: ViT\-B/16 and ResNet34 consistently occupy the upper\-right region associated with high EGiME scores and high accuracy, whereas AlexNet and VGG13 occupy the lower\-left region\. This separation is useful in practice because a high EGiME score is associated with stronger class compactness and clearer global separation, allowing weak backbones to be filtered out before full annotation\. It also provides a geometric explanation for the ranking behavior examined below: the score is aligned with the fully labeled criterion while remaining computable from limited labels\.

### 4\.3Experimental Analysis of Model Ranking Stability in Few\-Shot Scenarios

To rigorously evaluate the model ranking stability and sample efficiency of the proposed weighted entropy method in few\-shot scenarios, Table[3](https://arxiv.org/html/2610.27029#S4.T3)first presents the ground\-truth rankings of pre\-trained models on five benchmark datasets, serving as the absolute reference anchor for all subsequent comparative experiments\.

Table 3:Ground\-Truth Rankings of Pre\-trained Models on Five Benchmark Datasets, Obtained by Averaging the Accuracies of K\-means, KNN, and SVMIn the specific evaluation process, for the five selected datasets \(Mini\-val, STL\-10, CIFAR\-10, EuroSAT, and CIFAR\-100\), we randomly sample 100 images per class to construct the base evaluation pool\. To thoroughly eliminate random sampling bias and ensure the statistical significance of the results, for the proposed weighted entropy method, the two classical supervised baselines \(CV\(KNN\) and CV\(SVM\)\), and three recent transferability estimation metrics \(LEEP, LogME, and PACTran\), we independently repeat the model performance ranking experiments 100 times under different labeled sample proportions \(i\.e\., extracting a small number of samplesnshotn\_\{\\text\{shot\}\}per class\)\. The results of all cycles are aggregated to obtain the corresponding evaluation score of each model under each labeling proportion\. It is worth noting that in each independent sampling, apart from the extremely small number ofnshotn\_\{\\text\{shot\}\}labeled samples used for evaluation guidance or baseline classifier training, the remaining massive amount of unlabeled samples in the experimental pool \(accounting for 80%∼\\sim96%\) are completely retained and utilized to construct the global clustering manifold topological basis, thereby providing solid unsupervised statistical support for the calculation of weighted entropy\.

For the three transferability metrics, each is adopted in its original formulation: LEEP\[[26](https://arxiv.org/html/2610.27029#bib.bib19)\]is computed with a real ImageNet\-1000 source classifierθ\\theta, LogME\[[41](https://arxiv.org/html/2610.27029#bib.bib29)\]with the evidence of a Bayesian linear probe maximized over\(α,β\)\(\\alpha,\\beta\)by a fixed\-point iteration, and PACTran\[[11](https://arxiv.org/html/2610.27029#bib.bib41)\]with a PAC\-Bayes bound instantiated under a Gaussian prior\. Where the definition of a metric requires a labeled target set, that set is taken to be theKK\-shot labeled subset\.

It should be stressed that these three metrics are recent*general\-purpose*transferability estimators: they were proposed and validated on a fully labeled target set, none of them was designed for the few\-shot regime, and their behavior when only a handful of labeled samples per class is available has not been examined before\. The experiment reported below is therefore, to our knowledge, the first to place them under aKK\-shot budget\.

In the quantitative phase of the model ranking task, we adopt “Average Ranking Mistakes” as the core evaluation metric\. The specific operation is as follows: in each independent run, the rankings derived from the weighted entropy evaluation, the predicted accuracy\-based rankings of CV\(KNN\) and CV\(SVM\), and the score\-based rankings of LEEP, LogME, and PACTran are individually compared with the ground\-truth rankings in Table[3](https://arxiv.org/html/2610.27029#S4.T3)\. We precisely count the number of model pairs that exhibit inversions \(inconsistencies\) between the evaluated ranking and the ground\-truth ranking, and finally average the number of mistakes over the 100 independent experiments\. Table[4](https://arxiv.org/html/2610.27029#S4.T4)systematically presents the average ranking mistakes of all six methods on the five datasets under different labeling proportions\.

Table 4:Average Ranking Mistakes on Five Datasets Under Different Labeling ProportionsTable[4](https://arxiv.org/html/2610.27029#S4.T4)shows that, compared with CV\(KNN\) and CV\(SVM\), EGiME yields lower average ranking mistakes on all five datasets and at every labeling proportion considered\. This indicates that, in the tested few\-shot settings, using a small number of labeled samples as anchors while exploiting the unlabeled pool through clustering provides more stable model rankings than the two cross\-validation baselines\. Fig\.[4](https://arxiv.org/html/2610.27029#S4.F4)further visualizes the average ranking mistakes of all six methods on the five datasets\.

Beyond the two supervised baselines, three recent transferability estimation metrics, LEEP\[[26](https://arxiv.org/html/2610.27029#bib.bib19)\], LogME\[[41](https://arxiv.org/html/2610.27029#bib.bib29)\], and PACTran\[[11](https://arxiv.org/html/2610.27029#bib.bib41)\], are included as supplementary comparisons\. These metrics assign each model a single transferability score and rank the models by that score\. As shown in Table[4](https://arxiv.org/html/2610.27029#S4.T4), LEEP and PACTran generally produce higher average ranking mistakes, whereas LogME is more competitive but does not consistently outperform EGiME; in particular, LogME is lower than EGiME on STL\-10 at 4 and 6 shots\. These results should not be interpreted as evidence that the three metrics are generally ineffective, because they were originally designed and evaluated with fully labeled target sets\. Their comparison here is therefore limited to ranking performance under the few\-shot budget\.

\(A\) Mini\-val

\(B\) STL\-10

\(C\) CIFAR\-10

\(D\) EuroSAT

\(E\) CIFAR\-100

Figure 4:Average ranking mistakes of the six methods as a function of the number of labeled samples per class, drawn as waterfall surfaces; the vertical axis is logarithmic, since LEEP and PACTran exceed the remaining methods by up to an order of magnitude\. Within each panel the methods run from front \(EGiME\) to back \(LEEP\)\.Across the 100 repetitions, CV\(KNN\) and CV\(SVM\) show larger run\-to\-run fluctuations, whereas EGiME generally has a lower and more concentrated error distribution\. These results suggest that EGiME is less sensitive to random few\-shot sampling than the two cross\-validation baselines\.

Across the tested labeling proportions, EGiME remains below both cross\-validation baselines in most settings\. The gap is generally larger in the low\-label regime, while the curves approach one another on some datasets as the number of labels increases\. This trend is not monotonic on every dataset, so it is interpreted as a descriptive pattern rather than evidence of asymptotic convergence\.

To examine sample efficiency, we analyze CIFAR\-100, which has high feature dimensionality and fine\-grained categories\. From 4 to 20 shots, the average ranking mistake of EGiME decreases overall from 0\.95 to 0\.34, with small local increases at intermediate budgets\. CV\(KNN\) changes only modestly from 1\.66 to 1\.21, while CV\(SVM\) increases from 1\.60 to 3\.91\. These results suggest that EGiME benefits from additional labeled anchors, although the improvement is not strictly monotonic\.

These results are consistent with the known difficulty of purely supervised classifiers in few\-shot regimes\[[29](https://arxiv.org/html/2610.27029#bib.bib31),[39](https://arxiv.org/html/2610.27029#bib.bib23)\]\. With very few training samples, SVM decision boundaries can be sensitive to sampling noise\[[7](https://arxiv.org/html/2610.27029#bib.bib21)\]\. EGiME instead uses the unlabeled pool to regularize the score through the global cluster structure\. On CIFAR\-10, STL\-10, and Mini\-val, the errors of most methods tend to decrease as more labels are used, while EGiME remains lower than the two cross\-validation baselines in the tested settings\.

To test generalization under domain shift, we evaluate EuroSAT\. Because the pre\-trained models originate from natural images, their feature spaces are less aligned with remote\-sensing images, and ranking mistakes are generally higher than on the natural\-image datasets\. The EGiME error ranges from 4\.45 to 5\.86, compared with 6\.06 to 7\.54 for CV\(KNN\) and 8\.12 to 10\.70 for CV\(SVM\)\.

Across all nine labeling proportions on EuroSAT, EGiME has the lowest average ranking mistake among the six methods\. This result supports its usefulness under the evaluated domain shift, although the experiment covers one target domain and does not by itself establish general cross\-domain robustness\.

The ranking comparison shows whether a method orders the candidate models correctly; it does not show whether the score assigned to an individual model is stable across labeling conditions\. Fig\.[5](https://arxiv.org/html/2610.27029#S4.F5)and Fig\.[6](https://arxiv.org/html/2610.27029#S4.F6)present the per\-model score distributions of EGiME, CV\(KNN\), and CV\(SVM\) on the five datasets\. The three transferability metrics are not included because their scores are defined on different scales\. The distributions show that EGiME scores are generally more concentrated across labeling proportions than those of the two baselines in the examined settings\. These figures are therefore used as descriptive evidence of score stability\.

![Refer to caption](https://arxiv.org/html/2610.27029v1/mini_val_egi.png)

![Refer to caption](https://arxiv.org/html/2610.27029v1/mini_val_cvknn.png)

![Refer to caption](https://arxiv.org/html/2610.27029v1/mini_val_cvsvm.png)

\(A\) Mini\-ImageNet ![Refer to caption](https://arxiv.org/html/2610.27029v1/stl10_egi.png)![Refer to caption](https://arxiv.org/html/2610.27029v1/stl10_cvknn.png)![Refer to caption](https://arxiv.org/html/2610.27029v1/stl10_cvsvm.png) \(B\) STL\-10 ![Refer to caption](https://arxiv.org/html/2610.27029v1/cifar10_egi.png)![Refer to caption](https://arxiv.org/html/2610.27029v1/cifar10_cvknn.png)![Refer to caption](https://arxiv.org/html/2610.27029v1/cifar10_cvsvm.png) \(C\) CIFAR\-10

Figure 5:Score distributions across different labeled sample sizes\. \(A\) Mini\-ImageNet, \(B\) STL\-10, \(C\) CIFAR\-10\. Left: EGiME, Middle: CV\(KNN\), Right: CV\(SVM\)\.![Refer to caption](https://arxiv.org/html/2610.27029v1/eurosat_egi.png)

![Refer to caption](https://arxiv.org/html/2610.27029v1/eurosat_cvknn.png)

![Refer to caption](https://arxiv.org/html/2610.27029v1/eurosat_cvsvm.png)

\(D\) EuroSAT ![Refer to caption](https://arxiv.org/html/2610.27029v1/cifar100_25classes_egi.png)![Refer to caption](https://arxiv.org/html/2610.27029v1/cifar100_25classes_cvknn.png)![Refer to caption](https://arxiv.org/html/2610.27029v1/cifar100_25classes_cvsvm.png) \(E\) CIFAR\-100

Figure 6:Score distributions across different labeled sample sizes \(continued\)\. \(D\) EuroSAT, \(E\) CIFAR\-100\.
### 4\.4Ablation Experiment

To verify the critical role of each component in the EGiME framework, we design two ablation experiments: \(A\) replacing the K\-means clustering with equal partition to assess the contribution of the structural prior, and \(B\) decomposing the weighted entropy formula to assess the respective contributions of the inter\-cluster and intra\-cluster entropy terms\.

A\. Clustering\-based partitioning strategy\.We replace the K\-means clustering module in the EGiME framework with an “Equal Partition” strategy as the baseline\. The equal partition approach disregards the geometric structure and semantic distribution of the feature space, instead uniformly dividing samples into equally sized groups based on a fixed rule\. Since the subsequent entropy\-based evaluation formula remains identical under both strategies, this ablation experiment effectively isolates and quantifies the actual contribution of the structural prior\.

Table 5:Ablation Study \(A\): Ranking Errors of K\-means vs\. Equal PartitionTable[5](https://arxiv.org/html/2610.27029#S4.T5)shows that K\-means consistently yields lower average ranking errors than equal partition across all five datasets and labeling budgets\. For example, on CIFAR\-100, K\-means remains below 1\.0 while equal partition ranges from 13\.0 to 29\.0; on Mini\-val, equal partition remains at 35\.97, whereas K\-means reaches at most 2\.87\. As the number of labeled samples increases from 4 to 20, K\-means generally decreases \(e\.g\., CIFAR\-10 from 3\.66 to 2\.15 and Mini\-val from 2\.87 to 1\.38\), although the trend is not strictly monotonic; on STL\-10, for instance, it rises from 1\.11 to 1\.18 at the final budget\. These results support the importance of geometry\-aware partitioning for EGiME\.

B\. Entropy\.To further investigate the respective contributions of the two entropy terms, we decompose the EGiME formulationEr=Er\(inter\)/\(1\+Er\(intra\)\)E\_\{r\}=E\_\{r\}^\{\\mathrm\{\(inter\)\}\}/\(1\+E\_\{r\}^\{\\mathrm\{\(intra\)\}\}\)into two ablated variants\. Variant A \(inter\-cluster entropy only\) discards the intra\-cluster term by setting the denominator to 1, i\.e\.,ErA=Er\(inter\)E\_\{r\}^\{\\mathrm\{A\}\}=E\_\{r\}^\{\\mathrm\{\(inter\)\}\}, evaluating models purely by the uniformity of sample dispersion across clusters\. Variant B \(intra\-cluster entropy only\) discards the inter\-cluster term by setting the numerator to 1, i\.e\.,ErB=1/\(1\+Er\(intra\)\)E\_\{r\}^\{\\mathrm\{B\}\}=1/\(1\+E\_\{r\}^\{\\mathrm\{\(intra\)\}\}\), evaluating models purely by the concentration of labeled samples within clusters\. We compute the predicted ranking errorsR​EvariantRE\_\{\\mathrm\{variant\}\}across all five datasets byErAE\_\{r\}^\{\\mathrm\{A\}\}andErBE\_\{r\}^\{\\mathrm\{B\}\}, and compare them with the ranking errorsR​EFullRE\_\{\\mathrm\{Full\}\}obtained by the full EGiME byΔ​R​E=R​Evariant−R​EFull\\Delta RE=RE\_\{\\mathrm\{variant\}\}\-RE\_\{\\mathrm\{Full\}\}, as shown in Table[6](https://arxiv.org/html/2610.27029#S4.T6), where smallerΔ​R​E\\Delta REindicates better prediction\.

Table 6:The contribution ofErAE\_\{r\}^\{\\mathrm\{A\}\}andErBE\_\{r\}^\{\\mathrm\{B\}\}\. Increase in Ranking Error \(Δ​R​E=R​Evariant−R​EFull\\Delta RE=RE\_\{\\mathrm\{variant\}\}\-RE\_\{\\mathrm\{Full\}\}\) after Removing Each Entropy Component\. SmallerΔ​R​E\\Delta REindicates the removed component is less critical\.Table[6](https://arxiv.org/html/2610.27029#S4.T6)quantifies the increase in ranking error after removing each entropy component\. The inter\-cluster term is more influential on three of the five datasets, with the largest effect on EuroSAT \(Δ​R​E=\+10\.11\\Delta RE=\+10\.11\)\. CIFAR\-10 is an exception: removing the intra\-cluster term causes a larger increase \(\+3\.89\+3\.89\) than removing the inter\-cluster term \(\+3\.00\+3\.00\)\. On STL\-10, the two penalties are equal \(\+1\.33\+1\.33\)\. These results indicate that both entropy terms contribute, with a dataset\-dependent balance, rather than that one term dominates universally\.

## 5Conclusion

This paper addressed the challenge of evaluating pre\-trained neural networks in few\-shot transfer learning scenarios, where traditional supervised evaluation methods suffer from severe variance due to scarce labeled data\. We proposed EGiME, a semi\-supervised evaluation framework based on information\-theoretic criteria that leverages both limited labeled samples and abundant unlabeled data\. By reformulating model evaluation as the characterization of clustering structure in feature space, EGiME combines inter\-cluster entropy, which measures global class separability, with inverse\-frequency and intra\-cluster entropy, which capture local compactness of labeled anchors\. This design yields a training\-free, parameter\-light, and computationally efficient metric that scales favorably to large model zoos without requiring source\-domain data or auxiliary classifier training\.

Experiments on five few\-shot benchmarks demonstrated that the weighted entropy score correlates strongly with the ground\-truth accuracy, obtained by first computing the 10\-fold cross\-validation accuracy of K\-means, KNN, and SVM separately and then averaging the three accuracies with equal weights\. EGiME substantially outperforms cross\-validation competitors in ranking stability, particularly below 10% labeling\. Ablations confirmed that the adaptive K\-means partition is critical, whereas naive equal partitioning severely degrades ranking quality, establishing EGiME as a practical tool for pre\-trained model selection in label\-scarce regimes\.

Several limitations motivate future work: EGiME assumes a known number of target classes and that each class forms a single compact cluster, and the entropy formulation implicitly assumes balanced class distributions\. K\-means may also perform poorly in very high\-dimensional spaces\. Addressing these issues, extending the framework to multimodal settings, and investigating its theoretical connections to generalization bounds constitute promising directions\.

## Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 12671626, 12531019, the National Key R & D Program of China under Grant 2021YFA1003004, and the special foundation for Guangxi Ba Gui Scholars\.

## Data availability

The benchmark datasets used in this study \(Mini\-val, STL\-10, CIFAR\-10, EuroSAT, and CIFAR\-100\) are publicly available\. The evaluation code and the per\-run ranking results generated during this study are available from the corresponding author on reasonable request\.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper\.

## Author Biographies

Dequan Jinreceived the B\.Sc\., M\.Sc\., and Ph\.D\. degrees in mathematics from Xi’an Jiaotong University in 2005, 2008, and 2011\. He is currently a Professor with the School of Mathematics and Information Science, Guangxi University\. His research interests include mathematical theory, cognition learning, and data analysis\.

Tao Liis a graduate student at the School of Mathematics and Center for Applied Mathematics, Guangxi University, Nanning, China\. His research focuses on model evaluation in small\-sample scenarios\.

Lingfeng Xingis a graduate student at the School of Mathematics and Center for Applied Mathematics, Guangxi University, Nanning, China\. His research interests lie in 3D Gaussian Splatting and sparse\-view X\-ray computed tomography reconstruction\.

Shihui Yingreceived the B\.Eng\. degree in mechanical engineering and the Ph\.D\. degree in applied mathematics from Xi’an Jiaotong University in 2001 and 2008\. He is currently a Professor at Shanghai University\. His research interests include machine learning and medical image analysis\.

Hui Zhaoreceived the Ph\.D\. degree in finance from Guangxi University in 2017\. She was a visiting scholar at the University of Hong Kong and Fordham University\. She is currently a Professor with the School of Economics, Guangxi University\. Her research interests include data elements, data governance, and China\-ASEAN economic cooperation\.

## References

- \[1\]\(2021\)Beyond linear subspace clustering: a comparative study of nonlinear manifold clustering algorithms\.Computer Science Review42,pp\. 100435\.Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p1.1)\.
- \[2\]A\. Agostinelliet al\.\(2022\)How stable are transferability metrics evaluations?\.InECCV,Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p2.1)\.
- \[3\]M\. Ay, L\. Özbakır, S\. Kulluk, B\. Gülmez, G\. Öztürk, and S\. Özer\(2023\)FC\-Kmeans: fixed\-centered k\-means algorithm\.Expert Systems with Applications211,pp\. 118656\.Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p2.1)\.
- \[4\]Y\. Baoet al\.\(2019\)An information\-theoretic approach to transferability in task transfer learning\.InICIP,Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p2.1)\.
- \[5\]M\. Caronet al\.\(2018\)Deep clustering for unsupervised learning of visual features\.InECCV,Cited by:[§2\.2](https://arxiv.org/html/2610.27029#S2.SS2.p3.1)\.
- \[6\]M\. Caronet al\.\(2020\)Unsupervised learning of visual features by contrasting cluster assignments\.InNeurIPS,Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p1.1)\.
- \[7\]J\. Cervantes, F\. Garcia\-Lamont, L\. Rodríguez\-Mazahua, and A\. Lopez\(2020\)A comprehensive survey on support vector machine classification: applications, challenges and trends\.Neurocomputing408,pp\. 189–215\.Cited by:[§4\.3](https://arxiv.org/html/2610.27029#S4.SS3.p11.1)\.
- \[8\]O\. Chapelle, B\. Schölkopf, and A\. Zien\(2006\)Semi\-supervised learning\.MIT Press\.Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p1.1)\.
- \[9\]T\. Chenet al\.\(2020\)A simple framework for contrastive learning of visual representations\.InICML,Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p1.1)\.
- \[10\]T\. M\. Cover and J\. A\. Thomas\(2006\)Elements of information theory\.2nd edition,Wiley\.Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p3.1)\.
- \[11\]N\. Ding, X\. Chen, T\. Levinboim, S\. Changpinyo, and R\. Soricut\(2022\)PACTran: pac\-bayesian metrics for estimating the transferability of pretrained models to classification tasks\.InComputer Vision – ECCV 2022,Lecture Notes in Computer Science, Vol\.13694,pp\. 252–268\.External Links:[Document](https://dx.doi.org/10.1007/978-3-031-19830-4%5F15)Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p2.1),[§4\.3](https://arxiv.org/html/2610.27029#S4.SS3.p3.1),[§4\.3](https://arxiv.org/html/2610.27029#S4.SS3.p7.1)\.
- \[12\]A\. Dosovitskiyet al\.\(2021\)An image is worth 16x16 words: transformers for image recognition at scale\.InICLR,Cited by:[§1](https://arxiv.org/html/2610.27029#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p1.1)\.
- \[13\]C\. Finn, P\. Abbeel, and S\. Levine\(2017\)Model\-agnostic meta\-learning for fast adaptation of deep networks\.InICML,Cited by:[§2\.2](https://arxiv.org/html/2610.27029#S2.SS2.p1.1)\.
- \[14\]M\. Gholamiet al\.\(2023\)ETran: energy\-based transferability estimation\.InICCV,Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p2.1)\.
- \[15\]E\. Hancer, B\. Xue, and M\. Zhang\(2020\)A survey on feature selection approaches for clustering\.Artificial Intelligence Review\.Cited by:[§2\.2](https://arxiv.org/html/2610.27029#S2.SS2.p1.1)\.
- \[16\]K\. He, R\. Girshick, and P\. Dollár\(2019\)Rethinking ImageNet pre\-training\.InICCV,Cited by:[§1](https://arxiv.org/html/2610.27029#S1.p1.1)\.
- \[17\]K\. Heet al\.\(2020\)Momentum contrast for unsupervised visual representation learning\.InCVPR,Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p1.1)\.
- \[18\]K\. Heet al\.\(2022\)Masked autoencoders are scalable vision learners\.InCVPR,Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p1.1)\.
- \[19\]K\. He, X\. Zhang, S\. Ren, and J\. Sun\(2016\)Deep residual learning for image recognition\.InCVPR,Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p1.1)\.
- \[20\]S\. Ibrahim, N\. Ponomareva, and R\. Mazumder\(2023\)Newer is not always better: rethinking transferability metrics, their peculiarities, stability and performance\.InMachine Learning and Knowledge Discovery in Databases \(ECML PKDD 2023\),Lecture Notes in Computer Science,pp\. 693–709\.Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p2.1)\.
- \[21\]D\. Jinet al\.\(2021\)A weighting method for feature dimension by semisupervised learning with entropy\.IEEE Trans\. Neural Networks and Learning Systems\.Cited by:[§2\.2](https://arxiv.org/html/2610.27029#S2.SS2.p3.1)\.
- \[22\]P\. Khoslaet al\.\(2020\)Supervised contrastive learning\.InNeurIPS,Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p1.1)\.
- \[23\]R\. Kohavi\(1995\)A study of cross\-validation and bootstrap for accuracy estimation and model selection\.InProc\. IJCAI,Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p3.1)\.
- \[24\]A\. Krizhevsky, I\. Sutskever, and G\. E\. Hinton\(2012\)ImageNet classification with deep convolutional neural networks\.InNeurIPS,Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p1.1)\.
- \[25\]J\. MacQueen\(1967\)Some methods for classification and analysis of multivariate observations\.InProc\. Berkeley Symposium on Mathematical Statistics and Probability,Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p1.1),[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p2.1)\.
- \[26\]C\. Nguyen, T\. Hassner, M\. Seeger, and C\. Archambeau\(2020\)LEEP: a new measure to evaluate transferability of learned representations\.InProceedings of the 37th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.119,pp\. 7294–7305\.Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p2.1),[§4\.3](https://arxiv.org/html/2610.27029#S4.SS3.p3.1),[§4\.3](https://arxiv.org/html/2610.27029#S4.SS3.p7.1)\.
- \[27\]M\. Oquabet al\.\(2024\)DINOv2: learning robust visual features without supervision\.Transactions on Machine Learning Research\.Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p1.1)\.
- \[28\]V\. Papyan, X\. Y\. Han, and D\. L\. Donoho\(2020\)Prevalence of neural collapse during the terminal phase of deep learning training\.PNAS\.Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p2.1)\.
- \[29\]E\. Perez, D\. Kiela, and K\. Cho\(2021\)True few\-shot learning with language models\.InNeurIPS,Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p4.1),[§4\.3](https://arxiv.org/html/2610.27029#S4.SS3.p11.1)\.
- \[30\]A\. Radfordet al\.\(2021\)Learning transferable visual models from natural language supervision\.InICML,Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p1.1)\.
- \[31\]C\. E\. Shannon\(1948\)A mathematical theory of communication\.Bell System Technical Journal\.Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p3.1)\.
- \[32\]K\. Simonyan and A\. Zisserman\(2015\)Very deep convolutional networks for large\-scale image recognition\.InICLR,Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p1.1)\.
- \[33\]J\. Snell, K\. Swersky, and R\. Zemel\(2017\)Prototypical networks for few\-shot learning\.InNeurIPS,Cited by:[§2\.2](https://arxiv.org/html/2610.27029#S2.SS2.p1.1)\.
- \[34\]C\. Szegedyet al\.\(2015\)Going deeper with convolutions\.InCVPR,Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p1.1)\.
- \[35\]Y\. Tianet al\.\(2020\)Rethinking few\-shot image classification: a good embedding is all you need?\.InECCV,Cited by:[§2\.3](https://arxiv.org/html/2610.27029#S2.SS3.p1.1)\.
- \[36\]A\. Vabalaset al\.\(2019\)Machine learning algorithm validation with a limited sample size\.PLOS ONE\.Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p4.1)\.
- \[37\]L\. van der Maaten and G\. Hinton\(2008\)Visualizing data using t\-SNE\.Journal of Machine Learning Research\.Cited by:[§3\.1](https://arxiv.org/html/2610.27029#S3.SS1.p2.1)\.
- \[38\]O\. Vinyalset al\.\(2016\)Matching networks for one shot learning\.InNeurIPS,Cited by:[§2\.2](https://arxiv.org/html/2610.27029#S2.SS2.p1.1)\.
- \[39\]Y\. Wanget al\.\(2020\)Generalizing from a few examples: a survey on few\-shot learning\.ACM Computing Surveys\.Cited by:[§2\.2](https://arxiv.org/html/2610.27029#S2.SS2.p1.1),[§4\.3](https://arxiv.org/html/2610.27029#S4.SS3.p11.1)\.
- \[40\]Z\. Wang, Y\. Luo, L\. Zheng, Z\. Huang, and M\. Baktashmotlagh\(2023\)How far pre\-trained models are from neural collapse on the target dataset informs their transferability\.InICCV,pp\. 5526–5535\.Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p2.1)\.
- \[41\]K\. You, Y\. Liu, J\. Wang, and M\. Long\(2021\)LogME: practical assessment of pre\-trained models for transfer learning\.InProceedings of the 38th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.139,pp\. 12133–12143\.Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p2.1),[§4\.3](https://arxiv.org/html/2610.27029#S4.SS3.p3.1),[§4\.3](https://arxiv.org/html/2610.27029#S4.SS3.p7.1)\.
- \[42\]A\. R\. Zamiret al\.\(2018\)Taskonomy: disentangling task transfer learning\.InCVPR,Cited by:[§2\.1](https://arxiv.org/html/2610.27029#S2.SS1.p1.1)\.