Searching with Sparse Histories: Cross-Task Architecture Search from Incomplete Search Logs and Zero-Cost Proxies

# Cross-Task Architecture Searchfrom Incomplete Search Logs and Zero-Cost Proxies
Source: [https://arxiv.org/html/2610.32179](https://arxiv.org/html/2610.32179)
## Searching with Sparse Histories: Cross\-Task Architecture Search from Incomplete Search Logs and Zero\-Cost Proxies

Yuefeng XuAffiliation:Graduate School of Science and TechnologyAffiliation:Niigata University, Niigata, JapanEmail:[f25l005c@mail\.cc\.niigata\-u\.ac\.jp](mailto:)Jun Yu††thanks:Corresponding author\.Affiliation:Institute of Science and TechnologyAffiliation:Niigata University, Niigata, JapanEmail:[yujun@ie\.niigata\-u\.ac\.jp](mailto:)

###### Abstract

Neural architecture search \(NAS\) on a new task is expensive because every candidate must be trained to be evaluated\. Cross\-task NAS reduces this cost by reusing knowledge from previously searched tasks, but existing methods assume either dense performance tables on the source tasks, a single source task, or no source evaluations at all\. In practice, a team that has searched before holds*sparse search histories*: several past tasks, each with a small and different subset of evaluated architectures\. We study cross\-task NAS from such incomplete histories\. We first show that standard transfer heuristics break down in this regime, and that the two cheap sources of prior knowledge available to a practitioner—sparse historical logs and zero\-cost proxies—fail on*different*targets: proxies on dense\-prediction tasks, sparse logs on tasks that are negatively correlated with the sources\. We then propose SHINE, a risk\-robust transfer strategy that \(i\) starts the search from a portfolio covering the champions of each imputed source task and \(ii\) pretrains a pairwise ranker jointly on the sparse source tasks and on zero\-cost proxies treated as pseudo\-tasks, before adapting it to the target inside a weak\-predictor search\. SHINE is not uniformly better on average than a strong proxy\-only ranker; its value lies in substantially mitigating the severe failures of either knowledge source while retaining competitive average performance\. On TransNAS\-Bench\-101 with 50 target evaluations and 50 evaluated architectures per source, it mitigates the worst failure of each source \(Micro autoencoding: worst\-10% rank 290 with proxies alone*vs*\.43; Macro room layout: 465 with logs alone*vs*\.45\), and matches published methods in typical performance \(rank of mean 2\.00 on Macro*vs*\.2\.25 for TRP and, on the four metric\-compatible targets, better than Arch\-Graph on both spaces\)\. Over 13 cell\-space targets, including seven trained from scratch on new datasets and four targets run only after the analysis plan was fixed, SHINE improves over the proxy\-only ranker on 10 of 13 targets in both mean and worst\-case performance \(task\-levelp≤0\.016p\{\\leq\}0\.016; 4 of 4 on the new targets\) and is on par with source logs alone, which, however, fail severely on the macro space\.

## 1Introduction

Neural architecture search \(NAS\) has produced competitive networks for classification, dense prediction and many other vision problems\[[33](https://arxiv.org/html/2610.32179#bib.bib17),[21](https://arxiv.org/html/2610.32179#bib.bib15),[18](https://arxiv.org/html/2610.32179#bib.bib16),[7](https://arxiv.org/html/2610.32179#bib.bib14)\]\. Its central obstacle remains cost: the quality of an architecture is only known after it has been trained, so each search spends most of its budget on evaluations\. The cost is paid again for every new task, even when the search space and the training protocol are unchanged, because the ranking of architectures shifts from task to task\[[6](https://arxiv.org/html/2610.32179#bib.bib1),[31](https://arxiv.org/html/2610.32179#bib.bib29)\]\.

*Cross\-task*NAS addresses this by transferring knowledge from previously searched tasks\[[5](https://arxiv.org/html/2610.32179#bib.bib4),[13](https://arxiv.org/html/2610.32179#bib.bib5),[11](https://arxiv.org/html/2610.32179#bib.bib2),[15](https://arxiv.org/html/2610.32179#bib.bib3)\]\. Existing approaches differ in the kind of source knowledge they assume\. Benchmark\-driven analyses and direct\-transfer baselines assume*dense*source tables,*i\.e*\.every architecture has been evaluated on every source task\[[6](https://arxiv.org/html/2610.32179#bib.bib1)\]\. Arch\-Graph\[[11](https://arxiv.org/html/2610.32179#bib.bib2)\]pretrains a relation predictor on a*single*source task\. TRP\[[15](https://arxiv.org/html/2610.32179#bib.bib3)\]avoids source evaluations altogether and pretrains on*zero\-cost proxies*\[[1](https://arxiv.org/html/2610.32179#bib.bib19),[19](https://arxiv.org/html/2610.32179#bib.bib20)\]\. None of these matches the situation of a practitioner who has searched before: such a team owns*sparse search histories*—logs from several past tasks, each covering only the few dozen to few hundred architectures that happened to be trained on it, with little overlap between tasks\.

This paper asks how to best use such incomplete histories when searching on a new task under a small evaluation budget\. We find that the question is not answered by existing heuristics\. Aggregating source rankings into a consensus prior, a common default, performs poorly because high\-performing architectures are task specific\. Covering the champions of each source task is a far stronger strategy when source tables are dense, but its advantage collapses when each source is evaluated on onlym=50m\{=\}50architectures \([Sec\.3\.1](https://arxiv.org/html/2610.32179#S3.SS1)\)\. Zero\-cost proxies are a natural complement, as they are available for every architecture at negligible cost, yet we show that they are unreliable precisely on the dense\-prediction tasks where transfer matters most \([Sec\.5\.3](https://arxiv.org/html/2610.32179#S5.SS3)\)\.

We propose SHINE \(Sparse\-HistoryINformed sEarch\), a simple method that unifies the two cheap sources of knowledge\. SHINE treats every sparse source task and every zero\-cost proxy as a ranking task over the same architecture space and pretrains a pairwise ranker on all of them jointly\. The search starts from a portfolio that covers the champions of each source task after imputation, and it adapts the ranker on the target inside a weak\-predictor search loop that ends with an exploitation phase\. SHINE requires neither dense source tables nor task embeddings, and its overhead is negligible compared with a single architecture evaluation\. We position SHINE as a*risk\-robust*transfer strategy rather than a uniformly superior one: in typical runs it is on par with the better single knowledge source, and its benefit is that it substantially mitigates the severe failures that each source suffers on some target\.

Our contributions are:

- •We formulate cross\-task NAS from*sparse search histories*, a setting that reflects how search knowledge accumulates in practice, and show that consensus priors and source\-portfolio heuristics fail in it\.
- •We propose SHINE, which combines a source\-champion portfolio with multi\-task pairwise ranker pretraining over sparse source tasks and zero\-cost pseudo\-tasks\.
- •We show that historical logs and zero\-cost proxies fail on different targets, and that combining them mitigates the severe failures of either, reduces tail risk relative to proxies, while typical\-case performance stays on par with the better single source\.
- •On TransNAS\-Bench\-101\[[6](https://arxiv.org/html/2610.32179#bib.bib1)\], SHINE matches published methods in typical performance \(better than Arch\-Graph\[[11](https://arxiv.org/html/2610.32179#bib.bib2)\]on the metric\-compatible targets, on par with TRP\[[15](https://arxiv.org/html/2610.32179#bib.bib3)\]on Macro\); in a pooled analysis over 13 targets, seven of them trained from scratch on new datasets, with the analysis plan fixed before the last four were run, SHINE significantly improves over the proxy\-only ranker at the task level in both mean and worst\-case performance; MedMNIST\-NAS\-Bench\[[23](https://arxiv.org/html/2610.32179#bib.bib32)\]further validates the historical\-log component\.

## 2Related Work

#### Predictor\-based NAS\.

Performance predictors reduce the number of trained architectures by ranking candidates before evaluation\[[24](https://arxiv.org/html/2610.32179#bib.bib11),[29](https://arxiv.org/html/2610.32179#bib.bib10),[26](https://arxiv.org/html/2610.32179#bib.bib13)\]\. BANANAS\[[24](https://arxiv.org/html/2610.32179#bib.bib11)\]showed that path\-based encodings substantially improve predictors on cell search spaces; weakNAS\[[29](https://arxiv.org/html/2610.32179#bib.bib10)\]iteratively fits weak predictors and progressively narrows the sampling region; and simple local search is a strong baseline on tabular benchmarks\[[25](https://arxiv.org/html/2610.32179#bib.bib12)\]\. Pairwise and listwise ranking losses\[[4](https://arxiv.org/html/2610.32179#bib.bib23)\]are preferred over regression because search only needs the order of architectures\. SHINE uses the same search template but focuses on what the predictor is pretrained on\.

#### Cross\-task and transferable NAS\.

TransNAS\-Bench\-101\[[6](https://arxiv.org/html/2610.32179#bib.bib1)\]provides seven vision tasks over two shared search spaces and showed that direct architecture transfer is often worse than random search\. CATCH\[[5](https://arxiv.org/html/2610.32179#bib.bib4)\]meta\-learns a controller with task context, MetaD2A\[[13](https://arxiv.org/html/2610.32179#bib.bib5)\]generates architectures from dataset embeddings, and EMT\-NAS\[[16](https://arxiv.org/html/2610.32179#bib.bib8)\]and KTNAS\[[32](https://arxiv.org/html/2610.32179#bib.bib9)\]exchange knowledge between tasks searched simultaneously\. Arch\-Graph\[[11](https://arxiv.org/html/2610.32179#bib.bib2)\]pretrains a pairwise relation predictor on one source task with Task2Vec embeddings\[[2](https://arxiv.org/html/2610.32179#bib.bib30)\]and fine\-tunes it on 30 target evaluations\. TRP\[[15](https://arxiv.org/html/2610.32179#bib.bib3)\]pretrains a relativistic predictor on zero\-cost proxy votes and fine\-tunes it on a few dozen target evaluations\. Multi\-Predict\[[3](https://arxiv.org/html/2610.32179#bib.bib7)\]and HELP\[[14](https://arxiv.org/html/2610.32179#bib.bib6)\]transfer predictors across tasks or devices with few samples\. Unlike these works, we assume several*sparsely*evaluated source tasks and show how to combine them with proxies\.

#### Zero\-cost proxies\.

Zero\-cost proxies score untrained networks from a single minibatch\[[1](https://arxiv.org/html/2610.32179#bib.bib19),[19](https://arxiv.org/html/2610.32179#bib.bib20),[22](https://arxiv.org/html/2610.32179#bib.bib21),[17](https://arxiv.org/html/2610.32179#bib.bib22)\]\. NAS\-Bench\-Suite\-Zero\[[12](https://arxiv.org/html/2610.32179#bib.bib18)\]found that their reliability varies strongly across search spaces and tasks\. Proxies have also been used as input features of single\-task predictors\[[27](https://arxiv.org/html/2610.32179#bib.bib33)\]\. We confirm their unreliability on dense\-prediction tasks and use them as auxiliary pretraining tasks alongside sparse source logs rather than as a ranking\.

#### Warm\-starting from previous tasks\.

In hyperparameter optimisation, configurations that performed well on previous datasets are used to initialise the search\[[10](https://arxiv.org/html/2610.32179#bib.bib24),[28](https://arxiv.org/html/2610.32179#bib.bib25)\], often as a greedy portfolio\[[20](https://arxiv.org/html/2610.32179#bib.bib26),[8](https://arxiv.org/html/2610.32179#bib.bib27)\], and source surrogates can be ensembled with ranking\-based weights\[[9](https://arxiv.org/html/2610.32179#bib.bib28)\]\. SHINE adopts the portfolio idea for initialisation and extends it to sparse sources by imputation\.

## 3Cross\-Task NAS from Sparse Search Histories

#### Setting\.

Let𝒜\\mathcal\{A\}be a finite search space ofNNarchitectures and letyj​\(a\)y\_\{j\}\(a\)denote the quality of architecturea∈𝒜a\\in\\mathcal\{A\}on taskjj, oriented so that larger is better\. A*search history*consists ofSSsource tasks\. For sourcesswe observe only a subset𝒪s⊂𝒜\\mathcal\{O\}\_\{s\}\\subset\\mathcal\{A\}with\|𝒪s\|=m≪N\|\\mathcal\{O\}\_\{s\}\|=m\\ll Nand the corresponding values\{ys​\(a\)\}a∈𝒪s\\\{y\_\{s\}\(a\)\\\}\_\{a\\in\\mathcal\{O\}\_\{s\}\}; the subsets of different sources are drawn independently\. In addition,KKzero\-cost proxiesz1,…,zKz\_\{1\},\\dots,z\_\{K\}can be computed for every architecture at negligible cost\. On a new target tasktt, an algorithm sequentially selects architectures to train; afterBBevaluations it returns the best evaluated architecture\. No information aboutyty\_\{t\}is available except through theseBBevaluations\.

#### Evaluation\.

Following Arch\-Graph\[[11](https://arxiv.org/html/2610.32179#bib.bib2)\], we report the global rank of the returned architecture within𝒜\\mathcal\{A\}\(1 is best\), averaged over target tasks and random seeds\. For comparability with published tables, which report the metric value of the returned architecture averaged over trials, we also report the global rank of that average value\. Each target is held out in turn and the remaining tasks act as sources \(leave\-one\-task\-out\)\.

### 3\.1Why naive transfer fails with sparse histories

Before introducing our method we examine two natural heuristics on TransNAS\-Bench\-101 \(setup in[Sec\.5](https://arxiv.org/html/2610.32179#S5)\)\.*Consensus*ranks architectures by their mean percentile rank over sources\.*Champion portfolio*greedily selects architectures to minimise the summed regret over source tasks\[[28](https://arxiv.org/html/2610.32179#bib.bib25),[20](https://arxiv.org/html/2610.32179#bib.bib26)\], which effectively visits the best architecture of each source first\. With dense source tables, the champion portfolio is remarkably strong: after 50 target evaluations its average rank is 11\.0 on Macro and 15\.2 on Micro, close to Arch\-Graph \(5\.2 and 12\.2\), whereas consensus reaches only 90\.5 and 44\.2\. Top architectures are thus task specific, and averaging hides them\. However, when each source reveals onlym=50m\{=\}50random architectures, the portfolio degrades to an average rank of 24\.3 \(Macro\) and 44\.3 \(Micro\), and a rule that selects architectures with high cross\-source disagreement collapses entirely \(122\.6 and 516\.2\)\. A method for sparse histories must therefore generalise beyond the few observed source architectures and must not rely on the sources alone\.

## 4Method

Figure 1:Overview of SHINE\.Sparse source logs \(each past task has onlymmevaluated architectures\) are imputed per source to build a champion portfolio𝒬0\\mathcal\{Q\}\_\{0\}that initialises the search\. A pairwise ranker is pretrained jointly on all sparse source tasks and on zero\-cost proxies treated as pseudo\-tasks; on the new target it is repeatedly fine\-tuned on the evaluated architectures and used to sample the next ones from its predicted top\-kkuntil the budgetBBis spent\.SHINE has three components \([Fig\.1](https://arxiv.org/html/2610.32179#S4.F1)\): a champion portfolio built from imputed source rankings \([Sec\.4\.1](https://arxiv.org/html/2610.32179#S4.SS1)\), a pairwise ranker pretrained on sparse sources and zero\-cost pseudo\-tasks \([Sec\.4\.2](https://arxiv.org/html/2610.32179#S4.SS2)\), and a target\-side search that adapts this ranker \([Sec\.4\.3](https://arxiv.org/html/2610.32179#S4.SS3)\)\.

### 4\.1Champion portfolio from imputed sources

Each architecture is described by a structural encodingxax\_\{a\}\([Sec\.5](https://arxiv.org/html/2610.32179#S5)\)\. For every sourcesswe fit a ridge regressor with pairwise feature interactions to the percentile ranks observed on𝒪s\\mathcal\{O\}\_\{s\}and predict a complete rankingr^s​\(a\)\\hat\{r\}\_\{s\}\(a\); observed architectures keep their true order\. The initial target queries𝒬0\\mathcal\{Q\}\_\{0\},\|𝒬0\|=K0\|\\mathcal\{Q\}\_\{0\}\|=K\_\{0\}, greedily minimise the summed normalised regret over sources,

𝒬0=arg⁡min⁡∑s=1S\|𝒬\|=K0⁡\[maxa∈𝒜⁡r^s​\(a\)−maxa∈𝒬⁡r^s​\(a\)\],\\mathcal\{Q\}\_\{0\}=\\arg\\min\_\{\|\\mathcal\{Q\}\|=K\_\{0\}\}\\sum\_\{s=1\}^\{S\}\\Big\[\\max\_\{a\\in\\mathcal\{A\}\}\\hat\{r\}\_\{s\}\(a\)\-\\max\_\{a\\in\\mathcal\{Q\}\}\\hat\{r\}\_\{s\}\(a\)\\Big\],\(1\)with ties broken by the mean imputed rank\. The firstSSgreedy steps visit the \(imputed\) champion of each source, after which the portfolio refines the source\-specific top regions\. Unlike a consensus prior, the portfolio deliberately covers architectures that are excellent on*some*source, which is where target champions are most often found \([Sec\.3\.1](https://arxiv.org/html/2610.32179#S3.SS1)\)\.

### 4\.2Multi\-task pairwise ranker pretraining

We use a shared encoderfθf\_\{\\theta\}\(a two\-layer MLP\) with one linear head\(wj,bj\)\(w\_\{j\},b\_\{j\}\)per task,gj​\(a\)=wj⊤​fθ​\(xa\)\+bjg\_\{j\}\(a\)=w\_\{j\}^\{\\top\}f\_\{\\theta\}\(x\_\{a\}\)\+b\_\{j\}\. The pretraining tasks𝒥\\mathcal\{J\}comprise theSSsparse sources, each supervised only on𝒪s\\mathcal\{O\}\_\{s\}, and theKKzero\-cost proxies, each supervised on all of𝒜\\mathcal\{A\}by its own ranking\. Treating proxies as*pseudo\-tasks*lets the encoder learn architecture regularities from dense but noisy signals while the sparse sources contribute accurate but incomplete task\-specific rankings\. All tasks are trained with a pairwise logistic \(RankNet\) loss\[[4](https://arxiv.org/html/2610.32179#bib.bib23)\],

ℒpre=1\|𝒥\|​∑j∈𝒥𝔼\(a,b\)∼𝒫j​ℓ​\(δj​\(a,b\),gj​\(a\)−gj​\(b\)\),\\mathcal\{L\}\_\{\\text\{pre\}\}=\\frac\{1\}\{\|\\mathcal\{J\}\|\}\\sum\_\{j\\in\\mathcal\{J\}\}\\mathbb\{E\}\_\{\(a,b\)\\sim\\mathcal\{P\}\_\{j\}\}\\,\\ell\\big\(\\delta\_\{j\}\(a,b\),\\,g\_\{j\}\(a\)\-g\_\{j\}\(b\)\\big\),\(2\)whereℓ⁡\(δ,u\)=log⁡\(1\+e−δ​u\)\\ell\(\\delta,u\)=\\log\(1\+e^\{\-\\delta u\}\),δj​\(a,b\)=sign⁡\(yj​\(a\)−yj​\(b\)\)\\delta\_\{j\}\(a,b\)=\\operatorname\{sign\}\(y\_\{j\}\(a\)\-y\_\{j\}\(b\)\), and𝒫j\\mathcal\{P\}\_\{j\}samples pairs among the architectures labelled for taskjj\. Because only rankings are used, scales of different metrics and proxies need no calibration\. We train an ensemble ofE=4E\{=\}4members and initialise the target head of each member as the mean of its pretraining heads\. The proxy values are also appended toxax\_\{a\}as input features\.

### 4\.3Target adaptation and search

After observing the target set𝒬\\mathcal\{Q\}, each member is fine\-tuned from its pretrained state on target pairs with the pretraining loss as a regulariser,

ℒtgt=ℒrank​\(gt,\{\(a,yt​\(a\)\)\}a∈𝒬\)\+λ​ℒpre,\\mathcal\{L\}\_\{\\text\{tgt\}\}=\\mathcal\{L\}\_\{\\text\{rank\}\}\\big\(g\_\{t\};\\\{\(a,y\_\{t\}\(a\)\)\\\}\_\{a\\in\\mathcal\{Q\}\}\\big\)\+\\lambda\\,\\mathcal\{L\}\_\{\\text\{pre\}\},\(3\)and the ensemble\-averaged standardised scores rank the unobserved architectures\. Following the weak\-predictor paradigm\[[29](https://arxiv.org/html/2610.32179#bib.bib10)\], we alternate between refitting and querying: each round samples a batch ofbbarchitectures from the predicted top\-kkregion, withkkshrinking geometrically, and the lastFFevaluations exploit the predicted best architectures\. The regulariser keeps the ranker close to transferable structure when few target labels are available, while the target head absorbs task\-specific deviations\.[Algorithm1](https://arxiv.org/html/2610.32179#alg1)summarises the procedure\.

Algorithm 1SHINE1:sparse logs

\{\(𝒪s,ys\)\}s=1S\\\{\(\\mathcal\{O\}\_\{s\},y\_\{s\}\)\\\}\_\{s=1\}^\{S\}, proxies

\{zk\}k=1K\\\{z\_\{k\}\\\}\_\{k=1\}^\{K\}, budget

BB
2:

r^s←\\hat\{r\}\_\{s\}\\leftarrowridge imputation of each source;

𝒬←𝒬0\\mathcal\{Q\}\\leftarrow\\mathcal\{Q\}\_\{0\}via[Eq\.1](https://arxiv.org/html/2610.32179#S4.E1); evaluate

𝒬\\mathcal\{Q\}
3:pretrain ensemble

\{g\(e\)\}e=1E\\\{g^\{\(e\)\}\\\}\_\{e=1\}^\{E\}with[Eq\.2](https://arxiv.org/html/2610.32179#S4.E2)

4:for

ℓ=1,2,…\\ell=1,2,\\dotswhile

\|𝒬\|<B−F\|\\mathcal\{Q\}\|<B\-Fdo

5:fine\-tune each

g\(e\)g^\{\(e\)\}with[Eq\.3](https://arxiv.org/html/2610.32179#S4.E3);

p←1E​∑estd⁡\(gt\(e\)\)p\\leftarrow\\frac\{1\}\{E\}\\sum\_\{e\}\\mathrm\{std\}\(g^\{\(e\)\}\_\{t\}\)
6:sample

bbunseen architectures from top\-

kℓk\_\{\\ell\}of

pp; evaluate; add to

𝒬\\mathcal\{Q\}
7:endfor

8:while

\|𝒬\|<B\|\\mathcal\{Q\}\|<Bdo

9:fine\-tune; add the top\-

F/2F/2unseen architectures of

ppto

𝒬\\mathcal\{Q\}; evaluate

10:endwhile

11:return

arg⁡maxa∈𝒬​yt​\(a\)\\arg\\max\_\{a\\in\\mathcal\{Q\}\}y\_\{t\}\(a\)

#### Implementation\.

We useB=50B\{=\}50,K0=0\.2​BK\_\{0\}\{=\}0\.2B,F=0\.4​BF\{=\}0\.4B,b=5b\{=\}5,kkdecreasing from 400 to 20, hidden width 128, 400 pretraining steps, 120 fine\-tuning steps per round andλ=0\.3\\lambda\{=\}0\.3\. All hyper\-parameters are shared across search spaces and targets\. The full search, including all pretraining and fine\-tuning, takes about one GPU\-minute, negligible compared with a single architecture evaluation\.

## 5Experiments

### 5\.1Setup

#### Benchmarks\.

TransNAS\-Bench\-101\[[6](https://arxiv.org/html/2610.32179#bib.bib1)\]contains seven tasks derived from Taskonomy\[[31](https://arxiv.org/html/2610.32179#bib.bib29)\]—object and scene classification, room layout, jigsaw, semantic segmentation, surface normals and autoencoding—evaluated on a cell\-based Micro space \(4,096 architectures\) and a Macro skeleton space \(3,256 architectures\)\. Following Arch\-Graph\[[11](https://arxiv.org/html/2610.32179#bib.bib2)\], jigsaw is used only as a source and the remaining six tasks are targets; for each target all other tasks are sources\. For independent validation we use the public MedMNIST\-NAS\-Bench\[[23](https://arxiv.org/html/2610.32179#bib.bib32)\], a tabular benchmark on four MedMNIST v2 classification tasks\[[30](https://arxiv.org/html/2610.32179#bib.bib31)\]\(BreastMNIST, PneumoniaMNIST, DermaMNIST, BloodMNIST\) in which every architecture is trained from scratch for 100 epochs with three seeds\. Its Tiny space contains 768 architectures from three macro families and its Small space 5,016 architectures with additional templates, attention and down\-sampling choices\. Search observes validation AUROC, and we report the test AUROC of the architecture selected by validation\.

#### Protocol\.

Unless stated otherwise, the target budget isB=50B\{=\}50evaluations, the same as Arch\-Graph, and each source task revealsm=50m\{=\}50randomly chosen architectures\. Zero\-cost proxies are the twelve proxies of NAS\-Bench\-Suite\-Zero\[[12](https://arxiv.org/html/2610.32179#bib.bib18)\]computed on the object\-classification task, as in TRP\[[15](https://arxiv.org/html/2610.32179#bib.bib3)\]\. Five of them \(fisher, grad\-norm, grasp, snip, plain\) use the labels of one minibatch of 32 images; to avoid leaking target labels, we use proxies computed on scene classification when object classification is itself the target, and report the original setting as a control in[Sec\.5\.2](https://arxiv.org/html/2610.32179#S5.SS2)\. \(All Taskonomy tasks share the same images\.\) In the real\-training experiments \([Sec\.5\.7](https://arxiv.org/html/2610.32179#S5.SS7)\) the proxies never see target data\. Structural encodings are BANANAS\-style path encodings for the Micro cell\[[24](https://arxiv.org/html/2610.32179#bib.bib11)\]and per\-layer down\-sampling and widening indicators for Macro\. Results are averaged over 20 seeds that resample the source logs and the search randomness \(10 seeds for the source\-density sweep and the budget sweep\)\. SHINE was developed on TransNAS\-Bench\-101 and all hyper\-parameters were frozen before the final runs; MedMNIST\-NAS\-Bench was not used during development\. Code, source\-log samples and all real\-training records will be released\. The 20 seeds of a target share the same task, so seed\-level tests only capture robustness to search randomness, not generalisation to new tasks\. For TransNAS\-Bench\-101 comparisons we therefore report the number of targets \(out of six\) on which a method is better after averaging over seeds, a task\-level paired Wilcoxon test \(whose smallest attainable two\-sidedpp\-value with six targets is 0\.031\) and, where useful, a 95% hierarchical bootstrap interval of the mean difference \(resampling targets, then seeds\); ablationpp\-values are Holm\-corrected\. We also report the mean of the worst 10% of runs \(*worst\-10%*\) as a robustness measure\.

#### Baselines\.

We compare with the published results of single\-task NAS \(random search, REA\[[21](https://arxiv.org/html/2610.32179#bib.bib15)\], weakNAS\[[29](https://arxiv.org/html/2610.32179#bib.bib10)\]\), transferable NAS \(CATCH\[[5](https://arxiv.org/html/2610.32179#bib.bib4)\], transfer variants of weakNAS, Arch\-Graph\[[11](https://arxiv.org/html/2610.32179#bib.bib2)\]\) and TRP\[[15](https://arxiv.org/html/2610.32179#bib.bib3)\]\. To control for implementation details we also run, under our exact protocol, a ZC\-only ranker pretrained on proxy pseudo\-tasks without any source logs \(TRP\-style\), a sources\-only ranker without proxies, and tree\-based and ensemble variants\.

### 5\.2Main results

Table 1:Cross\-task search on TransNAS\-Bench\-101 with 50 target evaluations\.Global rank of the returned architecture \(lower is better\)\. “6 tasks”: mean rank over all six targets\. “4 tasks”: rank of the mean metric value over the four targets whose metric matches published tables \(object/scene classification, autoencoding, normals\), the convention of\[[11](https://arxiv.org/html/2610.32179#bib.bib2),[15](https://arxiv.org/html/2610.32179#bib.bib3)\]\. “Worst 10%”: mean rank of the worst 10% of runs\. Published numbers are taken from\[[11](https://arxiv.org/html/2610.32179#bib.bib2)\]; TRP numbers are converted from the metric values reported in\[[15](https://arxiv.org/html/2610.32179#bib.bib3)\]\. Our implementation:m=50m\{=\}50evaluated architectures per source, 20 seeds, leakage\-free proxies\.[Tab\.1](https://arxiv.org/html/2610.32179#S5.T1)summarises the results\. Against published methods, SHINE obtains a better rank of the mean value than Arch\-Graph on the four metric\-compatible targets in both spaces \(Macro 2\.00*vs*\.2\.75, Micro 9\.00*vs*\.10\.25\) and is on par with TRP on Macro \(2\.00*vs*\.2\.25\)\. Under the six\-target mean\-rank protocol it improves over Arch\-Graph on Macro \(4\.21*vs*\.5\.24\) but is clearly worse on Micro \(17\.84*vs*\.12\.20\); this average is dominated by room layout, whose metric differs from the published tables \(we use validation loss,\[[11](https://arxiv.org/html/2610.32179#bib.bib2)\]reports L2 error\)\. It also trails TRP on Micro \(9\.00*vs*\.7\.50\)\. Published numbers were obtained with different code, seeds and implementation details, so these comparisons are indicative rather than controlled\. Restricted to 50 evaluated jigsaw architectures—Arch\-Graph’s source budget, although with zero\-cost proxies instead of Task2Vec—SHINE is slightly better than Arch\-Graph on Macro \(4\.78*vs*\.5\.24; rank of mean 2\.50*vs*\.2\.75\) but worse on Micro \(rank of mean 10\.75*vs*\.10\.25; six\-target average 21\.27*vs*\.12\.20\)\.

Against our own ZC\-only ranker the picture is more nuanced\. In the typical case the two are on par: their per\-target median ranks average 3\.3*vs*\.3\.6 on Macro and 14\.8*vs*\.10\.5 on Micro, SHINE is better in mean rank on only 2 and 4 of the six targets, and the differences are not significant at either the seed or the task level\. The difference lies in the tail: the worst 10% of ZC\-only runs average rank 34\.8 on Macro and 152\.1 on Micro,*vs*\.20\.9 and 88\.9 for SHINE, and its single worst run on Micro ranks 545th of 4,096 \(SHINE: 185th\)\. The reduction is stable across tail levels \(CVaR at 5/10/20%:−46/−40/−30%\-46/\{\-40\}/\{\-30\}\\%on Macro,−48/−42/−38%\-48/\{\-42\}/\{\-38\}\\%on Micro\), but its sources differ: on Micro it appears on four of six targets and survives leaving out any single target \(worst\-10% 30\.4–98\.6*vs*\.102\.3–172\.8\), whereas on Macro it comes entirely from room layout, without which the two methods are on par \(worst\-10% 6\.9*vs*\.6\.5\)\. A hierarchical bootstrap over targets and seeds puts 78% \(Macro\) and 90% \(Micro\) of its mass on a lighter tail for SHINE, but its 95% intervals include zero; per\-target numbers are in[Tab\.8](https://arxiv.org/html/2610.32179#A4.T8)\(supplement\); the pooled analysis in[Sec\.5\.7](https://arxiv.org/html/2610.32179#S5.SS7)adds seven real\-training targets to increase task\-level power\. Sources\-only fails catastrophically on Macro instead \(worst 10%: 120\.2, worst run: 778th\)\. Combining both knowledge sources substantially mitigates the severe failures of either\.

#### Proxy leakage control\.

With the original object\-classification proxies, which see 32 target labels when object classification is the target \(as in TRP\), the results barely change: six\-target means 4\.30/17\.70 \(Macro/Micro\) for SHINE*vs*\.5\.42/25\.21 for ZC\-only, rank of mean 2\.25/9\.00*vs*\.1\.75/14\.00\. Excluding object classification altogether gives 4\.43/19\.53*vs*\.5\.99/26\.95\.

### 5\.3Complementarity of histories and proxies

Table 2:Per\-target mean rankof a ranker pretrained on zero\-cost proxies only \(ZC\), on sparse sources only \(Src\), and on both \(SHINE\)\. 20 seeds\.[Tab\.2](https://arxiv.org/html/2610.32179#S5.T2)breaks the results down by target\. Zero\-cost proxies alone are excellent on Macro, where the coarse skeleton choices that proxies capture \(depth, width, down\-sampling positions\) dominate performance\. On Micro they fail on several targets,*e\.g*\.autoencoding \(43\.2\) and scene classification \(20\.2\); sparse source logs correct exactly these targets\. Conversely, on Macro room layout—whose ranking is negatively correlated with the other tasks—sources alone are misleading \(74\.4\), yet the combination is the best of the three \(15\.5\), indicating that the shared ranker does not simply average the two signals\. In a development run we also computed the proxies on the target task instead of object classification; this degraded the ZC\-only ranker on Micro from 25\.2 to 36\.6 \(autoencoding: 93\.2\), confirming that proxies are unreliable on dense\-prediction targets and that historical logs are needed there\.

### 5\.4Ablations

Table 3:Ablations\(B=50B\{=\}50,m=50m\{=\}50, 20 seeds\)\. Mean rank over six targets, in parentheses the number of targets \(of six\) on which the full method is better after averaging over seeds, and mean rank of the worst 10% of runs\.[Tab\.3](https://arxiv.org/html/2610.32179#S5.T3)isolates the components of SHINE\. On Macro, removing ranker pretraining or the portfolio worsens all six targets; with only six targets, however, no ablation remains significant after Holm correction at the task level, so we read these results as consistent directions rather than established effects\. Removing either knowledge source sharply worsens the worst\-case runs\. On Micro, no component has a consistent effect across targets; notably, a ranker that uses the same features and portfolio but no pretraining is better on average \(14\.07, better on four of six targets\), suggesting that on cell spaces the pretrained prior can be too rigid for some targets\. From a risk perspective, however, no variant has a lighter tail than SHINE on both spaces: every variant that improves the Micro tail increases the Macro worst\-10% by at least 68%, and we keep the configuration frozen before the final runs rather than select one per space\. Deciding adaptively when to trust pretraining is a natural extension; an exploratory online gate \([AppendixB](https://arxiv.org/html/2610.32179#A2)\) did not reliably separate the two cases with only 50 evaluations\.

### 5\.5Target budget and source density

Figure 2:\(a,b\)Mean rank over six targets*vs*\.the number of target evaluationsBB\(10 seeds forB∈\{25,100\}B\\in\\\{25,100\\\}, 20 forB=50B\{=\}50\)\.\(c\)Mean rank of SHINE*vs*\.the number of evaluated architectures per sourcemm\(B=50B\{=\}50\)\.[Fig\.2](https://arxiv.org/html/2610.32179#S5.F2)\(a,b\) varies the target budget\. On Micro, the gap between SHINE and ZC\-only is largest when evaluations are scarce: atB=25B\{=\}25the six\-target mean rank is 39\.9*vs*\.57\.3, although SHINE is better on only three of six targets\. On Macro, ZC\-only is better atB=25B\{=\}25\(12\.6*vs*\.23\.2\), mostly because of room layout \(49\.5*vs*\.108\.7\); on the four metric\-matched targets the two are on par \(5\.8*vs*\.5\.6\)\. AtB=100B\{=\}100both methods find near\-optimal architectures \(Macro 2\.9*vs*\.3\.4, Micro 12\.8*vs*\.12\.6\)\.[Fig\.2](https://arxiv.org/html/2610.32179#S5.F2)\(c\) varies the source density: SHINE is robust tommbetween 20 and 200, with the best Micro result atm=200m\{=\}200\(14\.7\) and no consistent trend on Macro\.

#### Realistic search logs\.

So far each source revealed a uniformly random subset of architectures\. Real histories are biased towards good architectures, because they are the trajectories of earlier searches\. We therefore replaced each source log by the 50 architectures evaluated by a predictor\-guided search \(Extra\-Trees weak predictor, same search template\) on that source; for one seed on Macro, these logs lie on average at the 65th–85th percentile of their task \(full results in[Tab\.7](https://arxiv.org/html/2610.32179#A3.T7), supplement\)\. SHINE is robust to this shift: its mean rank changes from 4\.21 to 3\.78 on Macro and from 17\.84 to 20\.63 on Micro, neither difference being significant even at the seed level \(p=0\.56p\{=\}0\.56and0\.400\.40\), and it remains better than the ZC\-only ranker in mean rank and worst\-10% on both spaces \(Micro worst\-10%: 110\.8*vs*\.152\.1\)\. Biased logs mostly help the sources\-only ranker on Macro \(14\.26→\\to3\.64, worst\-10% 120\.2→\\to18\.0\), because past searches usually contain each source’s champions; with such logs, histories alone become as strong as SHINE on Macro\. On Micro, the rank of the mean value on the four matched targets rises to 11\.00, slightly behind Arch\-Graph \(10\.25\)\.

### 5\.6Independent validation on MedMNIST\-NAS\-Bench

Table 4:MedMNIST\-NAS\-Bench\(never used during development\)\. Normalised test regret \(lower is better\) and test rank of the architecture selected by validation AUROC, averaged over four tasks and 20 seeds\. No zero\-cost proxies are used here\.MedMNIST\-NAS\-Bench provides no zero\-cost proxies, so it isolates the value of sparse search histories on a search space and task family never used during development \([Tab\.4](https://arxiv.org/html/2610.32179#S5.T4)\)\. On Tiny, pretraining on three sparse sources withm=50m\{=\}50improves over the target\-only ranker on three of four tasks \(seed\-levelp=0\.03p\{=\}0\.03\), over random search on all four, and over the tree\-based transfer baseline on three of four\. On Small, histories withm=50m\{=\}50improve over the target\-only ranker on three of four tasks and match the tree baseline on average, and denser histories \(m=200m\{=\}200\) reduce the test regret by a further 24% and nearly halve the test rank\. Because selection is based on validation AUROC, test regret also reflects the validation–test gap, which is large for the small BreastMNIST and PneumoniaMNIST datasets\.

### 5\.7Validation with real training and pooled analysis

Table 5:Pooled analysis over 13 Micro targets\(analysis plan fixed before the last four targets were run\)\. Rank of the returned architecture as a percentage of the pool \(lower is better\): mean and CVaR10\(mean of the worst 10% of runs\)\. TN: TransNAS\-Bench\-101 \(B=50B\{=\}50, 20 seeds, pool 4,096\)\. RT: real training \(pool 300, averaged overB∈\{30,50\}B\{\\in\}\\\{30,50\\\}, 30 seeds each\)\. Bottom: number of targets on which SHINE is better, and task\-level Wilcoxonpp\-values \(Holm\-corrected over the two comparisons\)\.†Leakage\-free proxies \(computed on scene classification\)\.To test whether the conclusions carry over beyond tabular benchmarks, we trained 300 randomly drawn architectures from each TransNAS\-Bench\-101 space with the official model code on seven targets absent from Taskonomy, each for 20 epochs with one seed: classification on CIFAR\-100, Tiny\-ImageNet and Oxford\-IIIT Pets \(37 breeds\), and four dense\-prediction targets—reconstruction, denoising and inpainting on Tiny\-ImageNet and trimap segmentation on Pets—for which the encoder is followed by the official TransNAS autoencoder decoder with channel widths divided by four\. All inputs are64×6464\{\\times\}64; we search within these pools using the TransNAS tasks as sparse sources \(m=50m\{=\}50\) with the configuration frozen in[Sec\.5](https://arxiv.org/html/2610.32179#S5), atB∈\{30,50\}B\{\\in\}\\\{30,50\\\}\. We also trained both pools on SVHN\. Like other decade\-old digit benchmarks, it is saturated for modern CNNs: all architectures reach about 96% test accuracy and every method selects an architecture within 0\.3 points of the pool optimum, so test\-rank differences are dominated by noise \(the ZC\-only ranker happened to rank best on Micro\); we therefore replaced it by harder targets\. Each training protocol was fixed before its targets were trained\. The pooled analysis plan was fixed \(internal record, 2026\-10\-22\) after the six TransNAS targets and the CIFAR\-100 and Tiny\-ImageNet classification and reconstruction results had been seen, and before the four remaining targets \(denoising, inpainting, Pets classification and segmentation\) were run; we therefore report the new targets separately\. Per\-target results are in[Tab\.9](https://arxiv.org/html/2610.32179#A5.T9)\(supplement\)\.

On the Macro pools all transfer methods return architectures within a few ranks of the pool optimum \(mean rank at most 5\.3 of 300\), so these pools hardly discriminate between methods\. We therefore pool the six TransNAS Micro targets and the seven real\-training Micro targets \([Tab\.5](https://arxiv.org/html/2610.32179#S5.T5)\)\. Proxies fail on the dense\-prediction targets, where histories are the best single source, and they lose to SHINE on 10 of 13 targets both in mean and in worst\-case performance \(task\-level Holm\-correctedp=0\.012p\{=\}0\.012and0\.0160\.016; bootstrap interval of the mean CVaR10difference\[−2\.8,−0\.8\]\[\-2\.8,\-0\.8\]percentage points\)\. On the four targets run after the plan was fixed, SHINE beats the proxy\-only ranker on all four in both statistics \(the smallest attainableppwith four targets is 0\.125\), but histories alone have the lighter tail on all four\. The advantage over proxies does not hinge on a single data source: it holds on the seven real\-training targets alone \(6/7,p=0\.031p\{=\}0\.031and0\.0470\.047\), without the four Tiny\-ImageNet targets \(7/9,p=0\.020p\{=\}0\.020\), and when targets sharing a real dataset are averaged into one unit \(7/9,p=0\.020p\{=\}0\.020\); details are in[Tab\.10](https://arxiv.org/html/2610.32179#A6.T10)\(supplement\)\. Since the pools differ in size and budget fraction, the grouped results should also be read separately: on the six TransNAS targets SHINE beats proxies on 4/6 in both statistics\. Histories alone are on par with SHINE on these cell\-space targets \(7/13 and 6/13,p=0\.68p\{=\}0\.68and0\.450\.45\) and slightly lighter\-tailed on average, but they fail severely on Macro \([Tab\.1](https://arxiv.org/html/2610.32179#S5.T1): worst\-10% rank 120\.2*vs*\.20\.9\)\. Thus SHINE does not dominate either source; it substantially mitigates the severe failures observed with each of them while retaining competitive average performance\.

## 6Limitations and Conclusion

#### Limitations\.

Our main experiments use tabular benchmarks, which allow exhaustive evaluation and controlled leakage\-free analyses; our real\-training validation \([Sec\.5\.7](https://arxiv.org/html/2610.32179#S5.SS7)\) is limited to pools of 300 architectures, short schedules, a single training seed and64×6464\{\\times\}64inputs, and its Macro pools do not discriminate between methods\. The number of targets is small: six per TransNAS space, and the 13 pooled targets share four underlying datasets, so task\-level tests have limited power and are not fully independent\. Hyper\-parameters were developed on TransNAS\-Bench\-101; we therefore validate on MedMNIST\-NAS\-Bench, which was never used during development\. On Micro, SHINE still trails the reported results of TRP\. Finally, SHINE assumes that past tasks share the search space of the new task; transferring across different search spaces is left to future work\.

#### Conclusion\.

We studied cross\-task NAS from sparse search histories, the form in which search knowledge actually accumulates\. Consensus priors and source portfolios that work with dense tables fail in this regime, whereas zero\-cost proxies, though available everywhere, are unreliable on dense\-prediction tasks\. SHINE unifies both kinds of cheap knowledge by pretraining a pairwise ranker on sparse source tasks and proxy pseudo\-tasks and by initialising the search with a champion portfolio\. SHINE is best understood as a risk\-robust strategy: it substantially mitigates the severe failures that each knowledge source has on some target, reduces tail risk relative to proxies, while retaining competitive average performance\. Its advantage over proxies alone is small on the six TransNAS targets but significant at the task level when pooled over 13 cell\-space targets, whereas histories alone are on par with it on cell spaces but fail severely on the macro space\. Deciding adaptively how much to trust each source of prior knowledge, which could also recover the typical\-case gains of simpler variants on cell spaces, is a promising direction\. We hope that framing transfer around incomplete histories will make cross\-task NAS easier to deploy where past searches are already available\.

## Acknowledgments

This work was supported in part by JST SPRING, Grant Number JPMJSP2121 and JSPS Program for Forming Japan’s Peak Research Universities \(J\-PEAKS\), Grant Number JPJS00420240016\.

## References

- \[1\]\(2021\)Zero\-cost proxies for lightweight NAS\.InICLR,Cited by:[§1](https://arxiv.org/html/2610.32179#S1.p2.1),[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px3.p1.1)\.
- \[2\]A\. Achille, M\. Lam, R\. Tewari, A\. Ravichandran, S\. Maji, C\. C\. Fowlkes, S\. Soatto, and P\. Perona\(2019\)Task2Vec: task embedding for meta\-learning\.InICCV,Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px2.p1.1)\.
- \[3\]Y\. Akhauri and M\. S\. Abdelfattah\(2023\)Multi\-Predict: few shot predictors for efficient neural architecture search\.InInternational Conference on Automated Machine Learning \(AutoML\),Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px2.p1.1)\.
- \[4\]C\. Burges, T\. Shaked, E\. Renshaw, A\. Lazier, M\. Deeds, N\. Hamilton, and G\. Hullender\(2005\)Learning to rank using gradient descent\.InICML,Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2610.32179#S4.SS2.p1.1)\.
- \[5\]X\. Chen, Y\. Duan, Z\. Chen, H\. Xu, Z\. Chen, X\. Liang, T\. Zhang, and Z\. Li\(2020\)CATCH: context\-based meta reinforcement learning for transferrable architecture search\.InECCV,Cited by:[§1](https://arxiv.org/html/2610.32179#S1.p2.1),[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2610.32179#S5.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2610.32179#S5.T1.17.7.1.1)\.
- \[6\]Y\. Duan, X\. Chen, H\. Xu, Z\. Chen, X\. Liang, T\. Zhang, and Z\. Li\(2021\)TransNAS\-Bench\-101: improving transferability and generalizability of cross\-task neural architecture search\.InCVPR,Cited by:[4th item](https://arxiv.org/html/2610.32179#S1.I1.i4.p1.1),[§1](https://arxiv.org/html/2610.32179#S1.p1.1),[§1](https://arxiv.org/html/2610.32179#S1.p2.1),[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2610.32179#S5.SS1.SSS0.Px1.p1.1)\.
- \[7\]T\. Elsken, J\. H\. Metzen, and F\. Hutter\(2019\)Neural architecture search: a survey\.JMLR20\(55\),pp\. 1–21\.Cited by:[§1](https://arxiv.org/html/2610.32179#S1.p1.1)\.
- \[8\]M\. Feurer, K\. Eggensperger, S\. Falkner, M\. Lindauer, and F\. Hutter\(2022\)Auto\-sklearn 2\.0: hands\-free AutoML via meta\-learning\.JMLR23\(261\),pp\. 1–61\.Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px4.p1.1)\.
- \[9\]M\. Feurer, B\. Letham, F\. Hutter, and E\. Bakshy\(2018\)Practical transfer learning for Bayesian optimization\.arXiv preprint arXiv:1802\.02219\.Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px4.p1.1)\.
- \[10\]M\. Feurer, J\. T\. Springenberg, and F\. Hutter\(2015\)Initializing Bayesian hyperparameter optimization via meta\-learning\.InAAAI,Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px4.p1.1)\.
- \[11\]M\. Huang, Z\. Huang, C\. Li, X\. Chen, H\. Xu, Z\. Li, and X\. Liang\(2022\)Arch\-Graph: acyclic architecture relation predictor for task\-transferable neural architecture search\.InCVPR,Cited by:[4th item](https://arxiv.org/html/2610.32179#S1.I1.i4.p1.1),[§1](https://arxiv.org/html/2610.32179#S1.p2.1),[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2610.32179#S3.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2610.32179#S5.SS1.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2610.32179#S5.SS1.SSS0.Px3.p1.1),[§5\.2](https://arxiv.org/html/2610.32179#S5.SS2.p1.1),[Table 1](https://arxiv.org/html/2610.32179#S5.T1),[Table 1](https://arxiv.org/html/2610.32179#S5.T1.17.10.1.1)\.
- \[12\]A\. Krishnakumar, C\. White, A\. Zela, R\. Tu, M\. Safari, and F\. Hutter\(2022\)NAS\-Bench\-Suite\-Zero: accelerating research on zero cost proxies\.InNeurIPS Datasets and Benchmarks Track,Cited by:[Appendix A](https://arxiv.org/html/2610.32179#A1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px3.p1.1),[§5\.1](https://arxiv.org/html/2610.32179#S5.SS1.SSS0.Px2.p1.1)\.
- \[13\]H\. Lee, E\. Hyung, and S\. J\. Hwang\(2021\)Rapid neural architecture search by learning to generate graphs from datasets\.InICLR,Cited by:[§1](https://arxiv.org/html/2610.32179#S1.p2.1),[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px2.p1.1)\.
- \[14\]H\. Lee, S\. Lee, S\. Chong, and S\. J\. Hwang\(2021\)HELP: hardware\-adaptive efficient latency prediction for NAS via meta\-learning\.InNeurIPS,Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px2.p1.1)\.
- \[15\]N\. Li, B\. Xue, L\. Ma, and M\. Zhang\(2025\)Transferable relativistic predictor: mitigating cross\-task cold\-start issue in NAS\.InIJCAI,Cited by:[4th item](https://arxiv.org/html/2610.32179#S1.I1.i4.p1.1),[§1](https://arxiv.org/html/2610.32179#S1.p2.1),[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2610.32179#S5.SS1.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2610.32179#S5.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2610.32179#S5.T1),[Table 1](https://arxiv.org/html/2610.32179#S5.T1.17.11.1.1)\.
- \[16\]P\. Liao, Y\. Jin, and W\. Du\(2023\)EMT\-NAS: transferring architectural knowledge between tasks from different datasets\.InCVPR,Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px2.p1.1)\.
- \[17\]M\. Lin, P\. Wang, Z\. Sun, H\. Chen, X\. Sun, Q\. Qian, H\. Li, and R\. Jin\(2021\)Zen\-NAS: a zero\-shot NAS for high\-performance image recognition\.InICCV,Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px3.p1.1)\.
- \[18\]H\. Liu, K\. Simonyan, and Y\. Yang\(2019\)DARTS: differentiable architecture search\.InICLR,Cited by:[§1](https://arxiv.org/html/2610.32179#S1.p1.1)\.
- \[19\]J\. Mellor, J\. Turner, A\. Storkey, and E\. J\. Crowley\(2021\)Neural architecture search without training\.InICML,Cited by:[§1](https://arxiv.org/html/2610.32179#S1.p2.1),[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px3.p1.1)\.
- \[20\]F\. Pfisterer, J\. N\. van Rijn, P\. Probst, A\. C\. Müller, and B\. Bischl\(2021\)Learning multiple defaults for machine learning algorithms\.InProceedings of the Genetic and Evolutionary Computation Conference Companion \(GECCO\),External Links:[Document](https://dx.doi.org/10.1145/3449726.3459523)Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px4.p1.1),[§3\.1](https://arxiv.org/html/2610.32179#S3.SS1.p1.1)\.
- \[21\]E\. Real, A\. Aggarwal, Y\. Huang, and Q\. V\. Le\(2019\)Regularized evolution for image classifier architecture search\.InAAAI,Cited by:[§1](https://arxiv.org/html/2610.32179#S1.p1.1),[§5\.1](https://arxiv.org/html/2610.32179#S5.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2610.32179#S5.T1.17.4.1.1)\.
- \[22\]H\. Tanaka, D\. Kunin, D\. L\. K\. Yamins, and S\. Ganguli\(2020\)Pruning neural networks without any data by iteratively conserving synaptic flow\.InNeurIPS,Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px3.p1.1)\.
- \[23\]W\. Wang\(2026\)MedMNIST\-NAS\-Bench: a tabular neural architecture search benchmark on MedMNIST v2\.Note:Zenodo dataset, version 1\.0External Links:[Document](https://dx.doi.org/10.5281/zenodo.20364514)Cited by:[4th item](https://arxiv.org/html/2610.32179#S1.I1.i4.p1.1),[§5\.1](https://arxiv.org/html/2610.32179#S5.SS1.SSS0.Px1.p1.1)\.
- \[24\]C\. White, W\. Neiswanger, and Y\. Savani\(2021\)BANANAS: bayesian optimization with neural architectures for neural architecture search\.InAAAI,Cited by:[Appendix A](https://arxiv.org/html/2610.32179#A1.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2610.32179#S5.SS1.SSS0.Px2.p1.1)\.
- \[25\]C\. White, S\. Nolen, and Y\. Savani\(2021\)Exploring the loss landscape in neural architecture search\.InUAI,Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px1.p1.1)\.
- \[26\]C\. White, M\. Safari, R\. Sukthanker, B\. Ru, T\. Elsken, A\. Zela, D\. Dey, and F\. Hutter\(2023\)Neural architecture search: insights from 1000 papers\.arXiv preprint arXiv:2301\.08727\.Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px1.p1.1)\.
- \[27\]C\. White, A\. Zela, R\. Ru, Y\. Liu, and F\. Hutter\(2021\)How powerful are performance predictors in neural architecture search?\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px3.p1.1)\.
- \[28\]M\. Wistuba, N\. Schilling, and L\. Schmidt\-Thieme\(2015\)Sequential model\-free hyperparameter tuning\.InICDM,Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px4.p1.1),[§3\.1](https://arxiv.org/html/2610.32179#S3.SS1.p1.1)\.
- \[29\]J\. Wu, X\. Dai, D\. Chen, Y\. Chen, M\. Liu, Y\. Yu, Z\. Wang, Z\. Liu, M\. Chen, and L\. Yuan\(2021\)Stronger NAS with weaker predictors\.InNeurIPS,Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px1.p1.1),[§4\.3](https://arxiv.org/html/2610.32179#S4.SS3.p1.2),[§5\.1](https://arxiv.org/html/2610.32179#S5.SS1.SSS0.Px3.p1.1),[Table 1](https://arxiv.org/html/2610.32179#S5.T1.17.5.1.1)\.
- \[30\]J\. Yang, R\. Shi, D\. Wei, Z\. Liu, L\. Zhao, B\. Ke, H\. Pfister, and B\. Ni\(2023\)MedMNIST v2 – a large\-scale lightweight benchmark for 2D and 3D biomedical image classification\.Scientific Data10\(1\),pp\. 41\.Cited by:[§5\.1](https://arxiv.org/html/2610.32179#S5.SS1.SSS0.Px1.p1.1)\.
- \[31\]A\. R\. Zamir, A\. Sax, W\. Shen, L\. J\. Guibas, J\. Malik, and S\. Savarese\(2018\)Taskonomy: disentangling task transfer learning\.InCVPR,Cited by:[§1](https://arxiv.org/html/2610.32179#S1.p1.1),[§5\.1](https://arxiv.org/html/2610.32179#S5.SS1.SSS0.Px1.p1.1)\.
- \[32\]T\. Zhang and H\. Liu\(2025\)Multi\-task neural architecture search using architecture embedding and transfer rank\.arXiv preprint arXiv:2504\.00772\.Cited by:[§2](https://arxiv.org/html/2610.32179#S2.SS0.SSS0.Px2.p1.1)\.
- \[33\]B\. Zoph and Q\. V\. Le\(2017\)Neural architecture search with reinforcement learning\.InICLR,Cited by:[§1](https://arxiv.org/html/2610.32179#S1.p1.1)\.

Supplementary Material

## Appendix AImplementation details

#### Encodings\.

For the Micro cell \(four nodes, six edges with operations \{zero, skip,1×11\{\\times\}1conv,3×33\{\\times\}3conv\}\) we use one\-hot edge operations plus a BANANAS\-style path encoding\[[24](https://arxiv.org/html/2610.32179#bib.bib11)\]: each of the four input\-to\-output paths is active if it contains no zero operation and is encoded by the one\-hot index of its operation sequence \(3\+9\+9\+273\+9\+9\+27features\), complemented by the number of active paths, the number of3×33\{\\times\}3convolutions on active paths and a parameter\-count surrogate\. For Macro \(four to six layers of types 1–4, where even types double the width and types\>2\>2halve the resolution\) we use per\-layer one\-hot types, per\-layer down\-sampling and widening indicators, their cumulative sums, depth, and the mean positions of down\-sampling and widening layers\. All features are standardised\.

#### Zero\-cost proxies\.

We use the twelve proxies of NAS\-Bench\-Suite\-Zero\[[12](https://arxiv.org/html/2610.32179#bib.bib18)\]\(fisher, flops, grad\-norm, grasp, jacov, l2\-norm, nwot, params, plain, snip, synflow, zen\) computed on object classification, rank\-normalised to\[0,1\]\[0,1\]\.

#### Imputation\.

Each sparse source is imputed by ridge regression \(α=10\\alpha\{=\}10\) on the first\-order encoding and all pairwise products, fitted to the percentile ranks of itsmmobserved architectures\.

#### Compute\.

One full search \(B=50B\{=\}50\) takes about 77 s on an RTX 5090 shared by 12 processes; the ZC\-only ranker takes 54 s and the tree\-based variant under 1 s\.

## Appendix BExploratory: an online gate between pretrained and target\-only rankers

The ablation in[Tab\.3](https://arxiv.org/html/2610.32179#S5.T3)suggests that pretraining helps on Macro but can be too rigid on Micro\.*After*observing the final results, we therefore tested a prequential gate and fixed its rule and pass criteria before running it\. At every round we fit both the pretrained ranker \(A\) and a ranker with the same features and initialisation but without pretraining \(B\)\. After each new batch is evaluated, the predictions that each model made*before*observing the batch are scored by pairwise ordering accuracy on \(new, old\) and \(new, new\) pairs; with accumulated correct countsCA,CBC\_\{A\},C\_\{B\}overPPpairs, the weight isw=σ⁡\(2​\(CA−CB\)/P\)w=\\sigma\\big\(2\\,\(C\_\{A\}\-C\_\{B\}\)/\\sqrt\{P\}\\big\)and the search score isw​z​\(A\)\+\(1−w\)​z​\(B\)w\\,z\(A\)\+\(1\-w\)\\,z\(B\)\. A fixedw=0\.5w\{=\}0\.5ensemble served as control\.

Table 6:Exploratory gate\(20 seeds; run with the original object\-classification proxies\)\. TransNAS: mean rank over six targets / rank of mean value on the four matched targets\. MedMNIST: test regret \(m=50m\{=\}50, no proxies\)\. The gate did not meet its pre\-declared criterion \(within 1 rank, resp\. 0\.01 regret, of the better single model\)\.The gate missed its criterion on both TransNAS spaces \(by 0\.02 and 0\.07 ranks\) and on MedMNIST Tiny\. Its final weights stayed between 0\.4 and 0\.6 on average for every target,*i\.e*\.with 50 noisy evaluations the out\-of\-sample evidence was insufficient to separate the two rankers\. The fixed ensemble obtained the best metric\-matched TransNAS numbers \(2\.00 / 8\.00\), but since it was not a pre\-declared candidate we do not adopt it and report it only for completeness\. Reliably deciding*when*to trust transferred knowledge under very small budgets remains an open problem\.

## Appendix CSearch\-trajectory source logs

Table 7:Uniform*vs*\.search\-trajectory source logs\(B=50B\{=\}50,m=50m\{=\}50, 20 seeds\)\. Trajectory logs are the 50 architectures evaluated by an Extra\-Trees weak\-predictor search on each source\. Mean rank over six targets / rank of the mean value on the four matched targets / worst\-10%\.Paired over \(target, seed\), SHINE with trajectory logs does not differ significantly from SHINE with uniform logs \(p=0\.56p\{=\}0\.56on Macro,0\.400\.40on Micro\) or from the sources\-only ranker with trajectory logs \(p=0\.81p\{=\}0\.81and0\.300\.30\)\. The protocol of this experiment was fixed before it was run\.

## Appendix DTail robustness per target

Table 8:Worst\-10% rank per target\(mean of the two worst of 20 seeds;B=50B\{=\}50,m=50m\{=\}50\) for SHINE, the ZC\-only ranker and the sources\-only ranker, and pooled CVaR over all 120 runs with the 95% hierarchical bootstrap interval \(targets, then seeds\) of the difference SHINE−\-ZC\.
## Appendix EPer\-target real\-training results

Table 9:Real training, per target\.Test rank \(lower is better\) of the architecture selected on the validation set within a pool of 300 architectures, mean over 30 seeds, worst\-10% in parentheses\. Metrics: accuracy \(classification\), SSIM \(reconstruction, denoising\), PSNR inside the hole \(inpainting\), mIoU \(segmentation\)\.
## Appendix FSensitivity of the pooled analysis

Table 10:Sensitivity of the pooled comparison\(not part of the fixed plan\)\. Number of targets on which SHINE is better and task\-level Wilcoxonpp\(uncorrected\), for mean rank and CVaR10\(% of pool\)\.