GENEB: Why Genomic Models Are Hard to Compare

# GENEB: Why Genomic Models Are Hard to Compare
Source: [https://arxiv.org/html/2606.04525](https://arxiv.org/html/2606.04525)
###### Abstract

Progress in genomic foundation models is difficult to assess due to fragmented benchmarks, incompatible evaluation protocols, and task\-specific reporting\. As a result, claims of superiority or generality across models are often not directly comparable\. We introduce GENEB, a large\-scale diagnostic benchmark that evaluates frozen representations from 40 genomic foundation models across 100 tasks spanning 13 functional categories under a unified probing\-based protocol, including few\-shot regimes\. GENEB enables controlled comparison across model scale, architecture, tokenization, and pretraining data while explicitly exposing task\-level trade\-offs\. Our analysis shows that aggregate leaderboards are unstable: model rankings vary sharply across task categories, scale provides only modest and inconsistent gains, and architectural and pretraining alignment frequently outweigh parameter count\. These results highlight limitations of current evaluation practices and position GENEB as a reference framework for principled comparison and category\-aware model selection in genomic machine learning\.

Machine Learning, ICML

## 1Introduction

The genomic machine learning landscape has expanded rapidly over the past decade, producing a large and heterogeneous ecosystem of models, architectures, and training paradigms\. This expansion has not been accompanied by commensurate methodological infrastructure for comparison\. Figure[1](https://arxiv.org/html/2606.04525#S1.F1)illustrates the present state of the field: models are evaluated on disjoint benchmarks, compared under incompatible protocols, and frequently reported as state\-of\-the\-art within narrowly defined settings, leaving unclear how different models relate to one another or whether reported improvements reflect genuine progress\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/chaotic_models.png)Figure 1:Fragmented comparison landscape of genomic foundation models\.Each node represents a published model; directed edges denote models explicitly used as baselines or comparators in the corresponding paper\. The sparse, disconnected graph reflects the absence of unified cross\-model evaluation in genomic machine learning\.This fragmentation makes even basic questions difficult to answer\. Principled comparison between widely discussed models such asDNA\-GPT\(Zhanget al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib13)\),GenomeOcean\(Zhouet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib7)\), andEvo\(Nguyenet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib15)\)is currently not feasible: each is evaluated on different task sets, preprocessing pipelines, and evaluation protocols\. The same model is sometimes characterised as a major breakthrough in one context and as underperforming in another, reflecting not contradictory evidence but the absence of a common evaluation framework\.

This problem is compounded by the rapid growth in scale and visibility of genomic foundation models\. As models become larger and more expressive, claims of superiority and generality have grown correspondingly bolder, yet the methodological basis for adjudicating such claims has not kept pace\. The result is a widening gap between what is asserted about model capability and what can be reliably established through reproducible cross\-model evaluation\.

We introduceGENEB, a large\-scale benchmark evaluating 40 genomic foundation models on 100 tasks spanning 13 functional categories under a unified probing protocol\. GENEB is designed to enable controlled, systematic comparison and to expose performance trade\-offs obscured by fragmented evaluation practices; in spirit, it plays a role analogous to MTEB\(Muennighoffet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib3)\)in natural language processing, providing a unified evaluation framework rather than a single\-task leaderboard\. By making results directly comparable across models and tasks, GENEB establishes a shared reference point for assessing progress in genomic machine learning\.

##### Conflict of Interest Disclosure\.

The authors declare no financial conflicts of interest\. None of the 40 evaluated models was developed by the authors or their funders\.

## 2Related Work

The rapid growth of genomic foundation models has produced a heterogeneous landscape spanning diverse architectures, tokenization schemes, and pretraining strategies\.

##### Architectures\.

Early genomic models predominantly adopted Transformer encoders trained with masked language modeling\(Zhouet al\.,[2024a](https://arxiv.org/html/2606.04525#bib.bib2); Dalla\-Torreet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib18); Fishmanet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib16); Sanabriaet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib6)\)\. More recent work explored decoder\-only and generative architectures for unified sequence modeling and long\-context processing\(Zhanget al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib13); Nguyenet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib15); Wuet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib5); Zhouet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib7); Liet al\.,[2025b](https://arxiv.org/html/2606.04525#bib.bib10)\)\. To reduce attention complexity, alternative designs based on long convolutions and state\-space models have been proposed, alongside hybrid architectures combining multiple paradigms\(Nguyenet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib17); Schiffet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib12); Liuet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib14); Duanet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib8); Vishniakovet al\.,[2025b](https://arxiv.org/html/2606.04525#bib.bib25)\)\.

##### Tokenization and Pretraining\.

Tokenization strategies range from single\-nucleotide andkk\-mer representations to learned BPE vocabularies, each offering different trade\-offs between resolution and efficiency\(Zhouet al\.,[2024a](https://arxiv.org/html/2606.04525#bib.bib2),[2025](https://arxiv.org/html/2606.04525#bib.bib7)\)\. Pretraining data similarly varies from human\-only and species\-specific corpora to broad multi\-species and domain\-focused datasets, with prior studies suggesting potential benefits of both diversity and specialization depending on the task\(Dalla\-Torreet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib18); Wuet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib5); Avsecet al\.,[2021](https://arxiv.org/html/2606.04525#bib.bib1); Zhaiet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib22)\)\.

##### Benchmarks\.

Several benchmarks evaluate genomic foundation models, including Nucleotide Transformer tasks\(Dalla\-Torreet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib18)\), GUE/GUE\+\(Zhouet al\.,[2024a](https://arxiv.org/html/2606.04525#bib.bib2)\), Genomic Benchmarks\(Gresovaet al\.,[2022](https://arxiv.org/html/2606.04525#bib.bib45)\), BEND\(Marinet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib21)\), and DNALongBench\(Chenget al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib24)\)\. While these resources cover important regulatory, epigenetic, and cross\-species tasks, they differ in task design and evaluation protocols and typically assess only a limited subset of models, making cross\-paper comparison difficult\.

##### Comparative Benchmarking Studies\.

Recent studies have explored broader comparisons of genomic foundation models, but usually evaluate a small number of representative architectures\. For example,Wanget al\.\([2025b](https://arxiv.org/html/2606.04525#bib.bib49)\)focus on approximately ten model families and predominantly human\-centric tasks\. Platform\-based efforts such as OmniGenBench\(Wanget al\.,[2025a](https://arxiv.org/html/2606.04525#bib.bib48)\)provide dynamic leaderboards, but currently include a limited and evolving set of baselines, leaving many recent DNA\-specific models unevaluated\.

##### Positioning of GENEB\.

GENEB addresses these gaps by providing a large\-scale, controlled benchmark covering 40 genomic foundation models evaluated on 100 DNA classification tasks across 13 functional categories\. By evaluating all models on the full task suite under a unified probing\-based protocol, GENEB enables matched comparisons across architecture, tokenization, and pretraining data and yields a complete performance matrix that exposes task\-dependent trade\-offs\. We plan to release GENEB as a public benchmark with evaluations hosted on Hugging Face, serving as a community reference analogous to MTEB in NLP\(Muennighoffet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib3)\)\.

##### Extended Related Work\.

A detailed discussion of prior benchmarks, comparative studies, and architectural trends is provided in Appendix[A](https://arxiv.org/html/2606.04525#A1)\.

## 3Methodology

GENEB evaluates genomic foundation models \(see Appendix[C](https://arxiv.org/html/2606.04525#A3), Table[4](https://arxiv.org/html/2606.04525#A3.T4)\) using an embedding\-based*probing*protocol: frozen sequence representations are assessed with lightweight classifiers, isolating representation quality and enabling controlled comparison across architectures and training regimes\. The benchmark covers diverse genomic prediction tasks spanning multiple functional categories; full task definitions are provided in Appendix[B](https://arxiv.org/html/2606.04525#A2)\.

Probing setup\.For each task, frozen embeddings are used as features for logistic regression \(max\_iter=1000\) and evaluated in 1\-shot, 10\-shot, and full\-data regimes \(Figure[5](https://arxiv.org/html/2606.04525#S4.F5)\)\. Results are averaged over five fixed random seeds\{13,17,42,123,997\}\\\{13,17,42,123,997\\\}\. The stability of model rankings under non\-linear probing is verified empirically in Appendix[E\.1](https://arxiv.org/html/2606.04525#A5.SS1), and sensitivity of few\-shot conclusions to the choice of regularization strength is analyzed in Appendix[E\.2](https://arxiv.org/html/2606.04525#A5.SS2)\.

Metric and data\.We report Matthews Correlation Coefficient \(MCC\), which is robust to class imbalance and standard in genomic evaluation\. Tasks exceeding10510^\{5\}sequences are subsampled\. An empirical analysis usingGenomeOceanembeddings shows that MCC stabilizes beyond this size, motivating10510^\{5\}as a practical upper bound\.

## 4Aggregate Performance Analysis Across 100 Genomic Tasks

We present a systematic analysis of 40 DNA foundation models evaluated on 100 genomic prediction tasks spanning 13 functional categories\. Our goal is to characterize how model scale \(Figure[2](https://arxiv.org/html/2606.04525#S4.F2)\), architecture, tokenization, and pretraining data interact under a unified evaluation protocol, and to extract practically relevant patterns for model selection\. Unless stated otherwise, all statistics refer to MCC aggregated within the GENEB benchmark\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/image_per_size_v2.png)Figure 2:Pareto frontier of model efficiency: macro\-MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmic x\-axis and full\-shot macro\-average MCC on the y\-axis\. Marker size and color both encode macro\-MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\. Spearman correlation betweenlog⁡\(params\)\\log\(\\text\{params\}\)and macro\-MCC isρ=0\.565\\rho=0\.565\(p<0\.001p<0\.001\); excluding the prokaryotic\-only outlierEvo\-1\-131kraises this toρ=0\.686\\rho=0\.686\(p<0\.001p<0\.001\)\. While scale is a substantial predictor of aggregate performance, several large models fall below the frontier, indicating that architecture and pretraining choices can offset substantial scale differences \(see Section[4](https://arxiv.org/html/2606.04525#S4), Table[1](https://arxiv.org/html/2606.04525#S4.T1)\)\.##### The Scale–Performance Disconnect\.

Model size exhibits a statistically significant and substantial association with aggregate performance \(ρ=0\.565\\rho=0\.565,p<0\.001p<0\.001; Figure[2](https://arxiv.org/html/2606.04525#S4.F2)\), which strengthens further toρ=0\.686\\rho=0\.686\(p<0\.001p<0\.001\) once the prokaryotic\-only outlierEvo\-1\-131kis excluded \(see Domain Mismatch paragraph below\)\. Even with this strong aggregate trend, however, model selection is not reducible to parameter count: among the 36 in\-domain models \(excluding prokaryotic\-only, microbial\-only, and plant\-specific pretraining\), we identify 31 instances in which a model at least5×5\\timessmaller outperforms a larger counterpart in aggregate MCC, with this count being identical under micro\- and macro\-averaging\. A representative example isMutBERT\(86M, Transformer\-encoder\), which exceedseccDNAMamba\(1B, Mamba\-SSM\) by\+0\.110\+0\.110macro\-MCC despite an11\.611\.6\-fold size difference, illustrating that non\-scale design choices can offset substantial scale gaps in GENEB\. Category\-level scaling correlations are reported in Table[1](https://arxiv.org/html/2606.04525#S4.T1)\. We additionally verify that these aggregate statistics are robust to the choice of averaging scheme: macro\-averaged MCC \(weighting all 13 categories equally\) yields rankings that correlate with the micro\-averaged rankings reported here atρ=0\.988\\rho=0\.988\(Appendix[E\.4](https://arxiv.org/html/2606.04525#A5.SS4)\)\.

Table 1:Per\-category scaling correlations\.Spearman rank correlationρ\\rhobetweenlog10⁡\(parameter count\)\\log\_\{10\}\(\\text\{parameter count\}\)and macro\-MCC within each functional category \(n=40n=40models\)\.ρ\\rhonear\+1\+1indicates that larger models systematically outperform smaller ones;ρ\\rhonear0indicates no monotonic relationship between size and performance\. Thepp\-value tests whether the observedρ\\rhodiffers from zero; bolded values are significant atp<0\.05p<0\.05\. Rows sorted byρ\\rhodescending\. Scaling is significant in 11 of 13 categories, withρ\\rhoranging from0\.3470\.347\(DNA methylation\) to0\.5790\.579\(histone modifications\)\.
##### Architecture Comparison Under Controlled Conditions\.

To isolate architectural effects, we compare models matched by pretraining corpus \(multi\-species\) and tokenization \(BPE\), and focus on pairs where the remaining configuration differences are minimized\. Under these controlled conditions, Transformer models show substantial advantages over the state\-space model available in this controlled setting\. Specifically,Omni\-DNA\-1B\(Transformer\-decoder\) exceedseccDNAMamba\(Mamba\-SSM\) by\+0\.149\+0\.149macro\-MCC \(0\.568 vs\. 0\.419\), andGenomeOcean\-500Mshows a comparable\+0\.131\+0\.131gap over the same Mamba baseline \(0\.550 vs\. 0\.419\)\. Within Transformers, we also observe an encoder advantage in one matched comparison:GENA\-LM\-Large\-T2T\(Transformer\-encoder\) exceedsOmniNA\-220M\(Transformer\-decoder\) by\+0\.127\+0\.127MCC \(0\.552 vs\. 0\.425\) under matched multi\-species/BPE conditions\. These gaps hold under both micro\- and macro\-averaging \(Appendix[E\.4](https://arxiv.org/html/2606.04525#A5.SS4)\)\. The encoder\-vs\-decoder comparison, by contrast, is task\- and setting\-dependent\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/zip_images/iloveimg-compressed/main_hitmap.png)Figure 3:Model performance across task groups\.Heatmap shows full\-shot MCC averaged within each task group for 40 genomic foundation models, sorted by overall full\-shot macro\-average MCC\. Cell values report category\-level mean MCC, with colors ranging from red/orange for lower scores to green for higher scores\. The results reveal substantial task\-level heterogeneity: some categories, such as promoter, coding/non\-coding, and species\-classification tasks, are consistently easier, whereas DNA methylation, lncRNA, virus/phage, and regulatory tasks remain challenging\. This category\-specific structure shows that aggregate model rankings can hide important differences in downstream behavior\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/zip_images/iloveimg-compressed/image_radar_v2.png)Figure 4:Radar plots for category\-aware model selection\.Each subplot shows full\-shot macro\-MCC across the 13 GENEB task categories for a group of five models, grouped by overall macro\-MCC rank from strongest to weakest\. The plots expose category\-specific strengths not captured by aggregate rankings:Enformerhas a moderate overall rank but leads on TF binding \(0\.6980\.698\), enhancers \(0\.5390\.539\), and regulatory tasks \(0\.6040\.604\), and ranks second on mouse enhancers \(0\.6740\.674\) and third on chromatin accessibility \(0\.7110\.711\); theGenomeOceanfamily is particularly strong on virus/phage tasks; and plant\-oriented models such asPlantCaduceusandAgro\-NT\-1Bshow relative strength on lncRNA tasks\. These profiles motivate task\-specific model selection over global leaderboard position\.
##### Architecture Gaps Are Largest on Cross\-Species Regulatory Tasks\.

Architecture\-dependent gaps are particularly pronounced on tasks requiring cross\-species generalization \(Figure[3](https://arxiv.org/html/2606.04525#S4.F3); Figure[4](https://arxiv.org/html/2606.04525#S4.F4)\)\. On virus/phage,GenomeOcean\-500M\(Transformer\-decoder\) exceedseccDNAMamba\(Mamba\-SSM\) by\+0\.355\+0\.355macro\-MCC \(0\.6570\.657vs\.0\.3020\.302\)\. On mouse enhancers,Omni\-DNA\-1BexceedseccDNAMambaby\+0\.305\+0\.305\(0\.6750\.675vs\.0\.3700\.370\), andGENA\-LM\-Large\-T2TexceedsOmniNA\-220Mby\+0\.284\+0\.284under matched multi\-species/BPE conditions\. These category\-level gaps are several\-fold larger than the aggregate parameter\-tier gain \(\+0\.064\+0\.064macro\-MCC between models above11B and below200200M parameters\), reinforcing that architecture and pretraining alignment can exceed scale\-related differences on several categories\.

##### Chromatin Accessibility: A Domain Where SSM Models Become Competitive\.

Chromatin accessibility provides a notable exception to the general pattern of Transformer dominance\.eccDNAMamba\(Mamba\-SSM\) exceedsGenomeOcean\-500M\(Transformer\-decoder\) by\+0\.124\+0\.124macro\-MCC \(0\.5990\.599vs\.0\.4750\.475\) on this category\. BothJanusDNA\-72\-wandeccDNAMambaalso exhibit a substantial within\-model advantage on chromatin accessibility relative to their overall performance \(\+0\.200\+0\.200and\+0\.179\+0\.179MCC above their respective aggregate macro\-MCC\), a clear instance of category\-level specialization \(see Appendix[F\.11](https://arxiv.org/html/2606.04525#A6.SS11)\) within the SSM model family\. The controlled pretraining\-corpus analysis additionally shows a consistent\+0\.062\+0\.062macro\-MCC advantage of multi\-species over human\-only pretraining for chromatin accessibility \(66/66pairs; Table[2](https://arxiv.org/html/2606.04525#S4.T2)\)\. Within GENEB, these results suggest that chromatin accessibility may benefit from inductive biases captured by the evaluated SSM model family when combined with taxonomically diverse pretraining\.

Table 2:Transfer learning: human vs\. multi\-species pretraining\.Per\-categoryΔ\\DeltaMCC \(multi\-species−\-human\) averaged across 6 controlled model pairs matched on architecture, tokenization, and size \(±2×\\pm 2\\times\)\. For each pair, the within\-category MCC is computed by averaging over tasks; the table reports the cross\-pair average\. “Wins” indicates the number of pairs in which multi\-species exceeds human in that category\. “Overall” is the macro\-averaged aggregate \(Δ\\DeltaMCC averaged across the 13 categories\)\.Green: multi\-species advantage \(Δ\>\+0\.02\\Delta\>\+0\.02\)\.Gray: parity \(\|Δ\|≤0\.02\|\\Delta\|\\leq 0\.02\)\.Red: human advantage \(Δ<−0\.02\\Delta<\-0\.02\)\.

##### Tokenization Strategy Effects\.

Holding architecture and pretraining corpus fixed, we isolate the effect of tokenization scheme across 12 matched controlled pairs \(Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3.SSS0.Px5)\)\. No global ordering emerges; the preferred scheme varies with model family and pretraining setting\. Within Transformer\-decoders under matched multi\-species pretraining, BPE exceedskk\-mer on average \(\+0\.020\+0\.020macro\-MCC across 3 pairs\), but with substantial pair\-level variance:Omni\-DNA\-300MoutperformsGPT2\-Gene\-Multi\-v2by\+0\.071\+0\.071, whileOmniNA\-220Munderperforms the same baseline by−0\.042\-0\.042\. Within Transformer\-encoders, BPE andkk\-mer perform comparably \(\+0\.006\+0\.006across 5 pairs, with all pair\-level gaps below0\.020\.02MCC\)\. Under matched human pretraining, single\-nucleotide tokenization \(MutBERT\) exceeds BPE baselines in both available comparisons:\+0\.033\+0\.033overGENA\-LMand\+0\.038\+0\.038overGROVER\. Non\-standard vocabularies extend the same pattern:kk\-mer exceedsBioFM\-265M’s BioToken framework by\+0\.134\+0\.134in one Transformer\-decoder comparison, whileLucaOne’s mixed nucleotide–amino\-acid vocabulary exceedskk\-mer by\+0\.017\+0\.017over a matched Transformer\-encoder \(NT\-2\.5B\-MS\)\. Tokenization thus interacts with architecture, scale, and task structure rather than admitting a single global ordering in GENEB\.

##### Transfer Learning Analysis: Isolating Pretraining Corpus Effects\.

To disentangle pretraining data effects from architectural confounds, we restrict analysis to controlled pairs matched on architecture, tokenization, and size \(within±2×\\pm 2\\times; Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3)\)\.

Human vs\. multi\-species pretraining\(6 controlled pairs across Transformer\-encoders/decoders and BPE/kk\-mer\) shows a small aggregate effect: multi\-species pretraining yields an average\+0\.012\+0\.012macro\-MCC improvement over human\-only\. However, the effect is structured by task category \(Table[2](https://arxiv.org/html/2606.04525#S4.T2)\)\. Multi\-species pretraining shows its most consistent advantage on chromatin accessibility \(6/6 pairs;Δ=\+0\.062\\Delta=\+0\.062\), with additional positive shifts on splice sites \(Δ=\+0\.038\\Delta=\+0\.038\), species classification \(Δ=\+0\.031\\Delta=\+0\.031\), mouse enhancers \(Δ=\+0\.023\\Delta=\+0\.023\), and lncRNA \(Δ=\+0\.022\\Delta=\+0\.022\)\. In contrast, virus/phage tasks favor human\-only pretraining on average \(Δ=−0\.034\\Delta=\-0\.034\)\. Categories such as histone modifications, promoters, and TF binding are near parity under the\|Δ\|<0\.02\|\\Delta\|<0\.02criterion\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/zip_images/iloveimg-compressed/avg_mcc_fewshot.png)Figure 5:Few\-shot performance degradation\.Macro\-average MCC of genomic foundation models under full\-data, 10\-shot, and 1\-shot evaluation regimes\. Models are sorted by full\-data performance\. The top band reports the relative performance drop from full\-data to 10\-shot evaluation, highlighting the sensitivity of each model to limited supervision\.Multi\-species vs\. multi\-species\-microbial\(2 controlled pairs; Transformer\-encoder;kk\-mer\) yields the largest corpus effect observed in GENEB controlled comparisons\. Models pretrained on general multi\-species data that include eukaryotic genomes \(NT\-v2\-100M\-MS,Genomics\-FM\) exceed microbial\-focused pretraining \(DNABERT\-S\) by\+0\.084\+0\.084macro\-MCC on average \(2/2 pairs\)\. The largest category gaps align with known biological differences between microbial and eukaryotic genomes: splice sites \(Δ=\+0\.222\\Delta=\+0\.222\), species classification \(Δ=\+0\.130\\Delta=\+0\.130\), lncRNA \(Δ=\+0\.116\\Delta=\+0\.116\), and DNA methylation \(Δ=\+0\.108\\Delta=\+0\.108\)\. In GENEB, this is consistent with microbial\-focused pretraining being insufficient for eukaryotic genomic prediction tasks even under matched architectures and tokenization\.

Eukaryotic\-genes vs\. multi\-species\(1 controlled pair; 3B Transformer\-decoder;kk\-mer\) shows thatGENERator\-Eukaryote\-3B\(curated eukaryotic genes\) exceedsDNA\-GPT\-3B\-M\(broad multi\-species\) by\+0\.063\+0\.063macro\-MCC overall, with the largest advantages on chromatin accessibility \(\+0\.191\+0\.191\), lncRNA \(\+0\.142\+0\.142\), and mouse enhancers \(\+0\.124\+0\.124\)\. The only category favoring broad multi\-species in this comparison is regulatory element prediction \(−0\.040\-0\.040\)\. This conclusion rests on a single pair and may reflect model\-specific training choices \(Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3)\), but is consistent with corpus curation yielding gains beyond broad sequence diversity\.

What cannot be concluded from controlled comparisons\.Several pretraining corpus types lack matched architectural controls in the current model set and therefore cannot be attributed cleanly to data effects: plant\-genome models \(unique architectures and scale points\), human\-mouse epigenomic profile models \(Enformer,SPACE; CNN\-Transformer hybrids\), and the prokaryotic model \(Evo\-1\-131k; StripedHyena\)\. Apparent advantages or deficits for these groups should be interpreted as potentially confounded by architecture and training procedure rather than pretraining data alone\.

##### Few\-Shot Robustness Reveals an Inverse Performance Pattern\.

Across all 40 models, mean macro\-MCC degrades from0\.4880\.488\(full\-shot\) to0\.2530\.253\(10\-shot\) to0\.1060\.106\(1\-shot\) \(Figure[5](https://arxiv.org/html/2606.04525#S4.F5)\), corresponding to relative reductions of48\.2%48\.2\\%and78\.2%78\.2\\%, respectively\. Per\-model relative 10\-shot drops vary substantially, from35%35\\%forEvo\-1\-131kto65%65\\%forBioFM\-265M\(median48%48\\%\), reflecting both category\-level and model\-level effects that the aggregate trend does not capture\.

First, the degradation is structured by task category \(per\-category breakdown in Appendix[F](https://arxiv.org/html/2606.04525#A6)\)\. Promoter prediction retains38\.8%38\.8\\%of full\-shot macro\-MCC at 1\-shot and species classification retains30\.1%30\.1\\%, consistent with broadly distributed sequence\-composition signals \(GC content, codon usage,kk\-mer enrichment\) that pretraining captures and 1\-shot supervision preserves\. In contrast, three categories collapse to within0\.030\.03MCC of random performance at 1\-shot: virus/phage \(MCC1\-shot=0\.027\\text\{MCC\}\_\{\\text\{1\-shot\}\}=0\.027,93\.5%93\.5\\%drop\), DNA methylation \(0\.0150\.015,93\.2%93\.2\\%\), and lncRNA \(0\.0220\.022,91\.3%91\.3\\%\)\. These collapses align with under\-representation of viral genomes in eukaryotic pretraining corpora, the position\-specific nature of DNA methylation signals, and the low\-level sequence similarity of lncRNA to the broader non\-coding genome\.

Second, beyond this category\-level structure, few\-shot robustness is inversely aligned with full\-shot performance at the model level\. The five smallest absolute drops occur inEvo\-1\-131k\(Δ=0\.196\\Delta=0\.196\),Caduceus\-PH\-1k\(0\.2200\.220\),JanusDNA\-72\-wo\(0\.2720\.272\),Caduceus\-PS\-131k\(0\.2720\.272\), andJanusDNA\-72\-w\(0\.2750\.275\) – all among the weakest models in full\-shot\. Top full\-shot performers exhibit the opposite pattern:GENERator\-Eukaryote\-3B\(Δ=0\.489\\Delta=0\.489\),GENERator\-Eukaryote\-1\.2B\(0\.4630\.463\),LucaOne\(0\.4610\.461\), andNT\-2\.5B\-MS\(0\.4560\.456\) all exceed0\.420\.42in absolute drop\. This pattern does not indicate greater robustness in weaker models: a small absolute drop reflects a low full\-shot ceiling that leaves limited room for further degradation, not recovery of useful signal at 1\-shot\.Evo\-1\-131kis illustrative – its low ceiling stems from domain mismatch \(see Domain Mismatch paragraph below\), not from few\-shot regime itself\.

These observations expose a structural limitation of aggregate few\-shot leaderboards: they conflate category tractability with model quality, and reward models whose absolute drop is small because their full\-shot performance is already low\. Category\-resolved evaluation \(Appendix[F](https://arxiv.org/html/2606.04525#A6)\) is therefore the appropriate diagnostic in low\-supervision regimes\.

##### The Hard Frontier of GENEB: Where Scaling Does Not Help\.

A substantial fraction of GENEB tasks remains far from saturation:2828of100100tasks have mean MCC below0\.350\.35\(Figure[3](https://arxiv.org/html/2606.04525#S4.F3)\), dominated by 4mC methylation prediction \(*G\. subterraneus*0\.0610\.061;*E\. coli*0\.1030\.103;*G\. pickeringii*0\.1070\.107\) and plant lncRNA identification \(*S\. lycopersicum*0\.2210\.221;*G\. max*0\.2280\.228;*T\. aestivum*0\.2380\.238\)\. Even the strongest model on these categories,GENERator\-Eukaryote\-3B, reaches only0\.2060\.206–0\.4770\.477on 4mC tasks, andLucaOnereaches0\.4170\.417–0\.6290\.629on plant lncRNA\.

Category\-level scaling analysis \(Spearmanρ\\rhobetweenlog⁡\(params\)\\log\(\\text\{params\}\)and per\-category macro\-MCC across all 40 models; Table[1](https://arxiv.org/html/2606.04525#S4.T1)\) shows that this hardness is not purely a parameter\-count problem\. Eleven of the 13 categories show positive scaling significant atp<0\.05p<0\.05, but with substantial variation in strength \(ρ\\rhoranging from0\.3470\.347for DNA methylation to0\.5790\.579for histone modifications\)\. Two categories show no significant scaling: species classification \(ρ=0\.304\\rho=0\.304,p=0\.056p=0\.056\) and chromatin accessibility \(ρ=0\.245\\rho=0\.245,p=0\.128p=0\.128\)\. Even where scaling is statistically significant, the hard frontier of GENEB \(4mC methylation, plant lncRNA\) shows that scaling alone does not close the absolute performance gap; progress on these tasks will require complementary advances in pretraining\-corpus design, inductive biases, or task\-specific supervision\.

##### High\-Variance Tasks Reveal Decisive Design Patterns\.

The architectural and pretraining effects identified above are most pronounced on tasks where models disagree most: 13 GENEB tasks exhibit cross\-model standard deviation above0\.120\.12, indicating settings in which model choice has outsized impact on downstream performance \(Figure[6](https://arxiv.org/html/2606.04525#S4.F6)A\)\. Aggregating top\-3 and bottom\-3 placements across these tasks \(3939slots in each band\) reveals concentration effects \(Figure[6](https://arxiv.org/html/2606.04525#S4.F6)B\)\. By pretraining\-data type, multi\-species and eukaryotic\-gene pretraining together capture32/3932/39top\-3 placements \(2020and1212, respectively\), while human\-only pretraining is concentrated almost entirely in the bottom band \(29/3929/39bottom\-3, only1/391/39top\-3\)\. By architecture, the same imbalance holds: Transformer\-decoders and Transformer\-encoders together account for33/3933/39top\-3 placements \(1818and1515, respectively\), whereas Mamba\-SSM \(17/3917/39bottom\-3\), Hybrid\-Mamba\-MoE \(7/397/39\), and StripedHyena \(6/396/39\) dominate the bottom\. These high\-variance tasks thus operationalise the findings above into a practical diagnostic: in settings where GENEB models disagree most, pretraining scope and architectural family – not scale – predict whether a model will land in the top or bottom tier\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/smth2.png)Figure 6:High\-variance tasks reveal the role of pretraining data\.\(A\)GENEB tasks with cross\-model standard deviation above0\.120\.12, corresponding to settings where model selection most strongly affects downstream performance\.\(B\)Pretraining\-data composition of top\-3 and bottom\-3 placements across these tasks\. Multi\-species and eukaryotic\-gene pretraining dominate top placements, while human\-only, prokaryotic, and microbial pretraining are concentrated among bottom placements\. The result indicates that high\-variance tasks expose biologically meaningful differences in pretraining scope that are obscured by aggregate leaderboards\.
##### Domain Mismatch and Out\-of\-Domain Models\.

The poor aggregate performance ofEvo\-1\-131k– the largest model in GENEB \(77B parameters\), yet ranked among the weakest on TF binding, mouse enhancers, chromatin accessibility, species classification, and splice sites \(Figure[4](https://arxiv.org/html/2606.04525#S4.F4)\) – is best understood as a symptom of a broader issue: the GENEB task suite is heavily skewed toward eukaryotic genomic phenomena \(1212of1313categories\), creating a structural disadvantage for models pretrained on prokaryotic or microbial corpora\. This disadvantage is distinct from – and should not be conflated with – the effects of scale, architecture, or tokenization\. Recomputing the scale–performance correlation withoutEvo\-1\-131kraises Spearmanρ\\rhofrom0\.5650\.565to0\.6860\.686\(p<0\.001p<0\.001\) and widens the tier gap between models above11B and below200200M parameters from\+0\.064\+0\.064to\+0\.087\+0\.087macro\-MCC\.DNABERT\-S, pretrained on multi\-species microbial genomes, exhibits a milder version of the same pattern: overall macro\-MCC0\.4270\.427versus0\.5120\.512for the two eukaryotic multi\-species models under matched architecture and tokenization \(NT\-v2\-100M\-MS,Genomics\-FM; see Transfer Learning Analysis above\)\. We therefore treat aggregate rankings as a poor proxy for performance in prokaryotic or viral genomics, and encourage users of GENEB working in those domains to consult per\-task results directly rather than relying on the overall leaderboard\.

##### Practitioner Recommendations\.

The findings above motivate per\-category model selection rather than reliance on aggregate rankings \(Figure[4](https://arxiv.org/html/2606.04525#S4.F4)\)\. The per\-category recommendations below are organised around four recurring patterns\. For compact deployment under tight compute budgets,MutBERT\(86M, Transformer\-encoder, single\-nucleotide tokenization, human pretraining\) is the top sub\-100M model in 8 of 13 categories and the strongest≤100\\leq 100M model on overall macro\-MCC \(0\.5290\.529\)\. For epigenomic\-profile tasks \(TF binding, regulatory, enhancers\), the CNN–Transformer hybridsEnformerandSPACEare consistently among the top models, withEnformermatching the top modelOmni\-DNA\-1Bon mouse enhancers\. At the opposite end, hard regimes \(DNA methylation, plant lncRNA\) remain unsolved within GENEB: the best macro\-MCC is0\.4400\.440for DNA methylation \(GENERator\-Eukaryote\-3B\) and0\.5080\.508for plant lncRNA \(LucaOne\), with scaling weak for DNA methylation \(ρ=0\.347\\rho=0\.347, Table[1](https://arxiv.org/html/2606.04525#S4.T1)\)\. Finally, few\-shot evaluation reranks the best model in 8 of 13 categories, so the model winning under full supervision is not necessarily the model to deploy with only∼10\\sim 10labelled examples per task\.

Coding/non\-coding\.GENERator\-Eukaryote\-3B\(0\.9040\.904\) andLucaOne\(0\.9010\.901\) lead under full supervision; the most compact strong alternative isMutBERT\(0\.8940\.894, 86M\), which nearly matches the 3B\-parameter leader\. In 10\-shot deployments,MutBERTbecomes the top performer \(0\.7480\.748vs\.0\.6940\.694forGENERator\-Eukaryote\-3B\)\.

Promoter prediction\.GENERator\-Eukaryote\-3B\(0\.7740\.774\) leads, withGENERator\-Eukaryote\-1\.2B\(0\.7680\.768\) andOmni\-DNA\-1B\(0\.7590\.759\) close behind\. Under tighter compute budgets,Omni\-DNA\-300M\(0\.7400\.740\) andMutBERT\(0\.7390\.739, 86M\) provide strong compact options\.

Species classification\.The top three models –GenomeOcean\-4B\(0\.7620\.762\),GENERator\-Eukaryote\-3B\(0\.7610\.761\), andGENERator\-Eukaryote\-1\.2B\(0\.7570\.757\) – all exceed0\.750\.75and use multi\-species or eukaryotic\-gene pretraining\. For compute\-constrained deployment, theNT\-v2family \(NT\-v2\-250M\-MS0\.7470\.747,NT\-v2\-100M\-MS0\.7410\.741,NT\-v2\-50M\-3mer\-MS0\.7340\.734\) offers near\-equivalent performance at one\-tenth the parameter count\.

Chromatin accessibility\.GENERator\-Eukaryote\-3B\(0\.7280\.728\),Omni\-DNA\-1B\(0\.7140\.714\), andEnformer\(0\.7110\.711\) all exceed0\.710\.71;Enformer\(250M\) is the strongest sub\-300M option, andMutBERT\(86M,0\.6910\.691\) the strongest sub\-100M option\. The category also stands out as a setting in which the evaluated SSM model is competitive \(eccDNAMamba0\.5990\.599\)\. Under 10\-shot supervision the ranking reranks substantially:GENA\-LM\-Large\-T2Tbecomes the leader \(0\.5670\.567, down from0\.6450\.645at full\-shot\), whileGENERator\-Eukaryote\-3Bdrops to0\.5400\.540\.

TF binding\.Enformer\(0\.6980\.698\) leads by a wide margin, followed byOmni\-DNA\-1B\(0\.6470\.647\) andMutBERT\(0\.6460\.646, 86M\)\.MutBERTis therefore the recommended compact choice\.

Virus/phage classification\.GenomeOcean\-4B\(0\.6970\.697\) andGenomeOcean\-500M\(0\.6570\.657\) dominate this category, with a sharp drop toGENA\-LM\-Large\-T2T\(0\.5690\.569\); below 200M parameters performance drops further \(GROVER0\.5320\.532\)\.GenomeOcean\-4Bremains the 10\-shot leader \(0\.3770\.377\), though absolute scores collapse substantially\.

Mouse enhancers\.Omni\-DNA\-1B\(0\.6750\.675\) andEnformer\(0\.6740\.674\) are effectively tied at full supervision;Enformer\(250M\) is therefore the recommended compact choice\. Under 10\-shot supervision, however,SPACEbecomes the leader \(0\.3790\.379, from a full\-shot score of0\.6180\.618\), whileOmni\-DNA\-1Bdrops to0\.2530\.253– a substantial rerank that practitioners working with limited cross\-species labels should account for\.

Splice sites\.NT\-2\.5B\-MS\(0\.6520\.652\) andGENERator\-Eukaryote\-3B\(0\.6480\.648\) lead under full supervision\. The strongest sub\-300M alternative isEnformer\(0\.5860\.586\); the strongest sub\-100M alternative isMutBERT\(0\.5790\.579\)\. Microbial\-focused pretraining transfers poorly to this category, losing0\.2220\.222macro\-MCC to multi\-species in controlled comparisons \(see Transfer Learning Analysis above\)\. Under 10\-shot supervision,LucaOnebecomes the top model \(0\.2980\.298, from a full\-shot score of0\.6360\.636\), whileNT\-2\.5B\-MSdrops to0\.2420\.242\.

Regulatory\.Enformer\(0\.6040\.604\) andSPACE\(0\.5980\.598\) lead substantially, reflecting their epigenomic\-profile pretraining\. The strongest sub\-100M option isHyenaDNA\-Medium\-160k\(0\.4320\.432\)\. In 10\-shot,DNABERT\-2unexpectedly leads \(0\.2160\.216, from a full\-shot score of0\.4350\.435\), whileEnformerdrops to0\.1840\.184; we view this rerank as protocol\-fragile given the small absolute scores in this low\-data regime\.

Histone modifications\.TheGenomeOceanfamily leads \(GenomeOcean\-4B0\.5450\.545,GenomeOcean\-500M0\.5370\.537\), along withGENERator\-Eukaryote\-3B\(0\.5370\.537\)\. With 30 tasks, this is the largest category in GENEB and shows the strongest within\-category scaling \(ρ=0\.579\\rho=0\.579; Table[1](https://arxiv.org/html/2606.04525#S4.T1)\)\. The strongest sub\-100M option isMutBERT\(0\.5010\.501\)\. In 10\-shot,MutBERTalso leads \(0\.3000\.300\), making it the recommended choice across regimes when compute is constrained\.

Enhancers\.Enformer\(0\.5390\.539\) andSPACE\(0\.5260\.526\) lead, followed byMETAGENE\-1\(0\.5050\.505\)\. The strongest sub\-300M alternative isEnformeritself\. Under 10\-shot,GENA\-LM\-Large\-T2Tbecomes the top model \(0\.3720\.372, from0\.4880\.488at full\-shot\), a switch worth noting for low\-data deployments\.

lncRNA\.LucaOne\(0\.5080\.508\) leads by a substantial margin, followed byGENERator\-Eukaryote\-3B\(0\.4530\.453\) andGENERator\-Eukaryote\-1\.2B\(0\.4380\.438\)\. The strongest sub\-300M option isPlantCaduceus\(0\.3570\.357\), reflecting its plant\-specific pretraining\. This is a hard regime: no per\-task score exceeds0\.630\.63across any of the 6 plant lncRNA tasks, and 10\-shot performance is at or below0\.210\.21for every model\.

DNA methylation\.GENERator\-Eukaryote\-3B\(0\.4400\.440\) leads, withGENERator\-Eukaryote\-1\.2B\(0\.3970\.397\) andDNA\-GPT\-3B\-M\(0\.3670\.367\) following\. This is the hardest category in GENEB:66of88tasks have mean MCC below0\.250\.25; although scaling is statistically significant \(ρ=0\.347\\rho=0\.347,p=0\.028p=0\.028; Table[1](https://arxiv.org/html/2606.04525#S4.T1)\), no sub\-300M model exceeds0\.340\.34\. Few\-shot performance collapses to near\-random levels across all models, with no 1\-shot score exceeding0\.040\.04\.

## 5Conclusion

We introduced GENEB, a benchmark evaluating 40 genomic foundation models across 100 tasks from 13 functional categories under a unified linear\-probing protocol with macro\-MCC as the principal aggregation metric\. Our results show that while model scale shows a substantial aggregate association with performance \(ρ=0\.565\\rho=0\.565\), it remains an imperfect predictor of category\-level outcomes: architecture and pretraining alignment frequently offset substantial scale differences, and the model that wins under full supervision reranks under 10\-shot evaluation in 8 of 13 categories\. Transformer\-based models generally outperform the evaluated state\-space alternative, though domain\-specific exceptions exist \(e\.g\., chromatin accessibility\); tokenization effects interact with architecture rather than admitting a single global ordering; and microbial\-only corpora transfer poorly to eukaryotic tasks\.

Overall, these findings argue for category\-aware, controlled evaluation rather than aggregate leaderboards\. GENEB provides a reference framework to support principled model comparison and selection in genomic machine learning\.

## 6Limitations

GENEB has several limitations\.

Long\-range tasks\.GENEB underrepresents tasks requiring explicit modeling of very long\-range regulatory interactions \(\>10\>10kb\)\. As a result, models with explicit long\-context capability \(HyenaDNA\-Large\-1M,Caduceus\-PS\-131k,Evo\-1\-131k\) are not exercised on the regime where their architectural priors would most likely yield differentiating gains\. A detailed enumeration of considered\-but\-excluded long\-range datasets and model context\-length constraints is provided in Appendix[D\.3](https://arxiv.org/html/2606.04525#A4.SS3)\.

Task selection and curation\.Task selection is constrained by available datasets and existing benchmarks\. Some constituent tasks may be noisy or weakly defined, particularly in the hard regimes identified in Section[4](https://arxiv.org/html/2606.04525#S4)\(DNA methylation, plant lncRNA\), where label quality and supervision signal vary across sources\. Further task curation and refinement is needed as genomic benchmarks mature\.

Model coverage\.Not all genomic foundation models could be included due to unavailable weights, incompatible pipelines, or computational constraints\. Excluded models and inclusion criteria are discussed in Appendix[D](https://arxiv.org/html/2606.04525#A4)\.

Prokaryotic and viral task gap\.Of the 13 GENEB categories, only virus/phage classification reflects a non\-eukaryotic domain; prokaryotic gene prediction, microbial genome assembly verification, and CRISPR system characterization are not currently represented\. As a result, aggregate GENEB rankings are an unreliable proxy for performance in prokaryotic or viral genomics \(see Domain Mismatch paragraph, Section[4](https://arxiv.org/html/2606.04525#S4)\)\.

Frozen representations and pooling\-tokenization interactions\.GENEB evaluates frozen representations using linear probing, which enables controlled comparison of embedding quality across the 40\-model set but may underestimate the performance achievable with task\-specific fine\-tuning\. Empirical analysis in Appendix[E\.1](https://arxiv.org/html/2606.04525#A5.SS1)shows that model rankings under linear probing are highly consistent with those under non\-linear MLP probes; whether this stability extends to full task\-specific fine\-tuning remains an open question\. Additionally, single\-nucleotide andkk\-mer tokenizations produce substantially longer token sequences than BPE for a fixed input window, so the choice of pooling \(mean, attention\-weighted, or final\-token\) may favor different schemes\. We use mean pooling throughout; pooling\-tokenization interactions are not fully disentangled\.

Aggregate metric\.GENEB reports both micro\- and macro\-averaged MCC; histone modifications \(30 tasks\) and promoters \(22 tasks\) jointly account for over half of the per\-task evaluations, so micro\-averaging is structurally biased toward these two categories \(see Appendix[E\.4](https://arxiv.org/html/2606.04525#A5.SS4)\)\. We treat macro\-MCC as the principal aggregation, and recommend that category\-level results, rather than any single overall ranking, drive model selection\.

## 7Use of Large Language Models

Large language models were used as writing and editing assistants during the preparation of this manuscript\. Specifically, they were employed to improve clarity, organization, and phrasing of the text, as well as to assist with LaTeX formatting\. All experimental design, data processing, analysis, and interpretation of results were performed by the authors, and all reported findings were verified against the underlying benchmark outputs\.

## Acknowledgements

This work was supported by the Ministry of Economic Development of the Russian Federation \(agreement No\. 139\-15\-2025\-013, dated June 20, 2025, subsidy identifier 000000C313925P4B0002\)\.

## Impact Statement

GENEB is intended to improve the rigor of model comparison in genomic representation learning by replacing heterogeneous, single\-paper evaluations with a unified protocol across 40 models and 100 tasks\. We expect several positive effects on the field\. First, category\-aware evaluation reduces the risk that practitioners select models based on aggregate leaderboards that mask substantial heterogeneity across biological task types, particularly in clinically and agriculturally relevant domains \(e\.g\., regulatory element prediction, plant lncRNA, viral classification\)\. Second, our controlled comparisons isolate the contributions of architecture, tokenization, and pretraining corpus, which we hope will inform principled design of future genomic foundation models rather than indiscriminate scaling\.

We are mindful of several potential concerns\. Genomic models can in principle be applied to dual\-use research, including the design of pathogenic sequences; we believe a benchmark of representation quality on standard prediction tasks does not meaningfully shift this risk surface, but we note that responsible release practices for individual models remain the responsibility of their authors\. Additionally, GENEB inherits biases from its constituent datasets: the task suite is skewed toward eukaryotic and, within eukaryotes, toward human and well\-studied model organisms, which may under\-represent biologically and clinically important non\-model\-organism settings\. Users of GENEB should consult per\-task and per\-category results when applying findings to domains beyond those directly represented in the benchmark\.

## References

- Z\. Avsec, V\. Agarwal, D\. Visentin, J\. Ledsam, A\. Grabska\-Barwinska, K\. Taylor, Y\. Assael, J\. Jumper, P\. Kohli, and D\. Kelley \(2021\)Effective gene expression prediction from sequence by integrating long\-range interactions\.Nature Methods18,pp\. 1196–1203\.External Links:[Document](https://dx.doi.org/10.1038/s41592-021-01252-x)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px6.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Boshar, B\. Evans, Z\. Tang, A\. Picard, Y\. Adel, F\. K\. Lorbeer, C\. Rajesh, T\. Karch, S\. Sidbon, D\. Emms, J\. Mendoza\-Revilla, F\. Al\-Ani, E\. Seitz, Y\. Schiff, Y\. Bornachot, A\. Hernandez, M\. Lopez, A\. Laterre, K\. Beguir, P\. Koo, V\. Kuleshov, A\. Stark, B\. P\. de Almeida, and T\. Pierrot \(2025\)A foundational model for joint sequence\-function multi\-species modeling at scale for long\-range genomic prediction\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2025.12.22.695963),[Link](https://www.biorxiv.org/content/early/2025/12/25/2025.12.22.695963),https://www\.biorxiv\.org/content/early/2025/12/25/2025\.12\.22\.695963\.full\.pdfCited by:[§D\.1](https://arxiv.org/html/2606.04525#A4.SS1.SSS0.Px1.p1.1),[Table 5](https://arxiv.org/html/2606.04525#A4.T5.5.4.4.1)\.
- G\. Brixi, M\. G\. Durrant, J\. Ku, M\. Poli, G\. Brockman, D\. Chang, G\. A\. Gonzalez, S\. H\. King, D\. B\. Li, A\. T\. Merchant, M\. Naghipourfar, E\. Nguyen, C\. Ricci\-Tam, D\. W\. Romero, G\. Sun, A\. Taghibakshi, A\. Vorontsov, B\. Yang, M\. Deng, L\. Gorton, N\. Nguyen, N\. K\. Wang, E\. Adams, S\. A\. Baccus, S\. Dillmann, S\. Ermon, D\. Guo, R\. Ilango, K\. Janik, A\. X\. Lu, R\. Mehta, M\. R\.K\. Mofrad, M\. Y\. Ng, J\. Pannu, C\. Re, J\. C\. Schmok, J\. St\. John, J\. Sullivan, K\. Zhu, G\. Zynda, D\. Balsam, P\. Collison, A\. B\. Costa, T\. Hernandez\-Boussard, E\. Ho, M\. Liu, T\. McGrath, K\. Powell, D\. P\. Burke, H\. Goodarzi, P\. D\. Hsu, and B\. Hie \(2025a\)Genome modeling and design across all domains of life with evo 2\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.02.18.638918),[Link](https://www.biorxiv.org/content/early/2025/02/21/2025.02.18.638918),https://www\.biorxiv\.org/content/early/2025/02/21/2025\.02\.18\.638918\.full\.pdfCited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px2.p1.1)\.
- G\. Brixi, M\. G\. Durrant, J\. Ku, M\. Poli, G\. Brockman, D\. Chang, G\. A\. Gonzalez, S\. H\. King, D\. B\. Li, A\. T\. Merchant, M\. Naghipourfar, E\. Nguyen, C\. Ricci\-Tam, D\. W\. Romero, G\. Sun, A\. Taghibakshi, A\. Vorontsov, B\. Yang, M\. Deng, L\. Gorton, N\. Nguyen, N\. K\. Wang, E\. Adams, S\. A\. Baccus, S\. Dillmann, S\. Ermon, D\. Guo, R\. Ilango, K\. Janik, A\. X\. Lu, R\. Mehta, M\. R\.K\. Mofrad, M\. Y\. Ng, J\. Pannu, C\. Ré, J\. C\. Schmok, J\. St\. John, J\. Sullivan, K\. Zhu, G\. Zynda, D\. Balsam, P\. Collison, A\. B\. Costa, T\. Hernandez\-Boussard, E\. Ho, M\. Liu, T\. McGrath, K\. Powell, D\. P\. Burke, H\. Goodarzi, P\. D\. Hsu, and B\. L\. Hie \(2025b\)Genome modeling and design across all domains of life with evo 2\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.02.18.638918),[Link](https://www.biorxiv.org/content/early/2025/02/21/2025.02.18.638918),https://www\.biorxiv\.org/content/early/2025/02/21/2025\.02\.18\.638918\.full\.pdfCited by:[§D\.1](https://arxiv.org/html/2606.04525#A4.SS1.SSS0.Px4.p1.1),[Table 5](https://arxiv.org/html/2606.04525#A4.T5.5.16.16.1)\.
- W\. Cheng, Z\. Song, Y\. Zhang, S\. Wang, D\. Wang, M\. Yang, L\. Li, and J\. Ma \(2025\)DNALongBench: a benchmark suite for long\-range dna prediction tasks\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.01.06.631595),[Link](https://www.biorxiv.org/content/early/2025/01/08/2025.01.06.631595),https://www\.biorxiv\.org/content/early/2025/01/08/2025\.01\.06\.631595\.full\.pdfCited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px10.p2.1),[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px6.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px3.p1.1)\.
- H\. Dalla\-Torre, L\. Gonzalez, J\. Mendoza Revilla, N\. Lopez Carranza, A\. Henryk Grywaczewski, F\. Oteri, C\. Dallago, E\. Trop, H\. Sirelkhatim, G\. Richard,et al\.\(2023\)The nucleotide transformer: building and evaluating robust foundation models for human genomics\.bioRxiv,pp\. 2023–01\.Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px1.p1.2),[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px7.p1.1),[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px9.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px3.p1.1)\.
- Q\. Duan, B\. Huang, Z\. Song, I\. Lehmann, L\. Gu, R\. Eils, and B\. Wild \(2025\)JanusDNA: a powerful bi\-directional hybrid dna foundation model\.External Links:2505\.17257,[Link](https://arxiv.org/abs/2505.17257)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px4.p1.1),[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px7.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1)\.
- H\. Feng, L\. Wu, B\. Zhao, C\. Huff, J\. Zhang, J\. Wu, L\. Lin, P\. Wei, and C\. Wu \(2024\)Benchmarking dna foundation models for genomic sequence classification\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2024.08.16.608288),[Link](https://www.biorxiv.org/content/early/2024/08/18/2024.08.16.608288),https://www\.biorxiv\.org/content/early/2024/08/18/2024\.08\.16\.608288\.full\.pdfCited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px9.p2.1)\.
- V\. Fishman, Y\. Kuratov, M\. Petrov, A\. Shmelev, D\. Shepelin, N\. Chekanov, O\. Kardymon, and M\. Burtsev \(2023\)GENA\-lm: a family of open\-source foundational models for long dna sequences\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2023.06.12.544594),[Link](https://www.biorxiv.org/content/early/2023/06/13/2023.06.12.544594),https://www\.biorxiv\.org/content/early/2023/06/13/2023\.06\.12\.544594\.full\.pdfCited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px1.p1.2),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Gao, Q\. Liu, W\. Zeng, W\. H\. Wong, and R\. Jiang \(2023\)EpiGePT: a pretrained transformer model for epigenomics\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2023.07.15.549134),[Link](https://www.biorxiv.org/content/early/2023/07/18/2023.07.15.549134),https://www\.biorxiv\.org/content/early/2023/07/18/2023\.07\.15\.549134\.full\.pdfCited by:[§D\.1](https://arxiv.org/html/2606.04525#A4.SS1.SSS0.Px2.p1.1),[Table 5](https://arxiv.org/html/2606.04525#A4.T5.5.7.7.1)\.
- K\. Gresova, V\. Martinek, D\. Cechak, P\. Simecek, and P\. Alexiou \(2022\)Genomic benchmarks: a collection of datasets for genomic sequence classification\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2022.06.08.495248),[Link](https://www.biorxiv.org/content/early/2022/06/10/2022.06.08.495248),https://www\.biorxiv\.org/content/early/2022/06/10/2022\.06\.08\.495248\.full\.pdfCited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px9.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px3.p1.1)\.
- Y\. He, P\. Fang, Y\. Shan, Y\. Pan, Y\. Wei, Y\. Chen, Y\. Chen, Y\. Liu, Z\. Zeng, Z\. Zhou, F\. Zhu, E\. C\. Holmes, J\. Ye, J\. Li, Y\. Shu, M\. Shi, and Z\. Li \(2024\)LucaOne: generalized biological foundation model with unified nucleic acid and protein language\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2024.05.10.592927),[Link](https://www.biorxiv.org/content/early/2024/05/14/2024.05.10.592927),https://www\.biorxiv\.org/content/early/2024/05/14/2024\.05\.10\.592927\.full\.pdfCited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px5.p1.1)\.
- H\. Li, S\. Dey, B\. C\. Kwon, M\. Danziger, M\. Rosen\-Tzvi, J\. Hu, J\. Kozloski, C\. Tsou, B\. Dandala, and P\. Meyer \(2025a\)BMFM\-dna: a snp\-aware dna foundation model to capture variant effects\.External Links:2507\.05265,[Link](https://arxiv.org/abs/2507.05265)Cited by:[§D\.1](https://arxiv.org/html/2606.04525#A4.SS1.SSS0.Px3.p1.1),[Table 5](https://arxiv.org/html/2606.04525#A4.T5.5.14.14.1)\.
- S\. Li, Z\. Wang, Z\. Liu, D\. Wu, C\. Tan, J\. Zheng, Y\. Huang, and S\. Z\. Li \(2024\)VQDNA: unleashing the power of vector quantization for multi\-species genomic sequence modeling\.External Links:2405\.10812,[Link](https://arxiv.org/abs/2405.10812)Cited by:[Table 5](https://arxiv.org/html/2606.04525#A4.T5.5.13.13.1)\.
- Z\. Li, V\. Subasri, Y\. Shen, D\. Li, Y\. Zhao, G\. Stan, and C\. Shan \(2025b\)Omni\-dna: a unified genomic foundation model for cross\-modal and multi\-task learning\.External Links:2502\.03499,[Link](https://arxiv.org/abs/2502.03499)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Liu, J\. Li, and Z\. Zhang \(2025\)EccDNAMamba: a pre\-trained model for ultra\-long eccdna sequence analysis\.External Links:2506\.18940,[Link](https://arxiv.org/abs/2506.18940)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1)\.
- W\. Long, H\. Su, J\. Xiong, and Y\. Zhang \(2025\)MutBERT: probabilistic genome representation improves genomics foundation models\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.01.23.634452),[Link](https://www.biorxiv.org/content/early/2025/01/25/2025.01.23.634452),https://www\.biorxiv\.org/content/early/2025/01/25/2025\.01\.23\.634452\.full\.pdfCited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px8.p1.1)\.
- M\. Ma, G\. Liu, C\. Cao, P\. Deng, T\. Dao, A\. Gu, P\. Jin, Z\. Yang, Y\. Xia, R\. Luo, P\. Hu, Z\. Wang, Y\. Chen, H\. Liu, and T\. Qin \(2025\)HybriDNA: a hybrid transformer\-mamba2 long\-range dna language model\.External Links:2502\.10807,[Link](https://arxiv.org/abs/2502.10807)Cited by:[§D\.1](https://arxiv.org/html/2606.04525#A4.SS1.SSS0.Px2.p1.1),[Table 5](https://arxiv.org/html/2606.04525#A4.T5.5.11.11.1)\.
- A\. Malusare, H\. Kothandaraman, D\. Tamboli, N\. A\. Lanman, and V\. Aggarwal \(2024\)Understanding the natural language of dna using encoder–decoder foundation models with byte\-level precision\.Bioinformatics Advances4\(1\)\.External Links:ISSN 2635\-0041,[Link](http://dx.doi.org/10.1093/bioadv/vbae117),[Document](https://dx.doi.org/10.1093/bioadv/vbae117)Cited by:[§D\.1](https://arxiv.org/html/2606.04525#A4.SS1.SSS0.Px2.p1.1),[Table 5](https://arxiv.org/html/2606.04525#A4.T5.5.8.8.1)\.
- F\. I\. Marin, F\. Teufel, M\. Horlacher, D\. Madsen, D\. Pultz, O\. Winther, and W\. Boomsma \(2024\)BEND: benchmarking dna language models on biologically meaningful tasks\.External Links:2311\.12570,[Link](https://arxiv.org/abs/2311.12570)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px10.p2.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px3.p1.1)\.
- S\. M\. Mclaughlin, S\. H\. Ahanger, and D\. A\. Lim \(2024\)Nucleotide gpt: sequence\-based deep learning prediction of nuclear subcompartment\-associated genome architecture\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2024.11.27.625761),[Link](https://www.biorxiv.org/content/early/2024/12/03/2024.11.27.625761),https://www\.biorxiv\.org/content/early/2024/12/03/2024\.11\.27\.625761\.full\.pdfCited by:[§D\.1](https://arxiv.org/html/2606.04525#A4.SS1.SSS0.Px2.p1.1),[Table 5](https://arxiv.org/html/2606.04525#A4.T5.5.6.6.1)\.
- A\. Medvedev, K\. Viswanathan, P\. Kanithi, K\. Vishniakov, P\. Munjal, C\. Christophe, M\. A\. Pimentel, R\. Rajan, and S\. Khan \(2025\)BioToken and biofm – biologically\-informed tokenization enables accurate and efficient genomic foundation models\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.03.27.645711),[Link](https://www.biorxiv.org/content/early/2025/04/01/2025.03.27.645711),https://www\.biorxiv\.org/content/early/2025/04/01/2025\.03\.27\.645711\.full\.pdfCited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px7.p1.1)\.
- J\. Mendoza\-Revilla, E\. Trop, L\. Gonzalez, M\. Roller, H\. Dalla\-Torre, B\. P\. de Almeida, G\. Richard, J\. Caton, N\. Lopez Carranza, M\. Skwark,et al\.\(2023\)A foundational large language model for edible plant genomes\.bioRxiv,pp\. 2023–10\.Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px8.p1.1)\.
- N\. Muennighoff, N\. Tazi, L\. Magne, and N\. Reimers \(2023\)MTEB: massive text embedding benchmark\.External Links:2210\.07316,[Link](https://arxiv.org/abs/2210.07316)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px12.p3.1),[§1](https://arxiv.org/html/2606.04525#S1.p4.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px5.p1.1)\.
- E\. Nguyen, M\. Poli, M\. G\. Durrant, B\. Kang, D\. Katrekar, D\. B\. Li, L\. J\. Bartie, A\. W\. Thomas, S\. H\. King, G\. Brixi, J\. Sullivan, M\. Y\. Ng, A\. Lewis, A\. Lou, S\. Ermon, S\. A\. Baccus, T\. Hernandez\-Boussard, C\. Ré, P\. D\. Hsu, and B\. L\. Hie \(2024\)Sequence modeling and design from molecular to genome scale with evo\.Science386\(6723\),pp\. eado9336\.External Links:[Document](https://dx.doi.org/10.1126/science.ado9336),[Link](https://www.science.org/doi/abs/10.1126/science.ado9336)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2606.04525#S1.p2.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1)\.
- E\. Nguyen, M\. Poli, M\. Faizi, A\. Thomas, C\. Birch\-Sykes, M\. Wornow, A\. Patel, C\. Rabideau, S\. Massaroli, Y\. Bengio, S\. Ermon, S\. A\. Baccus, and C\. Ré \(2023\)HyenaDNA: long\-range genomic sequence modeling at single nucleotide resolution\.External Links:2306\.15794,[Link](https://arxiv.org/abs/2306.15794)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px3.p1.1),[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px7.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1)\.
- P\. F\. Nisantzis, C\. Gonçalves, and G\. de Polavieja \(2025\)C\.la\.p\.: enhancing transformer\-based genomic signal modeling by integrating dna sequences and chromatin accessibility data\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.02.19.638643),[Link](https://www.biorxiv.org/content/early/2025/02/23/2025.02.19.638643),https://www\.biorxiv\.org/content/early/2025/02/23/2025\.02\.19\.638643\.full\.pdfCited by:[§D\.1](https://arxiv.org/html/2606.04525#A4.SS1.SSS0.Px2.p1.1),[Table 5](https://arxiv.org/html/2606.04525#A4.T5.5.10.10.1)\.
- L\. Qiao, P\. Ye, Y\. Ren, W\. Bai, C\. Liang, X\. Ma, N\. Dong, and W\. Ouyang \(2024\)Model decides how to tokenize: adaptive dna sequence tokenization with mxdna\.External Links:2412\.13716,[Link](https://arxiv.org/abs/2412.13716)Cited by:[§D\.1](https://arxiv.org/html/2606.04525#A4.SS1.SSS0.Px3.p1.1),[Table 5](https://arxiv.org/html/2606.04525#A4.T5.5.12.12.1)\.
- G\. Richard, B\. P\. de Almeida, H\. Dalla\-Torre, C\. Blum, L\. Hexemer, P\. Pandey, S\. Laurent, M\. Lopez, A\. Laterre, M\. Lang, U\. Şahin, K\. Beguir, and T\. Pierrot \(2024\)ChatNT: a multimodal conversational agent for dna, rna and protein tasks\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2024.04.30.591835),[Link](https://www.biorxiv.org/content/early/2024/05/02/2024.04.30.591835),https://www\.biorxiv\.org/content/early/2024/05/02/2024\.04\.30\.591835\.full\.pdfCited by:[§D\.1](https://arxiv.org/html/2606.04525#A4.SS1.SSS0.Px5.p1.1),[Table 5](https://arxiv.org/html/2606.04525#A4.T5.5.18.18.1)\.
- M\. Sanabria, J\. Hirsch, P\. Joubert, and A\. Poetsch \(2024\)DNA language model grover learns sequence context in the human genome\.Nature Machine Intelligence6,pp\. 1–13\.External Links:[Document](https://dx.doi.org/10.1038/s42256-024-00872-0)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px1.p1.2),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Schiff, C\. Kao, A\. Gokaslan, T\. Dao, A\. Gu, and V\. Kuleshov \(2024\)Caduceus: bi\-directional equivariant long\-range dna sequence modeling\.External Links:2403\.03234,[Link](https://arxiv.org/abs/2403.03234)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1)\.
- X\. Shen and X\. Li \(2024\)OmniNA: a foundation model for nucleotide sequences\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2024.01.14.575543),[Link](https://www.biorxiv.org/content/early/2024/01/15/2024.01.14.575543),https://www\.biorxiv\.org/content/early/2024/01/15/2024\.01\.14\.575543\.full\.pdfCited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px5.p1.1)\.
- K\. Vishniakov, B\. B\. Amor, E\. Tekin, N\. A\. ElNaker, K\. Viswanathan, A\. Medvedev, A\. Singh, M\. Nadeem, M\. A\. Sayeed, P\. Kanithi, T\. Magalhaes, N\. Vassilieva, D\. Mahapatra, M\. Pimentel, and S\. Khan \(2025a\)Gene42: long\-range genomic foundation model with dense attention\.External Links:2503\.16565,[Link](https://arxiv.org/abs/2503.16565)Cited by:[§D\.1](https://arxiv.org/html/2606.04525#A4.SS1.SSS0.Px1.p1.1),[Table 5](https://arxiv.org/html/2606.04525#A4.T5.5.3.3.1)\.
- K\. Vishniakov, B\. B\. Amor, E\. Tekin, N\. A\. ElNaker, K\. Viswanathan, A\. Medvedev, A\. Singh, M\. Nadeem, M\. A\. Sayeed, P\. Kanithi, T\. Magalhaes, N\. Vassilieva, D\. Mahapatra, M\. Pimentel, and S\. Khan \(2025b\)Gene42: long\-range genomic foundation model with dense attention\.External Links:2503\.16565,[Link](https://arxiv.org/abs/2503.16565)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px4.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Wang, Y\. Jiao, Y\. Yu, T\. Qian, S\. Chen, J\. Chen, and Y\. Jiang \(2025a\)OmniGenBench: a benchmark for omnipotent multimodal generation across 50\+ tasks\.External Links:2505\.18775,[Link](https://arxiv.org/abs/2505.18775)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px11.p2.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px4.p1.1)\.
- Y\. Wang, Z\. Cai, Q\. Zeng, Y\. Gao, J\. Ouyang, Y\. Xu, S\. Yang, S\. He, Y\. Nie, Y\. Cai, F\. Zhou, C\. Jin, X\. Wang, Z\. Xie, D\. Zhu, T\. Xie, K\. Cheng, C\. Yang, X\. Fu, J\. Wang, K\. Zhang, J\. Yao, R\. Rabadan, and H\. Chen \(2025b\)Genomic touchstone: benchmarking genomic language models in the context of the central dogma\.bioRxiv preprint bioRxiv:2025\.06\.25\.661622\.Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px11.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px4.p1.1)\.
- W\. Wu, Q\. Li, M\. Li, K\. Fu, F\. Feng, J\. Ye, H\. Xiong, and Z\. Wang \(2025\)GENERator: a long\-context generative genomic foundation model\.External Links:2502\.07272,[Link](https://arxiv.org/abs/2502.07272)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px2.p1.1)\.
- H\. Yang, J\. Cole, Y\. Li, R\. Chen, G\. Min, and K\. Li \(2025a\)OmniGenBench: a modular platform for reproducible genomic foundation models benchmarking\.External Links:2505\.14402,[Link](https://arxiv.org/abs/2505.14402)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px9.p2.1)\.
- H\. Yang, M\. Chen, H\. Huang, J\. Duan, J\. Cao, Z\. Zhou, and R\. He \(2025b\)HAD: hybrid architecture distillation outperforms teacher in genomic sequence modeling\.External Links:2505\.20836,[Link](https://arxiv.org/abs/2505.20836)Cited by:[§D\.1](https://arxiv.org/html/2606.04525#A4.SS1.SSS0.Px2.p1.1),[Table 5](https://arxiv.org/html/2606.04525#A4.T5.5.9.9.1)\.
- Z\. Yang, J\. Zhu, and B\. Su \(2025c\)SPACE: your genomic profile predictor is a powerful dna foundation model\.External Links:2506\.01833,[Link](https://arxiv.org/abs/2506.01833)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px6.p1.1)\.
- P\. Ye, W\. Bai, Y\. Ren, W\. Li, L\. Qiao, C\. Liang, L\. Wang, Y\. Cai, J\. Sun, Z\. Yang, P\. Zheng, N\. Dong, T\. Chen, Z\. Wang, X\. Liu, X\. Ma, H\. Yan, Z\. Wang, S\. Wang, and W\. Ouyang \(2024\)Genomics\-fm: universal foundation model for versatile and data\-efficient functional genomic analysis\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2024.07.16.603653),[Link](https://www.biorxiv.org/content/early/2024/07/19/2024.07.16.603653),https://www\.biorxiv\.org/content/early/2024/07/19/2024\.07\.16\.603653\.full\.pdfCited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px8.p1.1)\.
- J\. Zhai, A\. Gokaslan, Y\. Schiff, A\. Berthel, Z\. Liu, W\. Lai, Z\. R\. Miller, A\. Scheben, M\. C\. Stitzer, M\. C\. Romay, E\. S\. Buckler, and V\. Kuleshov \(2025\)Cross\-species modeling of plant genomes at single\-nucleotide resolution using a pretrained dna language model\.Proceedings of the National Academy of Sciences122\(24\),pp\. e2421738122\.External Links:[Document](https://dx.doi.org/10.1073/pnas.2421738122),[Link](https://www.pnas.org/doi/abs/10.1073/pnas.2421738122),https://www\.pnas\.org/doi/pdf/10\.1073/pnas\.2421738122Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px8.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px2.p1.1)\.
- D\. Zhang, W\. Zhang, Y\. Zhao, J\. Zhang, B\. He, C\. Qin, and J\. Yao \(2023\)DNAGPT: a generalized pre\-trained tool for versatile dna sequence analysis tasks\.External Links:2307\.05628,[Link](https://arxiv.org/abs/2307.05628)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2606.04525#S1.p2.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1)\.
- X\. Zhang, M\. Yang, X\. Yin, Y\. Qian, and F\. Sun \(2024\)DeepGene: an efficient foundation model for genomics based on pan\-genome graph transformer\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2024.04.24.590879),[Link](https://www.biorxiv.org/content/early/2024/04/28/2024.04.24.590879),https://www\.biorxiv\.org/content/early/2024/04/28/2024\.04\.24\.590879\.full\.pdfCited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px8.p1.1)\.
- Z\. Zhou, Y\. Ji, W\. Li, P\. Dutta, R\. Davuluri, and H\. Liu \(2024a\)DNABERT\-2: efficient foundation model and benchmark for multi\-species genome\.External Links:2306\.15006,[Link](https://arxiv.org/abs/2306.15006)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px1.p1.2),[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px7.p1.1),[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px9.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px3.p1.1)\.
- Z\. Zhou, R\. Riley, S\. Kautsar, W\. Wu, R\. Egan, S\. Hofmeyr, S\. Goldhaber\-Gordon, M\. Yu, H\. Ho, F\. Liu, F\. Chen, R\. Morgan\-Kiss, L\. Shi, H\. Liu, and Z\. Wang \(2025\)GenomeOcean: an efficient genome foundation model trained on large\-scale metagenomic assemblies\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.01.30.635558),[Link](https://www.biorxiv.org/content/early/2025/02/05/2025.01.30.635558),https://www\.biorxiv\.org/content/early/2025/02/05/2025\.01\.30\.635558\.full\.pdfCited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px2.p1.1),[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px7.p1.1),[§1](https://arxiv.org/html/2606.04525#S1.p2.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2606.04525#S2.SS0.SSS0.Px2.p1.1)\.
- Z\. Zhou, W\. Wu, H\. Ho, J\. Wang, L\. Shi, R\. V\. Davuluri, Z\. Wang, and H\. Liu \(2024b\)DNABERT\-s: pioneering species differentiation with species\-aware dna embeddings\.External Links:2402\.08777,[Link](https://arxiv.org/abs/2402.08777)Cited by:[Appendix A](https://arxiv.org/html/2606.04525#A1.SS0.SSS0.Px8.p1.1)\.

## Appendix ARelated Work

The rapid development of foundation models for genomics has led to diverse architectural and methodological innovations \(see Figure[7](https://arxiv.org/html/2606.04525#A1.F7)\)\. We organize prior work by architectural design, tokenization, pretraining strategy, and benchmark scope\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/zip_images/iloveimg-compressed/fm_models.png)Figure 7:Architectural taxonomy of genomic foundation models\.##### Transformer\-Based Encoder Models\.

Early DNA language models predominantly adopted BERT\-style encoder architectures with masked language modeling \(MLM\) objectives\. DNABERT\-2\(Zhouet al\.,[2024a](https://arxiv.org/html/2606.04525#bib.bib2)\)addressed computational inefficiencies ofkk\-mer tokenization by replacing it with Byte\-Pair Encoding \(BPE\), achieving comparable performance with 21×\\timesfewer parameters\. The Nucleotide Transformer\(Dalla\-Torreet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib18)\)scaled this paradigm to 2\.5B parameters, demonstrating that multi\-species pretraining improves cross\-species generalization\. GENA\-LM\(Fishmanet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib16)\)extended context lengths to 36kb using sparse attention mechanisms \(BigBird\), while GROVER\(Sanabriaet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib6)\)introduced frequency\-balanced BPE vocabularies to mitigate the “rare word problem” in genomic sequences\.

##### Autoregressive and Generative Models\.

Decoder\-only architectures have gained prominence for their generative capabilities and unified training objectives\. DNA\-GPT\(Zhanget al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib13)\)introduced a comprehensive token language incorporating sequence, numerical, and instruction tokens for multi\-task genomic analysis\. Evo\(Nguyenet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib15)\)achieved genome\-scale sequence modeling using the StripedHyena architecture, processing up to 131kb contexts with single\-nucleotide resolution, with Evo 2\(Brixiet al\.,[2025a](https://arxiv.org/html/2606.04525#bib.bib23)\)extending this to all domains of life\. GENERator\(Wuet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib5)\)demonstrated that gene\-focused pretraining on eukaryotic sequences outperforms whole\-genome approaches for both understanding and generation tasks\. GenomeOcean\(Zhouet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib7)\)pioneered training on metagenomic assemblies, capturing rare biosphere diversity across 645Gbp of environmental sequences\. Omni\-DNA\(Liet al\.,[2025b](https://arxiv.org/html/2606.04525#bib.bib10)\)unified cross\-modal tasks \(DNA\-to\-text, DNA\-to\-image\) within a single autoregressive framework through vocabulary expansion\.

##### State Space Models and Efficient Architectures\.

To address the quadratic complexity of attention mechanisms, several works have explored sub\-quadratic alternatives\. HyenaDNA\(Nguyenet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib17)\)employed implicit long convolutions via the Hyena operator, enabling processing of sequences up to 1M nucleotides with𝒪​\(L​log⁡L\)\\mathcal\{O\}\(L\\log L\)complexity\. Caduceus\(Schiffet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib12)\)built upon the Mamba selective state space model, introducing bidirectional processing and reverse\-complement \(RC\) equivariance to respect DNA’s double\-stranded nature\. eccDNAMamba\(Liuet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib14)\)adapted the Mamba\-2 architecture specifically for extrachromosomal circular DNA, incorporating circular topology awareness through specialized augmentation strategies\.

##### Hybrid Architectures\.

Recent work has combined the strengths of different architectural paradigms\. JanusDNA\(Duanet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib8)\)introduced a bidirectional Mamba\-Attention\-MoE hybrid that achieves both autoregressive efficiency and bidirectional understanding, processing up to 1M base pairs at single\-nucleotide resolution\. Gene42\(Vishniakovet al\.,[2025b](https://arxiv.org/html/2606.04525#bib.bib25)\)extended dense attention to 192kb through incremental pretraining with adjusted RoPE frequencies\.

##### Multi\-Modal and Multi\-Species Models\.

Several recent models aim for broader biological understanding through multi\-modal integration\. LucaOne\(Heet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib30)\)unified nucleic acid and protein modeling within a single 1\.8B\-parameter encoder using a shared 39\-token vocabulary, demonstrating emergent understanding of the central dogma\. OmniNA\(Shen and Li,[2024](https://arxiv.org/html/2606.04525#bib.bib20)\)integrated nucleotide sequences with textual annotations, enabling natural language\-based genomic analysis across 17 diverse tasks\.

##### Long\-Range Genomic Modeling\.

Modeling distal regulatory interactions requires extended context lengths\. Enformer\(Avsecet al\.,[2021](https://arxiv.org/html/2606.04525#bib.bib1)\)combined convolutional towers with 11 transformer layers to capture interactions up to 100kb, significantly improving gene expression prediction from sequence\. DNALongBench\(Chenget al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib24)\)provides a comprehensive benchmark for evaluating long\-range dependencies up to 1M base pairs\. SPACE\(Yanget al\.,[2025c](https://arxiv.org/html/2606.04525#bib.bib19)\)proposed supervised pretraining on genomic profile prediction as an alternative to self\-supervised approaches, employing Mixture\-of\-Experts \(MoE\) for cross\-species modeling and profile\-grouped decoders for capturing inter\-track dependencies\.

##### Tokenization Strategies\.

Tokenization choices significantly impact model performance and efficiency\. Single\-nucleotide tokenization\(Nguyenet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib17); Duanet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib8)\)preserves fine\-grained resolution critical for SNP analysis but increases sequence length\.KK\-mer approaches\(Dalla\-Torreet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib18)\)capture local context but suffer from vocabulary explosion and sensitivity to mutations\. BPE\-based methods\(Zhouet al\.,[2024a](https://arxiv.org/html/2606.04525#bib.bib2),[2025](https://arxiv.org/html/2606.04525#bib.bib7)\)balance efficiency and biological relevance by learning data\-driven vocabularies\. BioToken\(Medvedevet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib27)\)introduced biologically\-informed tokenization that explicitly encodes structural annotations \(exon/intron boundaries\) and variant information within the token vocabulary, achieving strong performance with only 265M parameters\.

##### Specialized Domains\.

Beyond general\-purpose models, specialized architectures address specific genomic applications\. DNABERT\-S\(Zhouet al\.,[2024b](https://arxiv.org/html/2606.04525#bib.bib4)\)targets species differentiation through contrastive learning with Manifold Instance Mixup, doubling clustering performance over baselines\. DeepGene\(Zhanget al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib28)\)incorporates pan\-genome graph representations via Minigraph to capture population\-level genetic diversity\. MutBERT\(Longet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib31)\)represents the genome as probabilistic distributions over allele frequencies, explicitly modeling population\-level SNP variation\. Genomics\-FM\(Yeet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib29)\)employs an ensemble vocabulary strategy, pretraining on multiple tokenizations simultaneously and selectively activating task\-appropriate vocabularies during fine\-tuning\. PlantCaduceus\(Zhaiet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib22)\)adapts the Caduceus architecture for plant genomics, demonstrating remarkable cross\-species transferability across 160My of evolutionary divergence\. AgroNT\(Mendoza\-Revillaet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib11)\)provides a specialized foundation model for edible plant genomes\.

##### Benchmarks for Genomic Foundation Models\.

Standardized evaluation is essential for assessing the generalization capabilities of genomic foundation models, yet existing benchmarks typically focus on limited task families or biological domains\. The Nucleotide Transformer benchmark\(Dalla\-Torreet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib18)\)introduced a suite of DNA classification tasks spanning histone mark occupancy, regulatory element identification, and splice site prediction, establishing a common evaluation baseline for early Transformer\-based models\. The Genome Understanding Evaluation \(GUE/GUE\+\)\(Zhouet al\.,[2024a](https://arxiv.org/html/2606.04525#bib.bib2)\)extended this paradigm to multi\-species settings, including human, mouse, virus, and yeast, and to variable sequence lengths ranging from tens to thousands of base pairs\. Genomic Benchmarks\(Gresovaet al\.,[2022](https://arxiv.org/html/2606.04525#bib.bib45)\)further emphasized regulatory element classification, such as enhancers, promoters, coding/non\-coding regions, and open chromatin, across diverse organisms\.

Beyond these core resources, additional benchmarks target specific biological regimes\. Plant\-focused benchmarks evaluate long non\-coding RNA classification and transcriptional activity in crop genomes\(Yanget al\.,[2025a](https://arxiv.org/html/2606.04525#bib.bib46)\), while DNA methylation datasets assess epigenetic modifications \(4mC, 5mC, 6mA\) across a range of taxa\(Fenget al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib47)\)\. Collectively, these benchmarks have enabled substantial progress, but they remain fragmented in task scope, preprocessing assumptions, and evaluation protocols\.

##### Limitations of Existing Benchmarking Practices\.

A common limitation across prior benchmarks is the restricted set of evaluated models\. Most studies assess a small number of representative architectures, often selected for convenience or compatibility with a specific benchmark, rather than attempting exhaustive cross\-model evaluation\. As a result, comparisons across papers are difficult to interpret, and it is often unclear which models constitute appropriate baselines for a given task\. Moreover, reported state\-of\-the\-art results frequently depend on benchmark\-specific design choices, obscuring task\-level trade\-offs and failure modes\.

Recent benchmarking efforts such as BEND\(Marinet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib21)\)and DNALongBench\(Chenget al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib24)\)have highlighted important evaluation issues, including biologically meaningful task design and long\-range dependency modeling\. However, these benchmarks still evaluate limited model subsets and do not provide a unified performance matrix across a broad landscape of genomic foundation models\.

##### Comparative Benchmarking Studies\.

Several recent works have attempted broader comparative analysis of genomic foundation models\. For example,Wanget al\.\([2025b](https://arxiv.org/html/2606.04525#bib.bib49)\)evaluate approximately ten representative model families, primarily centered on human functional annotation tasks\. While such studies provide useful snapshots, the evaluated model sets remain limited and exclude many recently proposed architectures\. In addition, results are typically reported in static tables without unified leaderboards, making it difficult to place new models in a broader performance context\. As a consequence, researchers often lack clear guidance on which baselines are appropriate for a given task and how to position new models relative to existing work\.

Platform\-centric initiatives such as OmniGenBench\(Wanget al\.,[2025a](https://arxiv.org/html/2606.04525#bib.bib48)\)have made important strides toward reproducibility by providing modular infrastructure and dynamic leaderboards\. However, their reported baselines are often limited to a small set of established models \(e\.g\., DNABERT\-2, HyenaDNA, RNA\-FM\) and place substantial emphasis on RNA modalities\. Consequently, a large and rapidly evolving portion of the DNA\-specific foundation model landscape remains underexplored\.

##### Positioning of GENEB\.

GENEB is designed to complement and extend prior benchmarking efforts by enforcing exhaustive and controlled evaluation across a substantially broader model and task space\. GENEB aggregates 100 DNA classification tasks drawn from multiple established benchmarks, including Nucleotide Transformer tasks, Genomic Benchmarks, GUE/GUE\+, plant genomics benchmarks, and DNA methylation datasets, spanning 13 functional categories\. In contrast to prior work, GENEB evaluates all included models on the full task suite under a unified probing\-based protocol, regardless of the model’s original domain or anticipated strengths\.

This exhaustive cross\-evaluation yields a complete, static performance matrix over 40 genomic foundation models, including recent architectures such as BioFM, GENERator, JanusDNA, and DeepGene that are absent from many existing benchmarks\. By eliminating selective reporting and enforcing matched comparisons across architecture, tokenization, and pretraining data, GENEB exposes task\-dependent trade\-offs and failure modes that are often obscured in partial or benchmark\-specific evaluations\.

Beyond the static analysis presented in this work, we intend to release GENEB as a public benchmark with model evaluations hosted on Hugging Face, enabling transparent comparison and reproducible evaluation of future genomic foundation models\. In this sense, GENEB is conceptually aligned with MTEB\(Muennighoffet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib3)\)in natural language processing: a community\-facing benchmark that provides a shared reference point for evaluating representation quality across diverse tasks\. We hope that GENEB can serve a similar role for genomic foundation models, supporting more principled model comparison and task\-aware model selection\.

## Appendix BTask Taxonomy and Benchmark Composition

GENEB aggregates tasks from nine widely used genomic benchmarks to ensure broad biological coverage and comparability to prior work:NT\(Nucleotide Transformer original,1818tasks\) andNT\-rev\(its revised release,1818tasks\) covering histone marks, enhancers, promoters, and splice sites;GUE\(3131tasks\) covering empirically\-derived histone marks, TF binding, mouse enhancers, promoters, splice sites, phage fragments, and species classification;GB\(Genomic Benchmarks,99tasks\) covering enhancers, promoters, coding/non\-coding, regulatory elements, and open chromatin;PGB\(Plant Genomic Benchmark,77tasks\) covering plant lncRNA and transcriptional activity;iPro\-WAEL\(88tasks\) covering bacterial, plant, and human cell\-type\-specific promoter recognition;deep4mc\(66tasks\) andiDNA\-ABF\(22tasks\) covering DNA methylation \(4mC, 5mC, 6mA\) across taxa; andiDHS\-EL\(11task\) for chromatin accessibility\.

Our benchmark comprises100100classification tasks organized into1313functional categories spanning regulatory element prediction, epigenomic modification detection, and evolutionary sequence analysis\. This taxonomic organization reflects the diverse challenges facing DNA foundation models and enables systematic analysis across distinct biological domains\. Table[3](https://arxiv.org/html/2606.04525#A2.T3)summarizes the benchmark composition, with task counts ranging from single\-task categories \(Coding/Non\-coding, Chromatin Accessibility\) to large multi\-task groups \(Histone Modifications with3030tasks, Promoters with2222tasks\)\.

Table 3:Benchmark task organization across 13 functional categories\. Tasks span regulatory element prediction, epigenetic modification detection, and evolutionary sequence analysis across bacterial, plant, and mammalian systems\. Within each row, the source prefix \(e\.g\., NT, GUE, GB\) is given once and applies to all listed tasks\.CategorySubcategorynnRepresentative TasksHistone ModificationsOriginal \(NT\)10H3, H3K14ac, H3K36me3, H3K4me1/2/3, H3K79me3, H3K9ac, H4, H4acEmpirical \(GUE EMP\)10H3, H3K14ac, H3K36me3, H3K4me1/2/3, H3K79me3, H3K9ac, H4, H4acRevised \(NT\-rev\)10H2AFZ, H3K27ac, H3K27me3, H3K36me3, H3K4me1/2/3, H3K9ac, H3K9me3, H4K20me1PromotersHuman cell\-type \(iPro\)4iPro: GM12878, HUVEC, HeLa\-S3, NHEKPlant3iPro Arabidopsis \(TATA / no TATA\), PGB Promoter*M\. esculenta*Bacterial \(iPro\)2iPro:*B\. amyloliquefaciens*,*R\. capsulatus*General13GUE Prom 300 \(all / TATA / no TATA\), GUE Prom core \(all / TATA / no TATA\), NT Promoter \(all / TATA / no TATA\), NT\-rev Promoter \(all / TATA / no TATA\), GB Promoter \(no TATA\)EnhancersSpecies\-specific \(GB\)4GB: Drosophila enh\. \(STARK\), Mouse enh\. \(Ensembl\), Human enh\. \(Cohn\), Human enh\. \(Ensembl\)Type classification4NT Enhancers, NT Enhancers \(types\), NT\-rev Enhancers, NT\-rev Enhancers \(types\)DNA Methylation5mC / 6mA \(iDNA\)2iDNA: 5mC, 6mA4mC \(6 species\)64mC:*A\. thaliana*,*C\. elegans*,*D\. melanogaster*,*E\. coli*,*G\. pickeringii*,*G\. subterraneus*Splice SitesSite\-specific4NT Splice \(donors / acceptors\), NT\-rev Splice \(donors / acceptors\)Combined3NT Splice \(all\), NT\-rev Splice \(all\), GUE Splice reconstr\.lncRNAPlant \(6 species; PGB\)6PGB lncRNA:*G\. max*,*M\. esculenta*,*S\. bicolor*,*S\. lycopersicum*,*T\. aestivum*,*Z\. mays*Mouse EnhancersTissue\-specific \(GUE\)5GUE Mouse enh\.: \-0, \-1, \-2, \-3, \-4TF BindingHuman TFs \(GUE\)5GUE TF: \-0, \-1, \-2, \-3, \-4Species ClassificationBinary1GB Human\-or\-wormMulti\-class \(GUE\)2GUE: Fungi\-20, Virus\-40RegulatoryHuman regulatory \(GB\)2GB: Ensembl regulatory, OCR EnsemblVirus/PhageViral sequences \(GUE\)2GUE: Phage fragments, COVID variantsCoding/Non\-codingSequence type1GB Coding/Non\-codingChromatin AccessibilityDNase\-seq1iDHS DNase\-I
Source prefixes:NT = Nucleotide Transformer; NT\-rev = Nucleotide Transformer revised; GUE = Genome Understanding Evaluation; GUE EMP = GUE empirically\-validated subset; GB = Genomic Benchmarks; iPro = iPro\-WAEL promoter dataset; PGB = Plant Genomics Benchmark; iDNA = iDNA\-ABF; iDHS = iDHS\-EL; 4mC = deep4mc\.

##### Epigenomic and Chromatin Tasks\.

The largest task category, Histone Modifications \(n=30n=30\), encompasses prediction of post\-translational modifications across distinct histone marks including H3K4me1/2/3, H3K27ac, H3K36me3, H3K9ac, and H4 acetylation states\. The tasks draw from three sources:NT\(1010tasks: H3, H4, H3K4me1/2/3, H3K9ac, H3K14ac, H3K36me3, H3K79me3, H4ac\),NT\-rev\(1010tasks: revised versions of H3K4me1/2/3, H3K9ac, and H3K36me3 plus newly added H2AFZ, H3K27ac, H3K27me3, H3K9me3, and H4K20me1\), andGUE\(1010tasks: empirically derived subsets of the NT histone marks\)\. Chromatin Accessibility \(n=1n=1\) is theiDHS\-EL DNase\-Itask, providing a complementary measure of chromatin state\.

##### Regulatory Element Tasks\.

Promoter recognition \(n=22n=22\) draws from six sources:NTandNT\-rev\(66tasks total: TATA\-containing, TATA\-less, and combined human promoter classification\),GUE\(66tasks: core and 300\-bp promoter variants with TATA\-containing, TATA\-less, and combined splits\),GB\(11task: human non\-TATA promoters\),iPro\-WAEL\(88tasks spanning bacterialB\. amyloliquefaciensandR\. capsulatus, plantArabidopsisTATA/non\-TATA, and human cell\-type\-specific lines GM12878, HUVEC, HeLa\-S3, NHEK\), andPGB\(11task:M\. esculentatranscriptional activity\)\. Enhancer prediction \(n=8n=8\) draws fromNTandNT\-rev\(44tasks: combined and type\-stratified enhancer classification\) andGB\(44tasks:DrosophilaStark, mouse enhancers \(Ensembl\), and human enhancers \(Cohn and Ensembl\)\)\. General regulatory element tasks \(n=2n=2\) are both fromGB: human Ensembl regulatory region prediction and human open chromatin region prediction\.

##### Transcription Factor and Splicing Tasks\.

TF Binding prediction \(n=5n=5\) comprises fiveGUEhuman transcription factor binding tasks\. Splice Site detection \(n=7n=7\) combinesNT\(33tasks: donors, acceptors, combined\),NT\-rev\(33revised counterparts\), andGUE\(11reconstructed splice site task, an artificially difficult reconstruction of splice junctions\)\.

##### Sequence Modification Tasks\.

DNA Methylation prediction \(n=8n=8\) draws fromdeep4mc\(66tasks: 4mC detection inA\. thaliana,C\. elegans,D\. melanogaster,E\. coli,G\. pickeringii,G\. subterraneus\) andiDNA\-ABF\(22tasks: 5mC and 6mA modification site prediction\)\.

##### Non\-coding RNA and Species Classification Tasks\.

Long non\-coding RNA classification \(n=6n=6\) comprises the sixPGBplant lncRNA tasks: soybean \(G\. max\), cassava \(M\. esculenta\), sorghum \(S\. bicolor\), tomato \(S\. lycopersicum\), wheat \(T\. aestivum\), and maize \(Z\. mays\)\. Species Classification \(n=3n=3\) draws fromGB\(human\-versus\-worm discrimination\) andGUE\(2020\-way fungal species and4040\-way viral species classification\)\.

##### Additional Tasks\.

Mouse Enhancer prediction \(n=5n=5\) comprises fiveGUEmouse enhancer tasks \(tissue\-specific enhancer classification\)\. Virus/Phage detection \(n=2n=2\) comprises twoGUEtasks: phage fragment identification and COVID\-19 variant classification\. Coding/Non\-coding discrimination \(n=1n=1\) is theGBcoding\-vs\-intergenomic\-sequences task\. This diverse task composition enables evaluation of transfer learning across taxonomic boundaries \(human to plant, eukaryotic to prokaryotic\), assessment of specialization versus generalization trade\-offs, and identification of task\-specific model advantages\. The predominance of eukaryotic tasks \(1212of1313categories\) reflects current genomic research priorities while creating systematic disadvantages for prokaryotic\-focused models, a limitation we address in our analysis\.

## Appendix CModel Summary

Table[4](https://arxiv.org/html/2606.04525#A3.T4)summarizes the4040genomic foundation models evaluated in GENEB\. For each model we report its canonical name, architecture family, tokenization scheme, parameter count, and pretraining\-corpus type\.

Table 4:Summary of the 40 genomic foundation models evaluated in GENEB\.Sorted by parameter count \(descending\)\. Tokenization labels: SN = single\-nucleotide; BPE = byte\-pair encoding; k\-mer = fixed\-lengthkk\-mer; mixed = mixed nucleotide/amino\-acid vocabulary; BioToken = model\-specific tokenizer\. Architecture labels: T\-enc = Transformer\-encoder; T\-dec = Transformer\-decoder\.
## Appendix DModels Excluded from Evaluation

During our comprehensive survey of genomic foundation models, we identified several promising models that could not be included in our benchmark evaluation due to various technical and practical limitations\. We document these exclusions for transparency and to inform future benchmark efforts\. Table[5](https://arxiv.org/html/2606.04525#A4.T5)summarizes the excluded models and their exclusion reasons\.

Table 5:Genomic foundation models excluded from GENEB evaluation\.Models are grouped by exclusion category\. All models were considered for inclusion but could not be evaluated due to the stated limitations\.ModelCategoryExclusion ReasonUnavailable or Private WeightsGene42\(Vishniakovet al\.,[2025a](https://arxiv.org/html/2606.04525#bib.bib38)\)Private weightsModel weights not publicly releasedNTv3\(Bosharet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib39)\)Private weightsModel weights not publicly released despite HuggingFace placeholderCode/Infrastructure IssuesNucleotideGPT\(Mclaughlinet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib40)\)Broken codeCorrupted weights and buggy code; TPU\-only with no GPU support documentationEpiGePT\(Gaoet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib41)\)Missing extraction codeNo embedding extraction interface; requires extensive custom implementationENBED\(Malusareet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib42)\)Missing extraction codeNo embedding extraction code; unclear model checkpoint locationHAD\(Yanget al\.,[2025b](https://arxiv.org/html/2606.04525#bib.bib43)\)No codeNo public code repository availableC\.La\.P\.\(Nisantziset al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib44)\)No codeNo public code repository availableHybriDNA\(Maet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib32)\)No codeNo public code repository availableMxDNA\(Qiaoet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib33)\)Broken weightsIncompatible or corrupted model weightsVQDNA\(Liet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib34)\)No codeNo public code repository availableBMFM\-DNA\(Liet al\.,[2025a](https://arxiv.org/html/2606.04525#bib.bib35)\)Dependency conflictsLibrary version conflicts prevent executionComputational ConstraintsEvo2\(Brixiet al\.,[2025b](https://arxiv.org/html/2606.04525#bib.bib36)\)Hardware requirementsRequires H100/H200 GPUs exceeding our computational budgetArchitectural LimitationsChatNT\(Richardet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib37)\)Wrapper modelUses Nucleotide Transformer as encoder; not an independent foundation model### D\.1Detailed Exclusion Notes

##### Private or Unavailable Weights\.

Two models – Gene42 and NTv3 – were excluded because their pretrained weights have not been publicly released\. Gene42\(Vishniakovet al\.,[2025a](https://arxiv.org/html/2606.04525#bib.bib38)\)introduces a long\-range genomic foundation model with dense attention capable of processing sequences up to 192 kbp, representing a significant architectural advancement\. NTv3\(Bosharet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib39)\)extends the Nucleotide Transformer family with joint sequence\-function modeling\. Both models report strong benchmark results in their respective publications, but the lack of public weights prevents independent evaluation\.

##### Missing or Broken Code\.

Several models suffer from incomplete or non\-functional code releases\. NucleotideGPT\(Mclaughlinet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib40)\)provides weights that appear corrupted and code that only supports TPU execution without GPU alternatives\. EpiGePT\(Gaoet al\.,[2023](https://arxiv.org/html/2606.04525#bib.bib41)\)and ENBED\(Malusareet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib42)\)lack embedding extraction interfaces, making it infeasible to obtain sequence representations for downstream evaluation without substantial reverse\-engineering effort\. HAD\(Yanget al\.,[2025b](https://arxiv.org/html/2606.04525#bib.bib43)\), C\.La\.P\.\(Nisantziset al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib44)\), and HybriDNA\(Maet al\.,[2025](https://arxiv.org/html/2606.04525#bib.bib32)\)have no public code repositories despite published papers describing their architectures\.

##### Runtime and Dependency Issues\.

MxDNA\(Qiaoet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib33)\)provides code and weights but fail to execute correctly due to weight incompatibilities or undocumented runtime requirements\. BMFM\-DNA\(Liet al\.,[2025a](https://arxiv.org/html/2606.04525#bib.bib35)\)from IBM Research encounters library version conflicts that prevent successful model loading\.

##### Computational Requirements\.

Evo2\(Brixiet al\.,[2025b](https://arxiv.org/html/2606.04525#bib.bib36)\), the successor to the original Evo model included in our benchmark, requires H100 or H200 GPUs for inference due to its 40B parameter scale\. This exceeds our available computational resources \(A100 GPUs\) and represents a practical barrier for many research groups\.

##### Wrapper Models\.

ChatNT\(Richardet al\.,[2024](https://arxiv.org/html/2606.04525#bib.bib37)\)was excluded because it uses a frozen Nucleotide Transformer v2 as its DNA encoder, making it a wrapper rather than an independent foundation model\. Evaluating ChatNT would redundantly measure NTv2 performance while introducing confounding factors from the conversational interface\.

### D\.2Implications for Reproducibility

The prevalence of excluded models \(13 out of 53 initially surveyed, approximately 25%\) highlights a reproducibility challenge in genomic foundation model research\. We encourage future model releases to include: \(i\) publicly available pretrained weights, \(ii\) documented embedding extraction code, \(iii\) clear hardware requirements, and \(iv\) tested installation procedures across common environments\.

### D\.3Excluded Long\-Range Regulatory Tasks

GENEB does not include tasks that require explicit modeling of very long\-range regulatory interactions \(\>10\>10kb\)\. We considered the following candidate datasets but excluded them on the grounds outlined below:

##### Enhancer–promoter interaction prediction\.

Tasks based on predicting physical or functional contacts between enhancers and their target promoters at distances of5050–500500kb \(e\.g\., ChIA\-PET, BENGI, HiChIP\-derived datasets\)\. Most genomic foundation models in GENEB accept context windows below66kb, making fair evaluation impossible without arbitrary cropping\.

##### Three\-dimensional chromatin contact map prediction\.

Tasks based on predicting Hi\-C contact frequencies or topologically associating domain \(TAD\) boundaries from megabase\-scale sequence windows\. These require context lengths that exceed the input limits of all but a few of the evaluated models\.

##### Distal eQTL effect prediction\.

Tasks linking sequence variants to gene expression changes when the variant lies\>100\>100kb from the affected gene\. Direct sequence\-to\-effect formulations require long context; reduced formulations introduce confounds that defeat the purpose of fair cross\-model comparison\.

##### Whole\-locus expression quantification\.

Tasks predicting tissue\-specific expression from\>10\>10kb genomic windows around a gene of interest, including locus\-level enhancer collections\.

##### Models affected by these exclusions\.

Models with explicit long\-context capability – specificallyHyenaDNA\-Large\-1M\(11M tokens\),Caduceus\-PS\-131k\(131131k tokens\),Evo\-1\-131k\(131131k tokens\), andJanusDNA\-72\-w/JanusDNA\-72\-wo\(Hybrid\-Mamba\-MoE architecture\) – are not exercised on the regime where their architectural priors would most plausibly yield differentiating gains\. We treat this as a known limitation of the current benchmark snapshot; extending GENEB with long\-range regulatory tasks under a unified protocol is an important direction for future work\.

## Appendix EProbe Stability and Protocol Sensitivity Analysis

GENEB evaluates frozen representations with linear probing, which provides a controlled and interpretable measure of representation quality but in principle could obscure information accessible only via non\-linear readouts\. To verify that the rankings and conclusions reported in the main paper are not artifacts of this choice, we conduct two complementary stability analyses: \(i\) replacing the linear probe with a non\-linear MLP probe \(Section[E\.1](https://arxiv.org/html/2606.04525#A5.SS1)\), and \(ii\) varying the regularization strength of the linear probe across few\-shot regimes \(Section[E\.2](https://arxiv.org/html/2606.04525#A5.SS2)\)\. Both analyses are performed on a representative subset of1111models and1313tasks, where each task is drawn from a distinct functional category to ensure coverage of the benchmark’s diversity\. The selected models span the full range of architectures, tokenization schemes, pretraining corpora, and parameter scales evaluated in GENEB; the model and task subsets are summarized in Tables[6](https://arxiv.org/html/2606.04525#A5.T6)and[7](https://arxiv.org/html/2606.04525#A5.T7)\.

Table 6:Representative model subset used for probe stability and protocol sensitivity analyses\. The subset spans all major architectural families, tokenization schemes, pretraining corpora, and three orders of magnitude in parameter scale\.Table 7:Representative task subset used for probe stability and protocol sensitivity analyses\. One task is selected from each of the 13 functional categories in GENEB\.### E\.1Probe Stability Analysis

##### Setup\.

For each of the11×13=14311\\times 13=143model–task combinations, we compute two MCC values: one using the linear probe \(logistic regression\) employed throughout the main paper, and one using a non\-linear MLP probe consisting of a single hidden layer of 256 units with ReLU activation, trained with early stopping\. All other components of the evaluation pipeline – feature extraction, normalization, train/test splits, and random seeds – are held identical across the two probes\. We then compare the resulting rankings and absolute MCC values to assess whether the linear probe provides a faithful proxy for representation quality under non\-linear readouts\.

##### Rankings are highly stable across probes\.

Spearman rank correlation between the two probes isρ=0\.964\\rho=0\.964\(p<0\.001p<0\.001\) across all 143 model–task pairs, andρ=0\.973\\rho=0\.973\(p<0\.001p<0\.001\) when computed on per\-model average MCC\. The top\-3 and top\-5 models, ranked by mean MCC across the 13 tasks, are identical under both probes \(Table[8](https://arxiv.org/html/2606.04525#A5.T8)\)\. Per\-task Spearman correlations are positive for 12 of 13 tasks \(Table[9](https://arxiv.org/html/2606.04525#A5.T9)\), with a median ofρ=0\.855\\rho=0\.855\. The sole exception isGB Ensembl regulatory, where MCC values are tightly clustered across models, leaving rank ordering dominated by stochastic noise rather than substantive differences in representation quality\.

Table 8:Per\-model average MCC under linear and MLP probes across the 13 representative tasks\. Models are ordered by linear\-probe MCC\. The top\-5 positions are identical under both probes\. The largest absolute difference \(HyenaDNA\-Large\-1M,\+0\.052\+0\.052\) does not alter the model’s relative rank\.Table 9:Per\-task Spearman rank correlation between linear and MLP probe rankings across the 11 representative models\. Tasks are ordered by correlation strength\. Twelve of thirteen tasks exhibit strongly positive rank correlation\. The single negative value \(GB Ensembl regulatory\) corresponds to a task where MCC values are tightly clustered across models, leaving rank ordering dominated by noise rather than substantive performance differences\.
##### Absolute MCC shifts are small\.

The mean signed difference between linear and MLP probe MCC across the 11 models is\+0\.011\+0\.011, indicating only a marginal aggregate benefit from non\-linear readouts\. The largest single shift is observed forHyenaDNA\-Large\-1M\(\+0\.052\+0\.052\), which is consistent with the hypothesis that Hyena representations benefit modestly from non\-linear projection\. However, this shift does not change the model’s relative rank, and the overall ordering remains intact\.

##### Conclusion\.

These results indicate that the rankings reported in the main paper are robust to probe choice within the representative subset evaluated here: model orderings under linear probing constitute a reliable proxy for those obtained with non\-linear MLP probes\. The main empirical conclusions of GENEB – including the scale–performance disconnect, architectural dominance under matched conditions, and category\-dependent specialization – are therefore unlikely to be artifacts of the linear probing protocol\. We note that the present analysis addresses the stability of relative comparisons under frozen representations; the question of whether rankings remain stable under full task\-specific fine\-tuning is discussed in the main paper as a limitation\.

### E\.2Few\-Shot Protocol Sensitivity

##### Setup\.

A second potential source of artifact in our few\-shot conclusions is the choice of regularization strength in the linear probe, particularly in the low\-data 1\-shot and 10\-shot regimes where logistic regression behavior can be sensitive to hyperparameter settings\. To assess this, we sweep the inverse regularization strengthC∈\{0\.01,0\.1,1\.0,10\.0,100\.0\}C\\in\\\{0\.01,0\.1,1\.0,10\.0,100\.0\\\}across all three shot regimes \(1\-shot, 10\-shot, and full\-data\) for the same11×1311\\times 13model–task subset used in Section[E\.1](https://arxiv.org/html/2606.04525#A5.SS1)\. We sweepCCas the principal regularization hyperparameter of logistic regression; other settings \(maximum iterations, solver, convergence tolerance\) are held identical to the main\-paper protocol, as are feature normalization, train/test splits, and random seeds\. We then quantify both ranking stability and absolute MCC sensitivity as functions ofCC\.

##### Rankings are essentially invariant in the 1\-shot regime\.

Pairwise Spearman correlations between rankings at differentCCvalues are summarized in Table[10](https://arxiv.org/html/2606.04525#A5.T10)\. In the 1\-shot regime, mean pairwiseρ=0\.993\\rho=0\.993, with a minimum of0\.9820\.982across all\(52\)=10\\binom\{5\}\{2\}=10pairs ofCCvalues \(Table[11](https://arxiv.org/html/2606.04525#A5.T11)\)\. This near\-invariance indicates that rankings in the most data\-constrained regime are dominated by intrinsic representation quality rather than by regularization choice\.

Table 10:Ranking stability across regularization strengthsC∈\{0\.01,0\.1,1,10,100\}C\\in\\\{0\.01,0\.1,1,10,100\\\}\. Each row reports the mean, minimum, and maximum pairwise Spearman correlation between model rankings under different values ofCCwithin a given shot regime\. Rankings are highly stable in the 1\-shot regime and remain stable for adjacent values ofCCin the 10\-shot and full\-data regimes\.Table 11:Pairwise Spearman correlation between model rankings at different regularization strengths in the 1\-shot regime\. Rankings are nearly invariant \(ρ≥0\.982\\rho\\geq 0\.982\) across allCCpairs\.
##### Rankings remain stable for adjacentCCvalues in the 10\-shot regime\.

At 10\-shot, mean pairwiseρ=0\.805\\rho=0\.805, with adjacentCCvalues yieldingρ≥0\.9\\rho\\geq 0\.9\(Table[12](https://arxiv.org/html/2606.04525#A5.T12)\)\. Larger divergences appear only between extreme settings \(e\.g\.,C=0\.01C=0\.01vs\.C=100C=100,ρ=0\.582\\rho=0\.582\), reflecting the increased influence of regularization when small but non\-trivial amounts of supervision are available\. The full\-data regime exhibits a similar pattern \(meanρ=0\.766\\rho=0\.766; minρ=0\.436\\rho=0\.436\), indicating that ranking fluctuations are driven primarily by extreme regularization choices rather than by typical hyperparameter selections\.

Table 12:Pairwise Spearman correlation between model rankings at different regularization strengths in the 10\-shot regime\. AdjacentCCvalues yieldρ≥0\.9\\rho\\geq 0\.9, with substantial divergence appearing only between extreme settings\.
##### Absolute MCC sensitivity is bounded and concentrated in a small subset of models\.

Per\-model MCC ranges acrossCCvalues in the full\-data regime are reported in Table[13](https://arxiv.org/html/2606.04525#A5.T13)\. For most models, the range is well below0\.100\.10MCC, with the smallest sensitivity observed forGenomeOcean\-500M\(range0\.0140\.014\)\. The largest sensitivities are observed forLucaOne\(0\.1990\.199\),HyenaDNA\-Large\-1M\(0\.1440\.144\), andNT\-v2\-50M\-MS\(0\.1080\.108\), suggesting that representations from these models are somewhat more dependent on regularization tuning than those of the remaining models\. We note that absolute MCC sensitivity does not translate into ranking instability for typical regularization choices: across adjacentCCvalues, the relative ordering of models remains substantively unchanged \(Tables[11](https://arxiv.org/html/2606.04525#A5.T11)and[12](https://arxiv.org/html/2606.04525#A5.T12)\)\.

Table 13:Per\-model MCC range across regularization strengthsC∈\{0\.01,0\.1,1,10,100\}C\\in\\\{0\.01,0\.1,1,10,100\\\}in the full\-data regime\. The largest sensitivity is observed forLucaOne,HyenaDNA\-Large\-1M, andNT\-v2\-50M\-MS; the smallest forGenomeOcean\-500M\.
##### The principal few\-shot finding is protocol\-stable\.

Crucially, the central few\-shot conclusion reported in the main paper – the sharp degradation of mean MCC from full\-data to 1\-shot – is replicated at every value ofCCtested\. The magnitude of this degradation varies modestly across regularization choices, and its direction and severity are preserved without exception\.

##### Conclusion\.

These results indicate that the few\-shot findings reported in the main paper reflect properties of the evaluated representations under data\-constrained regimes, rather than artifacts of a particular regularization choice\. Rankings are essentially invariant under 1\-shot evaluation, remain stable for typical regularization settings under 10\-shot and full\-data evaluation, and the qualitative pattern of severe low\-data degradation persists uniformly across the regularization sweep\.

### E\.3Controlled\-Pair Comparisons and Residual Confounds

##### Methodology\.

Throughout the main paper, comparative claims about architecture, tokenization, and pretraining data are based on*matched pairs*of models that differ in exactly one factor of interest while holding others as constant as the available model set permits\. This controlled\-pair design substantially reduces the risk of attributing observed performance differences to the wrong cause and is preferable to unmatched comparisons across the full benchmark\. However, perfect isolation is impossible in practice: genomic foundation models differ along multiple correlated axes \(architecture, tokenization, training corpus, scale, training duration, and pretraining objective\), so even carefully matched pairs retain residual confounds\. To make these confounds explicit and to enable readers to assess the strength of each claim, we enumerate the full set of3030controlled pairs underlying the comparative analyses\.

Table 14:Summary of the3030controlled\-pair comparisons used in the analyses of architecture, pretraining data, and tokenization\. Each pair varies a single factor of interest while holding the others constant\.
##### Summary of comparisons\.

Table[14](https://arxiv.org/html/2606.04525#A5.T14)provides an overview of the matched pairs by factor type\. Architecture comparisons \(9 pairs\) hold tokenization and pretraining corpus type constant while varying the architectural family\. Pretraining\-data comparisons \(9 pairs\) hold architecture and tokenization constant while varying the corpus type\. Tokenization comparisons \(12 pairs\) hold architecture and pretraining corpus constant while varying the tokenization scheme\.

##### Architecture comparisons\.

Table[15](https://arxiv.org/html/2606.04525#A5.T15)enumerates the99matched pairs used to isolate architectural effects\. In each pair, tokenization and pretraining corpus type are held constant; the varied factor is architectural family\. For each pair we report the macro\-averaged MCC differenceΔ=MCCA−MCCB\\Delta=\\text\{MCC\}\_\{A\}\-\\text\{MCC\}\_\{B\}; positive values indicate that the architectural family listed first in the “Variable” column wins\. Residual confounds for this group of comparisons include model size differences \(constrained to within a factor of two where possible\), training duration, exact composition of the pretraining data, pretraining objective \(masked language modeling vs\. causal language modeling\), and depth\-to\-width ratio\.

Table 15:Architecture\-controlled pairs\. Each pair holds tokenization and pretraining corpus type constant and varies the architectural family\.Δ\\Deltamacro\-MCC is the difference between model A and model B; positive values indicate that the architectural family listed first in the “Variable” column wins\.
##### Pretraining\-data comparisons\.

Table[16](https://arxiv.org/html/2606.04525#A5.T16)enumerates the99matched pairs used to isolate the effect of pretraining corpus type\. In each pair, architecture and tokenization are held constant; the varied factor is the taxonomic composition of the pretraining data\. For each pair we report the macro\-averaged MCC differenceΔ=MCCA−MCCB\\Delta=\\text\{MCC\}\_\{A\}\-\\text\{MCC\}\_\{B\}; positive values indicate that the corpus type listed first in the “Variable” column wins\. Residual confounds include exact corpus scale and diversity, training duration, and learning\-rate schedules\. We additionally note that the*Eukaryotic\-genes vs\. Multi\-species*comparison is supported by a single matched pair \(GENERator\-Eukaryote\-3Bvs\.DNA\-GPT\-3B\-M\); claims derived from this comparison are accordingly flagged in the main text and should be interpreted with additional caution\.

Table 16:Pretraining\-data\-controlled pairs\. Each pair holds architecture and tokenization constant and varies the pretraining corpus type\.Δ\\Deltamacro\-MCC is the difference between model A and model B; positive values indicate the corpus type listed first in “Variable” wins\.Model AModel BControlledVariableΔ\\DeltaGenomics\-FMDNABERT\-SEncoder,kk\-merMulti vs\. Microbial\+0\.088\+0\.088NT\-v2\-100M\-MSDNABERT\-SEncoder,kk\-merMulti vs\. Microbial\+0\.081\+0\.081GENERator\-Eukaryote\-3BDNA\-GPT\-3B\-MDecoder,kk\-merEukaryotic vs\. Multi\+0\.063\+0\.063DNABERT\-2GENA\-LMEncoder, BPEMulti vs\. Human\+0\.014\+0\.014GENA\-LM\-T2T\-MultiGENA\-LMEncoder, BPEMulti vs\. Human\+0\.025\+0\.025DNABERT\-2GROVEREncoder, BPEMulti vs\. Human\+0\.019\+0\.019GENA\-LM\-T2T\-MultiGROVEREncoder, BPEMulti vs\. Human\+0\.030\+0\.030GPT2\-Gene\-Multi\-v2GPT2\-Gene\-v1Decoder,kk\-merMulti vs\. Human−0\.009\-0\.009DNA\-GPT\-0\.1B\-HGPT2\-Gene\-v1Decoder,kk\-merMulti vs\. Human−0\.007\-0\.007
##### Tokenization comparisons\.

Table[17](https://arxiv.org/html/2606.04525#A5.T17)enumerates the1212matched pairs used to isolate the effect of tokenization scheme\. In each pair, architecture and pretraining corpus type are held constant; the varied factor is the tokenization strategy\. For each pair we report the macro\-averaged MCC differenceΔ=MCCA−MCCB\\Delta=\\text\{MCC\}\_\{A\}\-\\text\{MCC\}\_\{B\}, where positive values indicate that the tokenization listed first in the “Variable” column wins\. Residual confounds include model size differences \(within±2×\\pm 2\\times\), vocabulary size \(tightly coupled with tokenization scheme\), and model\-specific training regimes\.

Aggregating across the 12 pairs reveals three regimes\. \(i\) In matched Transformer\-decoder comparisons with multi\-species pretraining, BPE exceedskk\-mer on average \(\+0\.020\+0\.020across 3 pairs\), with a large positive gap for one pair \(\+0\.071\+0\.071\) and one reversal \(−0\.042\-0\.042\)\. \(ii\) In matched Transformer\-encoder comparisons with multi\-species pretraining, BPE andkk\-mer perform comparably \(\+0\.006\+0\.006across 5 pairs, with all gaps within±0\.02\\pm 0\.02MCC\)\. \(iii\) In matched Transformer\-encoder comparisons with human pretraining, single\-nucleotide tokenization \(MutBERT\) consistently outperforms BPE baselines \(\+0\.033\+0\.033and\+0\.038\+0\.038overGENA\-LMandGROVER, respectively\)\. The two comparisons involving non\-standard tokenization schemes \(BioFM\-265M’s BioToken;LucaOne’s mixed vocabulary\) are reported for completeness but should be interpreted with caution, as these schemes introduce confounds beyond a simple choice of tokenization unit\.

Table 17:Tokenization\-controlled pairs\. Each pair holds architecture and pretraining corpus type constant and varies the tokenization scheme\.Δ\\Deltamacro\-MCC is the difference between model A and model B; positive values indicate that the tokenization listed first in the “Variable” column wins\.
##### Conclusion\.

Throughout the main paper, observations derived from these matched pairs are framed as*consistent with*or*associated with*the varied factor, rather than as*caused by*it\. Single\-pair comparisons – most notably the*Eukaryotic\-genes vs\. Multi\-species*contrast – are flagged as such on first appearance, and conclusions relying on them are stated with correspondingly reduced confidence\. We view this controlled\-pair methodology as a principled middle ground between unmatched comparisons across the full benchmark, which conflate multiple factors, and fully randomized causal experiments, which are infeasible at the scale of foundation\-model pretraining\.

### E\.4Micro\- vs\. Macro\-Averaged Aggregate Performance

The main paper reports macro\-averaged MCC as the principal aggregation metric, computed by first averaging within each of the 13 functional categories and then averaging across categories with equal weight per category\. This choice avoids implicitly overweighting categories with many tasks \(e\.g\., histone modifications with 30 tasks, promoters with 22 tasks\) and instead treats each functional category as a unit of biological interest\. To verify that the central findings of GENEB are robust to this weighting choice, we additionally compute*micro\-averaged*MCC by simple averaging across all 100 tasks and compare the two aggregation schemes\.

Figure[8](https://arxiv.org/html/2606.04525#A5.F8)compares the two aggregation schemes across all 40 models\. The Spearman rank correlation between micro\- and macro\-averaged orderings isρ=0\.988\\rho=0\.988\(p<0\.001p<0\.001\), indicating that the relative ordering of models is largely preserved\. The mean absolute shift in aggregate MCC is\|Δ\|=0\.009\|\\Delta\|=0\.009, and the top\-5 set is identical under both schemes \(within the top\-5,LucaOneandGENERator\-Eukaryote\-1\.2Bswap positions\)\. The largest individual shifts are concentrated in out\-of\-domain or specialized models:Evo\-1\-131k\(Δ=−0\.044\\Delta=\-0\.044\),Caduceus\-PS\-131k\(Δ=−0\.028\\Delta=\-0\.028\), andPlantCaduceus\(Δ=−0\.024\\Delta=\-0\.024\), all of which exhibit highly uneven category\-level performance that is amplified under category\-balanced averaging\.

The central empirical findings of GENEB – the substantial overall correlation between scale and performance, the instability of category rankings, and the dominance of architectural and pretraining alignment over parameter count – hold under both averaging schemes\. We view the sensitivity of out\-of\-domain models to weighting choice as itself consistent with the broader argument of this paper: single\-score leaderboards are an unreliable basis for genomic model selection\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/micro_vs_macro_v12.png)Figure 8:Robustness of GENEB aggregate rankings to averaging scheme\.For each of the 40 models,Δ=MCCmacro−MCCmicro\\Delta=\\text\{MCC\}\_\{\\text\{macro\}\}\-\\text\{MCC\}\_\{\\text\{micro\}\}is shown in the left panel; the side panel reports the underlying micro\- and macro\-averaged MCC values\. Models are sorted from largest negative shift to largest positive shift\. Out\-of\-domain models are highlighted: prokaryotic\-onlyEvo\-1\-131k\(red,Δ=−0\.044\\Delta=\-0\.044\), microbial\-onlyDNABERT\-S\(orange,Δ=\+0\.008\\Delta=\+0\.008\), and plant\-specificPlantCaduceusandAgro\-NT\-1B\(teal\)\. Across all 40 models the Spearman rank correlation between micro\- and macro\-averaged orderings isρ=0\.988\\rho=0\.988\(p<0\.001p<0\.001\); the mean absolute shift is\|Δ\|=0\.009\|\\Delta\|=0\.009MCC\.

## Appendix FResults Analysis

This section presents a systematic analysis of the benchmark results across 40 DNA foundation models evaluated on 100 genomic tasks spanning 13 functional categories\. We structure our analysis around three fundamental questions: \(1\) how do architectural choices influence model performance, \(2\) what role does tokenization strategy play, and \(3\) how does model scale interact with pretraining design decisions?

### F\.1Experimental Overview

Our benchmark encompasses substantial diversity in both models and tasks\. The 40 evaluated models span the following architectural families: Transformer\-encoder \(n=15n=15\), Transformer\-decoder \(n=13n=13\), Mamba\-SSM \(n=4n=4\), Hybrid\-Mamba\-MoE \(n=2n=2\), Hyena \(n=2n=2\), CNN\-Transformer hybrids \(n=2n=2, includingEnformerandSPACE\), Graph\-Transformer \(n=1n=1,DeepGene\), and StripedHyena \(n=1n=1,Evo\-1\-131k\)\. Model sizes range from under22M to77B parameters, with tokenization strategies includingkk\-mer \(n=13n=13\), BPE \(n=13n=13\), single\-nucleotide \(n=11n=11\), and three custom schemes \(BioFM\-265M’s BioToken,LucaOne’s mixed nucleotide–amino\-acid vocabulary, andGenomics\-FM’s ensemble of BPE andkk\-mer\)\. Pretraining corpora vary across multi\-species genomes \(n=20n=20\), human\-only \(n=12n=12\), eukaryotic gene sequences \(n=2n=2\), plant genomes \(n=2n=2\), human\-mouse epigenomic profiles \(n=2n=2\), prokaryotic sequences \(n=1n=1\), and multi\-species microbial genomes \(n=1n=1\)\.

The 100 downstream tasks are organized into 13 categories reflecting distinct aspects of genomic function: Histone Modifications \(30 tasks\), Promoters \(22\), Enhancers \(8\), DNA Methylation \(8\), Splice Sites \(7\), lncRNA \(6\), Mouse Enhancers \(5\), TF Binding \(5\), Species Classification \(3\), Regulatory \(2\), Virus/Phage \(2\), Coding/Non\-coding \(1\), and Chromatin Accessibility \(1\)\. The full task taxonomy is given in Appendix[B](https://arxiv.org/html/2606.04525#A2)\. This distribution enables fine\-grained analysis of model capabilities across regulatory, structural, and evolutionary prediction problems\.

### F\.2Aggregate Performance Patterns

Before examining specific architectural and design factors, we first characterize overall model performance across the benchmark\. Figure[9](https://arxiv.org/html/2606.04525#A6.F9)provides boxplot distributions of MCC scores within each task category for the top\-15 models per category, illustrating that performance variance differs substantially across categories\. Figure[10](https://arxiv.org/html/2606.04525#A6.F10)presents mean MCC scores for the top\-10 models across all 13 task categories, revealing substantial variation in category\-specific performance even among leading models\. Figure[11](https://arxiv.org/html/2606.04525#A6.F11)displays the distribution of model rankings across all 100 tasks, highlighting that even top\-ranked models exhibit considerable variance in per\-task rankings\. Finally, Figure[12](https://arxiv.org/html/2606.04525#A6.F12)shows the distribution of task\-level wins \(achieving the highest MCC on a given task\) and indicates that no single model dominates the benchmark: the top model wins only2020of100100tasks, with the remaining wins distributed across1515additional models\. This fragmentation underscores the importance of task\-aware model selection\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/zip_images/iloveimg-compressed/boxplot_per_task.png)Figure 9:Category\-wise MCC distributions across top\-performing models\.For each functional category, the figure reports the top\-15 models ranked by mean MCC\. Boxplots show the distribution of per\-task MCC values within the category: boxes denote the interquartile range, central lines indicate medians, whiskers show the non\-outlier range, and points mark outlier tasks\. For single\-task categories, individual MCC values are shown\. The dashed red line marks MCC=0\.5=0\.5\.
### F\.3Performance Landscape by Model Capacity

We stratify models into four capacity tiers to examine the relationship between parameter count and downstream performance\. Tiny models \(<<100M parameters,n=11n=11\) achieve a mean macro\-MCC of0\.4430\.443, withMutBERT\(86M\) leading the tier at0\.5290\.529\. Small models \(100M–500M,n=16n=16\) show modest improvement to0\.4850\.485mean macro\-MCC, topped byGENA\-LM\-Large\-T2T\(0\.5520\.552\) andEnformer\(0\.5490\.549\)\. Medium\-scale models \(500M–2B,n=6n=6\) reach0\.5260\.526mean macro\-MCC, withGENERator\-Eukaryote\-1\.2Bachieving the tier\-best0\.5810\.581\. Large models \(≥2\\geq 2B,n=7n=7\) attain0\.5330\.533mean macro\-MCC, led byGENERator\-Eukaryote\-3Bat0\.6050\.605\.

These tier\-level statistics reveal a substantial scaling pattern: the Spearman correlation betweenlog10⁡\(parameter count\)\\log\_\{10\}\(\\text\{parameter count\}\)and macro\-MCC isρ=0\.565\\rho=0\.565\(p<0\.001p<0\.001; see Section[4](https://arxiv.org/html/2606.04525#S4)and Figure[2](https://arxiv.org/html/2606.04525#S4.F2)\)\. Models above11B parameters achieve mean macro\-MCC of0\.5270\.527compared to0\.4630\.463for models below200200M, a gap of\+0\.064\+0\.064macro\-MCC\. While this association is statistically robust, it does not preclude substantial within\-tier variation: as shown in Section[4](https://arxiv.org/html/2606.04525#S4),3131in\-domain models demonstrate cases where a model at least5×5\\timessmaller outperforms a larger counterpart, indicating that scale alone is not a sufficient predictor of category\-level performance\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/image_per_task-3.png)Figure 10:Top\-10 model performance across task categories\.Mean MCC is shown for the 10 best\-performing models within each of the 13 functional task categories\. Models are ranked independently within each category by category\-level mean MCC, highlighting task\-specific leaders and performance differences across genomic prediction settings\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/rank_plot.png)Figure 11:Distribution of model ranks across benchmark tasks\.For each model, the boxplot shows the distribution of its task\-level ranks across 100 benchmark tasks, where lower rank indicates better performance\. Models are ordered by median rank, and the right column reports the average rank across all tasks\. The leading models combine low median rank with relatively compact rank distributions, indicating consistently strong cross\-task performance\. In contrast, models with wide interquartile ranges or numerous outliers show substantial task\-dependent variability, suggesting that aggregate performance can obscure pronounced category\- and task\-specific strengths\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/wins.png)Figure 12:Distribution of task\-level wins across models\.The figure reports the number of benchmark tasks, out of 100, on which each model achieves the highest MCC\. Only models with at least one task\-level win are included\. The left panel summarizes total wins per model, while the right panel decomposes these wins by functional task category\. The dispersed pattern of wins across models and categories indicates that benchmark performance is strongly task\-dependent and that no single model dominates uniformly across genomic prediction settings\.
### F\.4Architecture Comparison via Controlled Experiments

To isolate architectural effects from confounding factors, we identify model pairs sharing pretraining data and tokenization strategy while differing in architecture\. The full enumeration of controlled pairs is given in Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3)\. This controlled comparison reveals consistent patterns favoring attention\-based architectures over the evaluated state\-space alternative under matched conditions\.

Omni\-DNA\-1B\(Transformer\-decoder\) outperformseccDNAMamba\(Mamba\-SSM\) by\+0\.149\+0\.149macro\-MCC \(0\.5680\.568vs\.0\.4190\.419\), with both models trained on multi\-species data using BPE tokenization\. A second matched pair shows the same direction:GenomeOcean\-500M\(Transformer\-decoder\) exceedseccDNAMambaby\+0\.131\+0\.131macro\-MCC \(0\.5500\.550vs\.0\.4190\.419\) under identical pretraining and tokenization conditions\. This pattern, observed across two independent matched pairs, is consistent with attention\-based context modeling providing benefits over the evaluated state\-space architecture in this setting\.

Within the Transformer family, the encoder\-vs\-decoder ranking is less clear\-cut\.GENA\-LM\-Large\-T2T\(Transformer\-encoder\) outperformsOmniNA\-220M\(Transformer\-decoder\) by\+0\.127\+0\.127macro\-MCC \(0\.5520\.552vs\.0\.4250\.425\) under matched multi\-species/BPE conditions, andDNABERT\-2andGENA\-LM\-T2T\-Multisimilarly exceedOmniNA\-220Mby\+0\.086\+0\.086and\+0\.097\+0\.097macro\-MCC respectively\. However, as discussed in Section[4](https://arxiv.org/html/2606.04525#S4), the encoder–decoder comparison is task\- and setting\-dependent; the matched pairs reported here all share a single Transformer\-decoder baseline \(OmniNA\-220M\), and the pattern may not generalize to other Transformer\-decoder models\.

### F\.5Tokenization Strategy Effects

Tokenization represents a fundamental design choice that determines how nucleotide sequences are discretized for model consumption\. Our controlled comparisons reveal architecture\- and setting\-dependent effects rather than a universally optimal strategy \(Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3.SSS0.Px5)\)\.

GPT2\-Gene\-v1\(kk\-mer\) exceedsBioFM\-265M\(BioToken custom scheme\) by\+0\.134\+0\.134macro\-MCC \(0\.4760\.476vs\.0\.3420\.342\), with both sharing a Transformer\-decoder architecture and human pretraining\. However,kk\-mer is not uniformly preferred: under matched multi\-species pretraining,Omni\-DNA\-300M\(BPE\) outperformsGPT2\-Gene\-Multi\-v2\(kk\-mer\) by\+0\.071\+0\.071macro\-MCC \(0\.5380\.538vs\.0\.4670\.467\) within the Transformer\-decoder family\. Under matched human pretraining, single\-nucleotide tokenization \(MutBERT\) exceeds both BPE baselines, by\+0\.038\+0\.038overGROVERand\+0\.033\+0\.033overGENA\-LM\.

These mixed results indicate that tokenization interacts with other design choices – particularly pretraining data composition and model architecture – in ways that preclude a single global ordering\. The broader picture, including all 12 tokenization\-controlled pairs, is discussed in Section[4](https://arxiv.org/html/2606.04525#S4)and enumerated in Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3.SSS0.Px5)\.

### F\.6The Scale–Performance Paradox

A striking finding emerges from cross\-scale comparisons: numerous smaller models substantially outperform larger counterparts\. Among the3636in\-domain models \(excluding prokaryotic\-only, microbial\-only, and plant\-specific pretraining\), we identify3131instances where a model achieves superior aggregate macro\-MCC despite being at least5×5\\timessmaller than its comparison target\. Within the full 40\-model set, the count rises to7474, reflecting the additional contribution of cross\-domain cases\.

The most dramatic example involvesMutBERT\(86M parameters, macro\-MCC=0\.529=0\.529\) outperformingEvo\-1\-131k\(7B parameters, macro\-MCC=0\.298=0\.298\) by\+0\.231\+0\.231macro\-MCC despite an81×81\\timessize disadvantage\. The pattern repeats across cross\-domain comparisons withEvo\-1\-131kas the larger model:Omni\-DNA\-300MexceedsEvo\-1\-131kby\+0\.240\+0\.240macro\-MCC at a23×23\\timessize differential;GenomeOcean\-500Machieves a\+0\.252\+0\.252macro\-MCC advantage at14×14\\timessmaller scale; andOmni\-DNA\-1Boutperforms by\+0\.270\+0\.270macro\-MCC at7×7\\timesfewer parameters\. In\-domain reversals \(architectural rather than domain\-driven\) are illustrated in the main paper byMutBERToutperformingeccDNAMambaby\+0\.110\+0\.110macro\-MCC at an11\.6×11\.6\\timessize ratio \(Section[4](https://arxiv.org/html/2606.04525#S4)\)\.

The cross\-domain reversals are best interpreted as evidence of domain mismatch rather than as a universal critique of scaling\.Evo\-1\-131kwas pretrained exclusively on prokaryotic sequences, whereas1212of1313GENEB categories evaluate eukaryotic genomic functions \(Domain Mismatch paragraph, Section[4](https://arxiv.org/html/2606.04525#S4)\); the size disadvantages reported above therefore conflate scale with domain alignment\. The systematic analysis of pretraining data effects, restricted to controlled pairs matched on architecture and tokenization, is reported in Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3)\.

### F\.7Few\-Shot Learning Dynamics

We evaluate model robustness under data\-limited conditions through systematic few\-shot experiments\. Across all4040models, mean macro\-MCC degrades from0\.4880\.488\(full training data\) to0\.2530\.253\(1010\-shot\) to0\.1060\.106\(11\-shot\), corresponding to a78\.2%78\.2\\%performance reduction in the extreme low\-data regime \(Figure[5](https://arxiv.org/html/2606.04525#S4.F5)\)\.

Degradation patterns exhibit some heterogeneity across model capacity tiers but no monotonic trend\. Tiny models \(<<100M,n=11n=11\) decline from0\.4430\.443to0\.1090\.109\(75\.4%75\.4\\%degradation\); small models \(100M–500M,n=16n=16\) show a steeper decline from0\.4850\.485to0\.0930\.093\(80\.9%80\.9\\%\); medium models \(500M–2B,n=6n=6\) decline from0\.5260\.526to0\.1210\.121\(77\.1%77\.1\\%\); and large models \(≥2\\geq 2B,n=7n=7\) decline from0\.5330\.533to0\.1210\.121\(77\.4%77\.4\\%\)\. The lack of a clear size\-robustness trend at the tier level is consistent with the model\-level pattern reported below\.

Counterintuitively, the most few\-shot\-robust models in absolute terms are those with the weakest full\-data performance\. The five smallest absolute drops are observed forEvo\-1\-131k\(Δ=0\.196\\Delta=0\.196\),Caduceus\-PH\-1k\(0\.2200\.220\),JanusDNA\-72\-wo\(0\.2720\.272\),Caduceus\-PS\-131k\(0\.2720\.272\), andJanusDNA\-72\-w\(0\.2750\.275\), all of which rank among the weakest models in full\-shot evaluation\. Conversely, the strongest full\-shot performers exhibit the largest drops:GENERator\-Eukaryote\-3B\(Δ=0\.489\\Delta=0\.489\),GENERator\-Eukaryote\-1\.2B\(0\.4630\.463\),LucaOne\(0\.4610\.461\), andNT\-2\.5B\-MS\(0\.4560\.456\), all exceeding0\.420\.42in absolute drop\.

This inverse relationship should not be read as evidence of greater representational robustness in the lower\-performing models: a small absolute drop reflects a low full\-shot ceiling that leaves limited room for further degradation, not recovery of useful signal under 1\-shot supervision \(Section[4](https://arxiv.org/html/2606.04525#S4), Few\-Shot Robustness paragraph\)\. The practical implication is that aggregate few\-shot rankings conflate task tractability with model quality and should be interpreted alongside full\-shot performance, as discussed in Section[4](https://arxiv.org/html/2606.04525#S4)\.

### F\.8Pretraining Corpus Effects on Aggregate Performance

Grouping the 40 evaluated models by pretraining corpus type yields an ordering by mean macro\-MCC that is consistent with the controlled pretraining\-data comparisons reported in Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3)\. Eukaryotic gene\-focused pretraining yields the highest aggregate performance \(GENERator\-Eukaryote\-3B,GENERator\-Eukaryote\-1\.2B;n=2n=2, mean macro\-MCC=0\.593=0\.593\), followed by human\-mouse epigenomic profiles \(Enformer,SPACE;n=2n=2,0\.5440\.544\) and broad multi\-species corpora \(n=20n=20,0\.5220\.522\)\. Plant\-specific pretraining \(PlantCaduceus,Agro\-NT\-1B;n=2n=2,0\.4630\.463\), human\-only \(n=12n=12,0\.4300\.430\), and multi\-species microbial \(DNABERT\-S;n=1n=1,0\.4270\.427\) corpora form an intermediate band, with prokaryotic pretraining \(Evo\-1\-131k;n=1n=1,0\.2980\.298\) yielding the lowest aggregate macro\-MCC\.

We emphasize that this ordering is descriptive and confounded by architecture, tokenization, and parameter scale: only the controlled pairs reported in Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3)support clean attribution to pretraining corpus\. Within those controlled comparisons \(matched architecture and tokenization\), the largest effect observed in GENEB is the multi\-species vs\. microbial contrast:Genomics\-FM\(multi\-species\) exceedsDNABERT\-S\(multi\-species\-microbial\) by\+0\.088\+0\.088macro\-MCC under matched Transformer\-encoder/kk\-mer conditions, andNT\-v2\-100M\-MSsimilarly exceedsDNABERT\-Sby\+0\.081\+0\.081macro\-MCC\. The eukaryotic\-genes vs\. broad multi\-species contrast, supported by a single matched pair \(GENERator\-Eukaryote\-3Bvs\.DNA\-GPT\-3B\-M; both 3B, Transformer\-decoder,kk\-mer\), shows a\+0\.063\+0\.063macro\-MCC advantage for the eukaryotic\-genes corpus, but rests on insufficient data to support strong claims \(Transfer Learning Analysis paragraph, Section[4](https://arxiv.org/html/2606.04525#S4)\)\.

The ranking of aggregate means above is consistent with the biological expectation that taxonomic alignment between pretraining and downstream tasks supports transfer, since1212of1313GENEB categories evaluate eukaryotic genomic functions\. Models pretrained on prokaryotic or microbial corpora are correspondingly disadvantaged under aggregate evaluation, as discussed in the Domain Mismatch paragraph \(Section[4](https://arxiv.org/html/2606.04525#S4)\)\.

### F\.9Synthesis: Design Principles for DNA Foundation Models

Our systematic analysis yields several actionable principles for DNA foundation model development\. First, architectural choice matters but interacts with other factors: Transformer\-based models consistently outperform the evaluated state\-space alternative under controlled conditions, while the encoder vs\. decoder distinction is task\- and setting\-dependent \(Section[4](https://arxiv.org/html/2606.04525#S4), Architecture Comparison paragraph\)\. Second, tokenization effects are context\-dependent and cannot be optimized in isolation from architecture and pretraining choices\. Third, scale shows a substantial but non\-deterministic association with performance \(Spearmanρ=0\.565\\rho=0\.565,p<0\.001p<0\.001\): in\-domain cross\-scale reversals are common, and scale cannot compensate for data\-domain mismatch – an 86M\-parameter model trained on human sequences \(MutBERT\) outperforms a 7B\-parameter model trained on prokaryotic data \(Evo\-1\-131k\) by\+0\.231\+0\.231macro\-MCC\.

Most critically, pretraining corpus composition is a substantial contributor to downstream performance\. The descriptive ranking of aggregate means – eukaryotic genes\>\>human\-mouse profiles\>\>multi\-species\>\>plant\-specific\>\>human\-only≈\\approxmicrobial\>\>prokaryotic – reflects taxonomic alignment with evaluation tasks, although the ordering is confounded by architecture, tokenization, and parameter scale, and only the controlled pairs reported in Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3)support clean attribution to corpus type\. This pattern suggests that practitioners should prioritize domain\-appropriate pretraining alongside architectural and scale considerations when computational resources are constrained\.

### F\.10Transfer Learning Analysis

The diversity of pretraining data sources in our benchmark enables systematic investigation of cross\-domain transfer dynamics\. We analyze how representations learned from different taxonomic domains transfer to downstream tasks, revealing both positive and negative transfer phenomena with substantial practical implications\. All values in this section are macro\-averaged MCC \(per\-category averaging across the 13 functional categories of GENEB\) unless otherwise stated; the full enumeration of matched pairs underlying the controlled comparisons appears in Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3)\.

#### F\.10\.1Methodological Framework

To isolate pretraining\-data effects from confounding architectural and tokenization factors, we rely on controlled\-pair comparisons\. TheGENA\-LMpair provides an ideal natural experiment:GENA\-LM\(human\-only\) andGENA\-LM\-T2T\-Multi\(multi\-species\) share identical Transformer\-encoder architecture and BPE tokenization, differing principally in pretraining corpus\. This controlled setup enables more direct attribution of performance differences to data composition\. The full 30\-pair controlled\-comparison set is detailed in Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3)\.

#### F\.10\.2Controlled Comparison: Human versus Multi\-Species Pretraining

TheGENA\-LMcomparison reveals systematic advantages for multi\-species pretraining across the task spectrum\. Multi\-species training yields superior per\-category mean MCC in1111of1313task categories\. The largest advantages emerge for Chromatin Accessibility \(Δ=\+0\.123\\Delta=\+0\.123MCC:0\.5830\.583vs\.0\.4610\.461\), Mouse Enhancers \(\+0\.067\+0\.067:0\.5480\.548vs\.0\.4800\.480\), and Species Classification \(\+0\.033\+0\.033:0\.7070\.707vs\.0\.6740\.674\)\. Human\-only pretraining shows marginal advantages only for Enhancers \(\+0\.010\+0\.010MCC in favor of human\-only\) and Regulatory tasks \(\+0\.001\+0\.001\), with differences within noise margins\.

This pattern generalizes beyond theGENA\-LMpair\. Across all six available human\-vs\. multi\-species controlled comparisons \(Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3), Table[16](https://arxiv.org/html/2606.04525#A5.T16)\), multi\-species pretraining is favored for Chromatin Accessibility \(6/66/6pairs\), lncRNA \(5/65/6pairs\), Splice Sites \(4/64/6pairs\), and Mouse Enhancers \(4/64/6pairs\)\. Human\-only pretraining shows consistent advantage only for Virus/Phage \(4/64/6pairs, meanΔ=−0\.034\\Delta=\-0\.034MCC\), likely reflecting the predominance of human\-associated viral sequences in human\-genome training data\.

#### F\.10\.3Negative Transfer: Human to Plant Domains

The lncRNA task category provides a stringent test of cross\-kingdom transfer, comprising six plant\-specific classification tasks spanningGlycine max\(soybean\),Manihot esculenta\(cassava\),Sorghum bicolor\(sorghum\),Solanum lycopersicum\(tomato\),Triticum aestivum\(wheat\), andZea mays\(maize\)\. Human\-trained models exhibit substantial negative transfer on these tasks, achieving mean lncRNA MCC of only0\.1570\.157compared to0\.3470\.347for plant\-trained models – a deficit of0\.1900\.190MCC, or roughly a121%121\\%relative gain from domain\-appropriate pretraining\.

Task\-level analysis reveals consistent patterns across all six plant species\. OnG\. maxlncRNA classification, human\-trained models average0\.1200\.120MCC versus0\.3090\.309for plant\-trained models\. Similar gaps emerge forS\. bicolor\(0\.1720\.172vs\.0\.4080\.408\),M\. esculenta\(0\.1370\.137vs\.0\.4080\.408\), and the remaining three species\. Even the best\-performing human\-trained model \(MutBERT, lncRNA MCC=0\.260=0\.260\) substantially underperforms plant\-specialized models \(PlantCaduceus,0\.3570\.357;Agro\-NT\-1B,0\.3360\.336\)\.

Notably, multi\-species models achieve intermediate performance \(mean lncRNA MCC=0\.304=0\.304\), withLucaOnereaching0\.5080\.508– the best overall result on plant lncRNA tasks\. This suggests that broad taxonomic coverage partially compensates for the lack of plant\-specific pretraining, though dedicated plant models retain advantages on most individual tasks\.

#### F\.10\.4Positive Transfer: Multi\-Species to Human\-Specific Tasks

A counterintuitive finding emerges from analysis of predominantly human\-derived tasks: multi\-species pretraining consistently outperforms human\-only pretraining even on task categories whose underlying datasets come primarily from human genomic data\. This positive transfer phenomenon manifests across all five human\-centric task categories examined here\.

For Histone Modifications, multi\-species models achieve mean MCC of0\.4730\.473compared to0\.3970\.397for human\-trained models \(Δ=\+0\.076\\Delta=\+0\.076\)\. Chromatin Accessibility shows the largest gap at\+0\.097\+0\.097\(0\.6020\.602vs\.0\.5050\.505\)\. Regulatory element prediction exhibits\+0\.059\+0\.059advantage \(0\.4240\.424vs\.0\.3650\.365\), Enhancer detection\+0\.050\+0\.050\(0\.4620\.462vs\.0\.4120\.412\), and TF Binding\+0\.027\+0\.027\(0\.5690\.569vs\.0\.5420\.542\)\. The consistency of multi\-species advantages across functionally diverse task categories is consistent with exposure to evolutionarily conserved sequence patterns during pretraining supporting representation quality even for species\-specific downstream applications\.

#### F\.10\.5Catastrophic Transfer Failure: Prokaryotic to Eukaryotic Domains

The most extreme transfer failure involvesEvo\-1\-131k, a 7B\-parameter model pretrained exclusively on prokaryotic sequences\. Despite its substantial scale,Evo\-1\-131kachieves only0\.2980\.298overall macro\-MCC, ranking last \(4040of4040models\) and underperformingMutBERT\(86M parameters, human pretraining\) by0\.2310\.231macro\-MCC – an8181\-fold parameter disadvantage yielding inferior results\.

Category\-level analysis reveals the biological basis for this failure\.Evo\-1\-131kranks last \(40/4040/40\) on Splice Sites \(MCC=0\.160=0\.160\), TF Binding \(0\.1730\.173\), Species Classification \(0\.2850\.285\), and DNA Methylation \(0\.0730\.073\)\. These failures reflect fundamental differences between prokaryotic and eukaryotic genomic organization: prokaryotes lack spliceosomal introns, employ distinct transcription factor families, and utilize different DNA methylation machinery\. The only category whereEvo\-1\-131kachieves competitive performance is Coding/Non\-coding Classification \(MCC=0\.719=0\.719, rank36/4036/40\), reflecting the more universal nature of coding sequence signatures across domains of life\.

We emphasize three scope qualifiers for this observation\.*First*, the result reflects performance under frozen linear\-probing of pretrained representations – the evaluation protocol applied uniformly across all4040benchmark models\. It does not preclude the possibility that task\-specific full fine\-tuning ofEvo\-1\-131kon eukaryotic tasks would close part of the gap, and is not a statement aboutEvo\-1\-131k’s capabilities within its intended prokaryotic application domain, on which it was not evaluated here\.*Second*,Evo\-1\-131kis the only prokaryotic\-only model in GENEB; broader claims about prokaryotic\-to\-eukaryotic transfer would require additional prokaryotic\-pretrained models, which we flag as a coverage limitation\.*Third*, the magnitude of the failure on biologically structured tasks \(splicing, DNA methylation\) is consistent with the prior expectation that representations of prokaryotic sequence statistics carry limited information about eukaryotic\-specific molecular machinery, and we frame this observation as evidence of domain mismatch rather than a universal critique of the underlying model architecture or training methodology\.

#### F\.10\.6Partial Transfer: Microbial to Eukaryotic Domains

DNABERT\-S, pretrained on multi\-species microbial genomes, provides an intermediate case between prokaryotic\-only and eukaryotic multi\-species pretraining\. With overall macro\-MCC of0\.4270\.427,DNABERT\-Ssubstantially outperforms prokaryotic\-onlyEvo\-1\-131k\(0\.2980\.298\) but underperforms eukaryotic multi\-species models \(mean0\.5220\.522\)\.

Controlled comparison withNT\-v2\-100M\-MS\(eukaryotic multi\-species, matched architecture and tokenization\) reveals systematic deficits for the microbial corpus:NT\-v2\-100M\-MSexceedsDNABERT\-Sby\+0\.081\+0\.081overall macro\-MCC, with the largest per\-category gaps on Splice Sites \(\+0\.183\+0\.183\) and Species Classification \(\+0\.187\+0\.187\)\. The splice site deficit again reflects the absence of spliceosomal machinery in microbial training sequences\. Interestingly,DNABERT\-Sshows unusual strength on Regulatory tasks \(\+0\.069\+0\.069vs\.Genomics\-FM\), potentially reflecting transferable sequence features in regulatory regions across bacterial and eukaryotic genomes\. This intermediate\-corpus observation rests on a single microbial\-pretrained model in GENEB \(Appendix[D](https://arxiv.org/html/2606.04525#A4)\); broader claims about microbial\-to\-eukaryotic transfer await additional matched models\.

#### F\.10\.7Strong Positive Transfer: Eukaryotic Gene\-Focused Pretraining

TheGENERatormodels, pretrained on curated eukaryotic gene sequences, achieve the strongest overall transfer performance in our benchmark \(mean macro\-MCC=0\.593=0\.593\)\. Remarkably, these models exceed benchmark\-wide averages across all1313task categories, with largest advantages on DNA Methylation \(\+0\.200\+0\.200vs\. benchmark mean\), Splice Sites \(\+0\.198\+0\.198\), lncRNA \(\+0\.187\+0\.187\), Species Classification \(\+0\.137\+0\.137\), Mouse Enhancers \(\+0\.136\+0\.136\), and Chromatin Accessibility \(\+0\.117\+0\.117\)\.

Controlled comparison betweenGENERator\-Eukaryote\-3BandDNA\-GPT\-3B\-M\(both 3B parameters, Transformer\-decoder,kk\-mer tokenization\) isolates the effect of gene\-focused versus general multi\-species pretraining\. The gene\-focused approach yields\+0\.063\+0\.063overall macro\-MCC advantage, with particularly strong gains on Chromatin Accessibility \(\+0\.191\+0\.191\), lncRNA \(\+0\.142\+0\.142\), and Mouse Enhancers \(\+0\.124\+0\.124\)\. This pattern is consistent with curation of pretraining data toward functionally annotated genomic regions supporting downstream task performance beyond what raw sequence diversity provides\. We emphasize that this contrast rests on a single matched pair and should be interpreted with corresponding caution; see the Transfer Learning Analysis paragraph \(Section[4](https://arxiv.org/html/2606.04525#S4)\) and the single\-pair caveat in Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3)\.

#### F\.10\.8Specialized Transfer: Human\-Mouse Epigenomic Profiles

EnformerandSPACE, pretrained on human and mouse epigenomic profiles, exhibit a distinctive transfer pattern characterized by strong performance on regulatory tasks but deficits on sequence\-intrinsic features\. These models achieve mean macro\-MCC of0\.5440\.544overall, with substantial advantages on Regulatory tasks \(\+0\.194\+0\.194vs\. benchmark mean\), Mouse Enhancers \(\+0\.170\+0\.170\), Chromatin Accessibility \(\+0\.133\+0\.133\), and TF Binding \(\+0\.116\+0\.116\)\.

However, this specialization comes at a cost: human\-mouse\-profile models underperform on DNA Methylation \(−0\.104\-0\.104\), Coding/Non\-coding Classification \(−0\.061\-0\.061\), and lncRNA \(−0\.021\-0\.021\)\. This trade\-off reflects the nature of epigenomic profile pretraining, which emphasizes chromatin state and regulatory\-element patterns while potentially underweighting primary sequence features\. Practitioners should consider this specialization when selecting models for specific application domains \(see also Practitioner Recommendations, Section[4](https://arxiv.org/html/2606.04525#S4)\)\.

#### F\.10\.9Transfer Dynamics on Mouse Enhancer Tasks

The five Mouse Enhancer tasks provide a detailed view of how pretraining data influences performance on a homogeneous task category\. Human\-mouse\-profile models \(Enformer,SPACE\) achieve top\-1 performance on tasks0\(EnformerMCC=0\.667=0\.667\),22\(0\.8780\.878\), and33\(0\.7720\.772\), with substantial margins over human\-only models on these tasks\. The remaining two tasks \(1 and 4\) are won byOmni\-DNA\-1B\(Transformer\-decoder, multi\-species, BPE; MCC=0\.807=0\.807and0\.4880\.488respectively\)\. Eukaryotic\-geneGENERatormodels show consistent strong performance across all five tasks \(mean0\.6120\.612\) without claiming any individual top\-1 position\. Aggregating by pretraining\-corpus group, mean macro\-MCC on the Mouse Enhancer category is0\.6460\.646for human\-mouse\-profile models,0\.6120\.612forGENERator,0\.5140\.514for broad multi\-species,0\.3990\.399for plant\-trained, and0\.3950\.395for human\-only models\.

The superior performance of human\-mouse\-profile models on mouse enhancer tasks – despite these models not being trained on general genomic sequences – demonstrates that task\-relevant pretraining signals can outweigh broader sequence coverage when the downstream task aligns tightly with the pretraining target\. This pattern is consistent with the practitioner guidance that model selection should be informed by task–domain alignment rather than aggregate benchmark performance alone \(see Practitioner Recommendations, Section[4](https://arxiv.org/html/2606.04525#S4)\)\.

#### F\.10\.10Synthesis: Transfer Learning Principles

Our systematic analysis of cross\-domain transfer reveals five recurring patterns governing DNA foundation model generalization\.

First, taxonomic alignment between pretraining and downstream domains is critical for successful transfer\. Prokaryotic pretraining fails on eukaryotic tasks, while human pretraining shows negative transfer to plant\-specific tasks\. The magnitude of these failures – 7B parameters underperforming 86M parameters by0\.2310\.231macro\-MCC – underscores that domain mismatch cannot be overcome through scale alone\.

Second, taxonomic diversity in pretraining provides positive transfer even to ostensibly species\-specific tasks\. Multi\-species models outperform human\-only models on all five human\-specific task categories examined here, consistent with broad sequence exposure during pretraining supporting transferable representations\. This pattern argues against narrow species\-specific pretraining for general\-purpose foundation models\.

Third, specialized pretraining creates performance trade\-offs rather than uniform improvements\. Human\-mouse epigenomic profile models excel on regulatory tasks \(\+0\.194\+0\.194vs\. benchmark mean\) but underperform on DNA methylation \(−0\.104\-0\.104\)\. Gene\-focused eukaryotic pretraining achieves the highest overall performance but relies on curated training data\. These trade\-offs suggest that no single pretraining strategy optimally serves all downstream applications\.

Fourth, model scale cannot compensate for fundamental data\-domain mismatch\. The8181\-fold parameter advantage ofEvo\-1\-131koverMutBERTis entirely negated by the prokaryotic–eukaryotic domain gap\. This finding has practical resource\-allocation implications: investment in domain\-appropriate training data may yield higher returns than equivalent investment in model scale when domain mismatch exists\.

Fifth, the observed transfer patterns suggest that model selection should be informed by task–domain alignment rather than aggregate benchmark performance alone\. Different pretraining strategies excel on different task categories, motivating per\-category selection guidance: regulatory tasks favor epigenomic\-profile models \(Enformer,SPACE\); general eukaryotic tasks favor multi\-species or gene\-focused models; and for plant\-specific tasks, dedicated plant\-trained models substantially exceed human\-trained baselines \(mean lncRNA MCC0\.3470\.347vs\.0\.1570\.157\), while the best individual results come from select multi\-species models such asLucaOne\(lncRNA MCC=0\.508=0\.508\)\. The per\-category Practitioner Recommendations in Section[4](https://arxiv.org/html/2606.04525#S4)provide operational guidance consistent with these alignment principles\.

### F\.11Specialization Score: Formal Definition and Computation

For each modelmmand each taskttwe compute the per\-task rankrm,t∈\{1,…,40\}r\_\{m,t\}\\in\\\{1,\\ldots,40\\\}from full\-shot MCC, with rank11denoting the best\-performing model on tasktt\. For a categoryCCcontaining tasks𝒯C\\mathcal\{T\}\_\{C\}, the within\-category mean rank of modelmmis

r¯m,C=1\|𝒯C\|​∑t∈𝒯Crm,t,\\bar\{r\}\_\{m,C\}\\;=\\;\\frac\{1\}\{\|\\mathcal\{T\}\_\{C\}\|\}\\sum\_\{t\\in\\mathcal\{T\}\_\{C\}\}r\_\{m,t\},and the corresponding outside\-category mean rank is

r¯m,¬C=1\|𝒯∖𝒯C\|​∑t∉𝒯Crm,t,\\bar\{r\}\_\{m,\\neg C\}\\;=\\;\\frac\{1\}\{\|\\mathcal\{T\}\\setminus\\mathcal\{T\}\_\{C\}\|\}\\sum\_\{t\\notin\\mathcal\{T\}\_\{C\}\}r\_\{m,t\},where𝒯\\mathcal\{T\}is the full set of100100GENEB tasks\. The specialization score of modelmmon categoryCCis then

Δm,C=r¯m,¬C−r¯m,C\.\\Delta\_\{m,C\}\\;=\\;\\bar\{r\}\_\{m,\\neg C\}\\;\-\\;\\bar\{r\}\_\{m,C\}\.A positiveΔm,C\\Delta\_\{m,C\}indicates that modelmmranks better on categoryCCthan on the rest of the benchmark; values above∼5\\sim 5reflect substantial relative strength\. The largest specialization scores observed in GENEB areBioFM\-265Mon DNA methylation \(Δ=26\.3\\Delta=26\.3\),PlantCaduceuson lncRNA \(Δ=19\.0\\Delta=19\.0\),JanusDNA\-72\-won chromatin accessibility \(Δ=17\.9\\Delta=17\.9\), andeccDNAMambaon chromatin accessibility \(Δ=16\.5\\Delta=16\.5\)\. We use the score as a diagnostic measure throughout Appendix[F](https://arxiv.org/html/2606.04525#A6), particularly when discussing model families whose category\-level strengths differ from their aggregate ranking\.

### F\.12Task\-Category\-Specific Analysis

Having established general principles governing DNA foundation model performance, we now examine category\-specific patterns across six major task groups\. This fine\-grained analysis reveals substantial heterogeneity in optimal model configurations across functional genomic prediction problems\.

#### F\.12\.1Histone Modifications

The Histone Modifications category \(n=30n=30tasks\) spans diverse chromatin marks including H3K4me1, H3K4me2, H3K4me3, H3K27ac, H3K36me3, and H4 modifications\. Task difficulty varies substantially: the easiest individual task is GUE EMP H4 prediction \(mean MCC=0\.720=0\.720across the4040models\), while the most challenging is GUE EMP H3K4me3 prediction \(mean MCC=0\.243=0\.243\)\. Results for the 30 histone modification prediction tasks are presented in Figures[13](https://arxiv.org/html/2606.04525#A6.F13)–[15](https://arxiv.org/html/2606.04525#A6.F15)\.

Architecture comparisons under controlled conditions reveal substantial advantages for Transformer\-based models\.GenomeOcean\-500M\(Transformer\-decoder\) exceedseccDNAMamba\(Mamba\-SSM\-bidirectional\) by0\.1530\.153MCC \(0\.5370\.537vs\.0\.3840\.384\), with both models trained on multi\-species data using BPE tokenization\. A within\-Transformer contrast \(decoder vs\. encoder\) betweenGenomeOcean\-500M\(500M, decoder, BPE, multi\-species\) andGENA\-LM\-Large\-T2T\(336M, encoder, BPE, multi\-species\) shows a0\.0690\.069MCC advantage for the decoder model, although the two models differ in parameter count and the contrast is therefore not strictly matched on scale\.

Pretraining data effects follow the general hierarchy established in Section[F\.8](https://arxiv.org/html/2606.04525#A6.SS8)\. Eukaryotic gene\-focusedGENERator\-Eukaryote\-3Bexceeds multi\-speciesDNA\-GPT\-3B\-Mby0\.0600\.060MCC \(0\.5370\.537vs\.0\.4770\.477\) in controlled comparison \(both 3B, Transformer\-decoder,kk\-mer tokenization\)\. Multi\-species models consistently outperform microbial\-trained alternatives, withGenomics\-FMexceedingDNABERT\-Sby0\.0960\.096MCC \(0\.4690\.469vs\.0\.3730\.373\) under matched Transformer\-encoder/kk\-mer conditions\. Among small\-scale \(<<100M\) models,MutBERT\(86M\) achieves the highest Histone Modifications MCC \(0\.5010\.501\), ranking88th of4040overall on this category\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/kshot_histone.png)Figure 13:Few\-shot performance degradation on Histone Modifications\.For each of the 40 models, macro\-average MCC across 30 histone modification tasks under full\-shot, 10\-shot, and 1\-shot regimes; models ordered by full\-shot performance\. The top band shows the relative drop from full\-shot to 10\-shot per model\. Benchmark\-wide mean degradation:51\.0%51\.0\\%for 10\-shot,80\.3%80\.3\\%for 1\-shot\. The 1\-shot regime collapses to near\-random performance, while 10\-shot retains discriminative signal; full\-shot and 10\-shot rankings differ substantially \(Spearmanρ=0\.77\\rho=0\.77\)\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/pareto_histone.png)Figure 14:Pareto frontier for Histone Modifications: mean MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmic x\-axis and mean full\-shot Histone Modifications MCC on the y\-axis\. Marker size and color both encode MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\. Scale shows a positive but non\-deterministic association with Histone Modifications performance: several smaller models sit on or near the frontier, includingMutBERT\(86M, mean MCC=0\.501=0\.501\), which is the strongest sub\-100M model on this category\. Other small models such asJanusDNA\-72\-wandJanusDNA\-72\-wo\(both 1\.98M parameters\) sit well below the frontier, illustrating that small parameter count is neither sufficient nor consistent for competitive performance on this category\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/zip_images/iloveimg-compressed/histone_heatmap.png)Figure 15:Per\-task MCC for Histone Modifications\.Heatmap shows full\-shot MCC for each of the 40 genomic foundation models on the 30 histone modification tasks, with models sorted by mean Histone Modifications MCC\. Cell values report per\-task MCC, with colors ranging from red/orange for lower scores to green for higher scores\. Task difficulty varies substantially across marks: H4\-family tasks \(H4, H4ac, H4K20me1\) are consistently among the easiest, while H3K4me2/me3 tasks remain among the hardest across most models\. Tasks from the NT, NT\-revised, and GUE EMP sources show broadly similar within\-mark patterns, supporting the consolidation of these sources into a single category\.Few\-shot performance on histone modification tasks degrades severely\. The mean macro\-MCC across models drops from0\.4470\.447\(full data\) to0\.2190\.219\(1010\-shot\) to0\.0880\.088\(11\-shot\), corresponding to51\.0%51\.0\\%and80\.3%80\.3\\%relative reductions respectively\. We focus on the1010\-shot regime here as the11\-shot regime collapses to near\-random performance for all models \(no model exceeds0\.160\.16macro\-MCC at11\-shot\), limiting its discriminative value\.

At1010\-shot, the ranking of models differs substantially from full\-shot performance \(Spearmanρ=0\.77\\rho=0\.77across all 40 models\)\. The top1010\-shot performer isMutBERT\(86M parameters;1010\-shot MCC=0\.300=0\.300, full\-shot=0\.501=0\.501\), which ranks1111th at full\-shot, illustrating that compact, efficiency\-oriented models can retain operationally useful transferability under data\-limited conditions where larger gene\-focused models do not \(e\.g\.,GENERator\-Eukaryote\-3Bdrops to1010\-shot MCC=0\.246=0\.246from a full\-shot MCC of0\.5370\.537\)\. Of the top\-5 models by full\-shot MCC, only22remain in the top\-5 at1010\-shot \(GenomeOcean\-4B,GenomeOcean\-500M\)\. As discussed in Section[F\.7](https://arxiv.org/html/2606.04525#A6.SS7), this reranking reflects task\- and setting\-dependent few\-shot dynamics rather than a single property of model robustness\.

#### F\.12\.2Promoter Recognition

Promoter recognition tasks \(n=22n=22tasks\) span bacterial, plant, and mammalian systems with substantial difficulty variation\. Cell\-type\-specific human promoter tasks dominate the easy end \(iPro\-WAELHUVEC: mean MCC=0\.890=0\.890; HeLa\-S3:0\.8750\.875; GM12878:0\.8560\.856\), while bacterial promoter prediction forR\. capsulatusis the most challenging task \(0\.2740\.274\)\. Results for the 22 promoter prediction tasks are presented in Figures[16](https://arxiv.org/html/2606.04525#A6.F16)–[18](https://arxiv.org/html/2606.04525#A6.F18)\.

The scaling relationship is weaker for promoter tasks than for the benchmark overall\. The Spearman correlation betweenlog10⁡\(parameter count\)\\log\_\{10\}\(\\text\{parameter count\}\)and mean Promoter MCC isρ=0\.487\\rho=0\.487\(p=0\.001p=0\.001\), compared toρ=0\.565\\rho=0\.565across all categories \(Section[4](https://arxiv.org/html/2606.04525#S4)\)\. Models with at least 1B parameters achieve mean MCC of0\.7170\.717versus0\.6820\.682for models below 200M – a gap of0\.0350\.035MCC, narrower than the\+0\.064\+0\.064tier gap observed at the benchmark\-aggregate level\. This compressed performance range suggests that promoter sequence features are relatively accessible to smaller models\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/kshot_promoter.png)Figure 16:Few\-shot performance degradation on Promoter Recognition\.For each of the 40 models, macro\-average MCC across 22 promoter prediction tasks under full\-shot, 10\-shot, and 1\-shot regimes; models ordered by full\-shot performance\. The top band shows the relative drop from full\-shot to 10\-shot per model\. Benchmark\-wide mean degradation:30\.7%30\.7\\%for 10\-shot,61\.2%61\.2\\%for 1\-shot\. Unlike histone modifications, the 1\-shot regime retains operationally meaningful signal on this category \(maximum 1\-shot MCC=0\.363=0\.363\); full\-shot and 10\-shot rankings remain strongly correlated \(Spearmanρ=0\.85\\rho=0\.85\)\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/pareto_promoter.png)Figure 17:Pareto frontier for Promoter Recognition: mean MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmic x\-axis and mean full\-shot Promoter MCC on the y\-axis\. Marker size and color both encode MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\. Scaling is comparatively weak on this category \(Spearmanρ=0\.487\\rho=0\.487; tier gap≥1\\geq 1B vs\.<<200M of only0\.0350\.035MCC\), consistent with promoter sequence features being accessible to smaller models\.MutBERT\(86M, mean MCC=0\.739=0\.739, ranked99th overall on this category\) is the strongest sub\-100M model, sitting near the Pareto frontier alongside several multi\-billion\-parameter models\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/heatmap_promoter.png)Figure 18:Per\-task MCC for Promoter Recognition\.Heatmap shows full\-shot MCC for each of the 40 genomic foundation models on the 22 promoter prediction tasks, with models sorted by mean Promoter MCC\. Cell values report per\-task MCC, with colors ranging from red/orange for lower scores to green for higher scores\. Task difficulty varies substantially with sequence source and host species: cell\-type\-specific human promoter prediction \(HUVEC, HeLa\-S3, GM12878\) is consistently the easiest, while bacterial promoter prediction \(R\. capsulatus,B\. amyloliquefaciens\) and plant promoter prediction \(M\. esculenta\) are the most challenging\.Architecture comparisons maintain Transformer advantages over the evaluated state\-space alternative:Omni\-DNA\-1B\(Transformer\-decoder, 1B\) outperformseccDNAMamba\(Mamba\-SSM, 1B\) by0\.1070\.107MCC \(0\.7590\.759vs\.0\.6520\.652\), with both models trained on multi\-species data using BPE tokenization\. Within Transformer\-family comparisons,GENA\-LM\-Large\-T2T\(encoder, BPE, multi\-species, 336M\) exceedsOmniNA\-220M\(decoder, BPE, multi\-species, 220M\) by0\.0950\.095MCC \(0\.7450\.745vs\.0\.6500\.650\), although the two models differ in parameter count and the contrast is therefore not strictly matched on scale\. We note that this single\-pair encoder advantage on Promoters contrasts with the within\-Transformer decoder advantage observed on Histone Modifications, supporting the broader observation \(Section[4](https://arxiv.org/html/2606.04525#S4), Architecture Comparison paragraph\) that the encoder–decoder distinction is task\- and setting\-dependent\.

Pretraining data comparisons reveal nuanced patterns for promoter tasks\.GENA\-LM\-T2T\-Multi\(multi\-species\) exceedsGENA\-LM\(human\-only\) by only0\.0130\.013MCC \(0\.7100\.710vs\.0\.6970\.697\) under matched Transformer\-encoder/BPE conditions, a narrower advantage than the average multi\-vs\.\-human pretraining contrast reported in Appendix[E\.3](https://arxiv.org/html/2606.04525#A5.SS3)\. Notably, in the Transformer\-decoder/kk\-mer matched pair,GPT2\-Gene\-v1\(human\-only\) slightly exceedsGPT2\-Gene\-Multi\-v2\(multi\-species\) by0\.0160\.016MCC, consistent with the observation that the multi\-species advantage is not invariant to architecture and tokenization choice \(Section[F\.10](https://arxiv.org/html/2606.04525#A6.SS10)\)\.

Few\-shot performance on promoter tasks degrades more gracefully than on histone modifications\. Across the 40 models, mean MCC drops from0\.6930\.693\(full data\) to0\.4800\.480\(10\-shot\) to0\.2690\.269\(1\-shot\), corresponding to30\.7%30\.7\\%and61\.2%61\.2\\%relative reductions respectively\. Unlike histone modification, several models retain operationally meaningful 1\-shot performance \(maximum 1\-shot MCC =0\.3630\.363forGenomeOcean\-4B\)\. At 10\-shot, the top performers remainGENERator\-Eukaryote\-3B\(0\.5880\.588\),GENERator\-Eukaryote\-1\.2B\(0\.5800\.580\), andGENA\-LM\-Large\-T2T\(0\.5630\.563\); the full\-shot top\-5 and 10\-shot top\-5 overlap by 3 models, and full\-shot vs\. 10\-shot rankings correlate at Spearmanρ=0\.85\\rho=0\.85\.

#### F\.12\.3Enhancer Prediction

Enhancer prediction tasks \(n=8n=8tasks\) present moderate difficulty with mean MCC of0\.4460\.446across the4040models\. The category spans human enhancer prediction \(NT, NT\-rev, and GB Cohn and Ensembl subsets\), enhancer\-type classification \(NT and NT\-rev multi\-class variants\),Drosophilaenhancers \(GB Stark\), and a mouse enhancer task \(GB Ensembl\)\. Within\-category difficulty varies from a low of0\.3210\.321\(NT Enhancers \(types\), multi\-class\) to a high of0\.5670\.567\(GB Mouse enh\. \(Ensembl\)\)\. Results for the 8 enhancer prediction tasks are presented in Figures[19](https://arxiv.org/html/2606.04525#A6.F19)–[21](https://arxiv.org/html/2606.04525#A6.F21)\.

Human\-mouse epigenomic profile models demonstrate the strongest full\-shot performance on enhancer tasks, consistent with their specialized pretraining on regulatory elements\.Enformerachieves the top mean MCC on the category \(0\.5390\.539; rank11of4040by mean Enhancer MCC\) while ranking1010th of4040overall on the benchmark, a category\-specific advantage\.SPACEfollows closely \(mean MCC=0\.526=0\.526; rank22on Enhancers\)\. The advantage reflects the close relationship between enhancer sequences and the chromatin\-state signals present in human\-mouse epigenomic profile pretraining \(Section[F\.10](https://arxiv.org/html/2606.04525#A6.SS10)\)\.

Category\-favoring specialization \(defined as the gap between a model’s average per\-task rank on the 92 non\-enhancer tasks and its average per\-task rank on the 8 enhancer tasks\) is observed across diverse architectures rather than being confined to a single architectural family\. BeyondEnformer\(Δ=\+11\.3\\Delta=\+11\.3\) andSPACE\(Δ=\+7\.2\\Delta=\+7\.2\), notable specializers includeJanusDNA\-72\-wo\(Δ=\+7\.6\\Delta=\+7\.6; Hybrid\-Mamba\-MoE,1\.981\.98M parameters\),GENA\-LM\(Δ=\+7\.2\\Delta=\+7\.2; Transformer\-encoder, human\-only, 110M\),DNABERT\-2\(Δ=\+6\.4\\Delta=\+6\.4; Transformer\-encoder, multi\-species, 117M\), andGROVER\(Δ=\+5\.2\\Delta=\+5\.2; Transformer\-encoder, human\-only, 87M\)\. ThatJanusDNA\-72\-woappears among the top specializers at1\.981\.98M parameters is noteworthy given its very small scale; among sub\-100M models on this category, however,MutBERT\(86M, mean MCC=0\.475=0\.475\) achieves the highest full\-shot Enhancer MCC\. The architectural diversity of strong specializers argues against attributing enhancer specialization to any single design choice\.

Few\-shot performance degrades substantially on enhancer tasks\. Across the 40 models, mean MCC drops from0\.4460\.446\(full data\) to0\.2780\.278\(10\-shot\) to0\.1340\.134\(1\-shot\), corresponding to37\.6%37\.6\\%and70\.0%70\.0\\%relative reductions respectively\. The 1\-shot regime collapses to near\-random performance \(maximum 1\-shot MCC=0\.195=0\.195foreccDNAMamba\)\. The 10\-shot ranking differs sharply from the full\-shot ranking \(Spearmanρ=0\.64\\rho=0\.64; top\-5 overlap of only 1 model out of 5\), with both epigenomic\-profile specialists losing their full\-shot advantage:Enformerdrops from full\-shot mean MCC0\.5390\.539\(rank11\) to 10\-shot MCC0\.2520\.252\(rank3434\), andSPACEdrops from0\.5260\.526\(rank22\) to0\.3070\.307\(rank1010\), illustrating that strong full\-data category specialization need not translate to few\-shot transferability\. At 10\-shot, the top performers areGENA\-LM\-Large\-T2T\(0\.3720\.372\),Omni\-DNA\-300M\(0\.3420\.342\), andOmni\-DNA\-1B\(0\.3420\.342\)\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/kshot_enchancers.png)Figure 19:Few\-shot performance degradation on Enhancer Prediction\.For each of the 40 models, macro\-average MCC across 8 enhancer prediction tasks under full\-shot, 10\-shot, and 1\-shot regimes; models ordered by full\-shot performance\. The top band shows the relative drop from full\-shot to 10\-shot per model\. Benchmark\-wide mean degradation:37\.6%37\.6\\%for 10\-shot,70\.0%70\.0\\%for 1\-shot\. The 1\-shot regime collapses to near\-random performance \(maximum 1\-shot MCC=0\.195=0\.195\); the 10\-shot ranking diverges sharply from the full\-shot ranking \(Spearmanρ=0\.64\\rho=0\.64\), with epigenomic\-profile specialists losing their full\-shot advantage \(Enformer: full=0\.539=0\.539, 10\-shot=0\.252=0\.252;SPACE: full=0\.526=0\.526, 10\-shot=0\.307=0\.307\)\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/pareto_enhancers.png)Figure 20:Pareto frontier for Enhancer Prediction: mean MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmicxx\-axis and mean full\-shot Enhancer MCC on theyy\-axis\. Marker size and color both encode MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\. Scaling on this category is moderate \(Spearmanρ=0\.491\\rho=0\.491; tier gap≥1\\geq 1B vs\.<<200M of0\.0310\.031MCC\)\. Specialized human\-mouse epigenomic\-profile models occupy the frontier at moderate sizes \(Enformer,250250M, MCC=0\.539=0\.539;SPACE,589589M, MCC=0\.526=0\.526\), outperforming multi\-billion\-parameter generalist models\.MutBERT\(8686M, mean MCC=0\.475=0\.475\) is the strongest sub\-100M model on this category\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/heatmap_enhancers.png)Figure 21:Per\-task MCC for Enhancer Prediction\.Heatmap shows full\-shot MCC for each of the 40 genomic foundation models on the 8 enhancer prediction tasks, with models sorted by mean Enhancer MCC\. Cell values report per\-task MCC, with colors ranging from red/orange for lower scores to green for higher scores\. Task difficulty varies substantially: the multi\-class NT Enhancers \(types\) task is the most challenging \(mean MCC=0\.321=0\.321\), while GB Mouse enh\. \(Ensembl\) is the easiest \(mean MCC=0\.567=0\.567\)\. Human\-mouse epigenomic\-profile models \(Enformer,SPACE\) and Transformer\-encoder models \(DNABERT\-2,GENA\-LM,GROVER\) cluster at the top of the model ordering\.
#### F\.12\.4DNA Methylation

DNA methylation prediction \(n=8n=8tasks\) reveals the most pronounced specialization effects in our benchmark\. Task difficulty spans an extreme range: the iDNA\-ABF 6mA task achieves0\.4500\.450mean MCC while the hardest 4mC task \(G\. subterraneus\) approaches random performance at0\.0610\.061MCC\. Results for the 8 DNA methylation prediction tasks are presented in Figures[22](https://arxiv.org/html/2606.04525#A6.F22)–[24](https://arxiv.org/html/2606.04525#A6.F24)\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/dna_methylation_kshot_degradation_no_overlap.png)Figure 22:Few\-shot performance degradation on DNA Methylation\.For each of the 40 models, macro\-average MCC across 8 DNA methylation tasks under full\-shot, 10\-shot, and 1\-shot regimes; models ordered by full\-shot performance\. The top band shows the relative drop from full\-shot to 10\-shot per model\. Benchmark\-wide mean degradation:74\.7%74\.7\\%for 10\-shot,93\.2%93\.2\\%for 1\-shot – among the most severe of any task category\. Both 10\-shot \(maximum MCC=0\.097=0\.097\) and 1\-shot \(maximum=0\.033=0\.033\) regimes collapse to near\-random performance for all models; full\-shot and 10\-shot rankings correlate at Spearmanρ=0\.50\\rho=0\.50\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/pareto_dna_methylation.png)Figure 23:Pareto frontier for DNA Methylation: mean MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmic x\-axis and mean full\-shot DNA Methylation MCC on the y\-axis\. Marker size and color both encode MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\.NT\-v2\-50M\-3mer\-MS\(50M, mean MCC=0\.326=0\.326\) is the strongest sub\-100M model on this category, ranking66th of4040overall and sitting near the frontier alongside multi\-billion\-parameterGENERatormodels\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/dna_methylation_heatmap.png)Figure 24:Per\-task MCC for DNA Methylation\.Heatmap shows full\-shot MCC for each of the 40 genomic foundation models on the 8 DNA methylation tasks \(six 4mC, one 5mC, one 6mA\), with models sorted by mean DNA Methylation MCC\. Cell values report per\-task MCC, with colors ranging from red/orange for lower scores to green for higher scores\.Model specialization is particularly striking\.BioFM\-265Machieves rank10\.410\.4on methylation tasks versus rank36\.736\.7across other categories – a specialization score of26\.326\.3, the highest in our benchmark\. Similarly,DNA\-GPT\-0\.1B\-Hshows rank10\.610\.6on methylation versus27\.027\.0elsewhere \(score16\.416\.4\)\. All four NT\-v2 multi\-species variants demonstrate methylation specialization \(specialization scores ranging fromΔ=9\.7\\Delta=9\.7forNT\-v2\-50M\-MStoΔ=12\.9\\Delta=12\.9forNT\-v2\-100M\-MS\)\.

Scaling on methylation tasks is modest\. Spearman correlation between model size and MCC isρ=0\.347\\rho=0\.347\(p=0\.028p=0\.028\), weaker than the benchmark\-wideρ=0\.565\\rho=0\.565\. This suggests that methylation prediction depends more on specific sequence features captured during pretraining than on model capacity\.

Tokenization effects diverge from overall patterns\.kk\-mer tokenization shows clear advantages:DNA\-GPT\-3B\-M\(kk\-mer\) exceedsGenomeOcean\-4B\(BPE\) by0\.1360\.136MCC \(0\.3670\.367vs\.0\.2310\.231\) despite smaller size\.NT\-v2\-250M\-MS\(kk\-mer\) outperformsGENA\-LM\-Large\-T2T\(BPE\) by0\.1320\.132MCC \(0\.3390\.339vs\.0\.2070\.207\)\. This advantage likely reflects the importance of specifickk\-mer motifs in methylation site recognition\.

Few\-shot degradation is among the most severe in our benchmark:74\.7%74\.7\\%at 10\-shot \(0\.2190\.219to0\.0560\.056\) and93\.2%93\.2\\%at 1\-shot \(0\.2190\.219to0\.0150\.015\)\. Even top\-performing models collapse:GENERator\-Eukaryote\-3Bdegrades from0\.4400\.440to0\.0970\.097at 10\-shot and to0\.0210\.021at 1\-shot\. The 10\-shot ranking remains moderately correlated with full\-shot \(Spearmanρ=0\.50\\rho=0\.50\), withGENERator\-Eukaryote\-3B,DNA\-GPT\-3B\-M, andNT\-v2\-50M\-3mer\-MSoccupying the top three 10\-shot positions\.

#### F\.12\.5Splice Site Detection

Splice site detection \(n=7n=7tasks\) represents a critical test of genomic sequence understanding, as accurate splicing prediction requires recognition of complex sequence patterns spanning donor and acceptor sites\. Task difficulty varies from canonical donor/acceptor classification \(mean MCC≈0\.52\\approx 0\.52\) to the reconstructed splice site challenge \(mean MCC=0\.306=0\.306\)\. Results for the 7 splice site prediction tasks are presented in Figures[25](https://arxiv.org/html/2606.04525#A6.F25)–[27](https://arxiv.org/html/2606.04525#A6.F27)\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/splice_sites_kshot_degradation_no_overlap.png)Figure 25:Few\-shot performance degradation on Splice Site Detection\.For each of the 40 models, macro\-average MCC across 7 splice site tasks under full\-shot, 10\-shot, and 1\-shot regimes; models ordered by full\-shot performance\. The top band shows the relative drop from full\-shot to 10\-shot per model\. Benchmark\-wide mean degradation:67\.6%67\.6\\%for 10\-shot,86\.4%86\.4\\%for 1\-shot\. The 10\-shot regime retains discriminative signal \(maximum MCC=0\.298=0\.298forLucaOne\), while the 1\-shot regime collapses to near\-random performance \(maximum=0\.107=0\.107\)\. The 10\-shot ranking diverges noticeably from full\-shot \(Spearmanρ=0\.77\\rho=0\.77, top\-5 overlap22of55\), with compact models such asMutBERTentering the 10\-shot top\-5\.The scale\-performance paradox manifests dramatically for splice sites\.GENERator\-Eukaryote\-1\.2B\(1\.2B parameters\) exceedsEvo\-1\-131k\(7B parameters\) by0\.4690\.469MCC \(0\.6290\.629vs\.0\.1600\.160\) – among the largest pair\-wise gaps observed across any task category\. This result directly reflects the fundamental incompatibility between prokaryotic pretraining and eukaryotic splicing prediction: prokaryotes lack the spliceosomal machinery present in eukaryotes, renderingEvo\-1’s 7B parameters essentially uninformative for this task class\.

Architecture comparisons reveal the largest Transformer\-over\-SSM advantages observed in our benchmark\.Omni\-DNA\-1B\(Transformer\-decoder\) outperformseccDNAMamba\(Mamba\-SSM\) by0\.3520\.352MCC \(0\.6040\.604vs\.0\.2520\.252\) under matched conditions \(both 1B, multi\-species, BPE\), whileGenomeOcean\-500MexceedseccDNAMambaby0\.2990\.299MCC\. These substantial gaps suggest that attention mechanisms are particularly well\-suited for capturing the long\-range dependencies inherent in splice site recognition\.

Tokenization effects favor single\-nucleotide approaches\.MutBERT\(single\-nucleotide\) exceedsGROVER\(BPE\) by0\.1900\.190MCC \(0\.5790\.579vs\.0\.3900\.390\) under matched Transformer\-encoder/human\-pretraining conditions \(both≈87\\approx 87M\), and exceedsGENA\-LM\(BPE, Transformer\-encoder, human\) by0\.1590\.159MCC, although theMutBERT–GENA\-LMcontrast is not strictly matched on scale \(86M vs\. 110M\)\. These advantages likely reflect the importance of precise positional information at splice junctions, which coarser subword tokenization may obscure\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/pareto_splice_sites.png)Figure 26:Pareto frontier for Splice Site Detection: mean MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmic x\-axis and mean full\-shot Splice Sites MCC on the y\-axis\. Marker size and color both encode MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\. Scaling on this category is strong \(Spearmanρ=0\.535\\rho=0\.535,p<0\.001p<0\.001;ρ=0\.654\\rho=0\.654excluding prokaryoticEvo\-1\-131k\)\.MutBERT\(86M, mean MCC=0\.579=0\.579\) is the strongest sub\-100M model on this category, ranking99th of4040overall and sitting near the Pareto frontier alongside multi\-billion\-parameter models\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/splice_sites_heatmap.png)Figure 27:Per\-task MCC for Splice Site Detection\.Heatmap shows full\-shot MCC for each of the 40 genomic foundation models on the 7 splice site tasks, with models sorted by mean Splice Sites MCC\. Cell values report per\-task MCC, with colors ranging from red/orange for lower scores to green for higher scores\. Donor and acceptor classification tasks \(NT and NT\-revised sources\) are consistently easier than the joint splice\-sites\-all task or the GUE reconstructed splice site challenge\. Eukaryotic gene\-focused and multi\-species models dominate the top of the model ordering, with the human\-mouse epigenomic\-profile modelSPACEin the top\-5 andMutBERT\(86M\) entering the top\-10 despite its small scale\.Pretraining data effects are pronounced\. Multi\-speciesGenomics\-FMexceeds microbialDNABERT\-Sby0\.2610\.261MCC \(0\.4920\.492vs\.0\.2300\.230\) – the largest data\-source gap observed across task categories\. Eukaryotic gene\-focusedGENERator\-Eukaryote\-3Boutperforms multi\-speciesDNA\-GPT\-3B\-Mby0\.0780\.078MCC \(0\.6480\.648vs\.0\.5710\.571\) under matched Transformer\-decoder/kk\-mer conditions \(both 3B\), confirming the advantage of gene\-centric pretraining for splicing\-related tasks\.

#### F\.12\.6Long Non\-coding RNA Classification

The lncRNA category \(n=6n=6tasks\) exclusively comprises plant species classification problems spanningGlycine max\(soybean\),Manihot esculenta\(cassava\),Sorghum bicolor\(sorghum\),Solanum lycopersicum\(tomato\),Triticum aestivum\(wheat\), andZea mays\(maize\)\. This composition makes lncRNA tasks the primary testbed for plant\-specific transfer learning in our benchmark\. Results for the 6 lncRNA prediction tasks are presented in Figures[28](https://arxiv.org/html/2606.04525#A6.F28)–[30](https://arxiv.org/html/2606.04525#A6.F30)\.

Plant\-specialized models demonstrate clear advantages\.PlantCaduceusachieves average per\-task rank8\.08\.0on lncRNA tasks versus27\.027\.0across the remaining9494tasks, yielding a specialization gap ofΔ=19\.0\\Delta=19\.0– second only toBioFM\-265M’s methylation specialization \(Δ=26\.3\\Delta=26\.3\)\.Agro\-NT\-1Bshows a similar but smaller pattern \(Δ=10\.1\\Delta=10\.1\)\. These results validate the importance of taxonomically\-aligned pretraining for plant genomics applications\.

Human\-trained models exhibit substantial negative transfer\. The best human\-trained model \(MutBERT, MCC=0\.260=0\.260\) substantially underperforms plant\-trained alternatives \(PlantCaduceus, MCC=0\.357=0\.357\)\. Across all human\-trained models, mean lncRNA MCC is only0\.1570\.157compared to0\.3470\.347for plant\-trained models – a gap of0\.1900\.190MCC representing121%121\\%relative improvement\. The negative transfer is particularly striking forBioFM\-265M, the only model with a negative mean lncRNA MCC at full\-shot \(−0\.018\-0\.018\), indicating worse\-than\- random performance on plant lncRNA classification despite the model’s strong methylation specialization\.

Notably, the overall lncRNA leaderLucaOne\(multi\-species, mean MCC=0\.508=0\.508\) achieves the best per\-category performance, suggesting that sufficiently diverse pretraining can compensate for the lack of plant\-specific data\.LucaOne’s advantage overPlantCaduceusis consistent across all six individual plant species \(per\-taskΔ\\Deltaranging from\+0\.07\+0\.07onM\. esculentato\+0\.27\+0\.27onS\. bicolor\), indicating that broad taxonomic exposure can outweigh plant\-specific pretraining on this category even at the per\-task level\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/lncrna_kshot_degradation_no_overlap.png)Figure 28:Few\-shot performance degradation on lncRNA Classification\.For each of the 40 models, macro\-average MCC across 6 plant lncRNA tasks under full\-shot, 10\-shot, and 1\-shot regimes; models ordered by full\-shot performance\. The top band shows the relative drop from full\-shot to 10\-shot per model\. Benchmark\-wide mean degradation:79\.8%79\.8\\%for 10\-shot,91\.3%91\.3\\%for 1\-shot\. Both regimes collapse to near\-random performance \(10\-shot maximum MCC=0\.207=0\.207forPlantCaduceus; 1\-shot maximum=0\.055=0\.055\)\. Despite the collapse, the 10\-shot ranking remains strongly correlated with full\-shot \(Spearmanρ=0\.90\\rho=0\.90, top\-5 overlap44of55\); notably, the plant\-specializedPlantCaduceusrises from rank77at full\-shot to rank11at 10\-shot\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/pareto_lncrna_all_frontier_labels.png)Figure 29:Pareto frontier for lncRNA Classification: mean MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmic x\-axis and mean full\-shot lncRNA MCC on the y\-axis\. Marker size and color both encode MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\. Scaling on this category is moderate \(Spearmanρ=0\.565\\rho=0\.565,p<0\.001p<0\.001\)\. The frontier reflects the plant\-specific nature of the tasks:PlantCaduceus\(225M, MCC=0\.357=0\.357\) andAgro\-NT\-1B\(1B, MCC=0\.336=0\.336\) sit near the frontier despite modest scale, alongside multi\-species models with much larger parameter counts\. Among sub\-100M models,NT\-v2\-50M\-MS\(50M, MCC=0\.307=0\.307\) achieves the highest lncRNA MCC\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/lncrna_heatmap.png)Figure 30:Per\-task MCC for lncRNA Classification\.Heatmap shows full\-shot MCC for each of the 40 genomic foundation models on the 6 plant lncRNA classification tasks, with models sorted by mean lncRNA MCC\. Cell values report per\-task MCC, with colors ranging from red/orange for lower scores to green for higher scores\. Multi\-species \(LucaOne\) and eukaryotic gene\-focused \(GENERator\) models dominate the top of the ordering, with plant\-specialized models \(PlantCaduceus,Agro\-NT\-1B\) close behind\. Human\-pretrained models cluster near the bottom\.Few\-shot degradation is severe \(79\.8% relative drop at 10\-shot, 91\.3% at 1\-shot\), with all models collapsing to near\-random performance \(10\-shot maximum MCC=0\.207=0\.207\)\. The 10\-shot ranking remains strongly correlated with full\-shot \(Spearmanρ=0\.90\\rho=0\.90\), but the plant\-specializedPlantCaduceusrises from full\-shot rank77to 10\-shot rank11, whileLucaOnedrops to rank22– indicating that plant\-aligned pretraining provides somewhat greater robustness under data scarcity even when not yielding the highest full\-shot score\.

#### F\.12\.7Mouse Enhancer Prediction

Mouse Enhancer prediction \(n=5n=5tasks\) evaluates cross\-species regulatory element recognition, providing insight into transfer learning from human\-centric or multi\-species pretraining to a related but distinct mammalian system\. Task difficulty varies substantially across the 5 tasks: task 1 is the easiest \(mean MCC=0\.680=0\.680across the 40 models\) and task 4 is the hardest \(mean MCC=0\.308=0\.308\)\. Results for the 5 mouse enhancer prediction tasks are presented in Figures[31](https://arxiv.org/html/2606.04525#A6.F31)–[33](https://arxiv.org/html/2606.04525#A6.F33)\.

Scaling on Mouse Enhancers is moderate and statistically significant \(Spearmanρ=0\.474\\rho=0\.474,p=0\.002p=0\.002;ρ=0\.580\\rho=0\.580excluding the prokaryoticEvo\-1\-131k\)\. Models with at least 1B parameters average0\.5200\.520MCC versus0\.4380\.438for models below 200M – a gap of0\.0820\.082MCC, slightly larger than the benchmark\-wide\+0\.064\+0\.064tier gap\.

Architecture comparisons reveal substantial Transformer\-over\-SSM advantages\.Omni\-DNA\-1B\(Transformer\-decoder\) exceedseccDNAMamba\(Mamba\-SSM\) by0\.3050\.305MCC \(0\.6750\.675vs\.0\.3700\.370\) under matched conditions \(both 1B, multi\-species, BPE\) – the second\-largest Transformer\-over\-SSM gap observed across task categories \(after Splice Sites at0\.3520\.352\)\. A within\-Transformer encoder\-vs\-decoder contrast on the same category showsGENA\-LM\-Large\-T2T\(336M, encoder\) outperformingOmniNA\-220M\(220M, decoder\) by0\.2840\.284MCC \(0\.6150\.615vs\.0\.3320\.332\), although the contrast is not strictly matched on scale\. This direction is opposite to the encoder\-vs\-decoder result we observe on Histone Modifications and is consistent with the broader observation \(Section[4](https://arxiv.org/html/2606.04525#S4)\) that the encoder–decoder distinction is task\- and setting\-dependent\.

Pretraining data effects follow expected patterns with notable magnitude\. The controlledGENA\-LMcomparison shows multi\-species outperforming human\-only by0\.0670\.067MCC \(GENA\-LM\-T2T\-Multi=0\.548=0\.548vs\.GENA\-LM=0\.480=0\.480\)\. Eukaryotic gene\-focusedGENERator\-Eukaryote\-3Bexceeds multi\-speciesDNA\-GPT\-3B\-Mby0\.1240\.124MCC \(0\.6360\.636vs\.0\.5120\.512\) under matched Transformer\-decoder/kk\-mer/3B conditions, while multi\-speciesGenomics\-FMoutperforms microbialDNABERT\-Sby0\.1440\.144MCC under matched Transformer\-encoder conditions \(≈\\approx120M each\)\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/mouse_enhancers_kshot_degradation_no_overlap.png)Figure 31:Few\-shot performance degradation on Mouse Enhancer Prediction\.For each of the 40 models, macro\-average MCC across 5 mouse enhancer tasks under full\-shot, 10\-shot, and 1\-shot regimes; models ordered by full\-shot performance\. The top band shows the relative drop from full\-shot to 10\-shot per model\. Benchmark\-wide mean degradation:67\.4%67\.4\\%for 10\-shot,89\.2%89\.2\\%for 1\-shot\. Unlike on the \(human\-centric\) Enhancers category, human\-mouse epigenomic\-profile models retain their advantage under few\-shot conditions:SPACErises from full\-shot rank44to 10\-shot rank11\(10\-shot MCC=0\.379=0\.379\), andEnformerholds rank22at both regimes \(10\-shot MCC=0\.363=0\.363\)\. Full\-shot and 10\-shot rankings remain strongly correlated \(Spearmanρ=0\.90\\rho=0\.90, top\-5 overlap33of55\)\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/pareto_mouse_enhancers_all_frontier_labels.png)Figure 32:Pareto frontier for Mouse Enhancer Prediction: mean MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmic x\-axis and mean full\-shot Mouse Enhancer MCC on the y\-axis\. Marker size and color both encode MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\. Scaling on this category is moderate \(Spearmanρ=0\.474\\rho=0\.474,p=0\.002p=0\.002;ρ=0\.580\\rho=0\.580excluding prokaryoticEvo\-1\-131k\)\. Human\-mouse epigenomic\-profile models sit prominently on the frontier \(Enformer,250250M, MCC=0\.674=0\.674;SPACE,589589M, MCC=0\.618=0\.618\), andMutBERT\(86M, mean MCC=0\.559=0\.559\) is the strongest sub\-100M model on this category\.Human\-mouse epigenomic profile models show expected advantages given their training data includes mouse regulatory elements\.Enformer\(mean MCC=0\.674=0\.674, rank22of4040\) andSPACE\(mean MCC=0\.618=0\.618, rank44of4040\) rank among the category leaders, withEnformerachieving the notable result of outperformingEvo\-1\-131k\(7B parameters\) by0\.3870\.387MCC despite being28×28\\timessmaller\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/mouse_enhancers_heatmap.png)Figure 33:Per\-task MCC for Mouse Enhancer Prediction\.Heatmap shows full\-shot MCC for each of the 40 genomic foundation models on the 5 mouse enhancer tasks \(GUE mouse 0 through 4\), with models sorted by mean Mouse Enhancer MCC\. Cell values report per\-task MCC, with colors ranging from red/orange for lower scores to green for higher scores\. Tasks 1 and 2 are uniformly easier across models \(mean MCC\>0\.65\>0\.65\) than tasks 0, 3, and 4 \(mean MCC<0\.39<0\.39\)\. Human\-mouse epigenomic\-profile models \(Enformer,SPACE\), multi\-species models \(Omni\-DNA\-1B\), and eukaryotic gene\-focused models \(GENERator\) dominate the top of the model ordering\.
#### F\.12\.8Transcription Factor Binding

TF Binding prediction \(n=5n=5tasks\) represents a critical challenge in regulatory genomics, requiring recognition of sequence motifs that mediate protein\-DNA interactions\. This category exhibits one of the most striking scale\-performance paradoxes in our benchmark\. Results for the 5 transcription factor binding prediction tasks are presented in Figures[34](https://arxiv.org/html/2606.04525#A6.F34)–[36](https://arxiv.org/html/2606.04525#A6.F36)\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/tf_binding_kshot_degradation_no_overlap.png)Figure 34:Few\-shot performance degradation on TF Binding\.For each of the 40 models, macro\-average MCC across 5 TF binding tasks under full\-shot, 10\-shot, and 1\-shot regimes; models ordered by full\-shot performance\. The top band shows the relative drop from full\-shot to 10\-shot per model\. Benchmark\-wide mean degradation:62\.6%62\.6\\%for 10\-shot,85\.9%85\.9\\%for 1\-shot\. The 10\-shot regime retains discriminative signal \(maximum MCC=0\.394=0\.394forEnformer\), while 1\-shot collapses to near\-random performance \(maximum=0\.164=0\.164\)\. Full\-shot and 10\-shot rankings remain strongly correlated \(Spearmanρ=0\.88\\rho=0\.88, top\-5 overlap33of55\); notably,Enformerretains rank11at both regimes\.Scaling on TF Binding is modest but statistically detectable \(Spearmanρ=0\.358\\rho=0\.358,p=0\.023p=0\.023;ρ=0\.463\\rho=0\.463excluding the prokaryoticEvo\-1\-131k\)\. Models with at least 1B parameters average0\.5460\.546MCC versus0\.5460\.546for models below 200M – effectively no tier gap at the extremes, in contrast to the benchmark\-wide\+0\.064\+0\.064gap\. This flat\-at\-the\-extremes relationship reflects in large part the catastrophic failure ofEvo\-1\-131k\(7B parameters, mean MCC=0\.173=0\.173, rank4040of4040\), which pulls the≥1\\geq 1B group mean downward\.

The scale paradox reaches its apex for TF binding:Enformer\(250M parameters, mean MCC=0\.698=0\.698, rank11of4040\) exceedsEvo\-1\-131kby0\.5250\.525MCC – the largest performance gap between any small\-vs\-large model pair across our entire benchmark\. This28×28\\timessize disadvantage yielding4×4\\timesbetter performance starkly illustrates the dominance of pretraining data alignment over model scale\.

Architecture comparisons favor Transformer\-encoder variants on this category\.GENA\-LM\-Large\-T2T\(encoder, 336M\) outperformsOmniNA\-220M\(decoder, 220M\) by0\.2060\.206MCC \(0\.6380\.638vs\.0\.4330\.433\), and the smallerGENA\-LM\-T2T\-Multi\(encoder, 110M\) also outperformsOmniNA\-220M\(decoder, 220M\) by0\.1720\.172MCC\. The Transformer\-over\-SSM gap remains substantial under matched conditions:Omni\-DNA\-1BexceedseccDNAMambaby0\.1140\.114MCC \(0\.6470\.647vs\.0\.5320\.532\) at matched 1B/multi\-species/BPE\. These pairwise contrasts are consistent with the broader observation \(Section[4](https://arxiv.org/html/2606.04525#S4)\) that the encoder–decoder ranking on this benchmark is task\- and setting\-dependent; for TF Binding the encoder direction holds across both pairs we examined\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/pareto_tf_binding_all_frontier_labels.png)Figure 35:Pareto frontier for TF Binding: mean MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmic x\-axis and mean full\-shot TF Binding MCC on the y\-axis\. Marker size and color both encode MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\. Scaling on this category is modest \(Spearmanρ=0\.358\\rho=0\.358,p=0\.023p=0\.023;ρ=0\.463\\rho=0\.463excluding prokaryoticEvo\-1\-131k\), with models exceeding 1B parameters averaging essentially the same MCC as models below 200M \(0\.5460\.546vs\.0\.5460\.546\)\.MutBERT\(86M, mean MCC=0\.646=0\.646\) is the strongest sub\-100M model on this category and ranks33rd of4040overall, betweenEnformer\(250M\) andOmni\-DNA\-1B\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/tf_binding_heatmap.png)Figure 36:Per\-task MCC for TF Binding\.Heatmap shows full\-shot MCC for each of the 40 genomic foundation models on the 5 TF binding tasks \(GUE human TF 0 through 4\), with models sorted by mean TF Binding MCC\. Cell values report per\-task MCC, with colors ranging from red/orange for lower scores to green for higher scores\. Task difficulty varies from GUE TF\-3 \(mean MCC=0\.385=0\.385, hardest\) to GUE TF\-1 \(mean MCC=0\.652=0\.652, easiest\)\. Human\-mouse epigenomic\-profile models \(Enformer,SPACE\), Transformer\-encoder models \(GENA\-LM\-Large\-T2T,MutBERT\), and multi\-species Transformer\-decoder models \(Omni\-DNA\-1B\) dominate the top of the model ordering\.Human\-mouse epigenomic profile models again demonstrate specialized strength\.Enformerachieves the highest mean MCC on the category \(0\.6980\.698, rank11of4040\) andSPACEfollows closely \(0\.6360\.636, rank55of4040\); on per\-task average ranks these correspond to4\.04\.0and7\.07\.0respectively, reflecting consistent placement among the strongest models across the 5 TF binding tasks \(Enformer in top\-5 on44of55tasks; SPACE in top\-11 on all55\)\. This performance reflects the direct relevance of TF binding events to the chromatin\-state signals present in human\-mouse epigenomic profile pretraining \(Section[F\.10](https://arxiv.org/html/2606.04525#A6.SS10)\)\.

#### F\.12\.9Species Classification

Species Classification tasks \(n=3n=3tasks\) evaluate a model’s ability to distinguish genomic sequences from different organisms, testing whether pretraining captures species\-specific sequence signatures\. The category consists of one cross\-kingdom human/worm classification \(GB Human\-or\-worm\) and two fine\-grained taxonomic tasks \(GUE Fungi\-20 and GUE Virus\-40\)\. Task difficulty varies substantially: the cross\-kingdom task achieves mean MCC=0\.857=0\.857across the 40 models, while the GUE Virus\-40 task is the hardest at0\.3230\.323\. Results for the 3 species classification tasks are presented in Figures[37](https://arxiv.org/html/2606.04525#A6.F37)–[39](https://arxiv.org/html/2606.04525#A6.F39)\.

Scaling on Species Classification is among the weakest in our benchmark \(Spearmanρ=0\.304\\rho=0\.304,p=0\.056p=0\.056, marginally non\-significant;ρ=0\.406\\rho=0\.406,p=0\.010p=0\.010when excludingEvo\-1\-131k\)\. Models exceeding 1B parameters average0\.6570\.657MCC versus0\.6060\.606for models below 200M \(\+0\.050\+0\.050gap, slightly below the benchmark\-wide\+0\.064\+0\.064\)\. This compressed scaling reflects in part the strong performance of small multi\-species Nucleotide Transformer variants on this category, withNT\-v2\-50M\-3mer\-MS\(50M\) achieving0\.7340\.734MCC – within0\.0300\.030of the top full\-shot modelGenomeOcean\-4B\(4B, MCC=0\.762=0\.762\) at1/801/80the parameter count\.

The scale paradox is particularly visible for species classification, with126126pairwise cases where smaller models outperform larger counterparts by at least5×5\\timessize ratio\.NT\-v2\-50M\-3mer\-MS\(50M\) exceedsEvo\-1\-131k\(7B\) by0\.4490\.449MCC despite a140×140\\timessize disadvantage – one of the largest such gaps observed in our benchmark, comparable in magnitude to theEnformer–Evo\-1\-131kcontrast on TF Binding \(0\.5250\.525; Section[F\.10](https://arxiv.org/html/2606.04525#A6.SS10)\)\.

Architecture comparisons favor Transformer\-encoder variants\.NT\-v2\-250M\-MS\(encoder\) outperformsGPT2\-Gene\-Multi\-v2\(decoder\) by0\.1890\.189MCC \(0\.7470\.747vs\.0\.5590\.559\) under matched multi\-species pretraining andkk\-mer tokenization \(250M vs\. 200M; the contrast is not strictly matched on scale\)\. The same direction holds for the smallerNT\-v2\-100M\-MS, which outperformsGPT2\-Gene\-Multi\-v2by0\.1830\.183MCC at half the parameter count\. As elsewhere in the benchmark \(Section[4](https://arxiv.org/html/2606.04525#S4)\), the encoder–decoder ranking is task\- and setting\-dependent; for Species Classification the encoder direction holds across both pairs we examined\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/species_classification_kshot_degradation_no_overlap.png)Figure 37:Few\-shot performance degradation on Species Classification\.For each of the 40 models, macro\-average MCC across 3 species classification tasks under full\-shot, 10\-shot, and 1\-shot regimes; models ordered by full\-shot performance\. The top band shows the relative drop from full\-shot to 10\-shot per model\. Benchmark\-wide mean degradation:33\.0%33\.0\\%for 10\-shot,69\.9%69\.9\\%for 1\-shot – among the mildest few\-shot degradation of any task category in our benchmark\. Both 10\-shot \(maximum MCC=0\.676=0\.676forGenomeOcean\-4B\) and 1\-shot \(maximum=0\.382=0\.382, alsoGenomeOcean\-4B\) regimes retain substantial discriminative signal, indicating that species classification is relatively robust to data scarcity\. Full\-shot and 10\-shot rankings are very strongly correlated \(Spearmanρ=0\.91\\rho=0\.91, top\-5 overlap44of55\)\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/pareto_species_classification_all_frontier_labels_taller_v10_genomeocean500_lower.png)Figure 38:Pareto frontier for Species Classification: mean MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmic x\-axis and mean full\-shot Species Classification MCC on the y\-axis\. Marker size and color both encode MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\. Scaling on this category is among the weakest in our benchmark \(Spearmanρ=0\.304\\rho=0\.304,p=0\.056p=0\.056;ρ=0\.406\\rho=0\.406,p=0\.010p=0\.010excluding the prokaryoticEvo\-1\-131k\)\.NT\-v2\-50M\-3mer\-MS\(50M, mean MCC=0\.734=0\.734\) is the strongest sub\-100M model on this category, sitting near the Pareto frontier alongside multi\-billion\-parameter generalists\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/species_classification_heatmap.png)Figure 39:Per\-task MCC for Species Classification\.Heatmap shows full\-shot MCC for each of the 40 genomic foundation models on the 3 species classification tasks, with models sorted by mean Species Classification MCC\. Cell values report per\-task MCC, with colors ranging from red/orange for lower scores to green for higher scores\. The cross\-kingdom GB Human\-or\-worm task is uniformly easy across models \(mean MCC=0\.857=0\.857\), while the fine\-grained GUE Virus\-40 task is uniformly hard \(0\.3230\.323\)\. Multi\-species decoder models \(GenomeOcean\-4B,GENERator\-Eukaryote\-3B\) and Nucleotide Transformer encoder models cluster at the top of the model ordering\.Pretraining data effects are pronounced\. Multi\-species training strongly outperforms human\-only:DNA\-GPT\-0\.1B\-H\(multi\-species, 100M, Transformer\-decoder,kk\-mer\) exceedsGPT2\-Gene\-v1\(human, 200M, same architecture and tokenization\) by0\.1370\.137MCC \(0\.7140\.714vs\.0\.5760\.576\), despite the multi\-species model being half the size\. Multi\-species also outperforms microbial:NT\-v2\-100M\-MSoutperformsDNABERT\-S\(multi\-species\-microbial\) by0\.1870\.187MCC \(0\.7410\.741vs\.0\.5540\.554\) under matched Transformer\-encoder/kk\-mer conditions \(100M vs\. 117M\)\. These patterns logically reflect that species classification requires exposure to diverse taxonomic sequences during pretraining\.

The Nucleotide Transformer v2 multi\-species family shows clear specialization for this category, with all four NT\-v2 multi\-species variants exhibiting positive specialization \(NT\-v2\-100M\-MS: per\-task rank10\.710\.7on Species Classification vs\.19\.919\.9elsewhere,Δ=9\.2\\Delta=9\.2;NT\-v2\-250M\-MS:Δ=7\.7\\Delta=7\.7;NT\-v2\-50M\-3mer\-MS:Δ=5\.9\\Delta=5\.9;NT\-v2\-50M\-MS:Δ=5\.8\\Delta=5\.8\)\.

#### F\.12\.10Regulatory Element Prediction

Regulatory element prediction \(n=2n=2tasks: GB Ensembl regulatory and GB OCR Ensembl\) focuses on identifying genomic regions involved in transcriptional control\. This category reveals one of the largest domain\-specific advantages in our benchmark, with two human\-mouse epigenomic\-profile models substantially outperforming all generalists\. Results for the 2 regulatory element prediction tasks are presented in Figures[40](https://arxiv.org/html/2606.04525#A6.F40)–[42](https://arxiv.org/html/2606.04525#A6.F42)\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/regulatory_kshot_degradation_no_overlap.png)Figure 40:Few\-shot performance degradation on Regulatory Element Prediction\.For each of the 40 models, macro\-average MCC across 2 regulatory element tasks under full\-shot, 10\-shot, and 1\-shot regimes; models ordered by full\-shot performance\. The top band shows the relative drop from full\-shot to 10\-shot per model\. Benchmark\-wide mean degradation:62\.1%62\.1\\%for 10\-shot,81\.9%81\.9\\%for 1\-shot\. The 10\-shot ranking differs moderately from full\-shot \(Spearmanρ=0\.58\\rho=0\.58, top\-5 overlap33of55\):Enformerdrops from full\-shot rank11\(MCC=0\.604=0\.604\) to 10\-shot rank44\(MCC=0\.184=0\.184\), whileSPACEretains its top\-2 position \(full rank22→\\rightarrow10\-shot rank22, MCC=0\.206=0\.206\)\. Both 10\-shot \(maximum MCC=0\.216=0\.216forDNABERT\-2\) and 1\-shot \(maximum=0\.113=0\.113\) regimes collapse to near\-random performance across most models\.EnformerandSPACE, pretrained on human\-mouse epigenomic profiles, dominate this category with mean MCC of0\.6040\.604and0\.5980\.598respectively, substantially exceeding all other models\. The next\-best performer \(LucaOne, multi\-species,22B, MCC=0\.492=0\.492\) trails by over0\.110\.11MCC\. This exceptional performance directly reflects the alignment between pretraining data \(regulatory chromatin\-state signals across many cell types\) and task requirements \(regulatory element identification\)\.

Scaling on Regulatory is modest but statistically significant \(Spearmanρ=0\.378\\rho=0\.378,p=0\.016p=0\.016;ρ=0\.481\\rho=0\.481,p=0\.002p=0\.002excluding the prokaryoticEvo\-1\-131k\)\. Models exceeding 1B parameters average0\.4210\.421MCC versus0\.3920\.392for models below 200M \(\+0\.030\+0\.030gap, well below the benchmark\-wide\+0\.064\+0\.064\)\. The compressed tier gap reflects the dominance of medium\-scale specialized models \(Enformerat 250M,SPACEat 589M\) over larger general\-purpose alternatives, underscoring that domain\-appropriate pretraining trumps scale on this category\.

An unusual pattern emerges in pretraining\-data comparisons:DNABERT\-S\(multi\-species microbial\) outperformsGenomics\-FM\(multi\-species,≈\\approx120M each, matched Transformer\-encoder\) by0\.0690\.069MCC \(0\.4190\.419vs\.0\.3500\.350\) – the only category in our benchmark where this reversal occurs \(in all other 12 categoriesGenomics\-FMleads this pair\)\. This reversal may indicate that certain regulatory patterns are shared across prokaryotic and eukaryotic systems, or thatDNABERT\-S’s training procedure captures generalizable regulatory features despite its taxonomic focus; givenn=2n=2tasks and a single pairwise contrast, however, this remains an observation rather than a strong claim\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/pareto_regulatory_all_frontier_labels.png)Figure 41:Pareto frontier for Regulatory Element Prediction: mean MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmic x\-axis and mean full\-shot Regulatory MCC on the y\-axis\. Marker size and color both encode MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\. Scaling on this category is modest \(Spearmanρ=0\.378\\rho=0\.378,p=0\.016p=0\.016;ρ=0\.481\\rho=0\.481,p=0\.002p=0\.002excluding prokaryoticEvo\-1\-131k\)\. Human\-mouse epigenomic\-profile models occupy the frontier at moderate sizes \(Enformer,250250M, MCC=0\.604=0\.604;SPACE,589589M, MCC=0\.598=0\.598\), outperforming all multi\-billion\-parameter generalists\.HyenaDNA\-Medium\(14M, mean MCC=0\.432=0\.432\) is the strongest sub\-100M model on this category\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/heatmap_regulatory.png)Figure 42:Per\-task MCC for Regulatory Element Prediction\.Heatmap shows full\-shot MCC for each of the 40 genomic foundation models on the 2 regulatory element tasks, with models sorted by mean Regulatory MCC\. Cell values report per\-task MCC, with colors ranging from red/orange for lower scores to green for higher scores\. The GB Ensembl regulatory task is easier \(mean MCC=0\.471=0\.471\) than GB OCR Ensembl \(mean MCC=0\.344=0\.344\)\. Human\-mouse epigenomic\-profile models \(Enformer,SPACE\) dominate the top of the model ordering, with multi\-species models \(LucaOne,NT\-2\.5B\-MS,DNA\-GPT\-3B\-M\) following\.GENA\-LMshows notable specialization for regulatory tasks, achieving per\-task rank10\.510\.5on Regulatory versus20\.020\.0across the remaining9898tasks \(specializationΔ=9\.5\\Delta=9\.5\)\. The specialization is specific to the human\-pretrainedGENA\-LMbase variant: the multi\-speciesGENA\-LM\-T2T\-Multishows weaker specialization \(Δ=5\.8\\Delta=5\.8\) and the larger multi\-speciesGENA\-LM\-Large\-T2Tshows negative specialization \(Δ=−3\.2\\Delta=\-3\.2\)\. The pattern is consistent with the concentration of well\-annotated regulatory elements in human genomic resources, although withn=2n=2tasks the effect should be interpreted cautiously\.

#### F\.12\.11Virus and Phage Detection

Virus/Phage detection \(n=2n=2tasks: GUE Phage fragments and GUE COVID variants\) presents a distinctive prediction problem involving recognition of viral genomic signatures\. Task difficulty varies dramatically: phage fragment classification achieves0\.6330\.633mean MCC while COVID variant prediction proves challenging at0\.2000\.200MCC\. Results for the 2 virus and phage prediction tasks are presented in Figures[43](https://arxiv.org/html/2606.04525#A6.F43)–[45](https://arxiv.org/html/2606.04525#A6.F45)\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/virus_phage_kshot_degradation_no_overlap.png)Figure 43:Few\-shot performance degradation on Virus/Phage Detection\.For each of the 40 models, macro\-average MCC across 2 virus/phage tasks under full\-shot, 10\-shot, and 1\-shot regimes; models ordered by full\-shot performance\. The top band shows the relative drop from full\-shot to 10\-shot per model\. Benchmark\-wide mean degradation:71\.3%71\.3\\%for 10\-shot,93\.5%93\.5\\%for 1\-shot – the largest 1\-shot degradation of any task category in our benchmark\. The 10\-shot maximum is0\.3770\.377forGenomeOcean\-4B; 1\-shot maximum is0\.0810\.081\. Full\-shot and 10\-shot rankings remain strongly correlated \(Spearmanρ=0\.83\\rho=0\.83, top\-5 overlap44of55\)\.Scaling is statistically significant for this category \(Spearmanρ=0\.435\\rho=0\.435,p=0\.005p=0\.005\), with theGenomeOceanmodels achieving strong performance:GenomeOcean\-4Bleads the category \(MCC=0\.697=0\.697, rank11of4040\) andGenomeOcean\-500Mfollows at rank22\(MCC=0\.657=0\.657\)\.

A distinctive pattern emerges for this category: human\-trained models show competitive or superior performance compared to multi\-species alternatives\.GPT2\-Gene\-v1\(human\) outperforms bothDNA\-GPT\-0\.1B\-H\(multi\-species\) by0\.1140\.114MCC andGPT2\-Gene\-Multi\-v2\(multi\-species\) by0\.0490\.049MCC\. Similarly,GROVER\(human\) exceedsDNABERT\-2\(multi\-species\) by0\.0400\.040MCC\. This pattern likely reflects the predominance of human\-associated viral sequences in human genomic training data, providing relevant exposure to viral integration sites and endogenous retroviral elements\.

Architecture comparisons reveal large Transformer advantages\.GenomeOcean\-500M\(Transformer\-decoder\) exceedseccDNAMamba\(Mamba\-SSM, 1B\) by0\.3550\.355MCC \(0\.6570\.657vs\.0\.3020\.302\) despite being half the size, among the largest architectural gaps observed\.GENA\-LM\-Large\-T2T\(Transformer\-encoder\) outperformsOmniNA\-220M\(Transformer\-decoder\) by0\.1800\.180MCC \(0\.5690\.569vs\.0\.3890\.389\)\.

Tokenization effects show BPE advantages over single\-nucleotide approaches, reversing patterns observed for splice sites\.GROVER\(BPE\) exceedsMutBERT\(single\-nucleotide\) by0\.2090\.209MCC \(0\.5320\.532vs\.0\.3230\.323\), andGENA\-LM\(BPE\) shows similar advantage \(\+0\.157\+0\.157MCC\)\. This pattern suggests that viral sequence recognition benefits from the longer\-range patterns captured by subword tokenization\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/pareto_virus_phage_all_frontier_labels.png)Figure 44:Pareto frontier for Virus/Phage Detection: mean MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmic x\-axis and mean full\-shot Virus/Phage MCC on the y\-axis\. Marker size and color both encode MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\. TheGenomeOceanmulti\-species decoders dominate the frontier \(GenomeOcean\-4B,44B, MCC=0\.697=0\.697;GenomeOcean\-500M,500500M, MCC=0\.657=0\.657\)\.GROVER\(87M, mean MCC=0\.532=0\.532\) is the strongest sub\-100M model on this category\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/heatmap_virus.png)Figure 45:Per\-task MCC for Virus/Phage Detection\.Heatmap shows full\-shot MCC for each of the 40 genomic foundation models on the 2 virus/phage tasks, with models sorted by mean Virus/Phage MCC\. Cell values report per\-task MCC, with colors ranging from red/orange for lower scores to green for higher scores\. GUE Phage fragments \(mean MCC=0\.633=0\.633\) is substantially easier than GUE COVID variants \(0\.2000\.200\)\. Multi\-species decoder models \(GenomeOcean\-4B,GenomeOcean\-500M\) and Transformer\-encoder models dominate the top of the ordering\.Several models show specialization toward virus/phage detection\.Agro\-NT\-1Bachieves per\-task rank12\.512\.5on Virus/Phage versus20\.520\.5across the remaining9898tasks \(specializationΔ=8\.0\\Delta=8\.0\), potentially reflecting viral sequences embedded in plant genomic training data\.GROVER\(Δ=6\.5\\Delta=6\.5\) andGENA\-LM\(Δ=6\.0\\Delta=6\.0\) similarly specialize, both being human\-pretrained Transformer\-encoders\. The multi\-speciesGENA\-LM\-T2T\-MultiandGENA\-LM\-Large\-T2Tshow smaller positive specialization \(Δ=5\.3\\Delta=5\.3and5\.45\.4respectively\); the human\-only direction remains modest within the GENA\-LM family\.

Few\-shot degradation is severe for this category \(93\.5%93\.5\\%at 1\-shot,71\.3%71\.3\\%at 10\-shot\), with top performers showing the steepest absolute declines:GenomeOcean\-4Bdegrades by0\.6330\.633andGenomeOcean\-500Mby0\.5950\.595\. Despite these large absolute drops, bothGenomeOceanmodels retain their full\-shot rank \(top\-2 at both regimes\) with relative drops of46%46\\%and48%48\\%– well below the category mean of71\.3%71\.3\\%\. Some rank shifts emerge among the runners\-up:LucaOnerises from full\-shot rank77to 10\-shot rank33, whileGROVERdrops from66to2121\.

#### F\.12\.12Coding versus Non\-coding Classification

Coding/Non\-coding classification \(n=1n=1task: GB Coding/Non\-coding\) represents a fundamental sequence annotation problem with relatively high baseline performance \(mean MCC=0\.803=0\.803across the 40 models\)\. This task evaluates whether models capture the statistical signatures distinguishing protein\-coding from non\-coding sequences\. Results for this task are presented in Figures[46](https://arxiv.org/html/2606.04525#A6.F46)–[48](https://arxiv.org/html/2606.04525#A6.F48)\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/coding_non_coding_kshot_degradation_no_overlap.png)Figure 46:Few\-shot performance degradation on Coding/Non\-coding Classification\.For each of the 40 models, MCC on the coding vs\. non\-coding task under full\-shot, 10\-shot, and 1\-shot regimes; models ordered by full\-shot performance\. The top band shows the relative drop from full\-shot to 10\-shot per model\. Benchmark\-wide mean degradation:26\.6%26\.6\\%for 10\-shot,71\.7%71\.7\\%for 1\-shot – among the mildest 10\-shot degradation observed across task categories\.MutBERT\(86M, human, encoder\) achieves the highest 10\-shot MCC \(0\.7480\.748\) and the highest 1\-shot MCC \(0\.4170\.417\), rising from full\-shot rank33to 10\-shot rank11\. Full\-shot and 10\-shot rankings remain strongly correlated \(Spearmanρ=0\.86\\rho=0\.86, top\-5 overlap33of55\)\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/pareto_coding_non_coding_all_frontier_labels.png)Figure 47:Pareto frontier for Coding/Non\-coding Classification: MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmic x\-axis and full\-shot MCC on the y\-axis\. Marker size and color both encode MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\. Eukaryotic gene\-focusedGENERator\-Eukaryote\-3B\(3B, MCC=0\.904=0\.904\) leads, with multi\-speciesLucaOne\(2B, MCC=0\.901=0\.901\) close behind\.MutBERT\(86M, MCC=0\.894=0\.894\) is the strongest sub\-100M model on this category and ranks33rd of4040overall, demonstrating that for this task small well\-designed models remain highly competitive with much larger generalists\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/heatmap_coding.png)Figure 48:Per\-model MCC for Coding/Non\-coding Classification\.Heatmap shows full\-shot MCC for each of the 40 genomic foundation models on the single coding vs\. non\-coding task, with models sorted by MCC\. Cell values report per\-model MCC, with colors ranging from red/orange for lower scores to green for higher scores\. Multi\-species decoder models \(GENERator\-Eukaryote\-3B,LucaOne\), Transformer\-encoder models \(MutBERT\), and large multi\-species generalists cluster at the top of the model ordering\.Coding/Non\-coding classification shows significant positive scaling \(Spearmanρ=0\.480\\rho=0\.480,p=0\.002p=0\.002;ρ=0\.571\\rho=0\.571,p<0\.001p<0\.001excluding the prokaryoticEvo\-1\-131k\)\. Models exceeding 1B parameters achieve0\.8460\.846mean MCC compared to0\.7810\.781for models below 200M, a\+0\.065\+0\.065gap that matches the benchmark\-wide tier difference\.

GENERator\-Eukaryote\-3Bleads the category \(MCC=0\.904=0\.904\), followed closely byLucaOne\(MCC=0\.901=0\.901\) andMutBERT\(MCC=0\.894=0\.894\)\. The strong performance ofMutBERT\(86M parameters, rank33of4040\) demonstrates that even on a task that responds to scale, well\-designed smaller models remain competitive with multi\-billion\-parameter counterparts\.

Architecture comparisons favor Transformers on this category\.MutBERT\(Transformer\-encoder\) exceedsHyenaDNA\-Large\-1M\(Hyena\) by0\.1820\.182MCC \(0\.8940\.894vs\.0\.7120\.712\) under matched human\-pretraining and single\-nucleotide tokenization, although the contrast is not strictly matched on scale \(86M vs\. 55M\)\.Omni\-DNA\-1B\(Transformer\-decoder\) outperformseccDNAMamba\(Mamba\-SSM\) by0\.1440\.144MCC \(0\.8770\.877vs\.0\.7340\.734\) under matched 1B/multi\-species/BPE conditions\.

Pretraining data comparisons show advantages for both eukaryotic gene\-focused and multi\-species approaches over microbial training\.Genomics\-FM\(multi\-species\) exceedsDNABERT\-S\(multi\-species\-microbial\) by0\.1270\.127MCC \(0\.8450\.845vs\.0\.7180\.718\) under matched Transformer\-encoder conditions \(≈\\approx120M each\), whileGENERator\-Eukaryote\-3B\(eukaryotic\-gene\) outperformsDNA\-GPT\-3B\-M\(multi\-species\) by0\.0450\.045MCC under matched Transformer\-decoder/kk\-mer/3B conditions\.

Notably,Evo\-1\-131k\(prokaryotic pretraining\) achieves acceptable performance on this task \(MCC=0\.719=0\.719, rank3636of4040\), substantially better than its catastrophic results on Splice Sites \(MCC=0\.160=0\.160\) or TF Binding \(MCC=0\.173=0\.173\)\. This relative success may reflect the more universal nature of coding\-sequence signatures across prokaryotic and eukaryotic genomes, where codon usage patterns and open\-reading\-frame statistics share fundamental properties\.

#### F\.12\.13Chromatin Accessibility

Chromatin Accessibility prediction \(n=1n=1task: iDHS DNase\-I\) evaluates recognition of open chromatin regions, a key determinant of transcriptional potential\. This category shows distinctive patterns in both architecture and tokenization comparisons, and exhibits the weakest scaling effect of any category in our benchmark\. Results for this task are presented in Figures[49](https://arxiv.org/html/2606.04525#A6.F49)–[51](https://arxiv.org/html/2606.04525#A6.F51)\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/chromatin_accessibility_kshot_degradation_no_overlap.png)Figure 49:Few\-shot performance degradation on Chromatin Accessibility\.For each of the 40 models, MCC on the iDHS DNase\-I task under full\-shot, 10\-shot, and 1\-shot regimes; models ordered by full\-shot performance\. The top band shows the relative drop from full\-shot to 10\-shot per model\. Benchmark\-wide mean degradation:26\.8%26\.8\\%for 10\-shot,73\.6%73\.6\\%for 1\-shot – among the mildest 10\-shot degradation observed across task categories\. The 10\-shot and 1\-shot rankings reshuffle substantially relative to full\-shot \(Spearmanρ=0\.38\\rho=0\.38, top\-5 overlap22of55\):GENA\-LM\-Large\-T2Trises from full\-shot rank1212to 10\-shot rank11\(MCC=0\.567=0\.567\), andJanusDNA\-72\-w\(1\.98M parameters\) achieves the highest 1\-shot MCC \(0\.2740\.274\)\.Scaling effects are non\-significant on this category \(Spearmanρ=0\.244\\rho=0\.244,p=0\.128p=0\.128;ρ=0\.332\\rho=0\.332,p=0\.039p=0\.039excluding the prokaryoticEvo\-1\-131k\) – the weakest scaling relationship of any category in our benchmark\. Models exceeding 1B parameters achieve0\.5930\.593mean MCC versus0\.5470\.547for models below 200M \(a modest\+0\.046\+0\.046gap, well below the benchmark\-wide\+0\.064\+0\.064\)\.GENERator\-Eukaryote\-3Bleads the category \(MCC=0\.728=0\.728\), followed byOmni\-DNA\-1B\(MCC=0\.714=0\.714\) andEnformer\(MCC=0\.711=0\.711\)\.

Pretraining\-data effects are pronounced\. The controlledGENA\-LMcomparison shows multi\-species exceeding human\-only by0\.1230\.123MCC \(GENA\-LM\-T2T\-Multi=0\.583=0\.583vs\.GENA\-LM=0\.461=0\.461\) – the largest gap observed for this model pair across all1313task categories \(next\-largest is Mouse Enhancers at0\.0680\.068\)\. Eukaryotic gene\-focusedGENERator\-Eukaryote\-3Boutperforms multi\-speciesDNA\-GPT\-3B\-Mby0\.1910\.191MCC \(0\.7280\.728vs\.0\.5380\.538\) under matched 3B/decoder/kk\-mer conditions – also the largest such gap for this controlled pair across the benchmark\.

Tokenization comparisons reveal dramatic single\-nucleotide advantages\.MutBERT\(single\-nucleotide\) exceedsGENA\-LM\(BPE\) by0\.2310\.231MCC \(0\.6910\.691vs\.0\.4610\.461\) under matched Transformer\-encoder/human\-pretraining conditions \(86M vs\. 110M\) – the largest tokenization gap observed for this controlled pair across the benchmark\.MutBERTsimilarly outperformsGROVER\(BPE\) by0\.1710\.171MCC \(0\.6910\.691vs\.0\.5210\.521\) under matched conditions \(≈\\approx86–87M each\)\. This substantial advantage suggests that chromatin accessibility prediction benefits from fine\-grained positional information that coarser tokenization schemes may obscure\.

Architecture comparisons show mixed patterns for this category\. Under matched 1B/multi\-species/BPE conditions,Omni\-DNA\-1B\(Transformer\-decoder\) exceedseccDNAMamba\(Mamba\-SSM\) by0\.1160\.116MCC\. However,eccDNAMamba\(1B, MCC=0\.599=0\.599\) outperforms the smallerGenomeOcean\-500M\(Transformer\-decoder,500500M, MCC=0\.475=0\.475\) by0\.1240\.124MCC – a size\-favored outcome rather than a clean architectural reversal\. Additionally,Deepgene\(Graph\-Transformer, 85M\) exceedsGENA\-LM\(Transformer\-encoder, 110M\) by0\.1330\.133MCC\. These atypical patterns suggest that chromatin accessibility prediction may benefit from architectural features beyond standard attention mechanisms\.

Model specialization is striking for this category\. The smallest model in our benchmark,JanusDNA\-72\-w\(Hybrid\-Mamba\-MoE, 1\.98M parameters\), achieves per\-task rank15\.015\.0on Chromatin Accessibility versus32\.932\.9across the remaining9999tasks \(specializationΔ=17\.9\\Delta=17\.9\) – this Mamba/MoE hybrid at near\-toy scale outperforms most multi\-billion\-parameter generalists on this task\.eccDNAMamba\(1B Mamba\-SSM\) shows a similar pattern \(Δ=16\.5\\Delta=16\.5\)\. The relative strength of these two architectures on chromatin accessibility, despite their generally weaker overall performance, suggests that state\-space components may capture specific sequence features relevant to chromatin state prediction; however, other Mamba\-based models in our benchmark \(Caduceus\-PH\-1k,PlantCaduceus,Caduceus\-PS\-131k\) do not show this specialization, so the effect is not a uniform property of the Mamba family\.

![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/images/pareto_chromatin_accessibility_leftgreedy_manual_v5.png)Figure 50:Pareto frontier for Chromatin Accessibility: MCC vs\. parameter count\.Each point represents one of the 40 genomic foundation models, with parameter count on a logarithmic x\-axis and full\-shot MCC on the y\-axis\. Marker size and color both encode MCC\. The dashed line marks the Pareto frontier of best performance–size trade\-offs\.GENERator\-Eukaryote\-3B\(3B, MCC=0\.728=0\.728\) leads, withMutBERT\(86M, MCC=0\.691=0\.691, rank44of4040\) the strongest sub\-100M Transformer\-encoder\. Strikingly,JanusDNA\-72\-w\(1\.98M, MCC=0\.599=0\.599\) sits prominently on the frontier at rank1414of4040– the smallest model in the benchmark outperforming many counterparts that are over500×500\\timeslarger\.![Refer to caption](https://arxiv.org/html/2606.04525v1/all_images/heatmap_chromatin.png)Figure 51:Per\-model MCC for Chromatin Accessibility\.Heatmap shows full\-shot MCC for each of the 40 genomic foundation models on the single iDHS DNase\-I task, with models sorted by MCC\. Cell values report per\-model MCC, with colors ranging from red/orange for lower scores to green for higher scores\. Eukaryotic gene\-focused \(GENERator\-Eukaryote\-3B\), multi\-species decoders \(Omni\-DNA\-1B\), and human\-mouse epigenomic\-profile models \(Enformer,SPACE\) cluster at the top of the model ordering\.

### F\.13Task Difficulty and Model Differentiation

The 100 tasks in our benchmark span a wide difficulty spectrum, from near\-solved problems to challenges that remain largely intractable for current foundation models\. Understanding this difficulty landscape provides essential context for interpreting model comparisons and identifying promising directions for future research\.

#### F\.13\.1Easy Tasks: Approaching Ceiling Performance

We identify1818tasks achieving mean MCC exceeding0\.700\.70across all4040models, indicating problems where current approaches converge to robust solutions\. Human cell\-type\-specific promoter recognition tasks dominate this category: iPro HUVEC achieves0\.8900\.890mean MCC, iPro HeLa\-S3 reaches0\.8750\.875, and iPro GM12878 attains0\.8550\.855\. TheSPACEmodel achieves best performance on all three tasks, whileBioFM\-265Mconsistently ranks lowest, highlighting the importance of appropriate pretraining even for ostensibly solved problems\.

General promoter classification tasks \(NT Promoter \(all\), NT Promoter \(no TATA\), GUE Prom 300 \(no TATA\)\) cluster at0\.8530\.853–0\.8550\.855mean MCC, withGENA\-LM\-Large\-T2TandOmni\-DNA\-1Bamong the top performers\. The GB Human\-or\-worm species classification task reaches0\.8570\.857mean MCC, withMutBERTleading \(MCC=0\.948=0\.948\) andEvo\-1\-131kperforming worst – a pattern reflecting the prokaryotic model’s incompatibility with eukaryotic discrimination tasks\.

The GB Coding/Non\-coding task achieves0\.8030\.803mean MCC, representing a relatively accessible problem where sequence composition differences between coding and non\-coding regions provide strong discriminative signal\.GENERator\-Eukaryote\-3Bleads this task, whileCaduceus\-PH\-1kshows weakest performance\.

#### F\.13\.2Hard Tasks: Persistent Challenges

We identify2828tasks with mean MCC below0\.350\.35, representing problems where even state\-of\-the\-art models achieve limited success\. DNA methylation prediction dominates this category, with 4mC*G\. subterraneus*achieving only0\.0610\.061mean MCC \(maximum0\.2060\.206byGENERator\-Eukaryote\-3B\)\. 4mC*E\. coli*reaches0\.1030\.103mean MCC and 4mC*G\. pickeringii*achieves0\.1070\.107\. These results suggest that current sequence\-based approaches struggle to capture the contextual and enzymatic factors governing methylation site selection\.

Plant lncRNA classification tasks prove similarly challenging, with PGB lncRNA*S\. lycopersicum*\(tomato\) at0\.2210\.221mean MCC, PGB lncRNA*G\. max*\(soybean\) at0\.2280\.228, and PGB lncRNA*T\. aestivum*\(wheat\) at0\.2380\.238\.LucaOneachieves best performance across all66plant lncRNA tasks, reaching0\.5390\.539on*S\. lycopersicum*, demonstrating that sufficiently diverse multi\-species pretraining can partially address plant\-specific challenges even without dedicated plant data\.

Viral sequence classification presents unexpected difficulty, with the GUE COVID variants task achieving only0\.2000\.200mean MCC\.GENA\-LM\-Large\-T2Tleads at0\.4720\.472, suggesting that multi\-species genomic pretraining provides some transferable signal for viral sequence analysis despite the substantial evolutionary distance\.

#### F\.13\.3High\-Variance Tasks: Discriminating Model Capabilities

Tasks exhibiting high inter\-model variance \(standard deviation\>0\.12\>0\.12\) reveal where architectural and pretraining choices most strongly differentiate performance\. We identify1313such “controversial” tasks that serve as natural stress tests for model capabilities\.

The GUE Fungi\-20 task shows the highest variance \(std=0\.216=0\.216, range=0\.764=0\.764MCC\), withGenomeOcean\-4Bachieving0\.9390\.939MCC whileCaduceus\-PH\-1kreaches only0\.1750\.175\. The top performers uniformly employ multi\-species or eukaryotic\-gene pretraining \(GenomeOcean\-4B,GENERator\-Eukaryote\-3B,GENERator\-Eukaryote\-1\.2B\), while bottom performers use narrower data \(Caduceus\-PH\-1k,BioFM\-265M,JanusDNA\-72\-wo\)\.

Splice site detection tasks exhibit consistently high variance: NT Splice donors \(std=0\.195=0\.195, range=0\.666=0\.666\), NT Splice acceptors \(std=0\.186=0\.186, range=0\.643=0\.643\), and GUE Splice reconstr\. \(std=0\.170=0\.170, range=0\.597=0\.597\)\.NT\-2\.5B\-MS,LucaOne, andGENERator\-Eukaryote\-3Bconsistently rank among the top performers across splice tasks, whileEvo\-1\-131kconsistently ranks last – achieving negative MCC on GUE Splice reconstr\., indicating worse\-than\-random performance attributable to prokaryotic pretraining’s incompatibility with spliceosomal machinery\.

Mouse enhancer tasks show substantial variance \(std=0\.151=0\.151–0\.1750\.175\), withEnformerandSPACE\(human\-mouse epigenomic profiles\) dominating while human\-only trainedJanusDNAvariants andCaduceusvariants perform poorly\. This pattern supports the importance of taxonomically aligned pretraining for cross\-species regulatory element prediction\.

#### F\.13\.4Patterns in High\-Variance Task Performance

Systematic analysis of top\-3 and bottom\-3 performers across all1313high\-variance tasks reveals striking patterns in architecture and pretraining data\. Among top performers, Transformer\-decoder appears1818times and Transformer\-encoder1515times, with CNN\-Transformer architectures \(Enformer,SPACE\) appearing66times\. Among bottom performers, Mamba\-SSM architectures appear1717times, Hybrid\-Mamba\-MoE77times, and StripedHyena66times\. This distribution suggests that attention\-based architectures substantially outperform state\-space alternatives on the most discriminating benchmark tasks\.

Pretraining data patterns are even more pronounced\. Multi\-species training appears in top\-3 positions2020times and eukaryotic\-gene training1212times, while human\-mouse\-profiles appears66times\. In contrast, human\-only training appears in bottom\-3 positions2929times – more than all other categories combined\. Prokaryotic training appears66times in bottom positions despite comprising only one model \(Evo\-1\-131k\), indicating consistent failure across high\-variance tasks\. These patterns provide strong empirical support for prioritizing taxonomically diverse pretraining over species\-specific approaches\.