bioMoR: Biology-Guided Mixture-of-Recursions for Effective Genomic Learning

# bioMoR: Biology-Guided Mixture-of-Recursions for Effective Genomic Learning
Source: [https://arxiv.org/html/2608.06727](https://arxiv.org/html/2608.06727)
Koushik Howlader1, Tirtho Roy1, Md Tauhidul Islam2, Wei Le1 1Iowa State University, Iowa, USA 2Stanford University, CA, USA

###### Abstract

Transformer models for high\-dimensional omics analysis process thousands of genes or pathways, although only a subset requires deep computation\. Mixture\-of\-Recursions \(MoR\) improves efficiency through adaptive token\-choice or expert\-choice routing\. We propose bioMoR, which, to the best of our knowledge, is the first framework to apply MoR to gene\-level and pathway\-level learning\. Our contributions include identifying three locations for integrating structured biological knowledge within an MoR backbone: graph\-based information sharing refines token embeddings, a structural bias guides self\-attention toward biologically related tokens, and a graph\-aware router uses neighborhood information to determine each token’s recursion depth\. These techniques are centered on our insight that additional knowledge of token interaction can effectively help models construct embeddings and select which tokens should be learned more deeply\. Across eight benchmarks spanning diverse omics data types and evaluated under a unified five\-fold cross\-validation protocol, bioMoR improves average macro\-F1 by8\.28\.2percentage points and balanced accuracy by7\.17\.1percentage points over the strongest biology\-agnostic MoR baseline while using7575percent fewer parameters and up to5858percent fewer FLOPs than a non\-recursive Transformer\. The selected marker genes or pathways provide biological interpretability, while their token\-specific recursion depths reveal how computation is allocated\.

![Refer to caption](https://arxiv.org/html/2608.06727v1/x1.png)Figure 1:Overview of bioMoR\.Single\-cell and multi\-omics inputs are represented as gene or pathway tokens\. Fixed biological knowledge is incorporated at three sites:\(1\)embedding smoothing refines related tokens,\(2\)biology\-guided routing selects tokens for further computation, and\(3\)attention bias promotes interactions between biologically related tokens in the weight\-shared recursive Transformer block\.## 1Introduction

High\-throughput sequencing now measures thousands of genes, pathways, or molecular features per sample\. Transformers are attractive for these data because attention can model long\-range molecular dependencies; recent single\-cell foundation models, including scBERT \(?\), scGPT \(?\), scFoundation \(?\), and Geneformer \(?\), show strong results for annotation and representation learning\. However, the standard Transformers spend the same deep computation on every token, even though genomic signal is usually sparse: a cell type, tumor state, or metastatic phenotype is often driven by a small set of marker genes, regulatory programs, or pathways\. For computational efficiency and accuracy, a useful genomic model should not waste computation on weakly informative features while under\-refining the molecular programs that explain the phenotype\. The effective modeling thus should answer two questions at the same time: which genes or pathways should be kept, and which of those tokens should receive more model computation?

Guiding assumption\.We should perform computation and learning in a selective and structured manner, based on the semantics of omics data\. The model should retain biologically meaningful marker gene/pathway tokens and spend more recursive computation on the tokens whose biological neighborhoods are most predictive for the task\.

Existing methods address only part of this assumption\. Efficient attention methods such as Linformer \(?\), Performer \(?\), and Nyströmformer \(?\) reduce sequence cost through low\-rank or kernel approximations\. Adaptive computation methods such as ACT \(?\), mixture\-of\-experts \(?\), mixture\-of\-depths \(?\), and mixture\-of\-recursions \(?\) route tokens dynamically\. However, these approaches are biology\-agnostic, for example, they do not know whether two genes express together, share a pathway, or contribute to a same biological process\. Conversely, biology\-aware resources and models such as CellMarker \(?\), PanglaoDB \(?\), Reactome \(?\), genomic interaction graphs \(?\), Pathformer \(?\), and scBiGNN \(?\) provide and consider useful semantic structures of omics data, however, they are usually used for feature engineering, architectural constraints, or post\-hoc interpretation rather than for selecting tokens and deciding which tokens should receive deeper computation\.

We proposebioMoR, a biology\-guided Mixture\-of\-Recursions framework for genomic learning\. To the best of our knowledge, it is the first work that integrates biology knowledge and Mixture\-of\-Recursion architecture to drive gene or pathway token selections for adaptive learning\.

As shown in Figure[1](https://arxiv.org/html/2608.06727#S0.F1), bioMoR first compresses high\-dimensional inputs into marker\-gene or Reactome pathway tokens\. It then uses a shared recursive Transformer block and routes each token to an adaptive number of recursive steps\. The key change from biology\-agnostic MoR \(?\) is that we inject relevant biology knowledge into the model at three sites: it smooths token embeddings, biases attention toward related tokens, and guides the MoR router to decide token\-specific recursion depth\. Thus, biological structure and semantics becomes a control signal for adaptive computation, not only a static prior\.

This leads to four questions in our evaluation: does bioMoR improve predictive performance \(RQ1\), which bio\-knowledge injection site matters \(RQ2\), whether it indeed makes computation more efficient \(RQ3\), and which routing, marker budget, and model width settings are optimal \(RQ4\)\.

Across eight diverse genomic learning datasets, bioMoR improves average macro\-F1 by8\.28\.2points and balanced accuracy by7\.17\.1points, while using75%75\\%fewer parameters and up to58%58\\%fewer FLOPs than traditional Transformers\. It also achieved0\.920\.92linear\-probe accuracy and successfully selected biologically relevant pathways for deeper computation, confirmed by prior literature\.

In summary, our work makes the following contributions:

- •We introduce bioMoR, the first Mixture\-of\-Recursions \(MoR\) framework for gene\- and pathway\-level omics learning\.
- •We integrated biology knowledge, including gene co\-expression and pathway\-pathway relations, to improve representation learning, attention learning, and adaptive routing leveraging MoR backbone\.
- •We conducted extensive evaluation and show that our techniques significantly improves F1 and balanced accuracy and reduce model parameters as well as the FLOP cost, and they are generally applicable across single\-cell and multi\-omics benchmarks\.

## 2Related Work

#### Genomic representation learning\.

Transformer models such as scBERT \(?\), scGPT \(?\), and Geneformer \(?\) learn gene representations from large single\-cell datasets and support tasks such as cell\-type annotation\. These models are effective, but normally apply the same model depth to every input token\.

#### Biological knowledge in omics models\.

P\-NET uses known biological hierarchies for cancer prediction \(?\), while Pathformer \(?\) and scBiGNN \(?\) use pathway or gene\-interaction information to guide genomic learning\. These methods show the value of biological knowledge, but they do not use it to decide how much computation each token should receive\.

#### Adaptive computation\.

Adaptive Computation Time \(?\), mixture\-of\-experts \(?\), mixture\-of\-depths \(?\), and MoR \(?\) allow different tokens to receive different amounts of computation\. However, their routing decisions are biology\-agnostic\. bioMoR connects these two research directions by using biological knowledge to guide token representations, attention, and recursion depth\.

## 3Method

First, in*Genomic Tokenization*and*Mixture\-of\-Recursions*, we provide background on two existing components \(?;?;?\), on top of which we built bioMoR\. Second, in*bioMoR*, we explained our novel techniques of integrating biological knowledge to guide MoR learning\. See Figure[1](https://arxiv.org/html/2608.06727#S0.F1)for our overall workflow\.

### Genomic Tokenization

Letx∈ℝN×Cx\\in\\mathbb\{R\}^\{N\\times C\}denote one genomic sample withNNgenes andCCmolecular channels\. Single\-cell gene expression measures transcript abundance and hasC=1C\{=\}1\(?\)\. Multi\-omics inputs may additionally contain mutation indicators, which record DNA\-sequence alterations \(?\), and copy\-number variation \(CNV\), which records gains or losses of DNA segments \(?\)\.T=\[c1,…,cM\]∈ℝM×dT=\[c\_\{1\},\\ldots,c\_\{M\}\]\\in\\mathbb\{R\}^\{M\\times d\}\.

#### Single\-cell marker tokens\.

A single cell has thousands of genes, but only a few are informative\. We therefore turn theNNgenes intoM≪NM\\ll Nmarker tokens\. This makes the later attention cheap \(𝒪​\(M2​d\)\\mathcal\{O\}\(M^\{2\}d\)\) and lets each token be read as one named marker gene\. First, each geneiiis embedded as an identity vector plus its expression value,

ti=ei\+Wv​xi,ei∈ℝd,Wv∈ℝd×C,t\_\{i\}=e\_\{i\}\+W\_\{v\}x\_\{i\},\\qquad e\_\{i\}\\in\\mathbb\{R\}^\{d\},\\ W\_\{v\}\\in\\mathbb\{R\}^\{d\\times C\},\(1\)following the gene\-plus\-value scheme of single\-cell foundation models \(?;?\)\. We then keepMMlearnable queries; querymmsoftly picks genes through keyski=Wk​eik\_\{i\}=W\_\{k\}e\_\{i\},

wm​i=exp⁡\(qm⊤​ki/\(τ​d\)\)∑j=1Nexp⁡\(qm⊤​kj/\(τ​d\)\),w\_\{mi\}=\\frac\{\\exp\(q\_\{m\}^\{\\top\}k\_\{i\}/\(\\tau\\sqrt\{d\}\)\)\}\{\\sum\_\{j=1\}^\{N\}\\exp\(q\_\{m\}^\{\\top\}k\_\{j\}/\(\\tau\\sqrt\{d\}\)\)\},\(2\)and sums them into a tokencm=∑i=1Nwm​i​tic\_\{m\}=\\sum\_\{i=1\}^\{N\}w\_\{mi\}t\_\{i\}\. At test time, querymmcollapses to its top genegm=arg⁡maxi⁡wm​ig\_\{m\}=\\arg\\max\_\{i\}w\_\{mi\}, so every token names a marker gene\.

#### Pathway tokens\.

For multi\-omics cancer cohorts we instead group genes into Reactome pathways, which represent curated sets of genes participating in the same biological process \(?\)\. Each token is one pathway, built by pooling its member genes—mean pooling for dense expression and copy\-number channels, and sum/burden pooling for sparse mutation channels\. Every token is thus a named pathway\.

### Mixture\-of\-Recursions

LetT=\[c1,…,cM\]T=\[c\_\{1\},\\ldots,c\_\{M\}\]be the marker/pathway token matrix\. MoR applies one shared Transformer blockfθf\_\{\\theta\}up toKKtimes:

H\(0\)=T,H\(t\+1\)=fθ​\(H\(t\)\),t=0,…,K−1\.H^\{\(0\)\}=T,\\qquad H^\{\(t\+1\)\}=f\_\{\\theta\}\(H^\{\(t\)\}\),\\quad t=0,\\ldots,K\-1\.\(3\)Sharing makes parameter count independent ofKK, unlikeKKindependent Transformer blocks \(?;?;?\)\. A router decides which tokens continue\. Expert\-choice keeps the top\-⌈ct​M⌉\\lceil c\_\{t\}M\\rceiltokens at steptt; token\-choice lets each token self\-select a depth with load balancing \(?\)\. The number of active steps,

dm=∑t=1K𝟏​\{m​is kept at step​t\},d\_\{m\}=\\sum\_\{t=1\}^\{K\}\\mathbf\{1\}\\\{m\\ \\text\{is kept at step\}\\ t\\\},\(4\)is the token\-specific recursion depth\.

### bioMoR

Algorithm 1: bioMoR forward pass\.Input:xx, biological knowledgeBBtoken budgetMM, max depthKKT←T\\leftarrowmarker\-gene or pathway tokens fromxxA←A\\leftarrowrow\-normalized fixed knowledge matrix fromBBH←\(1−λ\)​T\+λ​A​TH\\leftarrow\(1\-\\lambda\)T\+\\lambda AT;𝒜←\{1,…,M\}\\mathcal\{A\}\\leftarrow\\\{1,\\ldots,M\\\}fort=1,…,Kt=1,\\ldots,Kwhile𝒜≠∅\\mathcal\{A\}\\neq\\emptysetdoH𝒜←fθ​\(H𝒜;A\)H\_\{\\mathcal\{A\}\}\\leftarrow f\_\{\\theta\}\(H\_\{\\mathcal\{A\}\};A\)reuse the same blockr𝒜←r\_\{\\mathcal\{A\}\}\\leftarrowbiology\-aware router scores𝒜next←\\mathcal\{A\}\_\{\\mathrm\{next\}\}\\leftarrowtokens routed to the next steptokens in𝒜∖𝒜next\\mathcal\{A\}\\setminus\\mathcal\{A\}\_\{\\mathrm\{next\}\}stop and keep current states𝒜←𝒜next\\mathcal\{A\}\\leftarrow\\mathcal\{A\}\_\{\\mathrm\{next\}\}return kept tokens to recursionend fory^←\\hat\{y\}\\leftarrowclassifier\(pool​\(H\)\)\\big\(\\mathrm\{pool\}\(H\)\\big\)Train:minimize class\-weighted CE plus auxiliary lossesOutput:y^\\hat\{y\}, markers/pathways, recursion depthsdmd\_\{m\}

#### Biological\-knowledge construction\.

Following Genomap \(?\), we construct a geneco\-expressionmatrix that summarizes pairwise gene relationships\. Its entries are partial correlations computed from the inverse covariance of the gene\-expression matrix and restricted to the selected markers; this matrix forms the single\-cell biological knowledgeBB\. For multi\-omics data, we use curated Reactome pathway relationships and treat them as undirected:Bi​j=Bj​i=1B\_\{ij\}=B\_\{ji\}=1when pathwaysiiandjjare related, and both entries are zero otherwise \(?\)\.

LetDDbe the diagonal matrix of row sums ofBB\. We use the fixed, row\-normalized matrix

The same biological knowledge is used for embedding smoothing \(Site 1\), attention bias \(Site 2\), and depth routing \(Site 3\)\.

#### Site 1: biology\-guided embedding smoothing\.

Before recursion, we smooth the tokens using the biological\-knowledge graphAA, so that information of biologically related tokens is also integrated into the embedding of the token:

T~=\(1−λ\)​T\+λ​A​T,\\widetilde\{T\}=\(1\-\\lambda\)T\+\\lambda AT,\(6\)whereλ\\lambdais learned\. This denoises each marker/pathway token toward its biological neighbours before the repeated recursive updates\.

#### Site 2: attention bias\.

Inside the recursive block, the*same*biological\-knowledge graphAAbiases self\-attention:

Attn​\(Q,K,V\)=softmax​\(Q​K⊤d\+λattn​𝟏​\[A\>0\]\)​V\.\\mathrm\{Attn\}\(Q,K,V\)=\\mathrm\{softmax\}\\\!\\left\(\\frac\{QK^\{\\top\}\}\{\\sqrt\{d\}\}\+\\lambda\_\{\\mathrm\{attn\}\}\\mathbf\{1\}\[A\>0\]\\right\)V\.\(7\)This encourages biologically related tokens to exchange information during refinement\.

#### Site 3: biology\-aware routing\.

The original MoR router uses the current token representationhm\(t\)h\_\{m\}^\{\(t\)\}to decide whether tokenmmshould continue to the next recursion step\. bioMoR keeps this original score and adds a learned correction based on the token’s biological neighborhood:

r~m\(t\)=wr⊤​hm\(t\)τr\+σ​\(γt\)​ϕψ​\(\[A​H\(t\),H\(t\)−A​H\(t\)\]m\)\.\\widetilde\{r\}\_\{m\}^\{\(t\)\}=\\frac\{w\_\{r\}^\{\\top\}h\_\{m\}^\{\(t\)\}\}\{\\tau\_\{r\}\}\+\\sigma\(\\gamma\_\{t\}\)\\phi\_\{\\psi\}\\\!\\left\(\[\\,AH^\{\(t\)\},\\ H^\{\(t\)\}\-AH^\{\(t\)\}\\,\]\_\{m\}\\right\)\.\(8\)The first term is the original MoR routing score\. In the second term,A​H\(t\)AH^\{\(t\)\}summarizes information from biologically related tokens, whileH\(t\)−A​H\(t\)H^\{\(t\)\}\-AH^\{\(t\)\}shows how the token differs from its biological neighborhood\. The small networkϕψ\\phi\_\{\\psi\}uses these two signals to adjust the routing score, andσ​\(γt\)\\sigma\(\\gamma\_\{t\}\)controls how much correction is added at recursion steptt\. Becauseϕψ\\phi\_\{\\psi\}is initialized to output zero, the correction is zero at the start of training and bioMoR initially behaves like biology\-agnostic MoR\. During training, the correction becomes nonzero when biological information helps reduce the task loss; otherwise, it can remain close to zero\. This design prevents noisy biological knowledge from affecting routing at the beginning while allowing the model to use it when helpful\.

#### Training and efficiency\.

The main task loss is class\-weighted cross\-entropy\. During training we minimize

ℒ=ℒCE\+α​ℒsuf\+β​ℒdiv\+γ​ℒcmp\+η​ℒrouter,\\mathcal\{L\}=\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\+\\alpha\\mathcal\{L\}\_\{\\mathrm\{suf\}\}\+\\beta\\mathcal\{L\}\_\{\\mathrm\{div\}\}\+\\gamma\\mathcal\{L\}\_\{\\mathrm\{cmp\}\}\+\\eta\\mathcal\{L\}\_\{\\mathrm\{router\}\},\(9\)where the auxiliary terms encourage marker sufficiency \(ℒsuf\\mathcal\{L\}\_\{\\mathrm\{suf\}\}\), marker diversity \(ℒdiv\\mathcal\{L\}\_\{\\mathrm\{div\}\}\), marker compression/sparsity \(ℒcmp\\mathcal\{L\}\_\{\\mathrm\{cmp\}\}\), and stable router logits and balanced token\-choice routing \(ℒrouter\\mathcal\{L\}\_\{\\mathrm\{router\}\}\)\. In expert\-choice routing, each recursion step selects a fixed number of the highest\-scoring tokens\. This fixed capacity directly controls how many tokens are processed at each step, so an additional load\-balancing loss is not needed \(?;?\)\. To quantify computational cost, we report FLOPs, a hardware\-independent count of the arithmetic operations performed by the Transformer stack \(?\):

Φeff=∑t=1K\(4​at2​d\+4​at​d​dff\),\\Phi\_\{\\mathrm\{eff\}\}=\\sum\_\{t=1\}^\{K\}\\left\(4a\_\{t\}^\{2\}d\+4a\_\{t\}dd\_\{\\mathrm\{ff\}\}\\right\),\(10\)whereata\_\{t\}is the mean number of active tokens at recursion steptt,ddis the model width, anddffd\_\{\\mathrm\{ff\}\}is the hidden dimension of the feed\-forward network\.

## 4Experimental Setup

#### Datasets and tasks\.

We evaluated bioMoR on88datasets: Genomap single\-cell data \(?\), Reactome\-based cancer data \(?\), and TCGA cohorts \(?\) downloaded through Xena \(?\)\. Single\-cell tasks are multi\-class cell\-type classification; cancer tasks include binary classification and survival prediction on HNSC \(head and neck squamous cell carcinoma\), UCEC \(uterine corpus endometrial carcinoma\), COAD \(colon adenocarcinoma\), and KIRP \(kidney renal papillary cell carcinoma\) \(?\), measured by Harrell’s C\-index \(?\)\. Gene expression measures RNA abundance \(?\), mutation denotes a DNA\-sequence alteration \(?\), and copy\-number variation \(CNV\) denotes gains or losses of DNA segments \(?\)\. Single\-cell inputs use gene tokens\. For cancer, we create each pathway token by pooling the features of genes assigned to that Reactome pathway: mean pooling for gene expression, CNV and summed mutation burden\. The pathway\-based dataset Pancancer has two variants: PAN\-2M uses mutation and CNV, whereas PAN\-3M additionally includes gene expression\. Additional datasets with full results are given in the supplement\.

#### Models and baselines\.

We first compare bioMoR with three relevant configurations of transformers:Vanillais the standard non\-recursive Transformer\.Recursivereuses one Transformer block for a fixed number of steps\.MoRadds adaptive recursion using expert\-choice or token\-choice routing \(?\)\.bioMoRuses the same recursive backbone as MoR, but integrates biological knowledge at the embedding, attention, and router sites\. We also compare with representative and state\-of\-the\-art models, including Random Forest \(?\), Nearest\-Centroid \(?\), scBiGNN \(?\), CNN\(ei\) \(?\), MOGONET \(?\), Pathformer \(?\), Geneformer \(?\), scGPT \(?\), and scTransformer\-style \(?\) baselines\. Please see the supplementary material for details about these models\.

#### Implementation and evaluation\.

Hyperparameters are fixed by modality, with no per\-dataset tuning\. The single\-cell configuration follows the Genomap benchmark protocol \(?\): AdamW, learning rate10−310^\{\-3\}, weight decay10−510^\{\-5\}, batch size128128,dmodel=96d\_\{\\mathrm\{model\}\}\{=\}96, andM=128M\{=\}128\. The pathway\-based multi\-omics configuration follows PATH \(?\): AdamW, learning rate3×10−43\\times 10^\{\-4\}, weight decay10−210^\{\-2\}, batch size3232,dmodel=128d\_\{\\mathrm\{model\}\}\{=\}128, andM=256M\{=\}256\. Both use10%10\\%linear warmup followed by cosine decay, gradient clipping at1\.01\.0, dropout0\.10\.1, at most100100epochs, and patience1515\. Results use paired55\-fold stratified cross\-validation with seed4242; each fold holds out20%20\\%for testing and10%10\\%of training for validation\. The default isK=4K\{=\}4, withK∈\{2,3,4\}K\\in\\\{2,3,4\\\}tested in RQ4\. TableLABEL:tab:selected\_resultsreports mean±\\pmSD\. See the supplement for the full hyperparameter configuration\.

## 5Results

We organize the results around four research questions\. RQ1 evaluates predictive gains, RQ2 tests which biological\-knowledge components matter, RQ3 measures whether the gains are computationally efficient, and RQ4 studies routing, depth, and model configurations\.

RQ1: Does bioMoR improve predictive performance?

In RQ1, we compare bioMoR with a controlled architecture ladder across eight datasets to determine whether biology\-guided routing improves predictive performance\. Because these datasets are class\-imbalanced, TableLABEL:tab:selected\_resultsreports macro\-F1 and balanced accuracy\. The architecture ladder consists of a vanilla Transformer, a fixed\-depth recursive Transformer, and biology\-agnostic MoR\. To align effective depth with the base Transformer, denoted Vanilla, all recursive models in TableLABEL:tab:selected\_resultsuseK=4K\{=\}4\. Both bioMoR routing variants outperform these baselines on average for both metrics\. The best bioMoR configuration improves average macro\-F1 by8\.28\.2percentage points and balanced accuracy by7\.17\.1percentage points over the strongest non\-bioMoR baseline\. Additional datasets and their performances are provided in the supplementary material\.

#### Survival prediction\.

To test whether the architecture transfers beyond classification, we evaluate survival\-risk prediction on four TCGA cohorts\. Figure[2](https://arxiv.org/html/2608.06727#S5.F2)compares bioMoR\-Expert and bioMoR\-Token with the Vanilla, Recursive, MoR\-Expert, and MoR\-Token baselines\. The bioMoR variants are competitive across HNSC, UCEC, COAD, and KIRP, showing that biology\-guided adaptive recursion also transfers to a time\-to\-event endpoint\.

![Refer to caption](https://arxiv.org/html/2608.06727v1/figs/survival_cindex.png)Figure 2:Survival prediction across TCGA cohorts\.Fold\-level C\-index distributions for HNSC, UCEC, COAD, and KIRP\. Higher values indicate better concordance between predicted risk and observed survival outcomes\.Beyond predictive performance, we next examine whether these gains are reflected in the learned representations\. Figure[3](https://arxiv.org/html/2608.06727#S5.F3)visualizes the frozen embeddings on Segerstolpe using UMAP \(?\) and evaluates their linear separability with a linear probe \(?\)\. As shown in Figure[3](https://arxiv.org/html/2608.06727#S5.F3), bioMoR forms cleaner cell\-type groups in the frozen embedding space, and the linear probe performs best on these embeddings\. This suggests that bioMoR improves the learned representation rather than only the final prediction layer\. Additional visualizations are provided in the supplement\.

Finally, to assess whether the gains extend beyond our internal architecture ladder, Figure[4](https://arxiv.org/html/2608.06727#S5.F4)compares bioMoR with dataset\-appropriate external state\-of\-the\-art baselines on PAN\-2M and Lung as representative datasets\. The full set of baselines is described underModels and baselines\. bioMoR obtains the highest mean macro\-F1 in both panels:0\.86±0\.010\.86\{\\pm\}0\.01on PAN\-2M, compared with0\.84±0\.010\.84\{\\pm\}0\.01for scBiGNN, and0\.80±0\.010\.80\{\\pm\}0\.01on Lung, compared with0\.78±0\.030\.78\{\\pm\}0\.03for TabNet\. Additional results for external baselines are provided in the supplement\.

![Refer to caption](https://arxiv.org/html/2608.06727v1/figs/umap_viz_old_600.png)Figure 3:Frozen\-embedding linear probe on Segerstolpe dataset\.UMAP visualization of penultimate per\-cell embeddings for Vanilla, Recursive, MoR, and bioMoR, colored by cell type\.![Refer to caption](https://arxiv.org/html/2608.06727v1/figs/sota_compare_600.png)Figure 4:Comparison with external baselines\.Mean macro\-F1±\\pmSD for bioMoR and dataset\-appropriate baselines on \(a\) pathway\-based PAN\-2M and \(b\) single\-cell Lung under the matched five\-fold evaluation setting\.
### Ablation Results

![Refer to caption](https://arxiv.org/html/2608.06727v1/x2.png)Figure 5:Injection\-site ablation\.Macro\-F1 across Spleen, Segerstolpe, Xin, and Pancancer\-3M with biological\-knowledge injection at the embedding, attention\-bias, router, or all three sites\. All experiments use expert\-choice routing withK=4K\{=\}4\. The inset shows the average rank across datasets \(11is best\)\.#### RQ2: In which component\(s\), the biological knowledge is most useful?

To evaluate the contribution of each biological injection site, we conduct an ablation study on the token embedding, attention bias, and router\. We ran all the 8 datasets, and due to the space, we showed results for four representative datasets in Figure[5](https://arxiv.org/html/2608.06727#S5.F5)\. Our results indicate that integrating biological knowledge at all three sites achieves the highest macro\-F1\. Among the single\-site variants, embedding injection performs best, followed by router\-only and attention\-bias\-only injection\. Overall, these results show that the three injection sites provide complementary information, with the strongest performance obtained when all three are used together\.

### Efficiency

![Refer to caption](https://arxiv.org/html/2608.06727v1/figs/t_cell_efficiency_600.png)Figure 6:Training and computational efficiency on T\-cell\.\(a\) Training loss and \(b\) validation macro\-F1 over epochs\. \(c\) Test macro\-F1 and training epoch at checkpoints selected by lowest validation loss \(∘\\circ\) or highest validation macro\-F1 \(⋆\\star\)\. \(d\) Test macro\-F1 versus FLOPs forK=2,3,4K\{=\}2,3,4; bubble size represents parameter count\.#### RQ3: What are the training and FLOP gains?

RQ3 asks whether bioMoR improves training behavior while reducing computational cost\. We evaluate computational efficiency across all datasets and provide the complete results in the supplementary material\. Due to the main\-paper space limit, we present T\-cell as a representative dataset from TableLABEL:tab:selected\_results\. With expert\-choice routing atK=4K\{=\}4, we compare learning curves and checkpoint\-selected test performance across the internal architecture ladder \(Figure[6](https://arxiv.org/html/2608.06727#S5.F6)a–c\)\. In panel \(a\), bioMoR reduces the training loss faster and reaches the lowest final value\. In panel \(b\), it achieves nearly70%70\\%validation macro\-F1, whereas the baselines remain near4040–50%50\\%\. Panel \(c\) confirms this advantage under both checkpoint\-selection rules, with bioMoR achieving the highest test macro\-F1 of69\.9%69\.9\\%\.

Panel \(d\) compares predictive performance and computational cost\. Each circle represents one model at a specific recursion depth\. Models closer to the upper\-left are preferable because they achieve higher macro\-F1 with fewer FLOPs\. bioMoR consistently maintains approximately6969–70%70\\%macro\-F1 across all recursion depths while requiring fewer FLOPs than Vanilla\. AtK=2K\{=\}2, bioMoR uses58%58\\%fewer FLOPs than Vanilla\.

### Configuration Studies

#### RQ4: Which configuration choices matter?

RQ4 examines whether bioMoR depends on a specific configuration\. We evaluate all datasets and report the complete results in the supplement; due to space limits, Table[1](https://arxiv.org/html/2608.06727#S5.T1)shows T\-cell as a representative case\. bioMoR remains strong under both routing policies and acrossK=2,3,4K\{=\}2,3,4, while Vanilla, Recursive, and MoR perform substantially worse\. Thus, biological guidance drives the main gain, while routing policy and depth adjust the accuracy–efficiency trade\-off\.

Table 1:RQ4 T\-cell configuration slice\.T\-cell macro\-F1 from Supplementary Table 1\. Each cell reports normalized FLOPs / macro\-F1 \(mean±\\pmSD\)\.Tables[2](https://arxiv.org/html/2608.06727#S5.T2)and[3](https://arxiv.org/html/2608.06727#S5.T3)examine the marker budgetMMand model widthdd, respectively\. Mean macro\-F1 remains within71\.871\.8–73\.573\.5asMMincreases from128128to20482048: T\-cell and Spleen improve, Lung remains stable, and Muraro and Segerstolpe show no consistent trend\. Asddincreases from9696to352352, mean macro\-F1 rises from70\.570\.5to72\.772\.7, although the change is not monotonic; Lung, T\-cell, and Spleen remain stable, Muraro generally improves, and Segerstolpe, BLCA, and PAN\-2M vary across settings\. Overall, bioMoR’s improvement does not depend on a specific configuration; the performance gain is primarily driven by biological knowledge\.

Table 2:RQ4 marker\-budget headroom\.bioMoR macro\-F1 \(mean±\\pmSD\) across the eight datasets shown in TableLABEL:tab:selected\_results\.Table 3:RQ4 model\-width sweep \(5\-fold CV\)\.bioMoR macro\-F1 \(mean±\\pmSD over the unified five folds\) acrossdmodel∈\{96,136,192,272,352\}d\_\{\\mathrm\{model\}\}\\in\\\{96,136,192,272,352\\\}at fixed marker budgetM=256M\{=\}256, on five single\-cell and three multi\-omics datasets\.![Refer to caption](https://arxiv.org/html/2608.06727v1/x3.png)Figure 7:Pathway\-prioritization case study\.Green boxes give examples of pathway tokens assigned to receive deeper computation; yellow boxes give examples that stop at the corresponding depth and retain their current representations\.

### Biological Case Study: Pathways Prioritized for Deeper Computation

We use PAN\-2M to examine which pathway tokens bioMoR processes more deeply for localized\-versus\-metastatic cancer classification\. Because each token represents a named biological pathway, the routing decisions can be compared with existing cancer studies\.

At each recursion, the router sends the top75%75\\%of active pathways to the next depth, while the remaining25%25\\%stop\. Thus, pathways reaching deeper levels receive more computation\. In Figure[7](https://arxiv.org/html/2608.06727#S5.F7), green boxes indicate continuing pathways and yellow boxes indicate stopped pathways\.

Pathways receiving the deepest computation include Wnt signaling, epithelial\-to\-mesenchymal transition \(EMT\), and PI3K–Akt signaling, all of which are linked to cancer invasion or metastasis \(?;?;?\)\. This agreement with prior studies indicates that bioMoR prioritizes pathways known to be relevant to metastatic cancer\.

## 6Conclusion

We developed bioMoR, a biological\-knowledge\-guided Mixture\-of\-Recursions framework and, to the best of our knowledge, the first application of MoR to genomic learning\. We integrate biological knowledge at three sites: to refine token embeddings, bias self\-attention, and route each token to an appropriate recursion depth\. This design directly addresses a central computational mismatch in omics modeling: most features should not require the same amount of deep computation, and the decision about where to spend computation should be informed by molecular structure rather than by token embeddings alone\. Our evaluation demonstrates that bioMoR significantly improves predictive performance while being more computationally efficient, as shown by fewer model parameters and reduced FLOPs\. These improvements are consistent across diverse genomic datasets and the extensive configurations we evaluated\. Overall, our results suggest that biological structure is useful not only as knowledge for representation learning, but also as a control signal for adaptive computation in genomic models\.

## References