PerturbCellRL: Verifier-Guided Reinforcement Learning for Single-Cell Perturbation Prediction

# PerturbCellRL: Verifier-Guided Reinforcement Learning for Single-Cell Perturbation Prediction
Source: [https://arxiv.org/html/2606.27752](https://arxiv.org/html/2606.27752)
Dongxia Wu∗ Stanford University Stanford, CA dowu@stanford\.edu &Mingyu Li∗ Peking University Beijing, China mingyulics@stu\.pku\.edu\.cn &Yuhui Zhang Stanford University Stanford, CA yuhuiz@stanford\.edu &Anurendra Kumar Stanford University Stanford, CA anurendk@stanford\.edu &Emma Lundberg Stanford University Stanford, CA emmalu@stanford\.edu &Serena Yeung\-Levy Stanford University Stanford, CA syyeung@stanford\.edu &Emily B\. Fox Stanford University Stanford, CA ebfox@stanford\.edu

###### Abstract

Single\-cell perturbation models can reduce costly wet\-lab screening by predicting how cells respond transcriptionally to interventions\. While recent generative models improve population\-level prediction, individual generated cells are not explicitly checked for biological consistency\. We introduce*PerturbCellRL*, a reinforcement learning \(RL\) framework that post\-trains a pretrained single\-cell transcriptomic generator using a suite of cell\-level verifiers as rewards\. These verifiers define four rewards: Pearson top\-kksimilarity, RMSE top\-kkproximity, DE Spearman, and Pathway activity\. The Pathway activity verifier rewards cells whose pathway responses match known perturbation biology\. We evaluate*PerturbCellRL*on multiple genetic and chemical perturbation benchmarks\. Across these benchmarks,*PerturbCellRL*improves over the pretrained flow\-matching generator on reward\-aligned evaluation metrics and a held\-out evaluation metric\. Moreover,*PerturbCellRL*remains competitive with state\-of\-the\-art methods on population\-level metrics\. Together, these results frame trustworthy single\-cell prediction as verifier\-guided generative alignment, moving beyond matching expression distributions toward predictions whose single\-cell perturbation effects are explicitly checked for biological consistency\.

††footnotetext:∗Equal contribution\.## 1Introduction

Single\-cell transcriptomic technologies make it possible to measure how genetic perturbations reshape cellular statesNormanet al\.\([2019](https://arxiv.org/html/2606.27752#bib.bib12)\); Replogleet al\.\([2022](https://arxiv.org/html/2606.27752#bib.bib4)\)\. These data support an increasingly important goal in computational biology: building*in silico*perturbation models that predict transcriptional responses before running expensive wet\-lab experiments\. Such models could accelerate target discovery, drug screening, combinatorial perturbation design, and therapeutic prioritization\. The recent virtual\-cell vision emphasizes that useful biological simulators should not only generate realistic measurements, but also support reliable scientific reasoning under interventionsBunneet al\.\([2024](https://arxiv.org/html/2606.27752#bib.bib44)\); Johnsonet al\.\([2023](https://arxiv.org/html/2606.27752#bib.bib46)\)\.

Perturbation prediction in single\-cell RNA\-seq is difficult because observations are sparse, noisy, high dimensional, and typically unpaired\. For a perturbationcc, we observe populations of control cells and perturbed cells, but not the before\-and\-after response of the same physical cell\. This makes the task fundamentally distributional: the model can only learn how to generate target expression distributions under a control state and perturbation condition\. Flow matching is well suited to this setting because it learns continuous generative dynamics from simple base distributions\. We utilize a single\-cell flow matching generator that predicts perturbed expression distributions conditioned on control states and perturbations\. scDFMYuet al\.\([2026](https://arxiv.org/html/2606.27752#bib.bib13)\)and related models provide strong distributional baselinesBunneet al\.\([2023](https://arxiv.org/html/2606.27752#bib.bib3)\); Kleinet al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib6)\), but their training objectives primarily reward population\-level agreement\.

![Refer to caption](https://arxiv.org/html/2606.27752v1/imgs/overview.png)Figure 1:Overview\.Current single\-cell perturbation generators can produce implausible individual responses\. For example, a generated cell may show perturbation effects inconsistent with the known pathway direction\. We design a suite of biologically meaningful verifiers serving in three roles: \(1\) as*evaluators*to assess single\-cell biological consistency, \(2\) as*reward signals*to align generation via RL, and \(3\) as*verification modules*to improve samples through test\-time scaling\.However, distributional realism does not by itself imply trustworthy generated cell profilesViñas Tornéet al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib2)\)\. A generated population can match aggregate target statistics while individual samples remain poorly aligned with plausible treatment responses, differentially expressed gene rankings, discriminative perturbation signatures, or pathway\-level responsesSchubertet al\.\([2018](https://arxiv.org/html/2606.27752#bib.bib98)\)\. These failures matter because downstream analyses often inspect individual generated cells or selected subpopulations when prioritizing perturbations, interpreting mechanisms, or choosing candidates for follow\-up experiments\. We propose single\-cell verifiers as biological guardrails: they check whether each generated profile remains compatible with plausible target responses, rather than only matching population\-level statistics\.

We further propose*PerturbCellRL*: reinforcement learning \(RL\) for trustworthy single\-cell transcriptomic prediction\. The key idea is to turn biological verifiers into reward functions for post\-training a pretrained generative model\. Our verifier suite scores each generated cell using four rewards: Pearson top\-kksimilarity, RMSE top\-kkproximity, DE Spearman, and Pathway activity\. These rewards measure target alignment, population placement, transcriptional ranking, and pathway\-level biological consistency\. Because these verifiers can be non\-differentiable and computed outside the generator, RL provides a natural mechanism for using them as direct optimization signalsZhenget al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib75)\); Liuet al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib71)\)\.

*PerturbCellRL*uses a pretrained flow\-matching generator as the base model and post\-trains its generative dynamics with a weighted sum of reward objectives\. At training time, the model generates several possible perturbed states conditioned on a control state and perturbation condition, scores them with the verifier suite, and updates the generator to increase the probability of high\-reward cells while regularizing against the pretrained policy\. At inference time, we use the pathway activity verifier for best\-of\-NNselection because it can be labeled from gene perturbation types without knowing the ground\-truth treatment response: multiple candidate responses are generated, scored, and filtered to select the most biologically plausible predictionSnellet al\.\([2024](https://arxiv.org/html/2606.27752#bib.bib78)\)\. This unifies evaluation, training, and inference\-time selection around the same transparent biological checks\.

We evaluate*PerturbCellRL*on multiple genetic and chemical perturbation benchmarks\. Across these benchmarks,*PerturbCellRL*improves over the pretrained flow\-matching generator on reward\-aligned evaluation metrics and a held\-out evaluation metric\. Best\-of\-NNselection further improves biological consistency, indicating that verifier\-guided test\-time scaling can extract better predictions from sampled candidate responses\. Importantly, these reward gains do not come at the cost of abandoning distributional quality:*PerturbCellRL*is competitive with state\-of\-the\-art perturbation models on existing population\-level evaluation metrics\.

In summary, our work identifies a key limitation of existing single\-cell perturbation modeling: the lack of explicit enforcement of biological consistency at the single cell level\. We address this limitation by incorporating biologically meaningful evaluators through RL, which substantially improves the plausibility of generated cell expressions\. We further show that these gains can be amplified via test\-time scaling\. Finally, our rewards serve as new metrics for the community to benchmark against\. Overall, our results move gene expression generation from “distributionally good” to being*biologically consistent*, supporting the downstream goal of drug discovery and personalized medicine with reduced reliance on costly wet\-lab experiments\.

## 2Related Work

#### Single\-cell perturbation prediction with large\-scale generative modeling\.

Single\-cell perturbation prediction aims to infer how cellular transcriptomes change under genetic or chemical interventions\. This problem has received substantial attention, driven by the increasing availability of large\-scale perturbation datasets and advances in generative modeling\.

On the data side, the NormanNormanet al\.\([2019](https://arxiv.org/html/2606.27752#bib.bib12)\), ComboSciPlexMathuret al\.\([2022](https://arxiv.org/html/2606.27752#bib.bib11)\)and Virtual Cell ChallengeRoohaniet al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib99)\)provide challenging genetic and chemical perturbation benchmarks\. scPerturbPeidliet al\.\([2024](https://arxiv.org/html/2606.27752#bib.bib52)\)highlights the broader availability of harmonized single\-cell perturbation data\.

On the modeling side, early deep generative approaches such as scVI established probabilistic representation learning for single\-cell transcriptomicsLopezet al\.\([2018](https://arxiv.org/html/2606.27752#bib.bib65)\)\. Subsequent perturbation prediction methods have used conditional autoencoders, graph neural networks, transformer architectures, and distributional generative models to predict responses under unseen perturbations or cellular contextsLotfollahiet al\.\([2023](https://arxiv.org/html/2606.27752#bib.bib8)\); Roohaniet al\.\([2024](https://arxiv.org/html/2606.27752#bib.bib9)\); Adduriet al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib7)\); Bereket and Karaletsos \([2023](https://arxiv.org/html/2606.27752#bib.bib82)\); Rampášeket al\.\([2019](https://arxiv.org/html/2606.27752#bib.bib83)\)\. More recently, state\-of\-the\-art methods have begun to adopt flow matchingLipmanet al\.\([2023](https://arxiv.org/html/2606.27752#bib.bib39),[2024](https://arxiv.org/html/2606.27752#bib.bib33)\); Liuet al\.\([2023](https://arxiv.org/html/2606.27752#bib.bib63)\), which provides a natural framework for conditional generation\. This is particularly well suited to perturbation prediction, where models generate perturbed cells conditioned on control states and perturbation labelsYuet al\.\([2026](https://arxiv.org/html/2606.27752#bib.bib13)\); Kleinet al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib6)\); Zhanget al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib17)\)\. In this work, we build on this line of flow\-matching models, using scDFMYuet al\.\([2026](https://arxiv.org/html/2606.27752#bib.bib13)\)in particular as our base generator, and further improve it through verifier\-guided post\-training on top of a strong flow\-matching backbone\.

#### Reinforcement learning and test\-time scaling for biological alignment\.

Although generative models can produce expression\-like cell profiles, incorporating biological priors into their predictions remains challenging\. This issue is especially important in single\-cell perturbation prediction, where a generated cell may look expression\-like while its perturbation effect is biologically inconsistent\. A trustworthy prediction should match observed population\-level expression patterns, correctly rank differentially expressed genes, and avoid unnecessary drift in stable genes\. However, these objectives are often non\-differentiable, making them difficult to optimize directly\.

One promising solution is RL, which has recently been used to align diffusion and flow models with human preferences or task\-specific rewardsBlacket al\.\([2023](https://arxiv.org/html/2606.27752#bib.bib69)\); Fanet al\.\([2023](https://arxiv.org/html/2606.27752#bib.bib70)\); Liuet al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib71)\); Xueet al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib73)\); Liet al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib74)\); Wuet al\.\([2026](https://arxiv.org/html/2606.27752#bib.bib1)\)\. Flow and diffusion models introduce additional challenges because exact sample likelihoods are often intractable\. Prior work such as FlowGRPO and MixGRPO formulates this problem as a Markov decision process, while recent methods such as DiffusionNFTZhenget al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib75)\)introduce forward\-process objectives for online RL, improving both training efficiency and stability\. In this work, we adapt this alignment perspective to transcriptomic perturbation prediction, where rewards are derived from biological verifiers rather than visual or human\-preference scores\.

Furthermore, when a verifier is available, inference can be improved by sampling multiple candidates and selecting the highest\-scoring output, an emerging direction known as test\-time scaling\. This best\-of\-NNstrategy is widely used in verifier\-guided reasoning systemsCobbeet al\.\([2021](https://arxiv.org/html/2606.27752#bib.bib76)\); Lightmanet al\.\([2023](https://arxiv.org/html/2606.27752#bib.bib77)\); Snellet al\.\([2024](https://arxiv.org/html/2606.27752#bib.bib78)\)and has also been explored as test\-time scaling in diffusion modelsMaet al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib72)\)\. In*PerturbCellRL*, best\-of\-NNselection uses the same normalized reward as RL post\-training\.

## 3Problem Formulation

We consider single\-cell perturbation prediction in transcriptomic space\. Letui∈ℝGu\_\{i\}\\in\\mathbb\{R\}^\{G\}denote a normalized control\-cell expression vector overGGgenes, and letci∈𝒞c\_\{i\}\\in\\mathcal\{C\}denote a perturbation, such as gene overexpression, CRISPR activation, or chemical treatment\. The goal is to learn a conditional generator that produces a perturbed expression profile

yi∼πθ\(⋅∣ui,ci\),y\_\{i\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid u\_\{i\},c\_\{i\}\),\(1\)whereyi∈ℝGy\_\{i\}\\in\\mathbb\{R\}^\{G\}is a generated perturbed transcriptome for control celluiu\_\{i\}under conditioncic\_\{i\}\.

In most single\-cell perturbation screens, control and perturbed cells are observed as unpaired populations\. For a perturbation conditioncc, let\{yc,jobs\}j=1nc\\\{y^\{\\mathrm\{obs\}\}\_\{c,j\}\\\}\_\{j=1\}^\{n\_\{c\}\}denote the real perturbed cells observed under that condition\. Because the same physical cell is not measured before and after perturbation, the model learns population\-level conditional generation rather than paired cell\-level responses\.

#### Base generative model\.

We use scDFMYuet al\.\([2026](https://arxiv.org/html/2606.27752#bib.bib13)\)as the base generative model, as it achieves strong performance in learning population\-level perturbation prediction\. Following conditional flow matching, scDFM learns a velocity fieldvθ​\(xt,t,ui,ci\)v\_\{\\theta\}\(x\_\{t\},t,u\_\{i\},c\_\{i\}\)that maps Gaussian base samples to perturbed expression states, conditioned on the control expression and perturbation\. Given a control celluiu\_\{i\}, perturbationcic\_\{i\}, Gaussian base samplex0∼𝒩​\(0,I\)x\_\{0\}\\sim\\mathcal\{N\}\(0,I\), and an observed perturbed cellyci,jobsy^\{\\mathrm\{obs\}\}\_\{c\_\{i\},j\}, a standard flow\-matching objective can be written as

ℒFM​\(θ\)\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{FM\}\}\(\\theta\)=𝔼ui,ci,x0,yci,jobs,t​‖vθ​\(xt,t,ui,ci\)−\(yci,jobs−x0\)‖22,\\displaystyle=\\mathbb\{E\}\_\{\\begin\{subarray\}\{c\}u\_\{i\},c\_\{i\},x\_\{0\},\\\\ y^\{\\mathrm\{obs\}\}\_\{c\_\{i\},j\},t\\end\{subarray\}\}\\left\\\|v\_\{\\theta\}\(x\_\{t\},t,u\_\{i\},c\_\{i\}\)\-\(y^\{\\mathrm\{obs\}\}\_\{c\_\{i\},j\}\-x\_\{0\}\)\\right\\\|\_\{2\}^\{2\},\(2\)xt\\displaystyle x\_\{t\}=\(1−t\)​x0\+t​yci,jobs,t∼𝒰​\[0,1\]\.\\displaystyle=\(1\-t\)x\_\{0\}\+ty^\{\\mathrm\{obs\}\}\_\{c\_\{i\},j\},\\qquad t\\sim\\mathcal\{U\}\[0,1\]\.
In this work, we treat scDFM as a pretrained policyπθ​\(yi∣ui,ci\)\\pi\_\{\\theta\}\(y\_\{i\}\\mid u\_\{i\},c\_\{i\}\)that can generate candidate perturbed transcriptomes from Gaussian base samples\.

#### Verifier objective\.

We are not only interested in average expression error or strong distributional matching\. We want generated cells to satisfy biological checks that matter for scientific use\. Let\{rk\}k=1K\\\{r\_\{k\}\\\}\_\{k=1\}^\{K\}be transcriptomic reward functions, where

rk:ℝG×ℝG×𝒞→ℝr\_\{k\}:\\mathbb\{R\}^\{G\}\\times\\mathbb\{R\}^\{G\}\\times\\mathcal\{C\}\\rightarrow\\mathbb\{R\}\(3\)scores a generated perturbed expressionyiy\_\{i\}, its control expressionuiu\_\{i\}, and conditioncic\_\{i\}along one axis of biological consistency\. The concrete rewards used in*PerturbCellRL*are defined in §[4\.1](https://arxiv.org/html/2606.27752#S4.SS1)\. The combined reward can be written abstractly as

R​\(yi,ui,ci\)=∑k=1Kwk​rk​\(yi,ui,ci\),R\(y\_\{i\},u\_\{i\},c\_\{i\}\)=\\sum\_\{k=1\}^\{K\}w\_\{k\}r\_\{k\}\(y\_\{i\},u\_\{i\},c\_\{i\}\),\(4\)wherewkw\_\{k\}are reward weights\.

The*PerturbCellRL*objective is to obtain a post\-trained generatorπθ′\\pi\_\{\\theta^\{\\prime\}\}that improves verifier scores while staying close to the pretrained scDFM generator:

maxθ′𝔼ui,ci,yi∼πθ′\(⋅∣ui,ci\)\[R\(yi,ui,ci\)\]s\.t\.𝔼ui,ciDKL\(πθ′\(⋅∣ui,ci\)∥πθ\(⋅∣ui,ci\)\)≤ϵ\.\\max\_\{\\theta^\{\\prime\}\}\\;\\mathbb\{E\}\_\{u\_\{i\},c\_\{i\},\\,y\_\{i\}\\sim\\pi\_\{\\theta^\{\\prime\}\}\(\\cdot\\mid u\_\{i\},c\_\{i\}\)\}\\left\[R\(y\_\{i\},u\_\{i\},c\_\{i\}\)\\right\]\\quad\\mathrm\{s\.t\.\}\\quad\\mathbb\{E\}\_\{u\_\{i\},c\_\{i\}\}D\_\{\\mathrm\{KL\}\}\\left\(\\pi\_\{\\theta^\{\\prime\}\}\(\\cdot\\mid u\_\{i\},c\_\{i\}\)\\,\\\|\\,\\pi\_\{\\theta\}\(\\cdot\\mid u\_\{i\},c\_\{i\}\)\\right\)\\leq\\epsilon\.\(5\)The KL constraint is important because biological verifiers are necessarily incomplete\. Regularizing toward scDFM helps preserve the model’s learned expression manifold and reduces reward hacking\.

## 4Method

*PerturbCellRL*post\-trains a pretrained single\-cell flow matching generator with biologically informed rewards\. The method has three components: a verifier suite, an RL update for flow matching models, and verifier\-guided inference\.

### 4\.1Reward Functions

![Refer to caption](https://arxiv.org/html/2606.27752v1/imgs/finall_00.png)Figure 2:*PerturbCellRL*Rewards\.Pearson top\-kkand RMSE top\-kkcompare each generated cell with nearby real target cells from the same perturbation condition\. The top\-kkdesign encourages predictions to lie near the target\-cell manifold while preserving cell\-level diversity, instead of collapsing all samples to a condition centroid\. Pathway activity and DE Spearman evaluate pathway directionality and differential\-expression ranking\.We use the notation from §[3](https://arxiv.org/html/2606.27752#S3)\. All rewards are computed within the current split, and we omit the split index for notation simplicity\. Let𝒢=\{1,…,G\}\\mathcal\{G\}=\\\{1,\\ldots,G\\\}denote the reward gene set\. When computing rewards, expressions are restricted to genes in𝒢\\mathcal\{G\}\. Letμ∈ℝG\\mu\\in\\mathbb\{R\}^\{G\}be the mean expression of all real target cells in the split\. Details are in Appendix[B](https://arxiv.org/html/2606.27752#A2)\.

#### Pearson top\-kksimilarity reward\.

This reward measures whether a generated cell is directionally similar to real target cells from the same perturbation\. After centering byμ\\mu, we find the top\-kkreal target cells with largest Pearson correlation toyiy\_\{i\}and average those correlations to obtainripearson∈\[−1,1\]r\_\{i\}^\{\\mathrm\{pearson\}\}\\in\[\-1,1\]:

ripearson=1\|𝒫i\|​∑j∈𝒫iρ​\(yi−μ,yci,jobs−μ\)\.r\_\{i\}^\{\\mathrm\{pearson\}\}=\\frac\{1\}\{\|\\mathcal\{P\}\_\{i\}\|\}\\sum\_\{j\\in\\mathcal\{P\}\_\{i\}\}\\rho\(y\_\{i\}\-\\mu,\\;y^\{\\mathrm\{obs\}\}\_\{c\_\{i\},j\}\-\\mu\)\.\(6\)

#### RMSE top\-kkproximity reward\.

This reward measures local proximity to the target\-cell manifold\. We compute the average RMSE fromyiy\_\{i\}to its top\-kknearest real target cells under conditioncic\_\{i\}, then map this distance torirmse​\-​topk∈\[0,1\]r\_\{i\}^\{\\mathrm\{rmse\}\\text\{\-\}\\mathrm\{topk\}\}\\in\[0,1\]using a condition\-specific leave\-one\-out normalization\.

rirmse​\-​topk=1−1\|𝒩i\|​∑j∈𝒩iRMSE⁡\(yi,yci,jobs\)Ucirmse​\-​topk\.r\_\{i\}^\{\\mathrm\{rmse\}\\text\{\-\}\\mathrm\{topk\}\}=1\-\\frac\{\\frac\{1\}\{\|\\mathcal\{N\}\_\{i\}\|\}\\sum\_\{j\\in\\mathcal\{N\}\_\{i\}\}\\operatorname\{RMSE\}\(y\_\{i\},y^\{\\mathrm\{obs\}\}\_\{c\_\{i\},j\}\)\}\{U\_\{c\_\{i\}\}^\{\\mathrm\{rmse\}\\text\{\-\}\\mathrm\{topk\}\}\}\.\(7\)

#### DE Spearman reward\.

This reward evaluates whether generated cells reproduce the rank ordering of perturbation effects on significant DE genes\. For each condition, we compute generated and real fold changes on the selected DE genes, rank\-transform both vectors, and use their Pearson correlation asrispearman∈\[−1,1\]r\_\{i\}^\{\\mathrm\{spearman\}\}\\in\[\-1,1\]:

rispearman=ρ​\(rank⁡\(F^i,𝒟ci\),rank⁡\(Fci,𝒟ci\)\)\.r\_\{i\}^\{\\mathrm\{spearman\}\}=\\rho\(\\operatorname\{rank\}\(\\widehat\{F\}\_\{i,\\mathcal\{D\}\_\{c\_\{i\}\}\}\),\\operatorname\{rank\}\(F\_\{c\_\{i\},\\mathcal\{D\}\_\{c\_\{i\}\}\}\)\)\.\(8\)

#### Pathway activity reward\.

The Pathway activity reward encodes prior biological knowledge about how a perturbation should impact genetic pathways\. Unlike other verifiers, it requires no ground\-truth target cells, making it applicable at test time without real perturbed data\. This reference\-free property enables verifier\-guided test\-time scaling; see §[4\.3](https://arxiv.org/html/2606.27752#S4.SS3)\.

PROGENy provides 14 curated signaling pathway signatures, each defined by weighted gene sets from perturbation experimentsSchubertet al\.\([2018](https://arxiv.org/html/2606.27752#bib.bib98)\)\. These signatures map gene expression to interpretable pathway activity scores\. We use a fold\-specific trained MLPfϕ:ℝK→ℝ14f\_\{\\phi\}:\\mathbb\{R\}^\{K\}\\to\\mathbb\{R\}^\{14\}to predict these PROGENy pathway activities, whereKKis the predictor gene set stored in the checkpoint\. The model projects the generated and control expressions onto this gene set and predicts pathway activities\. We use the*change*in pathway activity relative to the control cell rather than absolute activity\. This isolates the perturbation effect from baseline expression:

s^i=fϕ​\(yi\),si0=fϕ​\(ui\),Δ​si=s^i−si0\.\\hat\{s\}\_\{i\}=f\_\{\\phi\}\(y\_\{i\}\),\\qquad s\_\{i\}^\{0\}=f\_\{\\phi\}\(u\_\{i\}\),\\qquad\\Delta s\_\{i\}=\\hat\{s\}\_\{i\}\-s\_\{i\}^\{0\}\.\(9\)Herefϕf\_\{\\phi\}is a small trained MLP \(∼\\sim680K parameters\) used to predict PROGENy pathway scores\. Details of the predictor architecture and validation are in Appendix[C](https://arxiv.org/html/2606.27752#A3)\.

For a single\-gene perturbation with target genehh, the annotation table mapshhto pathwayp​\(h\)p\(h\), directiond​\(h\)∈\{\+1,−1\}d\(h\)\\in\\\{\+1,\-1\\\}, and confidence weightw​\(h\)≥0w\(h\)\\geq 0\. Confidence weights are fixed by annotation tier: High/Medium=1\.0=1\.0, Data\-derived=0\.8=0\.8, Low=0\.5=0\.5, and Ultra\-low=0\.2=0\.2\. The annotation table is constructed from literature curation and empirical PROGENy validation on the Norman dataset; details are in Appendix[D](https://arxiv.org/html/2606.27752#A4)\. The Pathway activity reward directly maps the annotated signed pathway change to\[0,1\]\[0,1\]:

ripathway=σ​\(w​\(h\)​d​\(h\)​Δ​si,p​\(h\)τpath\),ripathway∈\[0,1\]\.r\_\{i\}^\{\\mathrm\{pathway\}\}=\\sigma\\left\(\\frac\{w\(h\)\\,d\(h\)\\,\\Delta s\_\{i,p\(h\)\}\}\{\\tau\_\{\\mathrm\{path\}\}\}\\right\),\\qquad r\_\{i\}^\{\\mathrm\{pathway\}\}\\in\[0,1\]\.\(10\)
Thus, annotated up\-regulators are rewarded when the corresponding pathway delta is positive, while annotated down\-regulators are rewarded when it is negative\. Annotating pathway effects for combinatorial perturbations is difficult and left to future work\.

#### Reward normalization and combination\.

We map each reward to\[0,1\]\[0,1\]using its known range:\[−1,1\]\[\-1,1\]for Pearson top\-kkand DE Spearman, and\[0,1\]\[0,1\]for RMSE top\-kkand Pathway activity\. Letr~im\\widetilde\{r\}\_\{i\}^\{m\}denote the normalized reward\. Given reward setℳ\\mathcal\{M\}and weightsλm\\lambda\_\{m\}, the combined scalar reward is

r¯i=∑m∈ℳλm∑m′∈ℳλm′​r~im,r¯i∈\[0,1\]\.\\bar\{r\}\_\{i\}=\\sum\_\{m\\in\\mathcal\{M\}\}\\frac\{\\lambda\_\{m\}\}\{\\sum\_\{m^\{\\prime\}\\in\\mathcal\{M\}\}\\lambda\_\{m^\{\\prime\}\}\}\\widetilde\{r\}\_\{i\}^\{m\},\\qquad\\bar\{r\}\_\{i\}\\in\[0,1\]\.\(11\)We use equal weights for the four rewards by default\.

### 4\.2RL Post\-Training

![Refer to caption](https://arxiv.org/html/2606.27752v1/imgs/figure3.png)Figure 3:*PerturbCellRL*algorithm\.RL post\-training seeks to increase the likelihood of high\-reward samples and decrease the likelihood of low\-reward samples\. Therefore, the core training loop of*PerturbCellRL*consists of interleaved phases of sampling and training\. \(a\) Sampling: we generate multiple rollouts from a fixed control expression and perturbation condition, scoring each with the reward models\. \(b\) Training: because exact likelihoods in flow matching are intractable, we construct positive and negative velocities from the batch of rollouts and optimize them contrastively to achieve this goal, following DiffusionNFTZhenget al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib75)\)\.We describe how we optimize the pretrained base modelvθv\_\{\\theta\}with respect to the reward functions introduced in §[4\.1](https://arxiv.org/html/2606.27752#S4.SS1)\.

#### Objective\.

Our objective is to maximize the combined reward defined in Eq\. \([11](https://arxiv.org/html/2606.27752#S4.E11)\)\. Since these biological reward functions are non\-differentiable, standard backpropagation is inapplicable, necessitating an RL approach\. The core principle is to increase the generation likelihood of high\-reward samples while penalizing low\-reward ones\. We provide an overview of the algorithm in Figure[3](https://arxiv.org/html/2606.27752#S4.F3)\.

#### Algorithm overview\.

We adopt DiffusionNFTZhenget al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib75)\), a state\-of\-the\-art online RL algorithm for flow matching\. It operates on the flow’s forward process, avoiding intractable log likelihoods, and is built fromdistribution\-agnosticcomponents, hence extending naturally to our conditional Gaussian\-to\-expression flow matching setting without modification\. At each iteration, DiffusionNFT collects a batch of generated cell profiles, evaluates them with respect to the reward functions, and uses the rewards to define an improvement direction over the current policy\. The key idea is to split generated samples intopositive\(*high\-reward*\) andnegative\(*low\-reward*\) subsets and learn a contrastive update that moves the model towards the positive distribution\. Concretely, given a Gaussian startx0x\_\{0\}, generated cell profileyiy\_\{i\}, and optimality rewardr∈\[0,1\]r\\in\[0,1\], the training objective is \(detailed explanations in Appendix[A](https://arxiv.org/html/2606.27752#A1)\):

ℒ​\(θ\)=𝔼ui,ci,x0,tyi∼πold\(⋅∣x0,ui,ci\)\[r∥vθ\+\(xt,ui,ci,t\)−v∥22\+\(1−r\)∥vθ−\(xt,ui,ci,t\)−v∥22\]\+βDKL\(vθ∥vold\)\.\\begin\{split\}\\mathcal\{L\}\(\\theta\)=&\\mathbb\{E\}\_\{\\begin\{subarray\}\{c\}u\_\{i\},c\_\{i\},x\_\{0\},t\\\\ y\_\{i\}\\sim\\pi^\{\\mathrm\{old\}\}\(\\cdot\\mid x\_\{0\},u\_\{i\},c\_\{i\}\)\\end\{subarray\}\}\\Big\[r\\,\\\|v\_\{\\theta\}^\{\+\}\(x\_\{t\},u\_\{i\},c\_\{i\},t\)\-v\\\|\_\{2\}^\{2\}\\\\ &\\quad\+\(1\-r\)\\,\\\|v\_\{\\theta\}^\{\-\}\(x\_\{t\},u\_\{i\},c\_\{i\},t\)\-v\\\|\_\{2\}^\{2\}\\Big\]\+\\beta\\,D\_\{\\mathrm\{KL\}\}\\\!\\left\(v\_\{\\theta\}\\,\\\|\\,v^\{\\mathrm\{old\}\}\\right\)\.\\end\{split\}\(12\)

#### Rollout and advantage estimation\.

During sampling, we fix a perturbation conditioncic\_\{i\}and a source control celluiu\_\{i\}, draw Gaussian starts\{x0\(j\)\}j=1m\\\{x\_\{0\}^\{\(j\)\}\\\}\_\{j=1\}^\{m\}, and generate a group ofmmcandidate profiles\{yi\(j\)\}j=1m\\\{y\_\{i\}^\{\(j\)\}\\\}\_\{j=1\}^\{m\}\. Within each group, diversity comes from these Gaussian starts, whileuiu\_\{i\}andcic\_\{i\}remain fixed conditions\. We find this yields sufficient variation for DiffusionNFT to distinguish positive from negative generations\. Each candidate is scored by the reward functions, and the raw rewards are normalized within the group to obtain optimality probabilitiesr\(j\)∈\[0,1\]r^\{\(j\)\}\\in\[0,1\], following the advantage normalization schemeZhenget al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib75)\)\. The forward process is then applied between eachx0\(j\)x\_\{0\}^\{\(j\)\}and generated profile, and the loss in Eq\. \([12](https://arxiv.org/html/2606.27752#S4.E12)\) is computed over the group\.

### 4\.3Verifier\-Guided Inference

Explicit reward functions uniquely enable general test\-time scaling\. Among our proposed rewards, the Pathway activity reward operates without ground\-truth target gene expressions\. This allows it to directly evaluate biological feasibility at inference time\. In principle, any reference\-free verifier could guide test\-time scaling\. For example, a trained cell\-type classifier could score whether generated profiles preserve the expected identity, and housekeeping\-gene checks could penalize unnecessary drift\. We leverage this property to select among candidate generations at inference time\. This adapts the success of best\-of\-NNselection from reasoning modelsCobbeet al\.\([2021](https://arxiv.org/html/2606.27752#bib.bib76)\); Lightmanet al\.\([2023](https://arxiv.org/html/2606.27752#bib.bib77)\); Snellet al\.\([2024](https://arxiv.org/html/2606.27752#bib.bib78)\)to single\-cell perturbation prediction\.

Given a perturbation conditioncic\_\{i\}and a source control celluiu\_\{i\}, we generateNNcandidate profiles\{yi\(ℓ\)\}ℓ=1N\\\{y\_\{i\}^\{\(\\ell\)\}\\\}\_\{\\ell=1\}^\{N\}and select the one with the highest Pathway activity reward:

yi∗=yi\(ℓ∗\),ℓ∗=arg⁡maxℓ∈\{1,…,N\}⁡rpathway​\(yi\(ℓ\)\)\.y\_\{i\}^\{\*\}=y\_\{i\}^\{\(\\ell^\{\*\}\)\},\\qquad\\ell^\{\*\}=\\arg\\max\_\{\\ell\\in\\\{1,\\dots,N\\\}\}r^\{\\mathrm\{pathway\}\}\(y\_\{i\}^\{\(\\ell\)\}\)\.\(13\)This provides a simple, training\-free mechanism to improve prediction quality given additional inference compute\. Moreover, best\-of\-NNselection is complementary to RL post\-training: RL improves thebase distributionfrom which candidates are drawn, so that even modest values ofNNyield high\-quality outputs, while Pathway activity selection contributes additional gains\.

## 5Experiments

### 5\.1Experimental Setup

#### Datasets\.

![Refer to caption](https://arxiv.org/html/2606.27752v1/imgs/split.png)Figure 4:Norman additive and holdout split protocols\.Our experiments use NormanNormanet al\.\([2019](https://arxiv.org/html/2606.27752#bib.bib12)\)and ComboSciPlexMathuret al\.\([2022](https://arxiv.org/html/2606.27752#bib.bib11)\)datasets to test genetic and chemical perturbation predictions\. Following scDFMYuet al\.\([2026](https://arxiv.org/html/2606.27752#bib.bib13)\), we evaluate two Norman train\-test split protocols, illustrated in Figure[4](https://arxiv.org/html/2606.27752#S5.F4)\. In the additive split, all single\-gene perturbations and a subset of double\-gene perturbations are used for training, and the model predicts held\-out double\-gene perturbations\. In the holdout split, selected double\-gene perturbations and their constituent single\-gene perturbations are held out for testing, while the remaining perturbations are used for training\. We report holdout single and holdout double by averaging over the single\-gene and double\-gene perturbations in the holdout test set, respectively\. For the Norman additive and holdout results, we use four random train\-test folds and report averages across folds\.

#### Baselines\.

We compare against Control \(unperturbed cells\), Additive \(a task\-specific baseline for Norman additive that linearly superposes single\-gene effects\), GEARSRoohaniet al\.\([2024](https://arxiv.org/html/2606.27752#bib.bib9)\), CPALotfollahiet al\.\([2023](https://arxiv.org/html/2606.27752#bib.bib8)\), STATEAdduriet al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib7)\), CellFlowKleinet al\.\([2025](https://arxiv.org/html/2606.27752#bib.bib6)\), and scDFMYuet al\.\([2026](https://arxiv.org/html/2606.27752#bib.bib13)\)\. The primary comparison is against scDFM, which serves as the pretrained base model and the strongest flow\-matching baseline\.

#### Evaluation Metrics\.

We report both population\-level and single\-cell\-level metrics\. At the population level, we measure mean absolute error \(MAE\) between predicted and real pseudobulk means,PearsonΔ\\DeltaandPearsonΔ^\\hat\{\\Delta\}\(Pearson correlation of perturbation effects centered by control and training centroid, respectively\),DE\-Spearman LFC Sig\(Spearman correlation of log fold changes on statistically significant DE genes\), andDS\(Discrimination Score\), which ranks the predicted perturbation effect against all test perturbations byL1L\_\{1\}distance to the real effect\. We also report distribution\-level distances between predicted and real cell populations:MMD, using an RBF kernel wheres=0\.5s=0\.5setsσ2=s\\sigma^\{2\}=stimes the target\-cell median pairwise squared distance, andEnergy Distance\.DS,MMD, andEnergy Distanceare fully held\-out population\-level evaluation metrics\. They are not optimized as rewards, and therefore help test whether reward improvement comes from genuine biological alignment rather than reward hacking\.

At the single\-cell level, we evaluate using the four verifier rewards defined in §[4\.1](https://arxiv.org/html/2606.27752#S4.SS1)\. ThePearson top\-kksimilarity rewardmeasures average Pearson similarity between each generated cell’s perturbation effect and the top\-kkmost similar real target cells\. TheDE Spearman rewardmeasures Pearson correlation between rank\-transformed generated and real log fold changes on significant DE genes\. TheRMSE top\-kkproximity rewardmeasures the normalized top\-kkroot mean squared distance from each generated expression to real target cells from the same condition\. ThePathway activity rewardis the annotation\-weighted PROGENy pathway score from a fold\-specific MLP\. For reporting, we subtract the neutral value0\.50\.5from this reward, so zero indicates no annotated pathway\-direction evidence\. Note this verifier is only evaluated for single\-gene perturbations\. This is not a methodological limitation; we currently lack pathway annotations for the other settings\. We also reportDSas a held\-out single\-cell evaluation metric in addition to the four rewards\.

#### Implementation details\.

The base generator is the public scDFM checkpointYuet al\.\([2026](https://arxiv.org/html/2606.27752#bib.bib13)\), used as the reference model for RL fine\-tuning without retraining from scratch\. Each normalized reward is assigned weight11\. We use3232rollouts per group, sample batches of6464, and learning rate2×10−62\\times 10^\{\-6\}\. The KL weight is2\.02\.0for Norman and1\.21\.2for ComboSciPlex\. Each RL run is trained for16001600steps on one H100 GPU\.

### 5\.2Main Results

#### Single\-cell\-level performance\.

Figure[5](https://arxiv.org/html/2606.27752#S5.F5)compares pretrained scDFM with*PerturbCellRL*across Norman additive and holdout settings\. We track the four optimized single\-cell rewards together with single\-cell Discrimination Score \(DS\), which is held out from RL optimization\. Across both settings,*PerturbCellRL*improves the optimized rewards and also increases held\-out DS\. These results suggest that verifier\-guided post\-training improves single\-cell biological consistency without merely overfitting to the training rewards\.

![Refer to caption](https://arxiv.org/html/2606.27752v1/imgs/figure5.png)Figure 5:*PerturbCellRL*post\-training performance on Norman additive and holdout settings\.We report the four proposed single\-cell rewards and held\-out single\-cell Discrimination Score \(DS\) over16001600training steps\. Step0corresponds to the pretrained scDFM model\.
#### Population\-level performance\.

We next ask whether single\-cell reward gains preserve population\-level prediction quality across all benchmark settings\. Table[1](https://arxiv.org/html/2606.27752#S5.T1)reports Norman holdout single\-gene, Norman holdout double\-gene, Norman additive, and ComboSciPlex results, respectively\. All metrics in these tables are measured at the population level, unlike the per\-cell verifier rewards used for RL\. They aggregate cells within each perturbation or compare full predicted and real cell populations\. Thus, they evaluate distributional prediction quality rather than the single\-cell reward signals\. This separation partially addresses reward\-hacking concerns: a model could improve per\-cell reward signals while distorting the generated population distribution\. Moreover, PearsonΔ\\Delta, DS, MMD, and Energy Distance were not used by RL during training, and as such serve as fully held\-out evaluation metrics\.*PerturbCellRL*remains competitive with the state\-of\-the\-art scDFM across these population\-level metrics, and in many cases improves upon scDFM\. This indicates that improved verifier rewards do not sacrifice distributional quality\.

#### Test\-Time Scaling\.

![Refer to caption](https://arxiv.org/html/2606.27752v1/imgs/test_time_scaling_progeny_predictor_reward_mean_combined.png)Figure 6:Test\-time scaling with the PROGENy pathway verifier\.Best\-of\-NNselection improves pathway reward at both the single\-cell and population levels\.Figure[6](https://arxiv.org/html/2606.27752#S5.F6)shows that verifier\-guided best\-of\-NNselection produces a clear test\-time scaling trend\. As the number of candidate samples increases fromN=1N=1toN=8N=8, the PROGENy pathway reward increases monotonically at both evaluation levels\. At the single\-cell level, the reward rises from0\.0710\.071to0\.4030\.403\. At the population level, it rises from0\.1600\.160to0\.4560\.456\. The largest gain appears with only a small amount of extra inference compute, while largerNNvalues continue to improve the selected samples\. These results indicate that the pathway verifier can select generated responses whose predicted pathway changes better match the annotated perturbation direction, without retraining the generator\.

#### Visualization and case studies\.

To complement the scalar metrics, we visualize representative held\-out perturbations with target\-fitted UMAP projections\. This view directly compares whether predicted single\-cell populations occupy the same local manifold as the real target cells\. As shown in Figure[7](https://arxiv.org/html/2606.27752#S5.F7),*PerturbCellRL*better matches the geometry of the target cell distributions than scDFM\. Across representative single\- and double\-gene perturbations,*PerturbCellRL*predictions show tighter overlap with the target high\-density regions, whereas scDFM often spreads into displaced or peripheral areas of the UMAP space\. This suggests that verifier\-guided post\-training improves cell\-level perturbation consistency while preserving distributional alignment with the observed target populations\.

![Refer to caption](https://arxiv.org/html/2606.27752v1/imgs/image.png)Figure 7:Target\-fitted UMAP case studies on Norman holdout perturbations\.The left, middle, and right panels show cells from the same single\-gene perturbation, the same double\-gene perturbation, and single\-gene perturbations from the same pathway, respectively\. Blue, green, and orange densities denote real target cells, scDFM predictions, and*PerturbCellRL*predictions, respectively\.Table 1:Population\-level performance across Norman and ComboSciPlex settings\.Bold indicates best; underline indicates second best within each setting\. “–” indicates that the metric is not available for that setting\.SettingModelMAE↓\\downarrowDE\-Sp\. LFC Sig↑\\uparrowPearsonΔ^\\hat\{\\Delta\}↑\\uparrowPathway↑\\uparrowPearsonΔ\\Delta↑\\uparrowDS↑\\uparrowMMD↓\\downarrowEnergy↓\\downarrowHoldout SingleControl0\.0247–0\.26570\.0000–0\.52170\.26114\.6794GEARS0\.04660\.73560\.63560\.10020\.66460\.82710\.09794\.2827CPA0\.03770\.40820\.31540\.04880\.33360\.56160\.26734\.0323STATE0\.03400\.2969\-0\.01160\.10250\.36400\.51940\.08921\.1569CellFlow0\.02190\.75470\.45890\.06640\.54250\.56470\.05281\.4355scDFM0\.02030\.83650\.68490\.15640\.71830\.89190\.05810\.5641*PerturbCellRL*0\.01970\.84350\.70470\.16020\.73230\.89950\.05070\.5189Holdout DoubleControl0\.0414–\-0\.1412––0\.53330\.32246\.2272GEARS0\.07080\.80820\.6407–0\.75520\.87660\.11705\.3965CPA0\.05170\.35330\.2728–0\.48810\.61000\.29415\.1256STATE0\.04260\.24950\.0806–0\.48680\.53330\.10491\.6567CellFlow0\.03330\.83040\.3311–0\.71360\.56330\.06652\.2825scDFM0\.02510\.88470\.7433–0\.82790\.91220\.04550\.6396*PerturbCellRL*0\.02530\.88370\.7618–0\.83620\.92330\.04140\.6284AdditiveControl0\.0384–\-0\.1285––0\.51350\.32375\.9914Additive0\.02280\.69660\.8584–0\.90240\.96860\.20834\.5242GEARS0\.04000\.78240\.5755–0\.70810\.84820\.25825\.1080CPA0\.04370\.38440\.4000–0\.58250\.63390\.27704\.7087STATE0\.04080\.18060\.0926–0\.46680\.52670\.10261\.5350CellFlow0\.03030\.81250\.4618–0\.71680\.56740\.13042\.0537scDFM0\.02320\.90160\.8333–0\.88200\.97410\.04590\.5561*PerturbCellRL*0\.02380\.90190\.8512–0\.89420\.96820\.04070\.5507ComboSciPlexControl0\.0697–\-0\.3696––0\.57140\.20403\.1414GEARS0\.03890\.73490\.6383–0\.72210\.83670\.26434\.7818CPA0\.04410\.60940\.7415–0\.73720\.87760\.24643\.8947STATE0\.06710\.4112\-0\.3191–0\.35540\.57140\.18152\.7685CellFlow0\.02700\.85580\.7968–0\.84050\.81630\.13751\.4699scDFM0\.02420\.83580\.8419–0\.86810\.87760\.04610\.5170*PerturbCellRL*0\.02300\.84060\.8597–0\.88240\.87760\.03700\.4539

## 6Conclusion

We introduced*PerturbCellRL*, a verifier\-guided RL framework for aligning single\-cell perturbation generators with cell\-level biological checks\. Starting from a public scDFM checkpoint,*PerturbCellRL*optimizes four rewards: Pearson top\-kksimilarity, RMSE top\-kkproximity, DE Spearman, and Pathway activity\. Across Norman additive, Norman holdout, and ComboSciPlex settings,*PerturbCellRL*improves reward\-aligned metrics while remaining competitive on population\-level evaluation metrics\. The gains on held\-out DS, MMD, and Energy Distance suggest that post\-training does not simply exploit the optimized rewards\. At the same time, the framework depends on the quality and coverage of its verifiers\. Pathway activity is currently evaluated only where single\-gene pathway annotations are available, and broader annotations are needed for other settings\. Future work should expand reference\-free verifiers and validate high\-scoring predictions prospectively\. An exciting direction is therefore to collaborate with domain biologists to curate broader annotations, yielding more valuable rewards and extending verifier\-guided alignment to more biological settings\.

## Acknowledgement

This work was supported in part by ONR Grant N00014\-22\-1\-2110 and the Stanford Institute for Human\-Centered Artificial Intelligence \(HAI\)\. EBF, SY, EL are Biohub, San Francisco, Investigator\. E\.L\. and S\.Y\. were supported by the Stanford Institute for Human\-Centered AI\.

## References

- A\. K\. Adduri, D\. Gautam, B\. Bevilacqua, A\. Imran, R\. Shah, M\. Naghipourfar, N\. Teyssier, R\. Ilango, S\. Nagaraj, M\. Dong,et al\.\(2025\)Predicting cellular responses to perturbation across diverse contexts with state\.BioRxiv,pp\. 2025–06\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p3.1),[§5\.1](https://arxiv.org/html/2606.27752#S5.SS1.SSS0.Px2.p1.1)\.
- Modelling cellular perturbations with the sparse additive mechanism shift variational autoencoder\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 1–12\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p3.1)\.
- K\. Black, M\. Janner, Y\. Du, I\. Kostrikov, and S\. Levine \(2023\)Training diffusion models with reinforcement learning\.arXiv preprint arXiv:2305\.13301\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px2.p2.1)\.
- C\. Bunne, Y\. Roohani, Y\. Rosen, A\. Gupta, X\. Zhang, M\. Roed, T\. Alexandrov, M\. AlQuraishi, P\. Brennan, D\. B\. Burkhardt,et al\.\(2024\)How to build the virtual cell with artificial intelligence: priorities and opportunities\.Cell\.Cited by:[§1](https://arxiv.org/html/2606.27752#S1.p1.1)\.
- C\. Bunne, S\. G\. Stark, G\. Gut, J\. S\. Del Castillo, M\. Levesque, K\. Lehmann, L\. Pelkmans, A\. Krause, and G\. Rätsch \(2023\)Learning single\-cell perturbation responses using neural optimal transport\.Nature methods20\(11\),pp\. 1759–1768\.Cited by:[§1](https://arxiv.org/html/2606.27752#S1.p2.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.\(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px2.p3.2),[§4\.3](https://arxiv.org/html/2606.27752#S4.SS3.p1.1)\.
- Y\. Fan, O\. Watkins, Y\. Du, H\. Liu, M\. Ryu, C\. Boutilier, P\. Abbeel, M\. Ghavamzadeh, K\. Lee, and K\. Lee \(2023\)Dpok: reinforcement learning for fine\-tuning text\-to\-image diffusion models\.Advances in Neural Information Processing Systems36,pp\. 79858–79885\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px2.p2.1)\.
- G\. T\. Johnson, E\. Agmon, M\. Akamatsu, E\. Lundberg, B\. Lyons, W\. Ouyang, O\. A\. Quintero\-Carmona, M\. Riel\-Mehan, S\. Rafelski, and R\. Horwitz \(2023\)Building the next generation of virtual cells to understand cellular biology\.Biophysical Journal\.Cited by:[§1](https://arxiv.org/html/2606.27752#S1.p1.1)\.
- D\. Klein, J\. S\. Fleck, D\. Bobrovskiy, L\. Zimmermann, S\. Becker, A\. Palma, L\. Dony, A\. Tejada\-Lapuerta, G\. Huguet, H\. Lin,et al\.\(2025\)CellFlow enables generative single\-cell phenotype modeling with flow matching\.bioRxiv,pp\. 2025–04\.Cited by:[§1](https://arxiv.org/html/2606.27752#S1.p2.1),[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p3.1),[§5\.1](https://arxiv.org/html/2606.27752#S5.SS1.SSS0.Px2.p1.1)\.
- J\. Li, Y\. Cui, T\. Huang, Y\. Ma, C\. Fan, M\. Yang, and Z\. Zhong \(2025\)Mixgrpo: unlocking flow\-based grpo efficiency with mixed ode\-sde\.arXiv preprint arXiv:2507\.21802\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px2.p2.1)\.
- H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe \(2023\)Let’s verify step by step\.InThe twelfth international conference on learning representations,Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px2.p3.2),[§4\.3](https://arxiv.org/html/2606.27752#S4.SS3.p1.1)\.
- Y\. Lipman, R\. T\. Chen, H\. Ben\-Hamu, M\. Nickel, and M\. Le \(2023\)Flow matching for generative modeling\.InThe Eleventh International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p3.1)\.
- Y\. Lipman, M\. Havasi, P\. Holderrieth, N\. Shaul, M\. Le, B\. Karrer, R\. T\. Chen, D\. Lopez\-Paz, H\. Ben\-Hamu, and I\. Gat \(2024\)Flow matching guide and code\.arXiv preprint arXiv:2412\.06264\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p3.1)\.
- J\. Liu, G\. Liu, J\. Liang, Y\. Li, J\. Liu, X\. Wang, P\. Wan, D\. Zhang, and W\. Ouyang \(2025\)Flow\-grpo: training flow matching models via online rl\.arXiv preprint arXiv:2505\.05470\.Cited by:[§1](https://arxiv.org/html/2606.27752#S1.p4.2),[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px2.p2.1)\.
- X\. Liu, C\. Gong, and Q\. Liu \(2023\)Flow straight and fast: learning to generate and transfer data with rectified flow\.InICLR,Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p3.1)\.
- R\. Lopez, J\. Regier, M\. B\. Cole, M\. I\. Jordan, and N\. Yosef \(2018\)Deep generative modeling for single\-cell transcriptomics\.Nature methods15\(12\),pp\. 1053–1058\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p3.1)\.
- M\. Lotfollahi, A\. Klimovskaia Susmelj, C\. De Donno, L\. Hetzel, Y\. Ji, I\. L\. Ibarra, S\. R\. Srivatsan, M\. Naghipourfar, R\. M\. Daza, B\. Martin,et al\.\(2023\)Predicting cellular responses to complex perturbations in high\-throughput screens\.Molecular systems biology19\(6\),pp\. MSB202211517\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p3.1),[§5\.1](https://arxiv.org/html/2606.27752#S5.SS1.SSS0.Px2.p1.1)\.
- N\. Ma, S\. Tong, H\. Jia, H\. Hu, Y\. Su, M\. Zhang, X\. Yang, Y\. Li, T\. Jaakkola, X\. Jia,et al\.\(2025\)Inference\-time scaling for diffusion models beyond scaling denoising steps\.arXiv preprint arXiv:2501\.09732\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px2.p3.2)\.
- L\. Mathur, B\. Szalai, N\. Du, R\. Utharala, M\. Ballinger, J\. Landry, M\. Ryckelynck, V\. Benes, J\. Saez\-Rodriguez, and C\. A\. Merten \(2022\)Combi\-seq for multiplexed transcriptome\-based profiling of drug combinations using deterministic barcoding in single\-cell droplets\.Nature communications13\(1\),pp\. 4450\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p2.1),[§5\.1](https://arxiv.org/html/2606.27752#S5.SS1.SSS0.Px1.p1.1)\.
- T\. M\. Norman, M\. A\. Horlbeck, J\. M\. Replogle, A\. Y\. Ge, A\. Xu, M\. Jost, L\. A\. Gilbert, and J\. S\. Weissman \(2019\)Exploring genetic interaction manifolds constructed from rich single\-cell phenotypes\.Science365\(6455\),pp\. 786–793\.Cited by:[§1](https://arxiv.org/html/2606.27752#S1.p1.1),[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p2.1),[§5\.1](https://arxiv.org/html/2606.27752#S5.SS1.SSS0.Px1.p1.1)\.
- S\. Peidli, T\. D\. Green, C\. Shen, T\. Gross, J\. Min, S\. Garda, B\. Yuan, L\. J\. Schumacher, J\. P\. Taylor\-King, D\. S\. Marks,et al\.\(2024\)ScPerturb: harmonized single\-cell perturbation data\.Nature Methods21\(3\),pp\. 531–540\.Cited by:[Appendix C](https://arxiv.org/html/2606.27752#A3.p1.2),[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p2.1)\.
- L\. Rampášek, D\. Hidru, P\. Smirnov, B\. Haibe\-Kains, and A\. Goldenberg \(2019\)Dr\.vae: improving drug response prediction via modeling of drug perturbation effects\.Bioinformatics35\(19\),pp\. 3743–3751\.External Links:ISSN 1367\-4803,[Document](https://dx.doi.org/10.1093/bioinformatics/btz158)Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p3.1)\.
- J\. M\. Replogle, R\. A\. Saunders, A\. N\. Pogson, J\. A\. Hussmann, A\. Lenail, A\. Guna, L\. Mascibroda, E\. J\. Wagner, K\. Adelman, G\. Lithwick\-Yanai,et al\.\(2022\)Mapping information\-rich genotype\-phenotype landscapes with genome\-scale perturb\-seq\.Cell185\(14\),pp\. 2559–2575\.Cited by:[§1](https://arxiv.org/html/2606.27752#S1.p1.1)\.
- Y\. H\. Roohani, T\. J\. Hua, P\. Tung, L\. R\. Bounds, F\. B\. Yu, A\. Dobin, N\. Teyssier, A\. Adduri, A\. Woodrow, B\. S\. Plosky,et al\.\(2025\)Virtual cell challenge: toward a turing test for the virtual cell\.Cell188\(13\),pp\. 3370–3374\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p2.1)\.
- Y\. Roohani, K\. Huang, and J\. Leskovec \(2024\)Predicting transcriptional outcomes of novel multigene perturbations with gears\.Nature Biotechnology42\(6\),pp\. 927–935\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p3.1),[§5\.1](https://arxiv.org/html/2606.27752#S5.SS1.SSS0.Px2.p1.1)\.
- M\. Schubert, B\. Klinger, M\. Klünemann, A\. Sieber, F\. Uhlitz, S\. Sauer, M\. J\. Garnett, N\. Blüthgen, and J\. Saez\-Rodriguez \(2018\)Perturbation\-response genes reveal signaling footprints in cancer gene expression\.Nature communications9\(1\),pp\. 20\.Cited by:[§1](https://arxiv.org/html/2606.27752#S1.p3.1),[§4\.1](https://arxiv.org/html/2606.27752#S4.SS1.SSS0.Px4.p2.2)\.
- C\. Snell, J\. Lee, K\. Xu, and A\. Kumar \(2024\)Scaling llm test\-time compute optimally can be more effective than scaling model parameters\.arXiv preprint arXiv:2408\.03314\.Cited by:[§1](https://arxiv.org/html/2606.27752#S1.p5.1),[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px2.p3.2),[§4\.3](https://arxiv.org/html/2606.27752#S4.SS3.p1.1)\.
- R\. Viñas Torné, M\. Wiatrak, Z\. Piran, S\. Fan, L\. Jiang, S\. A\. Teichmann, M\. Nitzan, and M\. Brbić \(2025\)Systema: a framework for evaluating genetic perturbation response prediction beyond systematic variation\.Nature Biotechnology,pp\. 1–10\.Cited by:[§1](https://arxiv.org/html/2606.27752#S1.p3.1)\.
- D\. Wu, S\. Su, Y\. Zhang, E\. Sui, E\. Lundberg, E\. B\. Fox, and S\. Yeung\-Levy \(2026\)CellFluxRL: biologically\-constrained virtual cell modeling via reinforcement learning\.arXiv preprint arXiv:2603\.21743\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px2.p2.1)\.
- Z\. Xue, J\. Wu, Y\. Gao, F\. Kong, L\. Zhu, M\. Chen, Z\. Liu, W\. Liu, Q\. Guo, W\. Huang,et al\.\(2025\)Dancegrpo: unleashing grpo on visual generation\.arXiv preprint arXiv:2505\.07818\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px2.p2.1)\.
- C\. Yu, C\. Wang, B\. Liao, and T\. Wu \(2026\)Scdfm: distributional flow matching model for robust single\-cell perturbation prediction\.arXiv preprint arXiv:2602\.07103\.Cited by:[§1](https://arxiv.org/html/2606.27752#S1.p2.1),[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p3.1),[§3](https://arxiv.org/html/2606.27752#S3.SS0.SSS0.Px1.p1.5),[§5\.1](https://arxiv.org/html/2606.27752#S5.SS1.SSS0.Px1.p1.1),[§5\.1](https://arxiv.org/html/2606.27752#S5.SS1.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2606.27752#S5.SS1.SSS0.Px4.p1.7)\.
- Y\. Zhang, Y\. Su, C\. Wang, T\. Li, Z\. Wefers, J\. Nirschl, J\. Burgess, D\. Ding, A\. Lozano, E\. Lundberg,et al\.\(2025\)Cellflux: simulating cellular morphology changes via flow matching\.arXiv preprint arXiv:2502\.09775\.Cited by:[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px1.p3.1)\.
- K\. Zheng, H\. Chen, H\. Ye, H\. Wang, Q\. Zhang, K\. Jiang, H\. Su, S\. Ermon, J\. Zhu, and M\. Liu \(2025\)Diffusionnft: online diffusion reinforcement with forward process\.arXiv preprint arXiv:2509\.16117\.Cited by:[§1](https://arxiv.org/html/2606.27752#S1.p4.2),[§2](https://arxiv.org/html/2606.27752#S2.SS0.SSS0.Px2.p2.1),[Figure 3](https://arxiv.org/html/2606.27752#S4.F3),[§4\.2](https://arxiv.org/html/2606.27752#S4.SS2.SSS0.Px2.p1.3),[§4\.2](https://arxiv.org/html/2606.27752#S4.SS2.SSS0.Px3.p1.9)\.

## Appendix AAlgorithm Details

For Eq\. \([12](https://arxiv.org/html/2606.27752#S4.E12)\),β\\betais the KL divergence weight,x0∼𝒩​\(0,I\)x\_\{0\}\\sim\\mathcal\{N\}\(0,I\)is the Gaussian start,xt=\(1−t\)​x0\+t​yix\_\{t\}=\(1\-t\)x\_\{0\}\+ty\_\{i\}is the forward\-interpolated intermediate state,v=yi−x0v=y\_\{i\}\-x\_\{0\}is the corresponding velocity target, andvθ\+v\_\{\\theta\}^\{\+\},vθ−v\_\{\\theta\}^\{\-\}areimplicitpositive and negative policies defined as:

vθ\+​\(xt,ui,ci,t\)\\displaystyle v\_\{\\theta\}^\{\+\}\(x\_\{t\},u\_\{i\},c\_\{i\},t\):=\(1−γ\)​vold​\(xt,ui,ci,t\)\+γ​vθ​\(xt,ui,ci,t\),\\displaystyle:=\(1\-\\gamma\)\\,v^\{\\mathrm\{old\}\}\(x\_\{t\},u\_\{i\},c\_\{i\},t\)\+\\gamma\\,v\_\{\\theta\}\(x\_\{t\},u\_\{i\},c\_\{i\},t\),\(14\)vθ−​\(xt,ui,ci,t\)\\displaystyle v\_\{\\theta\}^\{\-\}\(x\_\{t\},u\_\{i\},c\_\{i\},t\):=\(1\+γ\)​vold​\(xt,ui,ci,t\)−γ​vθ​\(xt,ui,ci,t\)\.\\displaystyle:=\(1\+\\gamma\)\\,v^\{\\mathrm\{old\}\}\(x\_\{t\},u\_\{i\},c\_\{i\},t\)\-\\gamma\\,v\_\{\\theta\}\(x\_\{t\},u\_\{i\},c\_\{i\},t\)\.\(15\)Herevoldv^\{\\mathrm\{old\}\}is the data\-collection policy, a lagging copy ofvθv\_\{\\theta\}, andγ\>0\\gamma\>0controls guidance strength\. The implicit parameterization is central to the algorithm: rather than training separate positive and negative models, a single policyvθv\_\{\\theta\}is optimized such that its mixture withvoldv^\{\\mathrm\{old\}\}simultaneously fits high\-reward cells \(viavθ\+v\_\{\\theta\}^\{\+\}\) and avoids low\-reward ones \(viavθ−v\_\{\\theta\}^\{\-\}\)\. The optimal solution satisfiesvθ∗=vold\+2γ​Δv\_\{\\theta^\{\*\}\}=v^\{\\mathrm\{old\}\}\+\\tfrac\{2\}\{\\gamma\}\\Delta, whereΔ\\Deltais the reinforcement guidance direction pointing from the negative towards the positive distribution\. This formulation naturally regularizes the post\-trained model towards the pretrained policy: whenγ\\gammais large, the guidance strength2γ\\tfrac\{2\}\{\\gamma\}is small and the model stays close tovoldv^\{\\mathrm\{old\}\}; whenγ\\gammais small, the model may deviate more aggressively\. The data\-collection policyvoldv^\{\\mathrm\{old\}\}is updated via an exponential moving average ofvθv\_\{\\theta\}\.

Algorithm 1*PerturbCellRL*: Verifier\-Guided RL for scDFM1:Pretrained scDFM velocity

vθrefv\_\{\\theta\}^\{\\mathrm\{ref\}\}; reward weights

\{λm\}\\\{\\lambda\_\{m\}\\\}; perturbation dataset

𝒟\\mathcal\{D\}; group size

mm; guidance

γ\\gamma; KL weight

β\\beta
2:Initialize

vθ←vθrefv\_\{\\theta\}\\leftarrow v\_\{\\theta\}^\{\\mathrm\{ref\}\}and data\-collection policy

vold←vθrefv^\{\\mathrm\{old\}\}\\leftarrow v\_\{\\theta\}^\{\\mathrm\{ref\}\}
3:foreach RL iterationdo

4:foreach sampled

\(ui,ci\)∼𝒟\(u\_\{i\},c\_\{i\}\)\\sim\\mathcal\{D\}do

5:Draw Gaussian starts

\{x0\(j\)\}j=1m\\\{x\_\{0\}^\{\(j\)\}\\\}\_\{j=1\}^\{m\}
6:For each

jj, sample

yi\(j\)y\_\{i\}^\{\(j\)\}from

vold\(⋅∣x0\(j\),ui,ci\)v^\{\\mathrm\{old\}\}\(\\cdot\\mid x\_\{0\}^\{\(j\)\},u\_\{i\},c\_\{i\}\)
7:Score each candidate with the four reward functions and compute

r¯\(j\)\\bar\{r\}^\{\(j\)\}via Eq\. \([11](https://arxiv.org/html/2606.27752#S4.E11)\)

8:Normalize

r¯\(j\)\\bar\{r\}^\{\(j\)\}within the group to obtain optimality probabilities

r\(j\)∈\[0,1\]r^\{\(j\)\}\\in\[0,1\]
9:Compute forward interpolation

xt\(j\)=\(1−t\)​x0\(j\)\+t​yi\(j\)x\_\{t\}^\{\(j\)\}=\(1\-t\)x\_\{0\}^\{\(j\)\}\+ty\_\{i\}^\{\(j\)\}with velocity

v\(j\)=yi\(j\)−x0\(j\)v^\{\(j\)\}=y\_\{i\}^\{\(j\)\}\-x\_\{0\}^\{\(j\)\}
10:Compute implicit policies

vθ\+v\_\{\\theta\}^\{\+\},

vθ−v\_\{\\theta\}^\{\-\}via Eqs\. \([14](https://arxiv.org/html/2606.27752#A1.E14)\)–\([15](https://arxiv.org/html/2606.27752#A1.E15)\)

11:endfor

12:Update

vθv\_\{\\theta\}with the NFT loss in Eq\. \([12](https://arxiv.org/html/2606.27752#S4.E12)\)

13:Update

voldv^\{\\mathrm\{old\}\}via exponential moving average of

vθv\_\{\\theta\}
14:endfor

15:returnPost\-trained generator

vθv\_\{\\theta\}

## Appendix BVerifier Implementations

This appendix gives the expanded mathematical definitions for the verifier rewards used in the main text\. For top\-kkrewards, we usek=10k=10\.

#### Pearson top\-kksimilarity reward\.

For generated sampleii, define the centered generated expression and centered real target expression:

Δ^i=yi−μ,Δci,j=yci,jobs−μ\.\\widehat\{\\Delta\}\_\{i\}=y\_\{i\}\-\\mu,\\qquad\\Delta\_\{c\_\{i\},j\}=y^\{\\mathrm\{obs\}\}\_\{c\_\{i\},j\}\-\\mu\.\(16\)Let𝒫i\\mathcal\{P\}\_\{i\}be the top\-kkreal target cells from conditioncic\_\{i\}ranked by decreasingρ​\(Δ^i,Δci,j\)\\rho\(\\widehat\{\\Delta\}\_\{i\},\\Delta\_\{c\_\{i\},j\}\)\. The reward averages these nearest target similarities:

ripearson=1\|𝒫i\|​∑j∈𝒫iρ​\(Δ^i,Δci,j\),ripearson∈\[−1,1\]\.r\_\{i\}^\{\\mathrm\{pearson\}\}=\\frac\{1\}\{\|\\mathcal\{P\}\_\{i\}\|\}\\sum\_\{j\\in\\mathcal\{P\}\_\{i\}\}\\rho\(\\widehat\{\\Delta\}\_\{i\},\\Delta\_\{c\_\{i\},j\}\),\\qquad r\_\{i\}^\{\\mathrm\{pearson\}\}\\in\[\-1,1\]\.\(17\)

#### RMSE top\-kkproximity reward\.

For expressionsa,b∈ℝGa,b\\in\\mathbb\{R\}^\{G\}, define

RMSE⁡\(a,b\)=1G​∑g∈𝒢\(ag−bg\)2\.\\operatorname\{RMSE\}\(a,b\)=\\sqrt\{\\frac\{1\}\{G\}\\sum\_\{g\\in\\mathcal\{G\}\}\(a\_\{g\}\-b\_\{g\}\)^\{2\}\}\.\(18\)Let𝒩i\\mathcal\{N\}\_\{i\}be the top\-kkreal target cells from conditioncic\_\{i\}ranked by increasingRMSE⁡\(yi,yci,jobs\)\\operatorname\{RMSE\}\(y\_\{i\},y^\{\\mathrm\{obs\}\}\_\{c\_\{i\},j\}\)\. The generated top\-kkdistance is

dirmse​\-​topk=1\|𝒩i\|​∑j∈𝒩iRMSE⁡\(yi,yci,jobs\)\.d\_\{i\}^\{\\mathrm\{rmse\}\\text\{\-\}\\mathrm\{topk\}\}=\\frac\{1\}\{\|\\mathcal\{N\}\_\{i\}\|\}\\sum\_\{j\\in\\mathcal\{N\}\_\{i\}\}\\operatorname\{RMSE\}\(y\_\{i\},y^\{\\mathrm\{obs\}\}\_\{c\_\{i\},j\}\)\.\(19\)For each real target cellyc,jobsy^\{\\mathrm\{obs\}\}\_\{c,j\}, let𝒩c,j−j\\mathcal\{N\}\_\{c,j\}^\{\-j\}be its top\-kknearest neighbors among other real target cells from the same condition\. The condition\-specific upper bound is

Ucrmse​\-​topk=maxj⁡1\|𝒩c,j−j\|​∑ℓ∈𝒩c,j−jRMSE⁡\(yc,jobs,yc,ℓobs\)\.U\_\{c\}^\{\\mathrm\{rmse\}\\text\{\-\}\\mathrm\{topk\}\}=\\max\_\{j\}\\frac\{1\}\{\|\\mathcal\{N\}\_\{c,j\}^\{\-j\}\|\}\\sum\_\{\\ell\\in\\mathcal\{N\}\_\{c,j\}^\{\-j\}\}\\operatorname\{RMSE\}\(y^\{\\mathrm\{obs\}\}\_\{c,j\},y^\{\\mathrm\{obs\}\}\_\{c,\\ell\}\)\.\(20\)The reward maps distance to proximity:

rirmse​\-​topk=1−dirmse​\-​topkUcirmse​\-​topk,rirmse​\-​topk∈\[0,1\]\.r\_\{i\}^\{\\mathrm\{rmse\}\\text\{\-\}\\mathrm\{topk\}\}=1\-\\frac\{d\_\{i\}^\{\\mathrm\{rmse\}\\text\{\-\}\\mathrm\{topk\}\}\}\{U\_\{c\_\{i\}\}^\{\\mathrm\{rmse\}\\text\{\-\}\\mathrm\{topk\}\}\},\\qquad r\_\{i\}^\{\\mathrm\{rmse\}\\text\{\-\}\\mathrm\{topk\}\}\\in\[0,1\]\.\(21\)

#### DE Spearman reward\.

For conditioncic\_\{i\}, let𝒟ci\\mathcal\{D\}\_\{c\_\{i\}\}be the significant DE gene set:

𝒟ci=\{g∈𝒢:FDRci,g≤α\}\.\\mathcal\{D\}\_\{c\_\{i\}\}=\\\{g\\in\\mathcal\{G\}:\\mathrm\{FDR\}\_\{c\_\{i\},g\}\\leq\\alpha\\\}\.\(22\)The generated fold change is computed in linear space:

F^i,g=expm1⁡\(yi,g\)\+ϵexpm1⁡\(ui,g\)\+ϵ,g∈𝒟ci\.\\widehat\{F\}\_\{i,g\}=\\frac\{\\operatorname\{expm1\}\(y\_\{i,g\}\)\+\\epsilon\}\{\\operatorname\{expm1\}\(u\_\{i,g\}\)\+\\epsilon\},\\qquad g\\in\\mathcal\{D\}\_\{c\_\{i\}\}\.\(23\)The real fold change uses target and reference means:

Fci,g=Tci,g\+ϵRci,g\+ϵ,g∈𝒟ci\.F\_\{c\_\{i\},g\}=\\frac\{T\_\{c\_\{i\},g\}\+\\epsilon\}\{R\_\{c\_\{i\},g\}\+\\epsilon\},\\qquad g\\in\\mathcal\{D\}\_\{c\_\{i\}\}\.\(24\)The reward is Pearson correlation after rank transformation:

rispearman=ρ​\(rank⁡\(F^i,𝒟ci\),rank⁡\(Fci,𝒟ci\)\),rispearman∈\[−1,1\]\.r\_\{i\}^\{\\mathrm\{spearman\}\}=\\rho\(\\operatorname\{rank\}\(\\widehat\{F\}\_\{i,\\mathcal\{D\}\_\{c\_\{i\}\}\}\),\\operatorname\{rank\}\(F\_\{c\_\{i\},\\mathcal\{D\}\_\{c\_\{i\}\}\}\)\),\\qquad r\_\{i\}^\{\\mathrm\{spearman\}\}\\in\[\-1,1\]\.\(25\)

#### Pathway activity reward\.

Pathway annotations, confidence weights, PROGENy scoring, and unannotated perturbations are described in Appendix[C](https://arxiv.org/html/2606.27752#A3)and Appendix[D](https://arxiv.org/html/2606.27752#A4)\.

## Appendix CPROGENy Predictor MLP

We train a small fold/split\-specific MLP to directly predict 14 PROGENy pathway scores fromK=1000K\{=\}1000observed genes\. Training data is the scPerturb NormanWeissman2019 dataset\[[21](https://arxiv.org/html/2606.27752#bib.bib52)\]\(111,445 cells, 33,694 genes\), normalized to10410^\{4\}counts per cell and log1p\-transformed\. Ground\-truth targets are PROGENy scores computed on the full 33K expression using L2\-normalized PROGENy weights\.

Each MLP mapsℝ1000→ℝ14\\mathbb\{R\}^\{1000\}\\to\\mathbb\{R\}^\{14\}with hidden layers\[512,256,128\]\[512,256,128\], LayerNorm, ReLU, and Dropout\(p=0\.1p\{=\}0\.1\) \(∼\\sim680K parameters\)\. Models are trained with MSE loss, Adam \(lr=10−3\\,\{=\}10^\{\-3\}\), cosine annealing, and early stopping \(patience 5\)\. Eight models are trained in total, one per fold/split combination: folds\{0,1,2,3\}×\{train, test\}\\\{0,1,2,3\\\}\\times\\\{\\text\{train, test\}\\\}, each using its specific 1K gene set as input\.

Mean Pearson correlation between predicted and ground\-truth pathway scores across held\-out cells is0\.51±0\.010\.51\\pm 0\.01across all 8 configurations\. In a representative fold, strongest performance is on pathways annotated to target genes: TGFb \(r=0\.90r\{=\}0\.90\), JAK\-STAT \(r=0\.74r\{=\}0\.74\), and MAPK \(r=0\.68r\{=\}0\.68\)\.

## Appendix DPathway Annotation Table

We construct a tiered annotation table mapping each Norman perturbation gene to a PROGENy pathway, directiond​\(h\)∈\{\+1,−1\}d\(h\)\\in\\\{\+1,\-1\\\}, and confidence weightw​\(h\)w\(h\), covering 62/101 Norman genes\.

Annotations are assigned through literature curation \(High/Medium confidence\) and empirical validation using PROGENy delta scores on the full\-transcriptome Norman data\. Data\-derived annotations are accepted where\|δ\|\>0\.05\|\\delta\|\{\>\}0\.05and the top pathway is\>1\.5×\>1\.5\\timesthe second\-ranked\. Confidence weights follow: High/Medium=1\.0=1\.0, Data\-derived=0\.8=0\.8, Low=0\.5=0\.5, Ultra\-low=0\.2=0\.2\.

Rank\-1 agreement between literature annotations and data\-derived top pathway was 37% on held\-out canonical genes, reflecting genuine biological complexity in K562 CRISPRa rather than annotation error\.

The complete tiered annotation is reported in TableLABEL:tab:norman\_tiered\_annotation\. Rows with no pathway annotation are excluded from pathway\-specific analyses\.

Table 2:Norman tiered pathway annotation table\.GeneTierPathwayDir\.ConfidenceSourceIRF11JAK\-STATUpHighLiterature \+ DataFOXA12AndrogenUpHighLiterature onlyHK22HypoxiaUpHighLiterature onlyDUSP92MAPKDownHighLiterature onlyEGR12MAPKUpHighLiterature onlyETS22MAPKUpHighLiterature onlyFOSB2MAPKUpHighLiterature onlyJUN2MAPKUpHighLiterature onlyMAP2K32MAPKUpHighLiterature onlyMAP2K62MAPKUpHighLiterature onlyMAPK12MAPKUpHighLiterature onlyPTPN122MAPKDownHighLiterature onlySPI12NFkBUpHighLiterature onlyCBL2PI3KDownHighLiterature onlyFOXO42PI3KDownHighLiterature onlySGK12PI3KUpHighLiterature onlyCEBPB2TGFbUpHighLiterature \+ DataCOL1A12TGFbUpHighLiterature onlySNAI12TGFbUpHighLiterature onlyTGFBR22TGFbUpHighLiterature onlyBAK12TrailUpHighLiterature onlyBCL2L112TrailUpHighLiterature onlyCDKN1A2p53UpHighLiterature onlyTP732p53UpHighLiterature onlyAHR2JAK\-STATUpMediumLiterature \+ DataMAP4K32MAPKUpMediumLiterature onlyMAP4K52MAPKUpMediumLiterature onlyCEBPA2NFkBUpMediumLiterature onlyCEBPE2NFkBUpMediumLiterature onlyLYL12NFkBUpMediumLiterature onlyPTPN12PI3KUpMediumLiterature onlyPTPN132PI3KDownMediumLiterature onlyPTPN92PI3KDownMediumLiterature onlyCOL2A12TGFbUpMediumLiterature onlyFOXA32TGFbUpMediumLiterature onlyFOXF12TGFbUpMediumLiterature onlyKLF12TGFbUpMediumLiterature onlyRUNX1T12TGFbUpMediumLiterature onlyTBX22TGFbUpMediumLiterature onlyTBX32TGFbUpMediumLiterature onlyHES72WNTUpMediumLiterature onlyMAML22WNTUpMediumLiterature onlyCDKN1B2p53UpMediumLiterature onlyCDKN1C2p53UpMediumLiterature onlyCKS1B2p53DownMediumLiterature \+ DataKMT2A2p53UpMediumLiterature onlySET3HypoxiaDownData\-derivedData onlySLC4A13HypoxiaDownData\-derivedData onlyIER5L3MAPKDownData\-derivedData onlyMEIS13MAPKDownData\-derivedData onlyS1PR23TGFbUpData\-derivedData onlyBPGM3HypoxiaDownLowData only \(Low confidence\)HOXB93HypoxiaDownLowData only \(Low confidence\)IGDCC33HypoxiaDownLowData only \(Low confidence\)ZC3HAV13JAK\-STATUpLowData only \(Low confidence\)HOXC133MAPKDownLowData only \(Low confidence\)SAMD13MAPKDownLowData only \(Low confidence\)UBASH3A4HypoxiaDownUltra\-lowData only \(Ultra\-low confidence\)UBASH3B4HypoxiaDownUltra\-lowData only \(Ultra\-low confidence\)CNN14MAPKDownUltra\-lowData only \(Ultra\-low confidence\)ISL24MAPKDownUltra\-lowData only \(Ultra\-low confidence\)ZBTB254MAPKDownUltra\-lowData only \(Ultra\-low confidence\)ARID1A4–––UnannotatableARRDC34–––UnannotatableATL14–––UnannotatableBCORL14–––UnannotatableCBFA2T34–––UnannotatableCELF24–––UnannotatableCITED14–––UnannotatableCLDN64–––UnannotatableCNNM44–––UnannotatableCSRNP14–––UnannotatableDLX24–––UnannotatableFEV4–––UnannotatableFOXL24–––UnannotatableGLB1L24–––UnannotatableHNF4A4–––UnannotatableHOXA134–––UnannotatableIKZF34–––UnannotatableKIF18B4–––UnannotatableKIF2C4–––UnannotatableLHX14–––UnannotatableMAP7D14–––UnannotatableMIDN4–––UnannotatableNCL4–––UnannotatableNIT14–––UnannotatableOSR24–––UnannotatablePLK44–––UnannotatablePOU3F24–––UnannotatablePRDM14–––UnannotatablePRTG4–––UnannotatableRHOXF24–––UnannotatableRREB14–––UnannotatableSLC38A24–––UnannotatableSLC6A94–––UnannotatableSTIL4–––UnannotatableTMSB4X4–––UnannotatableTSC22D14–––UnannotatableZBTB14–––UnannotatableZBTB104–––UnannotatableZNF3184–––Unannotatable
## Appendix EDataset Details

We use the Norman and ComboSciPlex splits from scDFM\. For each split, training conditions are all benchmark conditions not listed as test conditions\.

#### Norman additive split\.

Each additive fold holds out 37 double\-gene perturbations\. All corresponding single\-gene perturbations remain in training\. Each fold therefore has 189 train conditions\. TableLABEL:tab:norman\_additive\_splitlists test conditions\.

Table 3:Norman additive held\-out double\-gene conditions\.FoldTest conditions0AHR\+FEV, BPGM\+SAMD1, CBL\+UBASH3A, CBL\+UBASH3B, CDKN1B\+CDKN1A, CEBPB\+CEBPA, CEBPB\+PTPN12, CEBPE\+CEBPA, CEBPE\+KLF1, CEBPE\+RUNX1T1, CNN1\+MAPK1, CNN1\+UBASH3A, DUSP9\+MAPK1, ETS2\+IGDCC3, ETS2\+PRTG, FEV\+ISL2, FOSB\+CEBPB, FOSB\+CEBPE, FOSB\+OSR2, FOXA3\+FOXA1, FOXA3\+HOXB9, IRF1\+SET, KIF18B\+KIF2C, KLF1\+MAP2K6, LYL1\+IER5L, MAP2K3\+IKZF3, MAP2K3\+MAP2K6, PTPN12\+OSR2, PTPN12\+SNAI1, PTPN12\+ZBTB25, SAMD1\+PTPN12, SET\+KLF1, TGFBR2\+ETS2, UBASH3B\+CNN1, UBASH3B\+PTPN12, UBASH3B\+UBASH3A, ZC3HAV1\+HOXC131AHR\+FEV, AHR\+KLF1, BCL2L11\+BAK1, BPGM\+ZBTB1, CBL\+PTPN9, CBL\+TGFBR2, CBL\+UBASH3A, CBL\+UBASH3B, CDKN1B\+CDKN1A, CDKN1C\+CDKN1A, CEBPB\+CEBPA, CEBPE\+CEBPB, CEBPE\+KLF1, DUSP9\+SNAI1, ETS2\+IKZF3, ETS2\+MAP7D1, FOSB\+CEBPE, FOXA3\+FOXF1, FOXA3\+FOXL2, FOXA3\+HOXB9, IGDCC3\+PRTG, KIF18B\+KIF2C, KLF1\+CEBPA, KLF1\+CLDN6, MAP2K3\+IKZF3, MAP2K3\+MAP2K6, MAP2K6\+SPI1, MAPK1\+PRTG, PTPN12\+PTPN9, SAMD1\+UBASH3B, SET\+CEBPE, SGK1\+S1PR2, SGK1\+TBX2, TGFBR2\+IGDCC3, UBASH3B\+CNN1, UBASH3B\+OSR2, ZC3HAV1\+CEBPE2BPGM\+ZBTB1, CBL\+CNN1, CBL\+PTPN12, CBL\+TGFBR2, CBL\+UBASH3B, CEBPE\+CEBPA, CEBPE\+RUNX1T1, CNN1\+MAPK1, CNN1\+UBASH3A, DUSP9\+KLF1, DUSP9\+SNAI1, ETS2\+MAP7D1, ETS2\+MAPK1, FEV\+CBFA2T3, FOSB\+IKZF3, FOSB\+OSR2, FOSB\+PTPN12, FOXA1\+HOXB9, FOXL2\+MEIS1, IGDCC3\+PRTG, JUN\+CEBPA, KLF1\+CEBPA, LYL1\+IER5L, MAP2K3\+IKZF3, MAP2K3\+MAP2K6, MAP2K6\+IKZF3, MAP2K6\+SPI1, MAPK1\+IKZF3, PTPN12\+PTPN9, SAMD1\+UBASH3B, TGFBR2\+ETS2, UBASH3B\+CNN1, UBASH3B\+PTPN9, UBASH3B\+UBASH3A, UBASH3B\+ZBTB25, ZC3HAV1\+CEBPE, ZNF318\+FOXL23AHR\+FEV, AHR\+KLF1, CBL\+UBASH3A, CDKN1C\+CDKN1A, CEBPE\+CNN1, CEBPE\+SPI1, CNN1\+MAPK1, DUSP9\+ETS2, DUSP9\+SNAI1, ETS2\+MAP7D1, ETS2\+PRTG, FEV\+CBFA2T3, FEV\+MAP7D1, FOSB\+CEBPB, FOSB\+CEBPE, FOSB\+OSR2, FOSB\+PTPN12, FOXA1\+FOXL2, IGDCC3\+PRTG, IRF1\+SET, KIF18B\+KIF2C, KLF1\+CLDN6, LYL1\+IER5L, MAPK1\+IKZF3, MAPK1\+PRTG, PTPN12\+SNAI1, PTPN12\+UBASH3A, PTPN12\+ZBTB25, SAMD1\+UBASH3B, SAMD1\+ZBTB1, SET\+CEBPE, SGK1\+S1PR2, SGK1\+TBX3, TGFBR2\+ETS2, UBASH3B\+ZBTB25, ZBTB10\+DLX2, ZBTB10\+SNAI1
#### Norman holdout split\.

Each holdout fold contains 15 held\-out double\-gene conditions\. Their constituent single\-gene perturbations are also held out\. The train condition counts are 188, 185, 190, and 188\. TableLABEL:tab:norman\_holdout\_splitlists test conditions\.

Table 4:Norman holdout held\-out conditions\.FoldSingle\-gene test conditionsDouble\-gene test conditions0BPGM, CEBPA, CEBPB, CEBPE, CNN1, ETS2, FEV, FOSB, FOXA1, FOXA3, HOXB9, HOXC13, IER5L, IGDCC3, ISL2, LYL1, PTPN12, SAMD1, SNAI1, UBASH3A, UBASH3B, ZBTB25, ZC3HAV1BPGM\+SAMD1, CEBPB\+PTPN12, CEBPE\+CEBPA, CNN1\+UBASH3A, ETS2\+IGDCC3, FEV\+ISL2, FOSB\+CEBPB, FOXA3\+FOXA1, FOXA3\+HOXB9, LYL1\+IER5L, PTPN12\+SNAI1, PTPN12\+ZBTB25, SAMD1\+PTPN12, UBASH3B\+UBASH3A, ZC3HAV1\+HOXC131BAK1, BCL2L11, CBL, CEBPA, CEBPB, CEBPE, CLDN6, ETS2, FOSB, IGDCC3, IKZF3, KIF18B, KIF2C, KLF1, MAP2K3, MAP2K6, MAP7D1, PRTG, S1PR2, SAMD1, SGK1, SPI1, TBX2, TGFBR2, UBASH3A, UBASH3BBCL2L11\+BAK1, CBL\+UBASH3A, CEBPB\+CEBPA, ETS2\+MAP7D1, FOSB\+CEBPE, IGDCC3\+PRTG, KIF18B\+KIF2C, KLF1\+CEBPA, KLF1\+CLDN6, MAP2K3\+IKZF3, MAP2K6\+SPI1, SAMD1\+UBASH3B, SGK1\+S1PR2, SGK1\+TBX2, TGFBR2\+IGDCC32CBL, CEBPA, CEBPE, CNN1, DUSP9, FOSB, FOXA1, HOXB9, IER5L, JUN, KLF1, LYL1, MAP2K3, MAP2K6, MAPK1, PTPN12, RUNX1T1, SNAI1, SPI1, TGFBR2, UBASH3BCBL\+CNN1, CBL\+PTPN12, CBL\+TGFBR2, CEBPE\+CEBPA, CEBPE\+RUNX1T1, CNN1\+MAPK1, DUSP9\+SNAI1, FOSB\+PTPN12, FOXA1\+HOXB9, JUN\+CEBPA, KLF1\+CEBPA, LYL1\+IER5L, MAP2K3\+MAP2K6, MAP2K6\+SPI1, UBASH3B\+CNN13AHR, CEBPE, CLDN6, CNN1, DLX2, ETS2, FEV, IER5L, IGDCC3, IKZF3, KLF1, LYL1, MAP7D1, MAPK1, PRTG, PTPN12, S1PR2, SGK1, SNAI1, SPI1, TBX3, UBASH3A, ZBTB10AHR\+FEV, CEBPE\+CNN1, CEBPE\+SPI1, CNN1\+MAPK1, ETS2\+PRTG, FEV\+MAP7D1, IGDCC3\+PRTG, KLF1\+CLDN6, LYL1\+IER5L, MAPK1\+IKZF3, PTPN12\+UBASH3A, SGK1\+S1PR2, SGK1\+TBX3, ZBTB10\+DLX2, ZBTB10\+SNAI1
#### ComboSciPlex split\.

ComboSciPlex uses the default scDFM split\. The seven held\-out conditions are listed in Table[5](https://arxiv.org/html/2606.27752#A5.T5)\. All other ComboSciPlex conditions are used for training\. This gives 25 train conditions\.

Table 5:ComboSciPlex held\-out test conditions\.Condition 1Condition 2PanobinostatCrizotinibPanobinostatCurcuminPanobinostatSRT1720PanobinostatSorafenibSRT2104AlvespimycincontrolAlvespimycincontrolDacinostat

## Appendix FAblation Study

We ablate the PROGENy predictor reward on the Norman holdout setting\. The full setting uses the same*PerturbCellRL*reward set reported in Table[1](https://arxiv.org/html/2606.27752#S5.T1)\. The ablated setting removes only the PROGENy predictor reward and keeps the Pearson top\-kk, DE Spearman, and RMSE top\-kkrewards\. Table[6](https://arxiv.org/html/2606.27752#A6.T6)reports population\-level mean metrics over four holdout folds\. Removing the PROGENy predictor reward leaves most holdout population metrics close to the full reward setting, while the holdout single pathway metric decreases from0\.16020\.1602to0\.15090\.1509\. This suggests that the pathway reward contributes directly to pathway\-aligned population behavior without substantially changing the other reported holdout metrics\.

Table 6:PROGENy predictor reward ablation on Norman holdout\.Each value is averaged over four holdout folds\. The MMD column follows Table[1](https://arxiv.org/html/2606.27752#S5.T1)and reports thes=0\.5s=0\.5RBF\-kernel MMD\. “–” indicates that the metric is not available for that setting\.SettingRewardMAE↓\\downarrowDE\-Sp\. LFC Sig↑\\uparrowPearsonΔ^\\hat\{\\Delta\}↑\\uparrowPathway↑\\uparrowPearsonΔ\\Delta↑\\uparrowDS↑\\uparrowMMD↓\\downarrowEnergy↓\\downarrowHoldout Singlew/ PROGENy predictor0\.01970\.84350\.70470\.16020\.73230\.89950\.05070\.5189w/o PROGENy predictor0\.01950\.83940\.71190\.15090\.73350\.89430\.04830\.5092Holdout Doublew/ PROGENy predictor0\.02530\.88370\.7618–0\.83620\.92330\.04140\.6284w/o PROGENy predictor0\.02530\.87830\.7610–0\.83590\.91670\.04120\.6295

### F\.1Test\-Time Scaling

We study test\-time scaling by varying the number of generated samples per conditionn∈\{1,2,4,8\}n\\in\\\{1,2,4,8\\\}and aggregating predictions across samples\. Table[7](https://arxiv.org/html/2606.27752#A6.T7)reports population\-level metrics on Norman holdout single\-gene perturbations, averaged over four holdout folds\. All reported metrics are population\-level\. At eachnn, we drawnncandidate populations per condition and keep the one with the highest pathway reward, so the pathway metric improves by construction \(from0\.1600\.160atn=1n\{=\}1to0\.4560\.456atn=8n\{=\}8\)\. The other population\-level metrics \(MAE, PearsonΔ\\Delta, MMD, Energy\) are off\-target for this selection rule: picking the pathway\-maximizing candidate biases the retained population toward strong pathway activity rather than toward matching the reference population’s per\-gene means and cell\-to\-cell spread, so these scores drift away from the reference asnngrows\.

Table 7:Test\-time scaling population\-level metrics on Norman holdout single\-gene perturbations\.Each value is averaged over four holdout folds\. The MMD column follows Table[1](https://arxiv.org/html/2606.27752#S5.T1)and reports thes=0\.5s=0\.5RBF\-kernel MMD\.\# SamplesMAE↓\\downarrowDE\-Sp\. LFC Sig↑\\uparrowPearsonΔ^\\hat\{\\Delta\}↑\\uparrowPathway↑\\uparrowPearsonΔ\\Delta↑\\uparrowDS↑\\uparrowMMD↓\\downarrowEnergy↓\\downarrow10\.01970\.84380\.70510\.15980\.73300\.89950\.05070\.518820\.02150\.84010\.66410\.34320\.71540\.89550\.05520\.591840\.02570\.82800\.59710\.42180\.66290\.87930\.06960\.789080\.02970\.81670\.54380\.45570\.62970\.88050\.08561\.0205

## Appendix GResponsible Use

*PerturbCellRL*is intended as a decision\-support method for prioritizing biological hypotheses, not as a replacement for experimental validation\. Predictions can be wrong when perturbations are outside the training distribution, when cell states are poorly represented, or when verifiers encode incomplete biological knowledge\. Any candidate therapeutic or biological conclusion suggested by the model should be validated with independent experiments\.