On the Design Space of Discrete Diffusion Online Adaptation for Molecular Optimization

# On the Design Space of Discrete Diffusion Online Adaptation for Molecular Optimization
Source: [https://arxiv.org/html/2607.02834](https://arxiv.org/html/2607.02834)
Trevor Chen1∗Ariel Dai1∗Jason Yang1Riccardo De Santi3,2\{\}^\{2\}\{\{\}^\{,\}\}^\{3\}Daniel Khalil1Wenda Chu1Nate Gruver4Pranav Murugan4Alexander F\. G\. Goldberg4Maruan Al\-Shedivat4Yisong Yue11Caltech2ETH Zurich3ETH AI Center4Genesis Molecular AI

###### Abstract

Molecular optimization often starts from a pretrained generative model that captures a broad prior over valid molecular structures\. At test time, however, the goal is not to sample from this prior, but to use a limited oracle budget to shift generation toward task\-specific high\-reward molecules\. We study this adaptation problem for discrete diffusion models\. Each online round couples several choices\. The loop must decide which candidates to evaluate, how rewards become model updates, which feedback to reuse, and how far to move beyond the pretrained prior\. These choices have mostly been studied in isolation, leaving open whether they complement one another, become redundant, or interfere inside a full online adaptation loop\. We conduct controlled studies across six small\-molecule binding\-affinity tasks and three protein\-fitness tasks\. We find that acquisition, reward shaping, and model debiasing provide complementary routes to higher reward, especially for small molecules\. Replay further stabilizes learning, while validity penalties keep small\-molecule exploration on the valid molecular manifold\. Together, these findings point to a practical recipe for feedback\-efficient molecular optimization: online fine\-tuning with acquisition, reward shaping, debiasing, replay, and validity control\. This recipe outperforms offline fine\-tuning and inference\-time search baselines under matched oracle\-call budgets and GPU\-hour accounting\. The gains are largest when high\-reward candidates require larger shifts from the pretrained prior\.

## 1Introduction

Discrete diffusion models have become a powerful tool for molecular generation, such as for small molecules\[[19](https://arxiv.org/html/2607.02834#bib.bib17),[42](https://arxiv.org/html/2607.02834#bib.bib3)\]and proteins\[[57](https://arxiv.org/html/2607.02834#bib.bib4)\], because they can model realistic chemical structures directly while producing diverse candidates\. In molecular design, however, unconditional generation is rarely useful\. A pretrained generatorpθ0p\_\{\\theta\_\{0\}\}encodes a broad molecular prior whose probability mass approximates the pretraining distribution, while a downstream design task defines a different target: sampling molecules achieving high rewards for a task\-specific oracler​\(x\)r\(x\)\. This oracle, which is typically costly to evaluate, might be a learned binding\-affinity model\[[16](https://arxiv.org/html/2607.02834#bib.bib34)\], a structure\-prediction, a docking pipeline\[[30](https://arxiv.org/html/2607.02834#bib.bib6)\], or a wet\-lab assay\. Under a fixed evaluation budgetBB, the typical goal of molecular optimization is to generate a moleculex∗x^\{\*\}best\-optimizing the given reward function:

x∗∈arg⁡maxx∈𝒳⁡r​\(x\)x^\{\*\}\\in\\arg\\max\_\{x\\in\\mathcal\{X\}\}\\;r\(x\)\(1\)where𝒳\\mathcal\{X\}denotes the valid chemical space under consideration\. This formulation corresponds to top\-11optimization of the unknown reward functionrr, a problem also denoted by best\-arm identification\[e\.g\.,[18](https://arxiv.org/html/2607.02834#bib.bib65)\]\. To tackle such generative optimization problems, the generator must be steered toward task\-relevant regions of chemical space while preserving prior validity\[[13](https://arxiv.org/html/2607.02834#bib.bib64),[23](https://arxiv.org/html/2607.02834#bib.bib44),[55](https://arxiv.org/html/2607.02834#bib.bib47)\]\. A powerful way to achieve this is via online model adaptation\[[60](https://arxiv.org/html/2607.02834#bib.bib22),[47](https://arxiv.org/html/2607.02834#bib.bib33),[54](https://arxiv.org/html/2607.02834#bib.bib43),[32](https://arxiv.org/html/2607.02834#bib.bib11),[48](https://arxiv.org/html/2607.02834#bib.bib39),[52](https://arxiv.org/html/2607.02834#bib.bib18),[60](https://arxiv.org/html/2607.02834#bib.bib22),[46](https://arxiv.org/html/2607.02834#bib.bib16),[47](https://arxiv.org/html/2607.02834#bib.bib33),[22](https://arxiv.org/html/2607.02834#bib.bib46),[29](https://arxiv.org/html/2607.02834#bib.bib48),[4](https://arxiv.org/html/2607.02834#bib.bib15)\], as described next\.

Online Model Adaptation for Optimization: Loop Structure\.Given a pre\-trained model, each roundttof online model adaptation alternates between two high\-level steps \(Fig\.[1](https://arxiv.org/html/2607.02834#S1.F1)\)\. In the first step, a batch of candidates is sampled frompθtp\_\{\\theta\_\{t\}\}, optionally via inference\-time steering, and candidates for oracle evaluation are selected\. In the second step, rewards observed are turned into training signals to update the model intopθt\+1p\_\{\\theta\_\{t\+1\}\}, before starting the next round\.

A Tightly Coupled Design Problem\.Online adaptation couples choices that are often studied separately: which candidates receive oracle evaluations, how rewards are converted into model updates, and how the generator is pushed beyond biases in the pretrained prior\. Thompson sampling can improve oracle\-call allocation through uncertainty\-aware exploration\[[54](https://arxiv.org/html/2607.02834#bib.bib43),[18](https://arxiv.org/html/2607.02834#bib.bib65)\]; top\-11/KKor CVaR\-style objectives can focus updates on the reward tail\[e\.g\.,[40](https://arxiv.org/html/2607.02834#bib.bib30),[58](https://arxiv.org/html/2607.02834#bib.bib66)\]; and negative\-score regularization can debias the generator away from over\-occupied modes\[[7](https://arxiv.org/html/2607.02834#bib.bib67),[40](https://arxiv.org/html/2607.02834#bib.bib30),[39](https://arxiv.org/html/2607.02834#bib.bib45)\]\. It is unclear whether these mechanisms complement one another, become redundant, or interfere inside a full online adaptation loop\.

To shed light on this problem, we perform an integrated study of online training loops for discrete\-diffusion\-based molecule & protein optimization to assess how representative design choices interact: whether choices for acquisition, feedback reuse, reward shaping, and prior debiasing provide distinct value, reinforce one another, or become redundant once combined\.

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/overview.jpg)Figure 1:Online test\-time feedback loop for molecular optimization\.Each round samples a candidate poolCtC\_\{t\}of sizeMMfrom the current discrete diffusion generatorpθtp\_\{\\theta\_\{t\}\}, selects a batchStS\_\{t\}of sizeKKfor oracle evaluation, appends the resulting pairs\(x,r​\(x\)\)\(x,r\(x\)\)to the evaluated set, converts scores into a training rewardr~​\(x\)\\tilde\{r\}\(x\), regularizes the update, and trainspθtp\_\{\\theta\_\{t\}\}intopθt\+1p\_\{\\theta\_\{t\+1\}\}before the next round\.Our experiments on 9 tasks spanning small\-molecule binding\-affinity and33protein\-fitness optimization show that these mechanisms provide complementary benefits, especially when high\-reward candidates require moving farther from the pretrained generative prior\. On small molecules, acquisition, CVaR reward shaping, and Density Entropy Regularization provide the clearest complementary gains: acquisition improves oracle\-call allocation, CVaR focuses updates on the reward tail, and Density Entropy Regularization pushes exploration away from over\-occupied prior modes\. Replay and validity control are more stabilizing, keeping useful feedback in the update distribution and preventing exploratory updates from leaving the valid molecular manifold\. On protein\-fitness tasks, ablation effects are weaker, consistent with family\-specific priors already covering high\-fitness regions\.

Overall, our contributions are:

- •We provide a controlled study of discrete\-diffusion online adaptation for molecular optimization under fixed oracle and runtime budgets, alongside offline fine\-tuning and inference\-time\-search based diagnostic baselines\.
- •We shed light on the interactions among simple*plug\-in components*added to the online optimization loop provide complementary benefits: Thompson sampling over the reward model improves oracle\-call allocation, replay stabilizes learning, CVaR reward shaping focuses updates on high\-reward candidates, Density Entropy regularization encourages exploration of modes, and invalid\-output penalties preserve molecular validity\.
- •We combine these components into a single online active fine\-tuning recipe that outperforms all baselines at small\-molecule and protein optimization under matched oracle\-call budgets\. We also elucidate that performance improvements are greater for molecular design solutions that lie further out of distribution from the original generative prior\.

## 2"Plug\-In" Design Components of Online Adaptation

We frame the loop in Fig\.[1](https://arxiv.org/html/2607.02834#S1.F1)as a*harness*: a fixed online schedule with swappable choices at the points where feedback is collected, transformed, and used for fine\-tuning\. Each round generates candidates, selects a subset for oracle evaluation, converts oracle scores into training rewards, regularizes the update, and fine\-tunes the generator\. The harness is finetuner\-agnostic: any objectiveℒft\\mathcal\{L\}\_\{\\text\{ft\}\}that consumes per\-sample log\-rewards and an off\-policy buffer can be used; we instantiate it with DDPP\-LB\[[32](https://arxiv.org/html/2607.02834#bib.bib11)\]and VIDD\[[48](https://arxiv.org/html/2607.02834#bib.bib39)\]\.

We organize the design choices by their role in the feedback loop\.*Steering strategies*affect which candidates receive oracle evaluations \(Sec\.[2\.1](https://arxiv.org/html/2607.02834#S2.SS1)\);*adaptation strategies*determine how observed rewards shape the model update \(Sec\.[2\.2](https://arxiv.org/html/2607.02834#S2.SS2)\); and*objective\-mismatch corrections*keep optimization aligned with accumulated feedback and valid molecules as exploration shifts the sampling distributionpθtp\_\{\\theta\_\{t\}\}\(Sec\.[2\.3](https://arxiv.org/html/2607.02834#S2.SS3)\)\. For each category, we instantiate one representative choice from prior work \(Table[1](https://arxiv.org/html/2607.02834#S2.T1)\)\. The per\-knob ablations in Sec\.[4](https://arxiv.org/html/2607.02834#S4)then test whether these choices retain distinct value when composed, or instead become redundant inside the full online loop\.

Table 1:Composable harness design choices evaluated in this paper\.Each knob layers on top of online DDPP\-LB and is ablated independently in Section[4](https://arxiv.org/html/2607.02834#S4), then combined into the final algorithm of Sec\.[4\.1](https://arxiv.org/html/2607.02834#S4.SS1)\. Search\-only and offline\-fine\-tuning baselines are used \(Sec\.[2](https://arxiv.org/html/2607.02834#S2)\)\.### 2\.1Steering Design Choices

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/diagram_thompson.png)

Figure 2:Thompson sampling\.AlthoughcAc\_\{A\}has the higher mean prediction,cBc\_\{B\}can be selected because its larger uncertainty gives it nonzero probability of drawing above the greedy threshold\.Thompson Sampling Acquisition\.Because the oracle budget is finite, the loop must choose a small subset of generated candidates for evaluation\. The simplest rule evaluates the candidates with the highest predicted reward under a surrogate model\. This greedy rule is purely exploitative: it spends oracle calls where the surrogate already predicts high reward\. We also consider uncertainty\-aware acquisition using Thompson sampling\[[51](https://arxiv.org/html/2607.02834#bib.bib49),[36](https://arxiv.org/html/2607.02834#bib.bib50)\]\. Given an ensemble surrogate with meanμi\\mu\_\{i\}and disagreementσi\\sigma\_\{i\}for candidatexi∈Ctx\_\{i\}\\in C\_\{t\}, Thompson sampling draws a sampled reward estimate:

r^i=μi\+εi​σi,εi∼𝒩​\(0,1\),\\hat\{r\}\_\{i\}=\\mu\_\{i\}\+\\varepsilon\_\{i\}\\sigma\_\{i\},\\qquad\\varepsilon\_\{i\}\\sim\\mathcal\{N\}\(0,1\),and evaluates the top\-KKcandidates byr^i\\hat\{r\}\_\{i\}, defining the selected batchStS\_\{t\}\. Fig\.[2](https://arxiv.org/html/2607.02834#S2.F2.5)depicts the intuition behind how Thompson sampling balances the explore/exploit trade\-off\. Candidates with high uncertainty can outrank candidates with higher predicted mean but lower uncertainty, i\.e\., this scheme incentivizes to explore seemingly lower quality but high uncertainty candidates\.

### 2\.2Adaptation Design Choices

CVaR\-Style Reward Shaping\.A second lever acts on the reward itself after oracle evaluation and before it enters the gradient\. We write the resulting shaped training reward asr~​\(x\)\\tilde\{r\}\(x\)\. Because the design objective in Eq\. \([1](https://arxiv.org/html/2607.02834#S1.E1)\) is controlled by the upper tail rather than the average generated molecule, we adapt the model via a CVaR\-style\[[34](https://arxiv.org/html/2607.02834#bib.bib52),[35](https://arxiv.org/html/2607.02834#bib.bib53)\]objective \(Fig\.[3](https://arxiv.org/html/2607.02834#S2.F3)\(a\)\), as introduced in prior work on diffusion model adaptation\[[40](https://arxiv.org/html/2607.02834#bib.bib30),[58](https://arxiv.org/html/2607.02834#bib.bib66)\]\. This objective allows to sacrifice the average sample reward to enhance the quality of top\-samples:

CVaR1−τ​\[r​\(x\)\]=𝔼​\[r​\(x\)∣r​\(x\)≥Qτ​\(r\)\]\.\\mathrm\{CVaR\}\_\{1\-\\tau\}\[r\(x\)\]=\\mathbb\{E\}\\\!\\left\[r\(x\)\\mid r\(x\)\\geq Q\_\{\\tau\}\(r\)\\right\]\.
![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/diagram_cvar.png)

\(a\)CVaR Shaping

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/diagram_debiasing.png)

\(b\)Density Entropy Regularization

Figure 3:Reward\-shaping design choices\.\(a\)CVaR upper\-tail shaping: the right\-tail of the reward distribution above theτ\\tau\-quantile \(dashed\) is shaded, and the solid red bar marksCVaR1−τ=𝔼​\[r∣r≥Qτ\]\\mathrm\{CVaR\}\_\{1\-\\tau\}=\\mathbb\{E\}\[r\\mid r\\geq Q\_\{\\tau\}\], the expected reward conditional on being in that tail\. Optimization concentrates gradient on this tail expectation, not on a hard top\-kkselection\.\(b\)Density Entropy Regularization \(model debiasing\): subtracting the current generator’s diffusion\-internal likelihoodlog⁡pθt​\(x\)\\log p\_\{\\theta\_\{t\}\}\(x\)from the log\-reward redistributes gradient mass off densely\-occupied modes and toward less\-frequented regions, exposing minor modes the unshaped reward would have kept under\-explored\.Density Entropy Regularization\.CVaR reward shaping focuses the update on the upper tail, but it does not distinguish between high\-reward regions that the generator already samples frequently and high\-reward regions that remain underexplored\. To expose this distinction independently of CVaR, practitioners can apply a density\-entropy regularizer\[[7](https://arxiv.org/html/2607.02834#bib.bib67),[40](https://arxiv.org/html/2607.02834#bib.bib30),[39](https://arxiv.org/html/2607.02834#bib.bib45)\]that uses the current model density as a measure of how occupied a region is under the generator and shapes the raw log\-reward as:

log⁡r~debias​\(x\)=log⁡r​\(x\)−γ​log⁡pθt​\(x\),\\log\\tilde\{r\}\_\{\\mathrm\{debias\}\}\(x\)=\\log r\(x\)\-\\gamma\\log p\_\{\\theta\_\{t\}\}\(x\),\(2\)illustrated in Fig\.[3](https://arxiv.org/html/2607.02834#S2.F3)\(b\), wherelog⁡pθt​\(x\)\\log p\_\{\\theta\_\{t\}\}\(x\)is the marginal\. In continuous\-diffusion analyses\[[7](https://arxiv.org/html/2607.02834#bib.bib67),[40](https://arxiv.org/html/2607.02834#bib.bib30)\], the corresponding negative\-score gradient∇xlog⁡pθt​\(x\)\\nabla\_\{x\}\\log p\_\{\\theta\_\{t\}\}\(x\)can be directly approximated by the available score network, but in masked discrete diffusion it is estimated by the marginalization:

pθt​\(x0\)=𝔼t,xt∼q​\(xt∣x0\)​\[pθt​\(x0∣xt\)\]⋅\(noise\-schedule weight\),p\_\{\\theta\_\{t\}\}\(x\_\{0\}\)\\;=\\;\\mathbb\{E\}\_\{t,\\,x\_\{t\}\\sim q\(x\_\{t\}\\mid x\_\{0\}\)\}\\\!\\bigl\[\\,p\_\{\\theta\_\{t\}\}\(x\_\{0\}\\mid x\_\{t\}\)\\,\\bigr\]\\cdot\\text\{\(noise\-schedule weight\)\},a sum over an exponentially\-large set of noised statesxtx\_\{t\}, making it intractable\. For discrete diffusion the proper asymptotic unbiased estimator is a Monte\-Carlo ELBO\[[26](https://arxiv.org/html/2607.02834#bib.bib70)\]\(LLaDA, Algorithm 3\), which costsnmcn\_\{\\text\{mc\}\}extra forward passes per evaluation\.

We replacelog⁡pθt​\(x\)\\log p\_\{\\theta\_\{t\}\}\(x\)in Eq\.[2](https://arxiv.org/html/2607.02834#S2.E2)with the cheapest such estimator:

log⁡pθt​\(x\)^≡log⁡pθt​\(x0∣xt\)=∑i:xti=\[MASK\]log⁡softmax​\(ℓi\)​\[x0i\]\.\\widehat\{\\log p\_\{\\theta\_\{t\}\}\(x\)\}\\;\\equiv\\;\\log p\_\{\\theta\_\{t\}\}\(x\_\{0\}\\mid x\_\{t\}\)\\;=\\;\\textstyle\\sum\_\{i\\,:\\,x\_\{t\}^\{i\}=\[\\text\{MASK\}\]\}\\,\\log\\mathrm\{softmax\}\(\\ell\_\{i\}\)\\bigl\[x\_\{0\}^\{i\}\\bigr\]\.This estimator is a cheap approximation of the diffusion ELBO, so it is not equivalent to the MCMC\-estimated continuous negative\-score regularizer or to the LLaDA\-styleL/lL/l\-weighted estimator in the limit\. We choose this estimator for computational reasons: the alternatives costnmc×n\_\{\\text\{mc\}\}\\\!\\timesextra forward passes per gradient step, which is prohibitive across thousands of online finetuning steps\. Despite the approximation, we find empirically that it induces the intended distribution shift and that the added noise leads to more exploration and overall better performance\. we find empirically that it achieves comparable oracle\-call efficiency to LLaDA\-Alg\. 3 variants withnmc∈\{1,4,8,16\}n\_\{\\text\{mc\}\}\\in\\\{1,4,8,16\\\}while being substantially faster per GPU hour\. Post\-hoc evaluation with the unbiased LLaDA\-Alg\. 3 estimator \(nmc=32n\_\{\\text\{mc\}\}\{=\}32\) confirms that the resulting NLL distributions from the cheap estimator follows closely to the LLaDa estimator with wider exploration \(Appendix Figs\.[35](https://arxiv.org/html/2607.02834#A6.F35)and[36](https://arxiv.org/html/2607.02834#A6.F36)\)\. Intuitively, this term favors candidates that are high reward but not already high likelihood under the generator, complementing Thompson\-sampling acquisition \(acquisition vs\. reward\-shape act on the loop at different points; see Appendix[B\.6](https://arxiv.org/html/2607.02834#A2.SS6)and Fig\.[3](https://arxiv.org/html/2607.02834#S2.F3)\)\. The estimated value is detached, so the parameter gradient enters the standard DDPP\-LB / VIDD policy\-gradient term as a scalar reward modifier \(Appendix[B](https://arxiv.org/html/2607.02834#A2)\)\.

### 2\.3Objective\-mismatch corrections

Once exploration starts shiftingpθtp\_\{\\theta\_\{t\}\}off the prior’s dominant modes, two gaps open between what the gradient actually optimizes and the true objective\. The next two components can close them\.

Replay Buffer\.A remaining choice is which evaluated samples appear in each update\. A purely on\-policy update trains only on the latest batch, so useful molecules discovered early stop influencing the generator once the sampling distribution moves on\. We therefore optionally train from a replay buffer that mixes fresh candidates with retained high\-reward samples\[[25](https://arxiv.org/html/2607.02834#bib.bib69),[41](https://arxiv.org/html/2607.02834#bib.bib68),[29](https://arxiv.org/html/2607.02834#bib.bib48)\]\. Concretely, the buffer is fixed\-capacity FIFO with priority eviction \(when full, the lowest\-reward sample leaves first, not the oldest\), and each minibatch is drawn by stratified sampling across reward bins so high\-reward outliers don’t get diluted by a larger volume of mid\-reward fresh candidates\.

Invalid\-output penalty\.To keep optimization on the molecular manifold, candidates that fail validity checks, ie RDKit parsing\[[17](https://arxiv.org/html/2607.02834#bib.bib54)\]in the case of small molecules, receive a fixed penaltyrinv<0r\_\{\\mathrm\{inv\}\}<0and enter training as low\-reward examples\. The mismatch this corrects is between the oracle’s reward \(defined for any string\) and the actual objective \(high reward on a valid molecule\); the penalty closes that gap by giving the gradient an explicit signal to pullpθtp\_\{\\theta\_\{t\}\}back toward the valid molecular manifold\.

## 3Experimental Setup

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures_source/Applications002.jpeg)Figure 4:In this study, we explore several tasks across two different domains, optimizing\(A\)small molecules for binding to protein targets and\(B\)proteins for various fitness metrics\.Tasks and oracles\.We evaluate whether the design choices in Section[2](https://arxiv.org/html/2607.02834#S2)improve discovery utility of finetuning methods under a fixed test\-time budget\. We study nine molecular optimization tasks across two domains\. The first domain is small\-molecule optimization for binding affinity to six protein targets \(2VT4, 5SDV, 6CM4, 7BKC, 7C7M, 7YLL from\[[63](https://arxiv.org/html/2607.02834#bib.bib29)\]\) with GenMol\[[19](https://arxiv.org/html/2607.02834#bib.bib17)\]as the generator and FlashAffinity \(FA\)\[[16](https://arxiv.org/html/2607.02834#bib.bib34)\]as the primary oracle, with selected comparisons re\-run under Boltz\-2\[[30](https://arxiv.org/html/2607.02834#bib.bib6)\]to verify conclusions hold for an oracle two orders of magnitude more expensive\. While many studies approach this from a structure\-based generative design perspective\[[63](https://arxiv.org/html/2607.02834#bib.bib29),[62](https://arxiv.org/html/2607.02834#bib.bib35),[43](https://arxiv.org/html/2607.02834#bib.bib7),[12](https://arxiv.org/html/2607.02834#bib.bib32),[33](https://arxiv.org/html/2607.02834#bib.bib26),[1](https://arxiv.org/html/2607.02834#bib.bib19)\], we use discrete diffusion due to the increased validity, synthesizability, and diversity of generations offered by these models\. The second domain is protein fitness optimization across three families \(GB1, CreiLOV, TrpB\) with a per\-family MDLM\-style discrete diffusion generator\[[38](https://arxiv.org/html/2607.02834#bib.bib2)\]and a per\-family fitness surrogate trained from experimental measurements as the oracle\[[60](https://arxiv.org/html/2607.02834#bib.bib22)\]\. For finetuning, we use either DDPP\[[32](https://arxiv.org/html/2607.02834#bib.bib11)\]or VIDD\[[48](https://arxiv.org/html/2607.02834#bib.bib39)\]\)\. The complete environment is detailed in Appendix[B\.8](https://arxiv.org/html/2607.02834#A2.SS8)\.

Experimental protocol\.The high\-level stages in Fig\.[1](https://arxiv.org/html/2607.02834#S1.F1)define our*full method*configuration for online adaptation, which serves as the upper\-bound reference point for every ablation in Sec\.[4](https://arxiv.org/html/2607.02834#S4)\. Each round generatesMMcandidates from the currentpθtp\_\{\\theta\_\{t\}\}, samples the top\-KKfor oracle evaluation, applies the invalid\-SMILES penalty to candidates that fail RDKit before they reach the oracle, refreshes the CVaR thresholdτ\\taufrom the buffer, and runsGGfine\-tuning gradient steps on stratified mini\-batches with both reward\-shape knobs \(CVaR\+\+Density Entropy Regularization\) applied to the log\-reward\. The ensemble surrogate is refit on the accumulated pairs at the end of each round\. Any fine\-tuning objectiveℒft\\mathcal\{L\}\_\{\\text\{ft\}\}that accepts a per\-sample log\-reward and an off\-policy buffer \(DDPP\-LB or VIDD\) plugs in unchanged\. Pseudocode for the loop is given in Appendix[C](https://arxiv.org/html/2607.02834#A3)as Algorithm[1](https://arxiv.org/html/2607.02834#alg1)\.

Evaluation protocol\.Each method is evaluated under the same oracle\-evaluation budget and random seeds\. We report performance both as a function of cumulative oracle calls and as a function of GPU hours\. Our primary metrics measure the upper tail of the evaluated set𝒟B\\mathcal\{D\}\_\{B\}\. We primarily report the best reward discovered so far,maxx∈𝒟B⁡r​\(x\)\\max\_\{x\\in\\mathcal\{D\}\_\{B\}\}r\(x\), as well as top\-kkand top\-k%k\\%means\. To compare across tasks, we normalize the improvement in reward and call this the reward "normalized reward" \(Eq\.[3](https://arxiv.org/html/2607.02834#A2.E3)\)\. Because reward optimization can exploit oracle artifacts or leave the valid molecular manifold, we additionally track secondary metrics over a sliding window of generated candidates \(parameters in Appendix[E](https://arxiv.org/html/2607.02834#A5)\): SMILES validity, pairwise Tanimoto diversity\[[2](https://arxiv.org/html/2607.02834#bib.bib55)\], QED\[[3](https://arxiv.org/html/2607.02834#bib.bib56)\], and SA\[[10](https://arxiv.org/html/2607.02834#bib.bib57)\]\. We compare the full online loop against unconditional sampling, inference\-time search variants, offline fine\-tuning, and online fine\-tuning without individual design choices\.

Advanced search\.In addition to the core acquisition choice above, we also test whether the candidate\-generation step benefits from inference\-time search\. Instead of plain ancestral sampling from the current generator, the loop can propose candidates using beam search\[[31](https://arxiv.org/html/2607.02834#bib.bib61)\], MCTS\[[50](https://arxiv.org/html/2607.02834#bib.bib28)\], or discrete Feynman\-Kac correction\[[14](https://arxiv.org/html/2607.02834#bib.bib36),[45](https://arxiv.org/html/2607.02834#bib.bib23)\]guided by the reward surrogate\[[60](https://arxiv.org/html/2607.02834#bib.bib22)\]\. In recent years, there have been significant related efforts combining inference time guidance with a fixed generator, which we do not fully explore here\[[8](https://arxiv.org/html/2607.02834#bib.bib5),[56](https://arxiv.org/html/2607.02834#bib.bib12),[44](https://arxiv.org/html/2607.02834#bib.bib20),[21](https://arxiv.org/html/2607.02834#bib.bib13),[27](https://arxiv.org/html/2607.02834#bib.bib10),[11](https://arxiv.org/html/2607.02834#bib.bib8),[59](https://arxiv.org/html/2607.02834#bib.bib37),[24](https://arxiv.org/html/2607.02834#bib.bib25),[52](https://arxiv.org/html/2607.02834#bib.bib18),[20](https://arxiv.org/html/2607.02834#bib.bib40),[15](https://arxiv.org/html/2607.02834#bib.bib38),[61](https://arxiv.org/html/2607.02834#bib.bib31),[49](https://arxiv.org/html/2607.02834#bib.bib1),[50](https://arxiv.org/html/2607.02834#bib.bib28),[60](https://arxiv.org/html/2607.02834#bib.bib22)\]\.

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/result1_combined_vs_baseline_1.png)Figure 5:Leave\-one\-out ablations reveal complementary roles most clearly on small\-molecule tasks\.Each panel removes one component from the full online loop and reports normalized maximum reward after 100, 1k, and 5k oracle calls, for DDPP\-LB \(top\) and VIDD \(bottom\), averaged over small\-molecule targets \(left\) and protein families \(right\)\. At later budgets, small\-molecule ablations diverge most strongly, with Density Entropy Regularization producing the largest drop; protein\-fitness ablations are weaker, with Thompson sampling giving the clearest consistent gain\. Error bars show±1\\pm 1SEM over target\-seed runs\. Reward is normalized per target before averaging; see Eq\.[3](https://arxiv.org/html/2607.02834#A2.E3)\. Tabular values are reported in Appendix[F](https://arxiv.org/html/2607.02834#A6)\.
## 4Results

Our main finding is that the online\-loop components are complementary, although not equally so across domains \(Fig\.[5](https://arxiv.org/html/2607.02834#S3.F5)\)\. On small molecules, acquisition, CVaR reward shaping, and Density Entropy Regularization produce the clearest reward gains, while replay and validity penalties stabilize the exploratory loop\. On protein\-fitness tasks, ablation effects are weaker and Thompson sampling gives the clearest consistent gain, suggesting that the family\-specific priors already cover high\-fitness regions more closely\. Sec\.[4\.1](https://arxiv.org/html/2607.02834#S4.SS1)breaks down the controlled ablations, and Sec\.[4\.2](https://arxiv.org/html/2607.02834#S4.SS2)analyzes how these components shift the fine\-tuned generator’s distribution\.

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/result2_combined_vs_search_1.png)Figure 6:Online adaptation outperforms inference\-time search most clearly on small\-molecule tasks\.Full loop \(black\) vs\. five search baselines at 100/1k/5k calls, over molecule targets \(left\) and protein families \(right\)\. On molecules the full loop leads at every budget and the gap widens with calls, as each gradient step compounds while search only re\-selects from a fixed generator\. On proteins the baselines reach the full loop by 5k, consistent with a prior already concentrated on high\-fitness regions, where adapting and selecting end up near\-equivalent\. Error bars: ±1 SEM over \(target, seed\); normalized reward per Eq\.[3](https://arxiv.org/html/2607.02834#A2.E3)### 4\.1Individual harness design components complement each other

Steering Design Choices\.Thompson sampling gives the clearest acquisition\-side gain in the main leave\-one\-out ablations \(Fig\.[5](https://arxiv.org/html/2607.02834#S3.F5)\)\. In the isolated Thompson ablation, its strongest effect appears in the top\-10% mean rather than in top\-1 alone, consistent with its role in diversifying which candidates receive oracle calls and shifting the generated distribution toward higher\-reward modes \(Appendix Fig\.[38](https://arxiv.org/html/2607.02834#A6.F38)\)\. In contrast, adding inference\-time search to the online loop \(best\-of\-NN, beam, MCTS, DFKC, or SMC\) does not improve over plain online fine\-tuning at fixed wall budget \(Appendix Fig\.[24](https://arxiv.org/html/2607.02834#A4.F24)\)\. The reason is primarily compute allocation: search proposesL×ML\\times Mcandidates per round instead ofMM, and the extra forward passes needed to score and prune them dominate the wall budget, leaving little time for fine\-tuning gradient steps \(Appendix Figs\.[25](https://arxiv.org/html/2607.02834#A4.F25),[26](https://arxiv.org/html/2607.02834#A4.F26),[28](https://arxiv.org/html/2607.02834#A4.F28)\)\. Under this budget, each gradient step extracts more reusable signal from an oracle call than one\-shot search does\.

Adaptation Design Choices\.CVaR reward shaping and Density Entropy Regularization provide the clearest reward gains on small\-molecule tasks \(Fig\.[5](https://arxiv.org/html/2607.02834#S3.F5)\)\. Appendix Fig\.[33](https://arxiv.org/html/2607.02834#A6.F33)compares CVaR\-on vs\. CVaR\-off across online and offline DDPP, while Appendix Fig\.[34](https://arxiv.org/html/2607.02834#A6.F34)isolates Density Entropy Regularization by holding CVaR off\. The two effects appear close to additive on maximum reward, consistent with the view that they target different failure modes: CVaR focuses updates on the reward tail, while Density Entropy Regularization pushes exploration away from modes already occupied by the current generator\. Sec\.[4\.2](https://arxiv.org/html/2607.02834#S4.SS2)analyzes these distributional effects directly\.

Objective\-Mismatch Corrections\.Replay and invalid\-output penalties have smaller and less uniform effects on top\-1 reward \(Fig\.[5](https://arxiv.org/html/2607.02834#S3.F5)\), but they stabilize failure modes induced by exploration\. The replay buffer has its clearest effect on top\-10% mean \(Appendix Fig\.[37](https://arxiv.org/html/2607.02834#A6.F37)\), consistent with aligning the gradient with the time\-averaged distribution of evaluated data rather than only the current on\-policy batch\. The invalid\-output penalty prevents a sharper failure: without a validity penalty \(rinv=0r\_\{\\text\{inv\}\}\{=\}0\), validity collapses within the first hour as the model concentrates mass on out\-of\-distribution but high\-surrogate\-reward strings\[[56](https://arxiv.org/html/2607.02834#bib.bib12),[53](https://arxiv.org/html/2607.02834#bib.bib62)\]\. This collapse is driven primarily by the likelihood penalty in Density Entropy Regularization \(Appendix[F](https://arxiv.org/html/2607.02834#A6)\), so validity control is best viewed as a guardrail for aggressive exploration rather than as a standalone reward booster\.

Online Adaptation Outperforms Inference\-Time Search\.Fig\.[6](https://arxiv.org/html/2607.02834#S4.F6)compares the full online loop against inference\-time search baselines that keep the generator fixed during selection \(best\-of\-NN, beam, MCTS, DFKC, and SMC\)\. The full loop dominates these baselines on small\-molecule tasks because oracle feedback is reused through model updates rather than spent only on one\-shot candidate selection\. Averaged across the six small\-molecule targets, the harness reaches the best search baseline’s final score in≈58×\\approx 58\\timesfewer oracle calls; on protein\-fitness tasks the corresponding speedup is≈16×\\approx 16\\times\. Replacing DDPP\-LB with VIDD asℒft\\mathcal\{L\}\_\{\\text\{ft\}\}gives similar trends

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures_source/combined_NLL.png)Figure 7:Distribution shifts inpθtp\_\{\\theta\_\{t\}\}explain performance gains for methods on tasks where high\-reward molecular design solutions lie outside the original generative priorpθtp\_\{\\theta\_\{t\}\}\.Each panel overlays the run\-end anchor\-NLL density of the*full method*\(blue\) against the corresponding leave\-one\-out*ablation cell*; the x\-axis \(−log⁡ppre​\(x\)\-\\log p\_\{\\text\{pre\}\}\(x\)\) measures how far each sample lies from the original generative prior\. Density Entropy Regularization pushes mass away from the priorpθtp\_\{\\theta\_\{t\}\}\. Overall, protein tasks experience weaker distribution shifts inpθtp\_\{\\theta\_\{t\}\}\. Results for the additional small\-molecule target 2VT4R are provided in Appendix Fig\.[32](https://arxiv.org/html/2607.02834#A5.F32)\.
### 4\.2Understanding performance improvements through distribution shifts inpθtp\_\{\\theta\_\{t\}\}

The domain dependence in Fig\.[5](https://arxiv.org/html/2607.02834#S3.F5)is largely explained by how far adaptation must move the generator away from its pretrained prior\. Fig\.[7](https://arxiv.org/html/2607.02834#S4.F7)measures this shift using anchor NLL under the frozen pretrained model,−log⁡ppre​\(x\)\-\\log p\_\{\\text\{pre\}\}\(x\)\. Density Entropy Regularization has the strongest and most direct effect on this distribution: removing it collapses mass back toward high\-density regions of the pretrained prior\. This matches its role in the objective, where the likelihood penalty discourages repeatedly reinforcing modes the current generator already occupies\.

CVaR shaping and Thompson sampling affect distribution shift more conditionally\. CVaR focuses the gradient on the upper reward tail, while Thompson sampling changes which uncertain candidates receive oracle calls\. Either mechanism can move the generator farther from the prior when high\-reward candidates lie in low\-density regions, but neither has a fixed direction of shift\. In Fig\.[7](https://arxiv.org/html/2607.02834#S4.F7), the no\-CVaR and full\-method densities nearly overlap on protein\-CreiLOV, where the prior already appears to cover the high\-reward basin\. On small\-molecule targets such as 5SDV and 6CM4, removing CVaR or Thompson produces a visible change in the high\-NLL tail, indicating that reward shaping and acquisition can help expose off\-prior high\-reward regions\.

Once acquisition and reward shaping movepθtp\_\{\\theta\_\{t\}\}into less prior\-dense regions, the main failure mode shifts from insufficient exploration to target mismatch\. A naive on\-policy update trains only on the current batch, so earlier high\-reward discoveries can stop influencing the generator\. The replay buffer addresses this by aligning the gradient with the reward\-tilted distribution over all evaluated data, not just the latest on\-policy samples\. A second mismatch appears when reward optimization moves probability mass off the valid molecular manifold\. The invalid\-SMILES penalty gives these failures a low\-reward training signal, pullingpθtp\_\{\\theta\_\{t\}\}back whenever exploration produces invalid strings\. These components are therefore best understood as guardrails for the exploratory loop rather than as the main drivers of distribution shift\.

This distributional view explains why the ablation effects are stronger on small\-molecule tasks and weaker on protein\-fitness tasks\. The small\-molecule generator is a broader prior, so high\-affinity candidates often require larger shifts away from its dominant modes\. The protein generators are family\-specific, so high\-fitness variants appear closer to regions already covered by the pretrained prior\. As a result, reward shaping and debiasing are less decisive on proteins, while acquisition can still help by allocating oracle calls more effectively\. This also explains why search baselines and ablations converge more quickly on the protein tasks: when the prior already places mass near high\-reward modes, adapting the generator and selecting from it become closer in effect\.

## 5Conclusion

This work studies how to use limited oracle feedback for molecular optimization with pretrained discrete diffusion models\. Our main finding is that online adaptation benefits from combining components that act on different parts of the feedback loop\. Acquisition, CVaR reward shaping, and Density Entropy Regularization provide the clearest reward gains on small\-molecule tasks, while replay and invalid\-output penalties stabilize the exploratory loop\. Together, these components form a practical recipe that outperforms offline fine\-tuning and inference\-time search baselines under matched oracle\-call budgets and GPU\-hour accounting\.

The results also clarify when this recipe is most useful\. Small\-molecule binding optimization often requires moving beyond high\-density regions of a broad pretrained prior, so shaping and debiasing matter more\. Protein\-fitness tasks show weaker ablation effects, consistent with family\-specific priors already covering high\-fitness regions more closely; in that regime, acquisition remains useful because oracle\-call allocation still matters, but large distribution shifts are less central\. Future work should extend this recipe to settings with less reliable learned oracles, multi\-objective design goals, and other discrete\-generation domains with non\-differentiable feedback\.

Acknowledgments\.This work was primarily conducted by researchers at Caltech, who were supported in part by NSF \#2505096, and gifts from Genesis Therapeutics, Point72, and OpenAI\. Jason Yang was supported in part by a Google Fellowship\. Riccardo De Santi was supported by and ETH AI Center doctoral fellowship, the Swiss National Science Foundation under NCCR Catalysis \(grants 180544 & 225147\) and NCCR Automation \(grant 51NF40 180545\), and Centres of Competence in Research funded by the Swiss National Science Foundation\.

## References

- \[1\]\(2025\-03\)ShEPhERD: Diffusing shape, electrostatics, and pharmacophores for bioisosteric drug design\.arXiv\(en\)\.Note:arXiv:2411\.04130 \[q\-bio\]External Links:[Link](http://arxiv.org/abs/2411.04130),[Document](https://dx.doi.org/10.48550/arXiv.2411.04130)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p1.1)\.
- \[2\]D\. Bajusz, A\. Rácz, and K\. Héberger\(2015\)Why is tanimoto index an appropriate choice for fingerprint\-based similarity calculations?\.Journal of Cheminformatics7,pp\. 20\.External Links:[Document](https://dx.doi.org/10.1186/s13321-015-0069-3)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p3.4)\.
- \[3\]G\. R\. Bickerton, G\. V\. Paolini, J\. Besnard, S\. Muresan, and A\. L\. Hopkins\(2012\)Quantifying the chemical beauty of drugs\.Nature Chemistry4\(2\),pp\. 90–98\.External Links:[Document](https://dx.doi.org/10.1038/nchem.1243)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p3.4)\.
- \[4\]N\. Blalock, S\. Seshadri, A\. Babber, S\. Fahlberg, and P\. Romero\(2025\)Functional Alignment of Protein Language Models via Reinforcement Learning with Experimental Feedback\.bioRxiv\(en\)\.External Links:[Link](https://www.biorxiv.org/content/10.1101/2025.05.02.651993),[Document](https://dx.doi.org/https%3A//doi.org/10.1101/2025.05.02.651993)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7)\.
- \[5\]O\. Chapelle and L\. Li\(2011\)An empirical evaluation of Thompson sampling\.InAdvances in Neural Information Processing Systems 24 \(NeurIPS\),pp\. 2249–2257\.External Links:[Link](https://papers.nips.cc/paper/4321-an-empirical-evaluation-of-thompson-sampling)Cited by:[§B\.6](https://arxiv.org/html/2607.02834#A2.SS6.SSS0.Px1.p1.6),[Table 1](https://arxiv.org/html/2607.02834#S2.T1.8.8.1.1.1)\.
- \[6\]H\. Chung, J\. Kim, M\. T\. McCann, M\. L\. Klasky, and J\. C\. Ye\(2023\)Diffusion posterior sampling for general noisy inverse problems\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/2209.14687)Cited by:[Figure 13](https://arxiv.org/html/2607.02834#A3.F13)\.
- \[7\]R\. De Santi\*, M\. Vlastelica\*, Y\. Hsieh, Z\. Shen, N\. He, and A\. Krause\(2025\-06\)Provable maximum entropy manifold exploration via diffusion models\.InProc\. International Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p3.2),[§2\.2](https://arxiv.org/html/2607.02834#S2.SS2.p2.2),[§2\.2](https://arxiv.org/html/2607.02834#S2.SS2.p2.5),[Table 1](https://arxiv.org/html/2607.02834#S2.T1.5.5.1.1.1)\.
- \[8\]K\. Didi, Z\. Zhang, G\. Zhou, D\. Reidenbach, Z\. Cao, S\. Cha, T\. Geffner, C\. Dallago, J\. Tang, M\. M\. Bronstein, M\. Steinegger, E\. Kucukbenli, A\. Vahdat, and K\. Kreis\(2026\)Scaling atomistic protein binder design with generative pretraining and test\-time compute\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=qmCpJtFZra)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[9\]B\. Efron\(2011\)Tweedie’s formula and selection bias\.Journal of the American Statistical Association106\(496\),pp\. 1602–1614\.External Links:[Document](https://dx.doi.org/10.1198/jasa.2011.tm11181)Cited by:[Figure 13](https://arxiv.org/html/2607.02834#A3.F13)\.
- \[10\]P\. Ertl and A\. Schuffenhauer\(2009\)Estimation of synthetic accessibility score of drug\-like molecules based on molecular complexity and fragment contributions\.Journal of Cheminformatics1,pp\. 8\.External Links:[Document](https://dx.doi.org/10.1186/1758-2946-1-8)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p3.4)\.
- \[11\]N\. Gruver, S\. Stanton, N\. C\. Frey, T\. G\. J\. Rudner, I\. Hotzel, J\. Lafrance\-Vanasse, A\. Rajpal, K\. Cho, and A\. G\. Wilson\(2023\-05\)Protein Design with Guided Discrete Diffusion\.InAdvances in Neural Information Processing Systems 36,\(en\)\.Note:arXiv:2305\.20009 \[cs, q\-bio\]External Links:[Link](http://arxiv.org/abs/2305.20009),[Document](https://dx.doi.org/https%3A//doi.org/10.48550/arXiv.2305.20009)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[12\]J\. Guan, W\. W\. Qian, X\. Peng, Y\. Su, J\. Peng, and J\. Ma\(2023\-03\)3D Equivariant Diffusion for Target\-Aware Molecule Generation and Affinity Prediction\.arXiv\(en\)\.Note:arXiv:2303\.03543 \[q\-bio\]External Links:[Link](http://arxiv.org/abs/2303.03543),[Document](https://dx.doi.org/10.48550/arXiv.2303.03543)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p1.1)\.
- \[13\]S\. Gutjahr, R\. De Santi, L\. Schaufelberger, K\. Jorner, and A\. Krause\(2025\)Constrained molecular generation via sequential flow model fine\-tuning\.InICML 2025 Generative AI and Biology \(GenBio\) Workshop,Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7)\.
- \[14\]M\. Hasan, V\. Ohanesian, A\. Gazizov, Y\. Bengio, A\. Aspuru\-Guzik, R\. Bondesan, M\. Skreta, and K\. Neklyudov\(2026\-01\)Discrete Feynman\-Kac Correctors\.arXiv\.Note:arXiv:2601\.10403 \[cs\]External Links:[Link](http://arxiv.org/abs/2601.10403),[Document](https://dx.doi.org/10.48550/arXiv.2601.10403)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[15\]Y\. Jian, C\. Wu, D\. Reidenbach, and A\. S\. Krishnapriyan\(2026\-02\)General Binding Affinity Guidance for Diffusion Models in Structure\-Based Drug Design\.Journal of Chemical Information and Modeling66\(3\),pp\. 1342–1352\.Note:arXiv:2406\.16821 \[cs\]External Links:ISSN 1549\-9596, 1549\-960X,[Link](http://arxiv.org/abs/2406.16821),[Document](https://dx.doi.org/10.1021/acs.jcim.5c01166)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[16\]S\. Jiang, Y\. Chen, Z\. Cao, and W\. Jin\(2025\)FlashAffinity: Bridging the Accuracy\-Speed Gap in Protein\-Ligand Binding Affinity Prediction\.bioRxiv\(en\)\.Note:bioRxiv preprint, December 2025External Links:[Link](https://www.biorxiv.org/content/10.64898/2025.12.22.695983v1),[Document](https://dx.doi.org/10.64898/2025.12.22.695983)Cited by:[§B\.8](https://arxiv.org/html/2607.02834#A2.SS8.SSS0.Px2.p1.3),[§1](https://arxiv.org/html/2607.02834#S1.p1.4),[§3](https://arxiv.org/html/2607.02834#S3.p1.1)\.
- \[17\]G\. Landrum and RDKit Contributors\(2024\)RDKit: open\-source cheminformatics software\.Note:[https://www\.rdkit\.org](https://www.rdkit.org/)Software release; canonical Zenodo DOI:[https://doi\.org/10\.5281/zenodo\.10099869](https://doi.org/10.5281/zenodo.10099869)Cited by:[§2\.3](https://arxiv.org/html/2607.02834#S2.SS3.p3.2)\.
- \[18\]T\. Lattimore and C\. Szepesvári\(2020\)Bandit algorithms\.Cambridge University Press\.External Links:[Document](https://dx.doi.org/10.1017/9781108571401)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7),[§1](https://arxiv.org/html/2607.02834#S1.p3.2)\.
- \[19\]S\. Lee, K\. Kreis, S\. P\. Veccham, M\. Liu, D\. Reidenbach, Y\. Peng, S\. Paliwal, W\. Nie, and A\. Vahdat\(2025\-01\)GenMol: A Drug Discovery Generalist with Discrete Diffusion\.arXiv\.Note:arXiv:2501\.06158 \[cs\]External Links:[Link](http://arxiv.org/abs/2501.06158),[Document](https://dx.doi.org/10.48550/arXiv.2501.06158)Cited by:[§B\.8](https://arxiv.org/html/2607.02834#A2.SS8.SSS0.Px2.p1.3),[§1](https://arxiv.org/html/2607.02834#S1.p1.4),[§3](https://arxiv.org/html/2607.02834#S3.p1.1)\.
- \[20\]X\. Li, M\. Uehara, X\. Su, G\. Scalia, T\. Biancalani, A\. Regev, S\. Levine, and S\. Ji\(2025\-06\)Dynamic Search for Inference\-Time Alignment in Diffusion Models\.arXiv\(en\)\.Note:arXiv:2503\.02039 \[cs\]External Links:[Link](http://arxiv.org/abs/2503.02039),[Document](https://dx.doi.org/10.48550/arXiv.2503.02039)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[21\]X\. Li, Y\. Zhao, C\. Wang, G\. Scalia, G\. Eraslan, S\. Nair, T\. Biancalani, A\. Regev, S\. Levine, and M\. Uehara\(2024\-10\)Derivative\-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value\-Based Decoding\.arXiv\(en\)\.Note:arXiv:2408\.08252 \[cs\]External Links:[Link](http://arxiv.org/abs/2408.08252)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[22\]Y\. Li, Z\. Jiang, E\. Dai, L\. Wang, W\. Ye, and L\. Liu\(2026\-04\)CAGenMol: Condition\-Aware Diffusion Language Model for Goal\-Directed Molecular Generation\.arXiv\.Note:arXiv:2604\.11483 \[cs\]External Links:[Link](http://arxiv.org/abs/2604.11483),[Document](https://dx.doi.org/10.48550/arXiv.2604.11483)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7)\.
- \[23\]J\. Listgarten and H\. Jiang\(2026\-04\)How artificial intelligence is reengineering protein engineering\.Science\(en\)\.External Links:[Link](https://www.science.org/doi/10.1126/science.aec8444),[Document](https://dx.doi.org/10.1126/science.aec8444)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7)\.
- \[24\]W\. S\. Lu, X\. Zhang, L\. S\. Mille\-Fragoso, H\. Dai, X\. J\. Gao, and W\. H\. Wong\(2025\-07\)ProVADA: Generation of Subcellular Protein Variants via Ensemble\-Guided Test\-Time Steering\.bioRxiv\(en\)\.Note:ISSN: 2692\-8205 Pages: 2025\.07\.11\.664238 Section: New ResultsExternal Links:[Link](https://www.biorxiv.org/content/10.1101/2025.07.11.664238v1),[Document](https://dx.doi.org/10.1101/2025.07.11.664238)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[25\]V\. Mnih, K\. Kavukcuoglu, D\. Silver, A\. A\. Rusu, J\. Veness, M\. G\. Bellemare, A\. Graves, M\. Riedmiller, A\. K\. Fidjeland, G\. Ostrovski, S\. Petersen, C\. Beattie, A\. Sadik, I\. Antonoglou, H\. King, D\. Kumaran, D\. Wierstra, S\. Legg, and D\. Hassabis\(2015\)Human\-level control through deep reinforcement learning\.Nature518\(7540\),pp\. 529–533\.External Links:[Document](https://dx.doi.org/10.1038/nature14236)Cited by:[§2\.3](https://arxiv.org/html/2607.02834#S2.SS3.p2.1),[Table 1](https://arxiv.org/html/2607.02834#S2.T1.12.12.1.1.1)\.
- \[26\]S\. Nie, F\. Zhu, Z\. You, X\. Zhang, J\. Ou, J\. Hu, J\. Zhou, Y\. Lin, J\. Wen, and C\. Li\(2025\)Large language diffusion models\.arXiv preprint arXiv:2502\.09992\.External Links:2502\.09992Cited by:[§2\.2](https://arxiv.org/html/2607.02834#S2.SS2.p2.4)\.
- \[27\]H\. Nisonoff, J\. Xiong, S\. Allenspach, and J\. Listgarten\(2025\)Unlocking Guidance for Discrete State\-Space Diffusion and Flow Models\.In13th International Conference on Learning Representations,\(en\)\.Note:arXiv:2406\.01572 \[cs\]External Links:[Link](http://arxiv.org/abs/2406.01572),[Document](https://dx.doi.org/https%3A//doi.org/10.48550/arXiv.2406.01572)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[28\]E\. Noutahi, C\. Gabellini, M\. Craig, J\. S\. C\. Lim, and P\. Tossou\(2023\-12\)Gotta be SAFE: A New Framework for Molecular Design\.arXiv\(en\)\.Note:arXiv:2310\.10773 \[cs\]External Links:[Link](http://arxiv.org/abs/2310.10773),[Document](https://dx.doi.org/10.48550/arXiv.2310.10773)Cited by:[§B\.3](https://arxiv.org/html/2607.02834#A2.SS3.p1.2)\.
- \[29\]M\. Olivecrona, T\. Blaschke, O\. Engkvist, and H\. Chen\(2017\-08\)Molecular De Novo Design through Deep Reinforcement Learning\.arXiv\(en\)\.Note:arXiv:1704\.07555 \[cs\]External Links:[Link](http://arxiv.org/abs/1704.07555),[Document](https://dx.doi.org/10.48550/arXiv.1704.07555)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7),[§2\.3](https://arxiv.org/html/2607.02834#S2.SS3.p2.1)\.
- \[30\]S\. Passaro, G\. Corso, J\. Wohlwend, M\. Reveiz, S\. Thaler, V\. R\. Somnath, N\. Getz, T\. Portnoi, J\. Roy, H\. Stark, D\. Kwabi\-Addo, D\. Beaini, T\. Jaakkola, and R\. Barzilay\(2025\)Boltz\-2: towards accurate and efficient binding affinity prediction\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.06.14.659707),[Link](https://www.biorxiv.org/content/10.1101/2025.06.14.659707)Cited by:[§B\.8](https://arxiv.org/html/2607.02834#A2.SS8.SSS0.Px2.p1.3),[§1](https://arxiv.org/html/2607.02834#S1.p1.4),[§3](https://arxiv.org/html/2607.02834#S3.p1.1)\.
- \[31\]V\. Rameshet al\.\(2025\)Test\-time scaling of diffusion models via noise trajectory search\.External Links:2506\.03164,[Link](https://arxiv.org/abs/2506.03164)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[32\]J\. Rector\-Brooks, M\. Hasan, Z\. Peng, Z\. Quinn, C\. Liu, S\. Mittal, N\. Dziri, M\. Bronstein, Y\. Bengio, P\. Chatterjee, A\. Tong, and A\. J\. Bose\(2024\-10\)Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction\.arXiv\(en\)\.Note:arXiv:2410\.08134 \[cs\]External Links:[Link](http://arxiv.org/abs/2410.08134)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7),[§2](https://arxiv.org/html/2607.02834#S2.p1.1),[§3](https://arxiv.org/html/2607.02834#S3.p1.1)\.
- \[33\]A\. Rekesh, M\. Cretu, D\. Shevchuk, V\. R\. Somnath, P\. Liò, R\. A\. Batey, M\. Tyers, M\. Koziarski, and C\. Liu\(2025\-07\)SynCoGen: Synthesizable 3D Molecule Generation via Joint Reaction and Coordinate Modeling\.arXiv\.Note:arXiv:2507\.11818 \[cs\]External Links:[Link](http://arxiv.org/abs/2507.11818),[Document](https://dx.doi.org/10.48550/arXiv.2507.11818)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p1.1)\.
- \[34\]R\. T\. Rockafellar and S\. Uryasev\(2000\)Optimization of conditional value\-at\-risk\.Journal of Risk2\(3\),pp\. 21–41\.External Links:[Document](https://dx.doi.org/10.21314/JOR.2000.038)Cited by:[§2\.2](https://arxiv.org/html/2607.02834#S2.SS2.p1.1)\.
- \[35\]R\. T\. Rockafellar and S\. Uryasev\(2002\)Conditional value\-at\-risk for general loss distributions\.Journal of Banking and Finance26\(7\),pp\. 1443–1471\.External Links:[Document](https://dx.doi.org/10.1016/S0378-4266%2802%2900271-6)Cited by:[§2\.2](https://arxiv.org/html/2607.02834#S2.SS2.p1.1)\.
- \[36\]D\. Russo, B\. Van Roy, A\. Kazerouni, I\. Osband, and Z\. Wen\(2018\)A tutorial on Thompson sampling\.Foundations and Trends in Machine Learning11\(1\),pp\. 1–96\.External Links:[Document](https://dx.doi.org/10.1561/2200000070),[Link](https://arxiv.org/abs/1707.02038)Cited by:[§2\.1](https://arxiv.org/html/2607.02834#S2.SS1.p1.3)\.
- \[37\]S\. S\. Sahoo, M\. Arriola, A\. Gokaslan, E\. M\. Marroquin, A\. M\. Rush, Y\. Schiff, J\. T\. Chiu, and V\. Kuleshov\(2024\)Simple and effective masked diffusion language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),External Links:[Link](https://arxiv.org/abs/2406.07524)Cited by:[§B\.3](https://arxiv.org/html/2607.02834#A2.SS3.p1.2)\.
- \[38\]S\. S\. Sahoo, M\. Arriola, Y\. Schiff, A\. Gokaslan, E\. Marroquin, J\. T\. Chiu, A\. Rush, and V\. Kuleshov\(2024\-11\)Simple and Effective Masked Diffusion Language Models\.arXiv\(en\)\.Note:arXiv:2406\.07524 \[cs\]External Links:[Link](http://arxiv.org/abs/2406.07524),[Document](https://dx.doi.org/10.48550/arXiv.2406.07524)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p1.1)\.
- \[39\]R\. D\. Santi, K\. Protopapas, Y\. Hsieh, and A\. Krause\(2026\-02\)Verifier\-Constrained Flow Expansion for Discovery Beyond the Data\.arXiv\(en\)\.Note:arXiv:2602\.15984 \[cs\]External Links:[Link](http://arxiv.org/abs/2602.15984),[Document](https://dx.doi.org/10.48550/arXiv.2602.15984)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p3.2),[§2\.2](https://arxiv.org/html/2607.02834#S2.SS2.p2.5)\.
- \[40\]R\. D\. Santi, M\. Vlastelica, and Y\. Hsieh\(2025\)Flow Density Control: Generative Optimization Beyond Entropy\-Regularized Fine\-Tuning\.Advances in Neural Information Processing Systems \(NeurIPS 2025\)\(en\)\.Note:arXiv:2511\.22640 \[cs\]External Links:[Link](http://arxiv.org/abs/2511.22640),[Document](https://dx.doi.org/10.48550/arXiv.2511.22640)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p3.2),[§2\.2](https://arxiv.org/html/2607.02834#S2.SS2.p1.1),[§2\.2](https://arxiv.org/html/2607.02834#S2.SS2.p2.2),[§2\.2](https://arxiv.org/html/2607.02834#S2.SS2.p2.5),[Table 1](https://arxiv.org/html/2607.02834#S2.T1.1.1.1.1.1)\.
- \[41\]T\. Schaul, J\. Quan, I\. Antonoglou, and D\. Silver\(2016\)Prioritized experience replay\.InInternational Conference on Learning Representations \(ICLR\),External Links:1511\.05952Cited by:[§2\.3](https://arxiv.org/html/2607.02834#S2.SS3.p2.1)\.
- \[42\]Y\. Schiff, S\. S\. Sahoo, H\. Phung, G\. Wang, S\. Boshar, H\. Dalla\-torre, B\. P\. d\. Almeida, A\. Rush, T\. Pierrot, and V\. Kuleshov\(2025\)Simple Guidance Mechanisms for Discrete Diffusion Models\.InThirteenth International Conference on Learning Representations,Note:arXiv:2412\.10193 \[cs\]External Links:[Link](http://arxiv.org/abs/2412.10193),[Document](https://dx.doi.org/10.48550/arXiv.2412.10193)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.4)\.
- \[43\]A\. Schneuing, Y\. Du, C\. Harris, A\. Jamasb, I\. Igashov, W\. Du, T\. Blundell, P\. Lió, C\. Gomes, M\. Welling, M\. Bronstein, and B\. Correia\(2022\-10\)Structure\-based Drug Design with Equivariant Diffusion Models\.arXiv\(en\)\.Note:arXiv:2210\.13695 \[cs, q\-bio\]External Links:[Link](http://arxiv.org/abs/2210.13695),[Document](https://dx.doi.org/10.48550/arXiv.2210.13695)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p1.1)\.
- \[44\]R\. Singhal, Z\. Horvitz, R\. Teehan, M\. Ren, Z\. Yu, K\. McKeown, and R\. Ranganath\(2025\-01\)A General Framework for Inference\-time Scaling and Steering of Diffusion Models\.arXiv\.Note:arXiv:2501\.06848 \[cs\]External Links:[Link](http://arxiv.org/abs/2501.06848),[Document](https://dx.doi.org/10.48550/arXiv.2501.06848)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[45\]M\. Skreta, T\. Akhound\-Sadegh, V\. Ohanesian, R\. Bondesan, A\. Aspuru\-Guzik, A\. Doucet, R\. Brekelmans, A\. Tong, and K\. Neklyudov\(2025\-06\)Feynman\-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts\.arXiv\.Note:arXiv:2503\.02819 \[cs\]External Links:[Link](http://arxiv.org/abs/2503.02819),[Document](https://dx.doi.org/10.48550/arXiv.2503.02819)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[46\]F\. Stocco, M\. Artigues\-Lleixà, A\. Hunklinger, T\. Widatalla, M\. Güell, and N\. Ferruz\(2025\)Guiding Generative Protein Language Models with Reinforcement Learning\.arXiv\(en\)\.External Links:[Link](http://arxiv.org/abs/2412.12979),[Document](https://dx.doi.org/https%3A//doi.org/10.48550/arXiv.2412.12979)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7)\.
- \[47\]F\. Stocco, M\. Garibbo, and N\. Ferruz\(2025\-11\)Guiding Generative Models for Protein Design: Prompting, Steering and Aligning\.arXiv\(en\)\.Note:arXiv:2511\.21476 \[q\-bio\]External Links:[Link](http://arxiv.org/abs/2511.21476),[Document](https://dx.doi.org/10.48550/arXiv.2511.21476)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7)\.
- \[48\]X\. Su, X\. Li, M\. Uehara, S\. Kim, Y\. Zhao, G\. Scalia, E\. Hajiramezanali, T\. Biancalani, D\. Zhi, and S\. Ji\(2025\-08\)Iterative Distillation for Reward\-Guided Fine\-Tuning of Diffusion Models in Biomolecular Design\.arXiv\(en\)\.Note:arXiv:2507\.00445 \[cs\]External Links:[Link](http://arxiv.org/abs/2507.00445),[Document](https://dx.doi.org/10.48550/arXiv.2507.00445)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7),[§2](https://arxiv.org/html/2607.02834#S2.p1.1),[§3](https://arxiv.org/html/2607.02834#S3.p1.1)\.
- \[49\]S\. Tang, Y\. Zhang, and P\. Chatterjee\(2024\-12\)PepTune: De Novo Generation of Therapeutic Peptides with Multi\-Objective\-Guided Discrete Diffusion\.arXiv\(en\)\.Note:arXiv:2412\.17780 \[q\-bio\]External Links:[Link](http://arxiv.org/abs/2412.17780),[Document](https://dx.doi.org/10.48550/arXiv.2412.17780)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[50\]S\. Tang, Y\. Zhu, M\. Tao, and P\. Chatterjee\(2025\-09\)TR2\-D2: Tree Search Guided Trajectory\-Aware Fine\-Tuning for Discrete Diffusion\.arXiv\.Note:arXiv:2509\.25171 \[cs\]External Links:[Link](http://arxiv.org/abs/2509.25171),[Document](https://dx.doi.org/10.48550/arXiv.2509.25171)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[51\]W\. R\. Thompson\(1933\)On the likelihood that one unknown probability exceeds another in view of the evidence of two samples\.Biometrika25\(3\-4\),pp\. 285–294\.External Links:[Document](https://dx.doi.org/10.1093/biomet/25.3-4.285)Cited by:[§2\.1](https://arxiv.org/html/2607.02834#S2.SS1.p1.3),[Table 1](https://arxiv.org/html/2607.02834#S2.T1.8.8.1.1.1)\.
- \[52\]M\. Uehara, X\. Su, Y\. Zhao, X\. Li, A\. Regev, S\. Ji, S\. Levine, and T\. Biancalani\(2025\-02\)Reward\-Guided Iterative Refinement in Diffusion Models at Test\-Time with Applications to Protein and DNA Design\.arXiv\.Note:arXiv:2502\.14944 \[q\-bio\]External Links:[Link](http://arxiv.org/abs/2502.14944),[Document](https://dx.doi.org/10.48550/arXiv.2502.14944)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7),[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[53\]M\. Uehara, Y\. Zhao, K\. Black, E\. Hajiramezanali, G\. Scalia, N\. L\. Diamant, A\. M\. Tseng, T\. Biancalani, and S\. Levine\(2024\)Fine\-tuning of continuous\-time diffusion models as entropy\-regularized control\.External Links:2402\.15194,[Link](https://arxiv.org/abs/2402.15194)Cited by:[Table 1](https://arxiv.org/html/2607.02834#S2.T1.13.13.1.1.1),[§4\.1](https://arxiv.org/html/2607.02834#S4.SS1.p3.1)\.
- \[54\]M\. Uehara, Y\. Zhao, K\. Black, E\. Hajiramezanali, G\. Scalia, N\. L\. Diamant, A\. M\. Tseng, S\. Levine, and T\. Biancalani\(2024\-07\)Feedback Efficient Online Fine\-Tuning of Diffusion Models\.arXiv\(en\)\.Note:arXiv:2402\.16359 \[cs\]External Links:[Link](http://arxiv.org/abs/2402.16359),[Document](https://dx.doi.org/10.48550/arXiv.2402.16359)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7),[§1](https://arxiv.org/html/2607.02834#S1.p3.2)\.
- \[55\]M\. Uehara, Y\. Zhao, C\. Wang, X\. Li, A\. Regev, S\. Levine, and T\. Biancalani\(2025\-01\)Inference\-Time Alignment in Diffusion Models with Reward\-Guided Generation: Tutorial and Review\.arXiv\.Note:arXiv:2501\.09685 \[cs\]External Links:[Link](http://arxiv.org/abs/2501.09685),[Document](https://dx.doi.org/10.48550/arXiv.2501.09685)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7)\.
- \[56\]C\. Wang, M\. Uehara, Y\. He, A\. Wang, T\. Biancalani, A\. Lal, T\. Jaakkola, S\. Levine, H\. Wang, and A\. Regev\(2024\-10\)Fine\-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design\.arXiv\(en\)\.Note:arXiv:2410\.13643 \[cs\]External Links:[Link](http://arxiv.org/abs/2410.13643)Cited by:[Table 1](https://arxiv.org/html/2607.02834#S2.T1.13.13.1.1.1),[§3](https://arxiv.org/html/2607.02834#S3.p4.1),[§4\.1](https://arxiv.org/html/2607.02834#S4.SS1.p3.1)\.
- \[57\]X\. Wang, Z\. Zheng, F\. Ye, D\. Xue, S\. Huang, and Q\. Gu\(2024\-02\)Diffusion Language Models Are Versatile Protein Learners\.InProceedings of the 41st International Conference on Machine Learning,pp\. 52309 – 52333\(en\)\.Note:arXiv:2402\.18567 \[cs, q\-bio\]External Links:[Link](http://arxiv.org/abs/2402.18567),[Document](https://dx.doi.org/10.1126/sciadv.adj3786)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.4)\.
- \[58\]Z\. Wang, R\. De Santi, X\. Mo, M\. M\. Zavlanos, A\. Krause, and K\. H\. Johansson\(2026\)Efficient tail\-aware generative optimization via flow model fine\-tuning\.arXiv preprint arXiv:2602\.16796\.Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p3.2),[§2\.2](https://arxiv.org/html/2607.02834#S2.SS2.p1.1),[Table 1](https://arxiv.org/html/2607.02834#S2.T1.1.1.1.1.1)\.
- \[59\]J\. Xiong, I\. Gaur, M\. Lukarska, H\. Nisonoff, L\. M\. Oltrogge, D\. F\. Savage, and J\. Listgarten\(2026\-01\)ProteinGuide: On\-the\-fly property guidance for protein sequence generative models\.arXiv\(en\)\.Note:arXiv:2505\.04823 \[cs\]External Links:[Link](http://arxiv.org/abs/2505.04823),[Document](https://dx.doi.org/10.48550/arXiv.2505.04823)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[60\]J\. Yang, W\. Chu, D\. Khalil, R\. Astudillo, B\. J\. Wittmann, F\. H\. Arnold, and Y\. Yue\(2025\-05\)Steering Generative Models with Experimental Data for Protein Fitness Optimization\.InAdvances in Neural Information Processing Systems,\(en\)\.External Links:[Link](http://arxiv.org/abs/2505.15093),[Document](https://dx.doi.org/https%3A//doi.org/10.48550/arXiv.2505.15093)Cited by:[§1](https://arxiv.org/html/2607.02834#S1.p1.7),[§3](https://arxiv.org/html/2607.02834#S3.p1.1),[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[61\]S\. Yang, D\. Hwang, S\. Lee, S\. Ryu, and S\. J\. Hwang\(2021\)Hit and Lead Discovery with Explorative RL and Fragment\-based Molecule Generation\.Advances in Neural Information Processing Systems 34 \(NeurIPS 2021\)\(en\)\.Note:arXiv:2110\.01219 \[cs\]External Links:[Link](http://arxiv.org/abs/2110.01219),[Document](https://dx.doi.org/10.48550/arXiv.2110.01219)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p4.1)\.
- \[62\]C\. Zeng, J\. Jin, C\. Ambrose, G\. Karypis, M\. Transtrum, E\. B\. Tadmor, R\. G\. Hennig, A\. Roitberg, S\. Martiniani, and M\. Liu\(2026\-01\)PropMolFlow: property\-guided molecule generation with geometry\-complete flow matching\.Nature Computational Science,pp\. 1–10\(en\)\.External Links:ISSN 2662\-8457,[Link](https://www.nature.com/articles/s43588-025-00946-y),[Document](https://dx.doi.org/10.1038/s43588-025-00946-y)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p1.1)\.
- \[63\]O\. Zhang, X\. Zhang, H\. Lin, C\. Tan, Q\. Wang, Y\. Mo, Q\. Feng, G\. Du, Y\. Yu, Z\. Jin, Z\. You, P\. Lin, Y\. Zhang, Y\. Tao, S\. Chen, J\. X\. Chen, C\. Hua, W\. Zhao, R\. Ma, Y\. Xia, K\. Ying, J\. Li, Y\. Zeng, L\. Lang, P\. Pan, H\. Cao, Z\. Song, B\. Qiang, J\. Wang, P\. Ji, L\. Bai, J\. Zhang, C\. Hsieh, P\. A\. Heng, S\. Sun, T\. Hou, and S\. Zheng\(2025\-10\)ODesign: A World Model for Biomolecular Interaction Design\.arXiv\.Note:arXiv:2510\.22304 \[q\-bio\]External Links:[Link](http://arxiv.org/abs/2510.22304),[Document](https://dx.doi.org/10.48550/arXiv.2510.22304)Cited by:[§3](https://arxiv.org/html/2607.02834#S3.p1.1)\.

## Appendix ADeclaration of LLM Usage

LLMs were used to write code, interpret results, assist with writing, and assist with figure making\.

## Appendix BMethod Details

This appendix records implementation specifics that are not stated in the main body\. Hyperparameter values are collected in Table[2](https://arxiv.org/html/2607.02834#A2.T2)\.

### B\.1Normalized Rewards

Across the bar plots in Sec\.[4](https://arxiv.org/html/2607.02834#S4)and the appendix grids we report a per\-target normalized reward, rescaling each target’s scores to a common \[0, 1\] range\.

liftt​\(x\)=rt​\(x\)−CtFt−Ct,\\mathrm\{lift\}\_\{t\}\(x\)\\;=\\;\\frac\{r\_\{t\}\(x\)\-C\_\{t\}\}\{F\_\{t\}\-C\_\{t\}\},\(3\)wherert​\(x\)r\_\{t\}\(x\)is the score of methodxxon targetttat a given slice, andCtC\_\{t\}/FtF\_\{t\}are the per\-targetmin\\minat call=1 andmax\\maxat the run end, taken over a figure\-specific method set: knobs∪\\cupbest\_of\_nfor Fig\.[5](https://arxiv.org/html/2607.02834#S3.F5), full method∪\\cupsearch baselines for Fig\.[6](https://arxiv.org/html/2607.02834#S4.F6)\. Per\-bar mean and symmetric SEM are pooled over \(target, seed\) draws ofliftt\\mathrm\{lift\}\_\{t\}\.

### B\.2Wall\-time accounting

Wall time for a single run decomposes as

twall=N⋅coracle\+Cmethod,t\_\{\\text\{wall\}\}\\;=\\;N\\cdot c\_\{\\text\{oracle\}\}\+C\_\{\\text\{method\}\},\(4\)whereNNis the number of oracle calls,coraclec\_\{\\text\{oracle\}\}is the per\-call oracle cost, andCmethodC\_\{\\text\{method\}\}collects all method\-side compute \(generation, fine\-tuning steps, surrogate updates\)\. Reporting against both axes \(oracle calls and GPU hours\) reveals whether one technique dominates another regardless of oracle cost, or only past a thresholdcoracle⋆c\_\{\\text\{oracle\}\}^\{\\star\}at which the two curves cross\. The retroactive100×100\\timesoracle\-cost rescaling used throughout the appendix figures simulates a switch from a fast oracle \(FA\) to a slow oracle \(Boltz\-2\-class\) by scalingcoraclec\_\{\\text\{oracle\}\}in this decomposition without re\-running the experiments\.

### B\.3Backbone and oracle internals

The GenMol checkpoint we use is the released MDLM\-style masked discrete diffusion model over SAFE\-tokenized SMILES\[[28](https://arxiv.org/html/2607.02834#bib.bib41),[37](https://arxiv.org/html/2607.02834#bib.bib63)\]with vocabulary size 1882 \(1880 SAFE tokens plus mask and end\-of\-sequence\)\. The forward noise schedule and architecture are unchanged from the release; only the policy weightsθ\\thetaare updated\. A FlashAffinity call decomposes as: parse the SMILES with RDKit; embed a 3D conformer with ETKDGv3 \(one retry on failure\); extract per\-atom features with the FA torchdrug parser; score the \(pocket, ligand\) pair with the released FA checkpoint\. Per\-pocket ESM3 protein features are pre\-computed once and cached\. Each FA call takes tens of milliseconds on the A100; Boltz\-2 calls take roughly2020seconds on the same hardware, two orders of magnitude slower\.

### B\.4Run logs

The compute\-breakdown plots \(Appendix Figs\.[25](https://arxiv.org/html/2607.02834#A4.F25)and[28](https://arxiv.org/html/2607.02834#A4.F28)\) and the retroactive100×100\\timesoracle\-cost rescaling are computed offline from two log files written during every run\.oracle\_timeline\.jsonlhas one line per oracle call \(cumulative call count, SMILES, score, best\-so\-far, elapsed seconds, RDKit\-derived QED and SA, epoch index\)\.active\_loop\_log\.jsonlhas one line per active\-loop round \(round number, top\-1 best\-so\-farf⋆f^\{\\star\}, mean ensemble disagreementσ\\sigma, Spearmanρ\\rhobetween ensemble and oracle on a held\-out20%20\\%of the buffer, cumulative wall seconds, and per\-phase wall timestgen,tsurrogate,toracle,tddpp,tensemblet\_\{\\text\{gen\}\},t\_\{\\text\{surrogate\}\},t\_\{\\text\{oracle\}\},t\_\{\\text\{ddpp\}\},t\_\{\\text\{ensemble\}\}\)\.

### B\.5Implementation subtleties

Three details are not derivable from Algorithm[1](https://arxiv.org/html/2607.02834#alg1)or Table[2](https://arxiv.org/html/2607.02834#A2.T2)and matter for reproducing our numbers\.

#### Gradient detach in model debiasing\.

log⁡pθ​\(x\)\\log p\_\{\\theta\}\(x\)in Eq\.[2](https://arxiv.org/html/2607.02834#S2.E2)is treated as a constant: gradient is not propagated through the debiasing term\. Passing gradient through it turns the loss into an entropy\-regularized policy gradient and breaks the posterior\-matching guarantee of DDPP\-LB\.

#### Thompson noise scope\.

At acquisition we draw oneε∼𝒩​\(0,1\)\\varepsilon\\sim\\mathcal\{N\}\(0,1\)per candidate, shared across allJJensemble heads, not one per \(candidate, head\) pair\. The per\-candidate scalar acts on the ensemble disagreementσi\\sigma\_\{i\}and is what produces the diversity push described in Sec\.[4](https://arxiv.org/html/2607.02834#S4)\.

#### CVaR threshold refresh\.

The thresholdτ\\tauis recomputed once per active\-loop epoch from the empiricalqq\-quantile of buffer rewards, not per gradient step\. Below\-threshold samples remain in the gradient with their shaped reward floored atϵ=10−8\\epsilon=10^\{\-8\}, which under the DDPP log\-reward becomes a strong negative signal that pushesqθq\_\{\\theta\}away from them rather than dropping them entirely\.

### B\.6Knob interactions: acquisition vs\. reward shaping

This subsection collects the longer descriptions of how Thompson\-sampling acquisition and model\-debiasing reward shaping individually act on the loop, moved here from Sec\.[2\.1](https://arxiv.org/html/2607.02834#S2.SS1)for brevity\.

#### Thompson sampling vs\. deterministic top\-μ\\muselection\.

The contrast with deterministic top\-μ\\muselection is sharp \(Fig\.[2](https://arxiv.org/html/2607.02834#S2.F2.5)\): under deterministic selection a candidatecBc\_\{B\}withμB<μA\\mu\_\{B\}<\\mu\_\{A\}never gets oracle\-evaluated, no matter how uncertain it is, so the training data\(x,r​\(x\)\)\(x,r\(x\)\)that downstream finetuning sees is concentrated on the current surrogate argmax\. As the ensemble disagreementσB\\sigma\_\{B\}grows away from zero, the Thompson rule smoothly transitions from this hard\-greedy regime to a stochastic exploration bonus, the standard way to guard against local optima\[[5](https://arxiv.org/html/2607.02834#bib.bib51)\]\. The acquisition step is itself modifying which samples frompθtp\_\{\\theta\_\{t\}\}enter the evaluated set and therefore which data the gradient computes against\.

#### Model debiasing as training\-dynamics shaping\.

At the outer\-loop level,xxnames the candidate being trained on; inside the diffusion objective, this term may be evaluated on the noised or partially denoised state used by the loss\. Model debiasing therefore acts on the diffusion training dynamics rather than introducing a separate density estimator over completed molecules\. This term favors candidates that are high reward but not already high likelihood under the current generator\. It therefore complements Thompson sampling rather than replacing it: Thompson sampling explores uncertainty in the reward surrogate before oracle evaluation, while model debiasing reshapes the training signal after oracle evaluation to avoid repeatedly reinforcing the generator’s current modes\. Fig\.[3](https://arxiv.org/html/2607.02834#S2.F3)illustrates the redistribution: under the unshaped reward the dominant prior mode keeps absorbing gradient mass; with debiasing the smaller modes get lifted above their prior heights\.

### B\.7Hyperparameters

Table[2](https://arxiv.org/html/2607.02834#A2.T2)lists the hyperparameter values used throughout the controlled experiments and the final\-comparison runs\. Defaults follow the recommended settings from the original DDPP\-LB and VIDD references where applicable; values for the active\-loop wrappers \(M,K,GM,K,G, ensemble size and training schedule\) follow our active\-loop reference implementation\. We did not run an exhaustive per\-method hyperparameter sweep\. The design space we ablate has thirteen\-plus methods \(Table[1](https://arxiv.org/html/2607.02834#S2.T1)\) crossed with five seeds and a two\-hour FA budget per cell, and the cost grows by another order of magnitude under the Boltz\-2 oracle; tuning each cell independently would multiply that cost by the size of the per\-method search grid and would also confound the design\-space comparison, since improvements would no longer be attributable to the knob under study versus to a more aggressive sweep on one cell than another\. We instead control everything except the knob being ablated, hold all other hyperparameters at the active\-loop reference defaults, and run every cell at the same wall budget\. The trade\-off is explicit: we measure the effect of each design choice at sensible\-default settings rather than at each method’s peak\-tuned setting\. Per\-knob sensitivity to the hyperparameter most likely to matter \(the invalid\-SMILES penalty magnitude\) is reported separately in Appendix Fig\.[40](https://arxiv.org/html/2607.02834#A6.F40)\.

GroupHyperparameterValueWall budget and seedsSeeds\{0,1,2,3,42\}\\\{0,1,2,3,42\\\}Wall budget per seed \(FA\)2 hoursWall budget per seed \(Boltz\-2\)10 hoursHardware1×\\timesNVIDIA A100 80GBActive loopGeneration batchMM1000Oracle batchKK25Fine\-tuning steps per roundGG50Buffer capacity10 000Buffer evictionpriority \(lowest reward first\)Fresh\-fraction0\.25Reward shapingCVaR quantileqq0\.8Reward floor10−810^\{\-8\}Model\-debiasing weightγ\\gamma1\.0Invalid\-SMILES penaltyrinvr\_\{\\text\{inv\}\}−5\-5Surrogate ensemble \(Thompson\)Ensemble sizeJJ10Hidden width256Training epochs / round100Learning rate10−310^\{\-3\}Mini\-batch size256Fine\-tuning \(DDPP\-LB\)Backbone learning rate5×10−55\\times 10^\{\-5\}LogZ head learning rate5×10−45\\times 10^\{\-4\}EMA decay0\.999Backbone warmup \(LogZ\-only steps\)200Search baselines \(when used as a knob\)Beam:NNbeams,LLbranches,KKstepsN=8N\{=\}8,L=4L\{=\}4,K=5K\{=\}5MCTS:cUCTc\_\{\\text\{UCT\}\},LL,KK1\.01\.0,44,55DFKC: particles,β\\beta, score\_start8, 2\.0, 0\.5Table 2:Hyperparameters used across all reported experiments\.Values are the active\-loop reference defaults; only the knob being ablated is varied within a comparison\.
### B\.8End\-to\-end reproduction recipe

This subsection lists, in order, every concrete asset and step required to reproduce one cell of our experimental table; combined with Algorithm[1](https://arxiv.org/html/2607.02834#alg1)\(the abstract loop\) and Table[2](https://arxiv.org/html/2607.02834#A2.T2)\(the numerical settings\), this is sufficient to rebuild the environment and re\-run\.

#### Software environment\.

Python 3\.10\. PyTorch 2\.x with CUDA 12\.x\. RDKit 2024\.x \(used for SAFE encoding/decoding, validity checking viaMolFromSmiles, and ETKDGv3 conformer embedding\)\. NumPy 1\.x\. Hydra for configuration\.boltzpackage version 2\.2\.1 \(PyPI\) for the Boltz\-2 oracle path\.

#### Pretrained assets \(downloaded once, cached locally\)\.

\(i\) TheGenMolmasked discrete\-diffusion checkpoint fromLeeet al\.\[[19](https://arxiv.org/html/2607.02834#bib.bib17)\]\(ICML 2025; arXiv:2501\.06158\), released by the authors at[https://github\.com/arielyyd/genmol](https://github.com/arielyyd/genmol); we use the second\-vocab\-fix release shipped asmodel\_v2\.ckpt\(1397 MB, vocabulary18821882=18801880SAFE tokens plus mask and end\-of\-sequence\)\. Forward noise schedule and architecture are unchanged from the release; only the policy weightsθ\\thetaare updated\. The model was pretrained on theSAFE dataset V2\([https://huggingface\.co/datasets/datamol\-io/safe\-drugs](https://huggingface.co/datasets/datamol-io/safe-drugs)\); we do not retrain\. \(ii\) TheFlashAffinity\(FA\) binary classifier checkpointFlashAffinity/checkpoints/binary\_1\.ckptfromJianget al\.\[[16](https://arxiv.org/html/2607.02834#bib.bib34)\]together with its torchdrug\-style featurizer; we score the binary head only\. \(iii\) For Boltz\-2 oracle runs, the released Boltz\-2 weights downloaded via theboltzpackage’s bootstrap \(structure checkpointboltz2\_conf\.ckpt\+ affinity headboltz2\_aff\.ckpt\),Passaroet al\.\[[30](https://arxiv.org/html/2607.02834#bib.bib6)\]\. \(iv\) Per\-pocketESM3protein\-feature tensors, computed once per PDB target and cached to disk \(FlashAffinity/data/protein\_test/repr/esm3\.lmdb\) so that no oracle call recomputes them\.

#### Per\-target inputs\.

Each PDB target \(2VT4, 5SDV, 6CM4, 7BKC, 7C7M, 7YLL\) contributes the receptor structure plus the binding\-pocket specification used by the FA / Boltz\-2 oracle\. We use the publicly released coordinates without modification\.

#### Seed handling\.

A run is identified by\(method, target, seed\)with seed∈\{0,1,2,3,42\}\\in\\\{0,1,2,3,42\\\}\. At process start the seed is propagated to:torch\.manual\_seed,torch\.cuda\.manual\_seed\_all,numpy\.random\.seed, Python’s built\-inrandom\.seed, and the RDKit conformer\-embedding seed passed toETKDGv3\.

#### Per\-cell launcher\.

Each cell is dispatched by a single launcher that takes\(method, target, seed, wall\_budget\_sec\)as arguments\. The launcher loads the GenMol checkpoint, attaches the configured component slots from Sec\.[2](https://arxiv.org/html/2607.02834#S2)\(acquisition rule, reward\-shape modifier, replay strategy, validity penalty, finetuning objective\), opens the two log files described in App\.[B](https://arxiv.org/html/2607.02834#A2)\(oracle\_timeline\.jsonlandactive\_loop\_log\.jsonl\), and runs the loop in Algorithm[1](https://arxiv.org/html/2607.02834#alg1)\.

#### Per\-round step counts\.

Each active\-loop epoch generatesM=1000M\{=\}1000candidate molecules from the currentqθq\_\{\\theta\}, scores them with the surrogate ensemble, picks the top\-K=25K\{=\}25for oracle evaluation under Thompson sampling, evaluates them, pushes each\(x,r\)\(x,r\)pair into the buffer \(withr=rinvr\{=\}r\_\{\\text\{inv\}\}for RDKit\-failed candidates\), refreshes the CVaR thresholdτ\\taufrom the buffer’s currentqq\-quantile, and runsG=50G\{=\}50DDPP\-LB gradient steps on stratified mini\-batches drawn from the buffer with fresh\-fraction0\.250\.25\. The ensemble surrogate is refit at the end of each epoch on all accumulated\(x,r\)\(x,r\)pairs for100100inner training epochs at learning rate10−310^\{\-3\}and mini\-batch size256256\.

#### Wall\-budget enforcement\.

A monotonic clock is checked after each round\. If the elapsed wall time exceeds the configuredwall\_budget\_sec\(2 h for FA, 10 h for Boltz\-2; a44\-h cap was applied to the small\-molecule batch1 runs documented in our latest data drop\), the loop terminates after writing one finalactive\_loop\_log\.jsonlentry and aresults\.jsonsummary\. If the runner is killed externally, only the per\-epoch and per\-call log files are guaranteed to be flushed; the partial\-summary case is described in App\.[B](https://arxiv.org/html/2607.02834#A2)\.

#### Output artifacts\.

Every cell produces a directory<TARGET\>/<METHOD\>\_s<SEED\>/containing exactly three files:results\.json\(post\-run summary or partial stub\),active\_loop\_log\.jsonl\(one JSON line per epoch\), andoracle\_timeline\.jsonl\(one JSON line per oracle call\)\. All paper figures are produced from these three file types alone; no additional artifacts are needed for plot regeneration\.

### B\.9Sources of stochasticity beyond the seed

Setting the seed described above pins all explicitly\-RNG\-driven sampling decisions: whichMMcandidates the diffusion sampler draws each epoch, the Thompson noiseε\\varepsilonat acquisition, mini\-batch composition during fine\-tuning, ensemble weight initialization, and the conformer seed handed to RDKit\. Several other sources of variation are*not*pinned by the seed, and we list them here so that bit\-level reproducibility is not expected even between two runs with identical\(method, target, seed\)\.

#### GPU floating\-point nondeterminism\.

cuBLAS / cuDNN convolution and matmul algorithms make algorithm\-selection decisions based on workspace memory and tensor shapes, and several reduction kernels \(e\.g\.,scatter\_add\_\) use atomic adds whose summation order is not deterministic across kernel launches\. We do not enabletorch\.use\_deterministic\_algorithms\(True\)because doing so would substitute slower kernels and change the wall\-time profile that our compute\-breakdown plots are measured against\. The practical effect is small bit\-level drift inqθq\_\{\\theta\}’s logits across nominally\-identical runs, which can cascade into different sampledxxat later epochs\.

#### ETKDGv3 conformer embedding\.

The RDKit conformer\-embedding routine has its own random restart policy when the first attempt fails to converge\. We pass a fixed embedding seed \(derived from the run seed\), but the number of restarts and the resulting conformer can still differ across re\-evaluations of the same SMILES if the underlying force\-field convergence is borderline\. Most molecules are unaffected; a small fraction \(typically<1%<1\\%\) yield slightly different FA scores on re\-evaluation\.

#### Async oracle ordering\.

Oracle calls are issued in batches ofKKbut completed asynchronously when the oracle is GPU\-bound \(Boltz\-2 in particular\)\. The order in which\(x,r\)\(x,r\)pairs land in the buffer therefore depends on per\-call latency, not just on the order they were dispatched\. This affects: \(i\) the buffer’s priority\-eviction tie\-breaks \(lowest reward first; ties broken by arrival order\), \(ii\) the empiricalqq\-quantile that the CVaR threshold refresh uses \(since the threshold is computed from buffer state at refresh time\), and \(iii\) which candidates appear in which mini\-batch under stratified sampling\.

#### Wall\-clock\-driven decisions\.

The active loop is wall\-budget\-bounded, not round\-count\-bounded\. Two runs with the same seed on different hardware \(or on the same hardware under different system load\) will execute a different number of epochs before the budget cutoff fires, and therefore terminate at different points along theqθq\_\{\\theta\}trajectory\. The 4\-hour cap on the small\-molecule batch1 runs makes this effect visible directly in the per\-target completion table reported in App\.[F](https://arxiv.org/html/2607.02834#A6)\.

#### DataLoader worker ordering\.

When the buffer’s mini\-batch sampler runs with PyTorchDataLoaderworkers\>0\>0, the order in which prefetched batches arrive at the training loop depends on worker\-process scheduling\. We use a single\-worker loader for the fine\-tuning step to remove this source, but the surrogate\-ensemble retraining loop uses multiple workers for throughput; ensemble outputs are therefore not bit\-exact across runs\.

#### Practical implication\.

For all reported metrics in the paper \(top\-1, top\-10%, validity, etc\.\) we average across five seeds and report95%95\\%CI bands\. The bit\-level drift above is dominated by the seed\-driven variation in every comparison we have inspected; the conclusions in Sec\.[4](https://arxiv.org/html/2607.02834#S4)\(which knobs are complementary, where the failure modes fire\) are stable under the level of nondeterminism listed here\. We surface these sources nonetheless because anyone trying to bit\-match a single curve from our logs will not be able to, even with the seed and full hyperparameter set in hand\.

## Appendix CMethod and concept illustrations

This appendix collects schematic diagrams for the methods and concepts introduced in Sec\.[2](https://arxiv.org/html/2607.02834#S2)and Sec\.[4](https://arxiv.org/html/2607.02834#S4)\. Each diagram is referenced from the body where the corresponding concept is first introduced; this section serves as a single place to look up the visual\.

### C\.1Inference\-time search families

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/diagram_uncond.png)Figure 8:Unconditional sampling\.The simplest baseline: sample frompθp\_\{\\theta\}and score every sample under the oracle, with no exploration policy\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/diagram_beam.png)Figure 9:Beam search\.At each denoising step, the top\-KKpartial trajectories by score are kept and expanded; the rest are pruned\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/diagram_mcts.png)Figure 10:Monte Carlo Tree Search\.Each node is a partial denoising state; UCB selects the next expansion based on visit counts and value estimates from rollouts\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/diagram_dfkc.png)Figure 11:Discrete Feynman\-Kac correctors \(DFKC\) / SMC\.A particle population is reweighted at each denoising step by a tilt proportional to the \(estimated\) reward, then resampled when effective sample size drops below threshold\.
### C\.2Surrogate and Tweedie shortcuts

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/diagram_surrogate.png)Figure 12:Surrogate screening\.A cheap learned surrogater^​\(xt\)\\hat\{r\}\(x\_\{t\}\)scores intermediate states in place of the expensive oracle, reducing per\-output oracle cost fromO​\(L\)O\(L\)toO​\(1\)O\(1\)\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/diagram_tweedie.png)Figure 13:Tweedie estimator\.The posterior\-mean denoised statex^0=𝔼​\[x0∣xt\]\\hat\{x\}\_\{0\}=\\mathbb\{E\}\[x\_\{0\}\\mid x\_\{t\}\]is computed in a single denoising pass and scored directly, replacing a full rollout\[[9](https://arxiv.org/html/2607.02834#bib.bib58),[6](https://arxiv.org/html/2607.02834#bib.bib59)\]\.
### C\.3Fine\-tuning objective and exploration knobs

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/diagram_ddpp.png)Figure 14:DDPP\-LB loss\.Trajectory\-balance squared residual matchingqθq\_\{\\theta\}to the reward\-tilted posteriorp⋆∝ppre⋅rp^\{\\star\}\\propto p\_\{\\text\{pre\}\}\\cdot r, with a learned partition\-function headZ^ϕ\\hat\{Z\}\_\{\\phi\}trained jointly withθ\\theta\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/synthetic_distribution_evolution.png)Figure 15:Reward shapes on a synthetic 25\-mode landscape\.\(a\) Priorpanchorp\_\{\\text\{anchor\}\}: 25 roughly Gaussian modes\. \(b\) Each mode is assigned a score in\[0,1\]\[0,1\]\. \(c\) Vanilla DDPP\-LB with raw rewardr​\(x\)r\(x\)puts mass on modes proportional to their reward weight \(ideal under unlimited sampling, but compute\-inefficient under limited oracle calls\)\. \(d\) CVaR shaping concentrates most mass on ther≥0\.3r\\geq 0\.3modes, enabling deeper search of the top\-reward modes\. \(e\) Model debiasinglog⁡r~​\(x\)=log⁡r~cvar​\(x\)−γ​log⁡pθt​\(x\)\\log\\tilde\{r\}\(x\)=\\log\\tilde\{r\}\_\{\\mathrm\{cvar\}\}\(x\)\-\\gamma\\log p\_\{\\theta\_\{t\}\}\(x\)flattens probability mass across rewarded modes, allowing rare high\-scoring molecules in less\-likely modes to be discovered\. \(f\) CVaR\+\+model debiasing yields a roughly uniform distribution over the top\-τ\\taumodes, preventing CVaR’s early collapse while preserving its tail focus\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/diagram_thompson.png)Figure 16:Thompson sampling\.Acquisition rule: a per\-candidate sampled surrogate rewardr^i=μi\+εi​σi\\hat\{r\}\_\{i\}=\\mu\_\{i\}\+\\varepsilon\_\{i\}\\sigma\_\{i\}over the ensemble’s mean/disagreement, oracle\-evaluating the top\-KKbyr^i\\hat\{r\}\_\{i\}\. Different draws ofε\\varepsilonfavor different molecular regions\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/diagram_buffer.png)Figure 17:Replay buffer\.Each generated\(x,r\)\(x,r\)pair is stored in a fixed\-capacity buffer with priority eviction; mini\-batches are drawn from the buffer \(with a fixed fresh\-fraction\) so the gradient sees a time\-averaged version of the policy distribution rather than the sharp on\-policy one\.
### C\.4Active loop

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/diagram_active_loop.png)Figure 18:Active loop\.The high\-level structure shared by every fine\-tuning method we evaluate: generate candidates, score them under the oracle, then refine the backbonepθp\_\{\\theta\}and/or the surrogate before generating again\.Algorithm 1Online active loop with the five composable knobs \(finetuner\-agnostic\)\. Schematic in Fig\.[1](https://arxiv.org/html/2607.02834#S1.F1)\.1:Pretrained

pprep\_\{\\text\{pre\}\}, oracle

r​\(⋅\)r\(\\cdot\), fine\-tuning loss

ℒft\\mathcal\{L\}\_\{\\text\{ft\}\}, wall budget

TwallT\_\{\\text\{wall\}\}
2:Generation batch

MM, oracle batch

KK, fine\-tuning steps per round

GG
3:Reward\-shape: CVaR quantile

qq, debiasing

γ\\gamma, invalid penalty

rinvr\_\{\\text\{inv\}\}; ensemble size

JJ
4:Initialize

pθ0←pprep\_\{\\theta\_\{0\}\}\\leftarrow p\_\{\\text\{pre\}\}; buffer

ℬ←∅\\mathcal\{B\}\\leftarrow\\emptyset; ensemble

\{r^ϕj\}j=1J\\\{\\hat\{r\}\_\{\\phi\_\{j\}\}\\\}\_\{j=1\}^\{J\}random

5:whilewall time

<Twall<T\_\{\\text\{wall\}\}do

6:

Ct←C\_\{t\}\\leftarrowsample

MMcandidates from

pθtp\_\{\\theta\_\{t\}\}
7:

\(μi,σi\)←\(\\mu\_\{i\},\\sigma\_\{i\}\)\\leftarrowensemble mean/std on

xi∈Ctx\_\{i\}\\in C\_\{t\}
8:

r^i∼𝒩​\(μi,σi2\)\\hat\{r\}\_\{i\}\\sim\\mathcal\{N\}\(\\mu\_\{i\},\\sigma\_\{i\}^\{2\}\)⊳\\triangleright*\(4\) Thompson sampling*

9:

St←S\_\{t\}\\leftarrowtop\-

KKof

CtC\_\{t\}by

r^i\\hat\{r\}\_\{i\}
10:for

x∈Stx\\in S\_\{t\}do

11:

rx←rinvr\_\{x\}\\leftarrow r\_\{\\text\{inv\}\}if

xxfails RDKit parseelse

r​\(x\)r\(x\)⊳\\triangleright*\(3\) invalid\-SMILES penalty*

12:

ℬ←ℬ∪\{\(x,rx\)\}\\mathcal\{B\}\\leftarrow\\mathcal\{B\}\\cup\\\{\(x,r\_\{x\}\)\\\}⊳\\triangleright*\(5\) replay buffer*

13:endfor

14:

τ←Qq​\(\{r:\(⋅,r\)∈ℬ\}\)\\tau\\leftarrow Q\_\{q\}\(\\\{r:\(\\cdot,r\)\\in\\mathcal\{B\}\\\}\)
15:for

g=1,…,Gg=1,\\ldots,Gdo

16:sample mini\-batch

ℬb⊂ℬ\\mathcal\{B\}\_\{b\}\\subset\\mathcal\{B\}
17:

r∗​\(x\)←\(rx−τ\)\+/\(1−q\)r^\{\*\}\(x\)\\leftarrow\(r\_\{x\}\-\\tau\)\_\{\+\}/\(1\-q\)⊳\\triangleright*\(1\) CVaR shaping*

18:

log⁡r~​\(x\)←log⁡r∗​\(x\)−γ​log⁡pθt​\(x\)\\log\\tilde\{r\}\(x\)\\leftarrow\\log r^\{\*\}\(x\)\-\\gamma\\log p\_\{\\theta\_\{t\}\}\(x\)⊳\\triangleright*\(2\) model debiasing*

19:update

pθtp\_\{\\theta\_\{t\}\}via

ℒft\\mathcal\{L\}\_\{\\text\{ft\}\}on

ℬb\\mathcal\{B\}\_\{b\}using

log⁡r~\\log\\tilde\{r\}as the per\-sample reward

20:endfor

21:refit ensemble

\{r^ϕj\}\\\{\\hat\{r\}\_\{\\phi\_\{j\}\}\\\}on

ℬ\\mathcal\{B\}
22:endwhile

23:return

arg⁡maxx∈ℬ⁡rx\\arg\\max\_\{x\\in\\mathcal\{B\}\}r\_\{x\}

## Appendix DSearch comparisons and compute breakdowns

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/final_comparison.png)Figure 19:Final comparison\.Top\-10 mean FA score vs\. \(a\) cumulative oracle calls and \(b\) GPU hours, comparing the full online active loop \(Our method: DDPP\-LB\+\+CVaR\+\+debiasing\+\+Thompson\+\+buffer\+\+invalid penalty\) against the strongest pure\-search baseline \(Beam Search\) and untreated online finetuning \(DDPP Online No\-CVaR\)\. The full stack dominates both baselines under matched compute on the FA oracle\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/surrogate_search_a.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/surrogate_search_b.png)

Figure 20:Surrogate screening on inference\-time search\.Beam, MCTS, and DFKC each compared against their surrogate\-pre\-filter variant \(cheap MLP scores all candidates, oracle only sees the top\-KK\)\. Solid: oracle\-on\-everything\. Dashed: surrogate pre\-filter\. Seed\-averaged \(n=5n\{=\}5\) top\-1 best FA against cumulative oracle calls \(left\) and GPU hours \(right\)\. Surrogate screening dominates plain search on both axes across all three families: each oracle call is spent on a candidate that already cleared the cheap filter, so the per\-call yield is higher\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/tweedie_search_a.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/tweedie_search_b.png)

Figure 21:Tweedie vs\. completion \(inference\-time search\)\.Same beam/MCTS/DFKC families as above\. Solid: completion \(full denoising rollout to score\)\. Dashed: Tweedie posterior\-mean estimatex^0=𝔼​\[x0∣xt\]\\hat\{x\}\_\{0\}=\\mathbb\{E\}\[x\_\{0\}\\mid x\_\{t\}\]in place of the rollout\. Seed\-averaged \(n=5n\{=\}5\)\. The Tweedie estimator’s noise in the discrete setting is reflected directly in score: every Tweedie variant underperforms its completion counterpart, and the gap is largest for DFKC\. The diffusion\-pass savings do not translate into top\-1 oracle score because the per\-step score estimate is too noisy to steer the search\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/fig_tweedie_inference_compute.png)Figure 22:Compute breakdown for inference\-time search, completion vs\. Tweedie\.Stacked bars are seed\-averaged \(n=5n\{=\}5\) wall\-clock spend at the22\-hour FA budget; oracle compute is estimated asncalls×cFAn\_\{\\text\{calls\}\}\\times c\_\{\\text\{FA\}\}withcFA≈50c\_\{\\text\{FA\}\}\\approx 50ms \(matched to the per\-phase log on the active runs\)\. Tweedie shifts the per\-family balance but does not recover enough wall time to compensate for the score loss in Fig\.[21](https://arxiv.org/html/2607.02834#A4.F21): for MCTS it produces more oracle calls in the same budget, for DFKC it produces fewer\. Generation dominates total wall time in every cell\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/online_offline_a.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/online_offline_b.png)

Figure 23:Online vs\. offline DDPP\-LBon top\-1 best FA score so far, against cumulative oracle calls \(left\) and GPU hours \(right\)\. Both runs use the same DDPP\-LB loss and no reward shaping; the only difference is whether the buffer of\(x,r​\(x\)\)\(x,r\(x\)\)pairs is collected up front and trained on once \(offline\) or refreshed continuously by the currentqθq\_\{\\theta\}between gradient steps \(online\)\. Seed\-averaged \(n=5n\{=\}5\) bands\. Online dominates on both axes: the larger number of gradient updates and the larger volume of policy\-correlated data pushed through them push the sampling distribution closer top⋆p^\{\\star\}than a one\-shot offline schedule does\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/tweedie_active_a.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/tweedie_active_b.png)

Figure 24:Tweedie vs\. completion \(active hybrids\)\.Same Tweedie/completion comparison but applied inside the online DDPP\-LB hybrids of Sec\.[4](https://arxiv.org/html/2607.02834#S4)\(beam and MCTS only; DFKC is not paired here\)\. Solid: completion\. Dashed: Tweedie\. Seed\-averaged \(n=5n\{=\}5\)\. The Tweedie variants underperform their completion counterparts here as well, despite the saved generation passes nominally freeing time for fine\-tuning\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/fig_search_overhead.png)Figure 25:Compute breakdown under a 2\-hour wall budget for plain online finetuning \(left\) versus online finetuning augmented with beam search \(middle\) or MCTS \(right\)\. All numbers are seed\-averaged \(n=5n\{=\}5\)\. Plain online finetune spends roughly half its budget on fine\-tuning \(∼\\sim1\.0 hr\); both search variants spend almost the entire budget on candidate generation, leaving nearly no time for fine\-tuning gradient steps \(∼\\sim0\.04 hr for beam,∼\\sim0\.21 hr for MCTS\)\. The reduction in fine\-tuning wall time is the proximate reason search\-augmented online finetuning fails to improve over the plain variant\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/fig_tweedie_active_compute.png)Figure 26:Compute breakdown for active hybrids, completion vs\. Tweedie\.Per\-phase wall times pulled fromactive\_loop\_log\.jsonl; seed\-averaged \(n=5n\{=\}5\)\. Generation dominates for both completion and Tweedie variants, leaving very little wall time for fine\-tuning \(≤0\.1\\leq 0\.1hr\) regardless of whether the search uses the Tweedie shortcut or completion\. The fine\-tuning budget reduction observed for completion\-based hybrids in Fig\.[25](https://arxiv.org/html/2607.02834#A4.F25)is not relieved by the Tweedie shortcut\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/oracle_rescale_cheaper.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/oracle_rescale_expensive.png)

Figure 27:Retroactive oracle\-cost rescaling on online search hybrids\.Same three online methods \(ddpp\_online\_cvarvs\.ddpp\_beam\_cvarvs\.ddpp\_mcts\_cvar\) plotted on the GPU\-hours axis with the per\-call oracle cost rescaled by1/1001/100\(left\) and by100100\(right\)\. Rescaling shifts the GPU\-hours axis by four orders of magnitude in either direction without altering the relative ordering: plain online DDPP\-LB matches or exceeds both search\-augmented variants under both regimes, supporting the claim in Sec\.[4](https://arxiv.org/html/2607.02834#S4)that the marginal value of inference\-time search on top of online finetuning is robust to oracle expense\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/fig9_ab.png)Figure 28:Compute breakdown\.Per\-method decomposition of wall\-clock spend at standard FA oracle cost \(left,1×1\\times\) and under a retroactive100×100\\timesoracle\-cost rescaling \(right\)\. Pure search spends almost all of its budget on oracle calls, so its absolute compute scales nearly linearly withcoraclec\_\{\\text\{oracle\}\}\. Online finetuning amortizes generation and gradient updates across the same oracle budget, so its share of oracle compute is smaller and the absolute compute under the100×100\\timesrescaling grows much less\.
## Appendix EPer\-cell secondary\-metric traces

These panels report sliding\-window means of pairwise Tanimoto diversity, QED, SA, and SMILES validity for the per\-knob ablation cells of Appendix[F](https://arxiv.org/html/2607.02834#A6)\. Window size 300, stride 200; one curve per cell, x\-axis = molecules evaluated\.

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/aux_rewardshape_validity.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/aux_rewardshape_tanimoto.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/aux_rewardshape_qed.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/aux_rewardshape_sa.png)

Figure 29:CVaR ablation, secondary metrics\.Same four cells as Fig\.[33](https://arxiv.org/html/2607.02834#A6.F33)\(offline / online×\\timesCVaR\-on / CVaR\-off\)\. Top\-left: validity\. Top\-right: Tanimoto diversity\. Bottom\-left: QED\. Bottom\-right: SA\. The top\-1 FA gains in Fig\.[33](https://arxiv.org/html/2607.02834#A6.F33)are not accompanied by validity collapse or QED/SA degradation, ruling out reward hacking on the CVaR knob\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/aux_debiasing_validity.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/aux_debiasing_tanimoto.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/aux_debiasing_qed.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/aux_debiasing_sa.png)

Figure 30:Model\-debiasing ablation, secondary metrics\.Same two cells as Fig\.[34](https://arxiv.org/html/2607.02834#A6.F34)\(online no\-CVaR with vs\. without Density Entropy Regularization\)\. Same panel layout as above\. The top\-1 FA gain from debiasing is not accompanied by validity or QED/SA degradation\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/aux_buffer_validity.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/aux_buffer_tanimoto.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/aux_buffer_qed.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/aux_buffer_sa.png)

Figure 31:Replay\-buffer ablation, secondary metrics\.Same four cells as Fig\.[37](https://arxiv.org/html/2607.02834#A6.F37)\(buffer×\\timesCVaR\)\. Top\-left: validity\. Top\-right: Tanimoto diversity\. Bottom\-left: QED\. Bottom\-right: SA\. The buffer\-on cells preserve diversity and SA more reliably than the buffer\-off cells, consistent with the buffer’s role as an implicit normalizer over the historical sample distribution \(Sec\.[4](https://arxiv.org/html/2607.02834#S4)\)\.The validity panel for the invalid\-SMILES penalty sweep is reported directly in Fig\.[40](https://arxiv.org/html/2607.02834#A6.F40); per\-cell QED/SA/Tanimoto for that sweep are not informative because the cells differ only in the validity\-restoration penalty\.

#### Run\-end anchor\-NLL distribution\.

The per\-knob traces above measure how the secondary metrics evolve over time\. A complementary view is the end\-of\-run snapshot of how far each cell’s policyqθq\_\{\\theta\}has drifted from the frozen priorpprep\_\{\\text\{pre\}\}, measured by the anchor NLL−log⁡ppre​\(x0∣xt\)\-\\log p\_\{\\text\{pre\}\}\(x\_\{0\}\\mid x\_\{t\}\)averaged over masked tokens \(Fig\.[32](https://arxiv.org/html/2607.02834#A5.F32)\)\. Cells here are the leave\-one\-out variants from Section[4](https://arxiv.org/html/2607.02834#S4)\(full method minus one knob at a time\), not the per\-knob composition cells of Appendix[F](https://arxiv.org/html/2607.02834#A6)\.

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/anchor_nll_per_cell.png)Figure 32:Anchor\-NLL distribution per leave\-one\-out ablation cell\.Histogram of per\-token anchor NLL−log⁡ppre​\(x0∣xt\)\-\\log p\_\{\\text\{pre\}\}\(x\_\{0\}\\mid x\_\{t\}\)at run end, computed over all valid samples \(n in title\)\. Top\-left is the full method; the other five panels each remove one knob and overlay the resulting distribution against the full\-method reference \(blue\)\. Removing the density entropy regularization / debiasing knob \(bottom\-left\) collapses mass back toward the prior \(low anchor NLL\) far more than removing any other knob, identifying debiasing as the primary driver of distribution shift away frompprep\_\{\\text\{pre\}\}\. Removing CVaR \(top\-right\) reduces the high\-NLL tail, consistent with CVaR’s role in pushing the policy toward the upper\-reward region of the distribution\. The remaining knobs \(invalid penalty, replay buffer, Thompson\) leave the bimodal shape qualitatively intact\.

## Appendix FPer\-knob individual controlled experiments

Each figure isolates one design\-choice knob from group \(iii\) of Table[1](https://arxiv.org/html/2607.02834#S2.T1), holding the other knobs at their reference defaults \(Table[2](https://arxiv.org/html/2607.02834#A2.T2)\)\. Every figure is a3×23\\times 2grid: rows are top\-1 best\-so\-far, top\-10 mean, and top\-10% mean; columns are cumulative oracle calls \(left\) and GPU hours \(right\)\.

#### Reading the top\-10% row\.

For inference\-time search comparisons \(Appendix[D](https://arxiv.org/html/2607.02834#A4), e\.g\. Figs\.[20](https://arxiv.org/html/2607.02834#A4.F20)and[21](https://arxiv.org/html/2607.02834#A4.F21)\) the underlying generative distribution is fixed and only the selection procedure changes\. Top\-10% mean in those figures reflects sampling efficiency from a static distribution and saturates as the upper tail ofpprep\_\{\\text\{pre\}\}is exhausted\. In the active ablations below the policyqθq\_\{\\theta\}is itself updated, so a continued rise in top\-10% mean is direct evidence that the sampling distribution is shifting — not just that we are picking better samples from the same distribution\.

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/fig_cvar_3x2.png)Figure 33:CVaR ablation\.Four cells on DDPP\-LB: solid = CVaR\-on, dotted = CVaR\-off; green = offline, teal = online\. Seed\-averaged \(n=5n\{=\}5\) with±1\\pm 1std bands\. CVaR lifts all three metrics in both regimes; the gain on top\-10% mean \(bottom row\) for the online\+\+CVaR cell — which continues to climb to∼0\.5\\sim 0\.5rather than saturating — confirms that the CVaR\-shaped gradient shifts the policy distribution itself rather than only re\-ranking samples from a fixed distribution\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/fig_debiasing_3x2.png)Figure 34:Model\-debiasing ablation\.Two cells, both online DDPP\-LB with CVaR off: solid \(with debiasing\) vs\. dotted \(without\)\. Seed\-averaged \(n=5n\{=\}5\)\. Holding CVaR off isolates the debiasing knob from the upper\-tail reweighting in Fig\.[33](https://arxiv.org/html/2607.02834#A6.F33)\. Debiasing lifts top\-1 by∼\+0\.6\\sim\+0\.6and top\-10% mean by a comparable margin, again indicating a genuine shift inqθq\_\{\\theta\}and not just better sample selection\.
#### Cheap estimator vs\. LLaDA\-Alg\. 3\.

We compare five density\-entropy\-regularization variants: the single\-pass cheap estimator and LLaDA\-Alg\. 3 withnmc∈\{1,4,8,16\}n\_\{\\text\{mc\}\}\\in\\\{1,4,8,16\\\}, for both DDPP\-LB and VIDD on FA/2VT4\. Fig\.[35](https://arxiv.org/html/2607.02834#A6.F35)shows best\-so\-far oracle score per oracle call and per GPU hour\. The cheap estimator matches or exceeds all LLaDA variants on oracle\-call efficiency and is substantially faster per GPU hour due to requiring only a single forward pass\. Fig\.[36](https://arxiv.org/html/2607.02834#A6.F36)shows the distribution of NLL under the pretrained model for the top\-200 discovered molecules per variant, evaluated post\-hoc via LLaDA\-Alg\. 3 \(nmc=32n\_\{\\text\{mc\}\}\{=\}32\)\. The cheap estimator induces at least as much off\-prior shift as the LLaDA variants, confirming that the single\-pass approximation is a computationally sound substitute\.

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/negscore_estimator_ddpp.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/negscore_estimator_vidd.png)

Figure 35:Best\-so\-far oracle curves: cheap estimator vs\. LLaDA\-Alg\. 3 variants \(FA/2VT4, 2 hr wall\)\.Each subplot shows best FA score per oracle call \(left\) and per GPU hour \(right\) for five density\-entropy\-regularization variants: single\-pass cheap estimator \(black\) and LLaDA\-Alg\. 3 withnmc∈\{1,4,8,16\}n\_\{\\text\{mc\}\}\\in\\\{1,4,8,16\\\}\(colored\)\.Top:DDPP\-LB\.Bottom:VIDD\. The cheap estimator matches or exceeds LLaDA variants on oracle calls and is far more or at least equally efficient per GPU hour\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/negscore_nll_dist_ddpp.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/negscore_nll_dist_vidd.png)

Figure 36:NLL under the pretrained model for the top\-200 discovered molecules, per estimator \(FA/2VT4\)\.NLL evaluated post\-hoc via LLaDA\-Alg\. 3 \(nmc=32n\_\{\\text\{mc\}\}\{=\}32\); dashed line = median\.Top:DDPP\-LB\.Bottom:VIDD\. The cheap estimator produces distributions shifted at least as far off the prior as the LLaDA variants, validating that the single\-pass approximation achieves the intended exploration pressure\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/fig_buffer_2line_3x2.png)Figure 37:Replay\-buffer ablation\.Two cells, both online DDPP\-LB\+\+CVaR: solid \(with replay buffer\) vs\. dashed \(no buffer\)\. Seed\-averaged \(n=5n\{=\}5\)\. Top\-1 separates the cells modestly\. The clearest signal is in top\-10% mean \(bottom row\), where the buffer\-on cell continues to climb past−0\.4\-0\.4while the buffer\-off cell plateaus near−0\.7\-0\.7\. The buffer is shifting where the policy puts mass over the run, not merely re\-presenting a fixed set of samples to the gradient\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/fig_thompson_3x2.png)Figure 38:Thompson\-sampling ablation\.Pink: Thompson on top of online DDPP\-LB\+\+CVaR\. Cyan: same configuration with deterministic top\-KKselection by ensemble mean instead of Thompson draws\. Seed\-averaged \(n=5n\{=\}5\)\. Top\-1 is comparable; top\-10% mean \(bottom row\) is where the contrast is sharpest — Thompson reaches∼−0\.05\\sim\-0\.05vs\.∼−0\.4\\sim\-0\.4for deterministic selection, a clear distribution shift toward higher\-reward modes that comes from Thompson diversifying which candidates the oracle is spent on rather than from any change to the gradient itself\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/fig_penalty_3x2.png)Figure 39:Invalid\-SMILES penalty sweeprinv∈\{0,−1,−3,−5,−7\}r\_\{\\text\{inv\}\}\\in\\\{0,\-1,\-3,\-5,\-7\\\}\.On top of Approach A\+\+Thompson\. Light\-to\-dark purple = increasing penalty magnitude\. Top\-1 and top\-10 narrow the gap between cells over time; top\-10% mean separates them most clearly, with smaller penalties \(−1\-1,−3\-3\) reaching slightly higher upper\-tail FA but at the cost of validity \(Fig\.[40](https://arxiv.org/html/2607.02834#A6.F40)\)\.rinv=−5r\_\{\\text\{inv\}\}=\-5achieves the best validity / top\-1 FA trade\-off across the sweep\. Effect is not very signifcant on regular DDPP\-LB but becomes much more significant with other exploratory methods, as seen in the ablation figures\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/fig17_FA__b_h.png)Figure 40:Validity panel paired with top\-1 FA for the same penalty sweep as Fig\.[39](https://arxiv.org/html/2607.02834#A6.F39), used in the body to motivate the choice ofrinv=−5r\_\{\\text\{inv\}\}=\-5\. Therinv=0r\_\{\\text\{inv\}\}\{=\}0line shows grammar collapse; all non\-zero penalties restore validity\.

## Appendix GMulti\-knob ablation: DER; CVaR; Thompson Sampling

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/multiknob_ablation_lift_curves.png)Figure 41:Multi\-knob ablation: lift\-fraction trajectories\.Normalized reward \(1=1=FULL final\) vs\. cumulative oracle calls, averaged across the six FA\-oracle small\-molecule targets and five seeds; pair removals dashed, triple removal dotted\. Removing additional exploration / shaping knobs degrades lift roughly monotonically the triple sits lowest at every budget on DDPP\-LB, with VIDD showing the same ordering at compressed spread\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/multiknob_ablation_lift_bars.png)Figure 42:Multi\-knob ablation: normalized reward at oracle\-call slices\.Same normalization as Fig\.[5](https://arxiv.org/html/2607.02834#S3.F5); six FA\-oracle targets averaged, error bars±1\\pm 1SEM across targets\. No pair removal sits above either of its constituent single removals at the 5 000\-call slice joint losses compound rather than substitute\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/multiknob_ablation_anchor_nll_2vt4_ddpp.png)Figure 43:Anchor\-NLL distribution per multi\-knob ablation cell \(DDPP\-LB, 2VT4\)\.Per\-token anchor NLL−log⁡ppre​\(x0∣xt\)\-\\log p\_\{\\text\{pre\}\}\(x\_\{0\}\\mid x\_\{t\}\)under the frozen pretrained generator on all oracle\-evaluated SMILES \(5 seeds\), rough estimator withT=10T=10random\-mask MC averaging; FULL \(steelblue\) overlaid as reference\. Every cell with Density Entropy Regularization removed collapses mass back toward the prior, identifying DE as the dominant driver of distribution shift away frompprep\_\{\\text\{pre\}\}regardless of the other knobs\.
## Appendix HCross\-finetuner ablation: VIDD

Table 3:Ablation normalized reward \(mean±\\pms\.e\.\)\.Each row removes one design choice from the full method\. Values are averaged across 6 molecular targets \(Mol\) and 3 protein families \(Prot\), normalised by the per\-target range\[Ct,Ft\]\[C\_\{t\},F\_\{t\}\]shared with the bar chart in Fig\.[5](https://arxiv.org/html/2607.02834#S3.F5)\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/vidd_only_grid_norm.png)Figure 44:VIDD harness ablation across all99tasks\.Mirror of Fig\.[48](https://arxiv.org/html/2607.02834#A9.F48)with VIDD substituted for DDPP\-LB as the finetuning objectiveℒft\\mathcal\{L\}\_\{\\text\{ft\}\}in the harness; normalized reward is per\-target\(1=\(1=full VIDD\)\)\. Knob ordering matches DDPP\-LB on most targets, supporting the harness claim of Sec\.[2](https://arxiv.org/html/2607.02834#S2)that knob intent transfers across finetuners\. Spread between ablation cells is consistently smaller under VIDD: the student\-teacher distillation step damps the policy drift that the exploration / objective\-mismatch knobs amplify under DDPP\-LB, so each knob’s marginal contribution is also damped\.
## Appendix IPer\-task side\-by\-side ablation grids

The main\-text figures \(Fig\.[48](https://arxiv.org/html/2607.02834#A9.F48), Fig\.[49](https://arxiv.org/html/2607.02834#A9.F49), and Fig\.[44](https://arxiv.org/html/2607.02834#A8.F44)in Appendix[H](https://arxiv.org/html/2607.02834#A8)\) hold the finetuner / search\-family fixed and step through the99tasks\. The figures below transpose that view: each3×33\\times 3grid groups rows by task and places DDPP\-LB, VIDD, and search\-only baselines side by side per task\. Same lift\-fraction normalization and conventions as Sec\.[3](https://arxiv.org/html/2607.02834#S3)\.

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/mol_ablations_grid_set1_norm.png)Figure 45:Per\-task ablation, small molecules \(set 1: 2VT4, 5SDV, 6CM4\)\.Rows are FA\-oracle targets; columns are DDPP\-LB ablations \(left\), VIDD ablations \(centre\), search baselines \(right\)\. The VIDD column is consistently more compressed than DDPP\-LB on the same target, matching the cross\-finetuner observation in Fig\.[44](https://arxiv.org/html/2607.02834#A8.F44)\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/mol_ablations_grid_set2_norm.png)Figure 46:Per\-task ablation, small molecules \(set 2: 7BKC, 7C7M, 7YLL\)\.Continuation of Fig\.[45](https://arxiv.org/html/2607.02834#A9.F45)for the remaining three FA\-oracle targets\. The 7C7M row is extreme: removing\- density entropy regularizationdrops DDPP\-LB to a small fraction of the full method, while the same removal under VIDD has a much smaller effect — consistent with the student\-teacher damping of policy drift discussed in Appendix[H](https://arxiv.org/html/2607.02834#A8)\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/protein_ablations_grid_norm.png)Figure 47:Per\-task ablation, protein\-fitness targets \(GB1, CreiLOV, TrpB\)\.Rows are protein\-fitness families; columns match Fig\.[45](https://arxiv.org/html/2607.02834#A9.F45)\. CI bands are wider than the small\-molecule rows because each family has a single trained surrogate / generator pair \(no within\-family pooling\)\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/ddpp_only_grid_norm.png)Figure 48:DDPP\-LB harness ablation across all99tasks\.Normalized reward\(1=\(1=full DDPP\-LB on that target\)\)for each leave\-one\-out ablation of the knobs in Table[1](https://arxiv.org/html/2607.02834#S2.T1); six FA\-oracle small molecules \(top two rows\) and three protein\-fitness families \(bottom row\)\. Seeds, CI bands, and budget cap follow Sec\.[3](https://arxiv.org/html/2607.02834#S3)\. Each knob’s contribution varies across tasks, but knob ordering is broadly preserved across targets\. VIDD twin: Appendix[H](https://arxiv.org/html/2607.02834#A8)\. Per\-task side\-by\-side: Appendix[I](https://arxiv.org/html/2607.02834#A9)\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/search_only_grid_norm.png)

Figure 49:Inference\-time search\-only baselines across all99tasks\.Same lift\-fraction normalization as Fig\.[48](https://arxiv.org/html/2607.02834#A9.F48)applied to the search\-only methods \(best\-of\-NN, beam, MCTS, DFKC, SMC\) at matched cumulative\-oracle budget\.

### I\.1Breakdown of Compute Spend

To pull apart why test\-time finetuning beats inference\-time search, it helps to look at where each method actually spends its wall\-clock budget\. We log per\-call elapsed time alongside the oracle score in every run \(Sec\.[3](https://arxiv.org/html/2607.02834#S3)\), which lets us decompose each method’s wall time into the components of Eq\.[4](https://arxiv.org/html/2607.02834#A2.E4): oracle evaluation \(N⋅coracleN\\cdot c\_\{\\text\{oracle\}\}\), candidate generation, fine\-tuning gradient steps, surrogate ensemble training, and bookkeeping\. The same per\-call timeline supports a retroactive cost analysis: scaling the per\-call oracle costcoraclec\_\{\\text\{oracle\}\}by100×100\\timessimulates a switch from a fast oracle \(FA\) to a slow oracle \(Boltz\-2\-class\) without re\-running anything\. Appendix Fig\.[28](https://arxiv.org/html/2607.02834#A4.F28)shows the resulting breakdowns side\-by\-side\.

The takeaway is that finetuning’s advantage over search comes mostly from not spending the budget on oracle calls\. Each gradient step extracts more usable signal per oracle call than a one\-shot search evaluation does, so the same oracle budget does more work\. The advantage widens further as the oracle gets more expensive: the100×100\\timespanel is closer to where Boltz\-2\-class oracles actually live, and there the full\-loop compute breakdown is dominated by generation and surrogate training rather than oracle evaluation\.

## Appendix JAggregated ablation grids

The per\-task figures above retain the cross\-target spread that motivates the manifold / mode\-density discussion in Sec\.[4](https://arxiv.org/html/2607.02834#S4)\. The figures below collapse that spread by averaging the lift\-fraction curves across the66FA\-oracle small molecules \(top row\) and across the33protein\-fitness families \(bottom row\), giving a single curve per knob per axis\-type\. CI bands are across\-target \(±1\.96⋅SEM\\pm 1\.96\\cdot\\mathrm\{SEM\}\), so they reflect target\-level rather than seed\-level variability\.

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/avg_grid_norm.png)Figure 50:Aggregated ablation grid, oracle\-call view\.2×32\\times 3grid: row11averages over all six FA\-oracle small molecules, row22averages over the three protein\-fitness families; columns are DDPP\-LB ablations, VIDD ablations, and search\-only baselines\. Same lift\-fraction normalization as Fig\.[48](https://arxiv.org/html/2607.02834#A9.F48)\. Across\-target CI bands \(target\-level variability\) are wider than the per\-target seed bands in the figures above\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/avg_grid_norm_gpuhrs.png)Figure 51:Aggregated ablation grid, GPU\-hours view\.GPU\-hours twin of Fig\.[50](https://arxiv.org/html/2607.02834#A10.F50); same averaging\.
## Appendix KGPU\-hour view of the ablation grids

The main\-text and per\-task ablation grids \(Figs\.[48](https://arxiv.org/html/2607.02834#A9.F48),[49](https://arxiv.org/html/2607.02834#A9.F49),[44](https://arxiv.org/html/2607.02834#A8.F44),[45](https://arxiv.org/html/2607.02834#A9.F45),[46](https://arxiv.org/html/2607.02834#A9.F46),[47](https://arxiv.org/html/2607.02834#A9.F47)\) use cumulative oracle calls on the x\-axis to isolate feedback efficiency\. This appendix replots the same data against GPU hours, including method\-side cost \(generation, scoring, surrogate update, finetuning\); see Sec\.[3](https://arxiv.org/html/2607.02834#S3)for the motivation\. Same lift\-fraction normalization throughout\.

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/ddpp_only_grid_norm_gpuhrs.png)Figure 52:DDPP\-LB harness ablation, GPU\-hours view\.GPU\-hours twin of Fig\.[48](https://arxiv.org/html/2607.02834#A9.F48)\. Knob ordering is largely preserved, but the gap between the full method and\- Thompson samplingnarrows in wall time — the Thompson step adds non\-trivial method\-side cost\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/search_only_grid_norm_gpuhrs.png)Figure 53:Search\-only baselines, GPU\-hours view\.GPU\-hours twin of Fig\.[49](https://arxiv.org/html/2607.02834#A9.F49)\. Plain best\-of\-NNwins on wall time more often than on oracle\-call efficiency — no per\-call surrogate / search overhead\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/vidd_only_grid_norm_gpuhrs.png)Figure 54:VIDD harness ablation, GPU\-hours view\.GPU\-hours twin of Fig\.[44](https://arxiv.org/html/2607.02834#A8.F44)\. The distillation step adds a fixed per\-round cost, so GPU\-hour caps are tighter than the corresponding DDPP\-LB caps on the same target; knob ordering is unchanged\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/mol_ablations_grid_set1_norm_gpuhrs.png)Figure 55:Per\-task ablation \(mol set 1\), GPU\-hours view\.GPU\-hour twin of Fig\.[45](https://arxiv.org/html/2607.02834#A9.F45)\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/mol_ablations_grid_set2_norm_gpuhrs.png)Figure 56:Per\-task ablation \(mol set 2\), GPU\-hours view\.GPU\-hour twin of Fig\.[46](https://arxiv.org/html/2607.02834#A9.F46)\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/protein_ablations_grid_norm_gpuhrs.png)Figure 57:Per\-task ablation \(proteins\), GPU\-hours view\.GPU\-hour twin of Fig\.[47](https://arxiv.org/html/2607.02834#A9.F47)\.
## Appendix LReproductions of the body bar plots

Reproducing the body bar plots here so downstream cross\-references \(per\-finetuner discussion, GPU\-hours twins, etc\.\) resolve to a single canonical pair of labels\.

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/result1_combined_vs_baseline.png)Figure 58:DDPP\-LB and VIDD harness ablation, lift\-fraction bar version\.Reproduces Fig\.[5](https://arxiv.org/html/2607.02834#S3.F5)for appendix cross\-referencing\. See body for the full caption and discussion\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/ablations/result2_combined_vs_search.png)Figure 59:Full method vs\. inference\-time search baselines, lift\-fraction bar version\.Reproduces Fig\.[6](https://arxiv.org/html/2607.02834#S4.F6)for appendix cross\-referencing\. See body for the full caption and discussion\.
## Appendix MBoltz\-2 oracle results

The body claims that the design\-space conclusions transfer from FlashAffinity \(FA\) to a structure\-prediction\-based oracle that is two orders of magnitude more expensive \(Sec\.[3](https://arxiv.org/html/2607.02834#S3)\)\. This appendix replicates each comparison from Appendices[D](https://arxiv.org/html/2607.02834#A4)and[F](https://arxiv.org/html/2607.02834#A6)under Boltz\-2, with a1010\-hour wall budget per seed\. The Boltz\-2 panels use the same five seeds \(\{0, 1, 2, 3, 42\}\) and the same per\-knob configurations as their FA counterparts; ablation cells that did not get re\-run on Boltz\-2 \(annealed\-CVaR variants, search\-side Tweedie variants\) are omitted\. We show top\-1 best\-so\-far against cumulative oracle calls \(left\) and GPU hours \(right\) only — the top\-10 / top\-10% rows from Appendix[F](https://arxiv.org/html/2607.02834#A6)are not separately rendered for Boltz\-2\. Across every panel below the relative ordering of methods is preserved, supporting the claim that the recipe is robust to oracle expense\.

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_online_offline_a.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_online_offline_b.png)

Figure 60:Online vs\. offline DDPP\-LB — Boltz\-2\.Companion to Fig\.[23](https://arxiv.org/html/2607.02834#A4.F23)\. Same conclusion: online dominates offline at matched compute on top\-1 best\-so\-far Boltz\-2 score\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_cvar_a.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_cvar_b.png)

Figure 61:CVaR ablation — Boltz\-2\.Companion to Fig\.[33](https://arxiv.org/html/2607.02834#A6.F33), online cells only \(offline DDPP was not re\-run on Boltz\-2\)\. CVaR\-on solid, CVaR\-off dotted\. CVaR continues to lift top\-1 by a comparable margin under Boltz\-2 as under FA\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_buffer_a.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_buffer_b.png)

Figure 62:Replay\-buffer ablation — Boltz\-2\.Companion to Fig\.[37](https://arxiv.org/html/2607.02834#A6.F37)\. Same four cells as the FA version \(buffer×\\timesCVaR\)\. The buffer\-on / CVaR\-on cell continues to lead\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_thompson_a.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_thompson_b.png)

Figure 63:Thompson\-sampling ablation — Boltz\-2\.Companion to Fig\.[38](https://arxiv.org/html/2607.02834#A6.F38)\. Thompson on top of online DDPP\-LB\+\+CVaR vs\. deterministic top\-KKselection\. Thompson retains its lead on top\-1 under the slower oracle\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_surrogate_a.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_surrogate_b.png)

Figure 64:Surrogate screening on inference\-time search — Boltz\-2\.Companion to Fig\.[20](https://arxiv.org/html/2607.02834#A4.F20)\. Beam, MCTS, and DFKC each compared against their surrogate\-pre\-filter variant\. Surrogate screening continues to dominate plain search across all three families under Boltz\-2\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_oracle_rescale_1x_a.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_oracle_rescale_1x_b.png)

Figure 65:Online search hybrids at standard Boltz\-2 oracle cost\.Plain online DDPP\-LB\+\+CVaR vs\. DDPP\+\+beam\+\+CVaR vs\. DDPP\+\+MCTS\+\+CVaR\. Plain online matches or exceeds both search\-augmented variants on top\-1 Boltz\-2, mirroring the FA conclusion in Sec\.[4](https://arxiv.org/html/2607.02834#S4)\.![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_oracle_rescale_cheaper_b.png)

![Refer to caption](https://arxiv.org/html/2607.02834v1/figures/boltz_oracle_rescale_expensive_b.png)

Figure 66:Retroactive Boltz\-2 oracle\-cost rescaling\.Companion to Fig\.[27](https://arxiv.org/html/2607.02834#A4.F27), restricted to the GPU\-hours axis\. Per\-call Boltz\-2 cost rescaled by1/1001/100\(left\) and100100\(right\)\. The rescaling shifts the GPU\-hours axis by four orders of magnitude in either direction without altering the relative ordering: plain online DDPP\-LB\+\+CVaR continues to match or exceed the search\-augmented variants under both regimes, supporting the claim that the marginal value of inference\-time search on top of online finetuning is robust to oracle expense even when the oracle is itself the expensive variant\.
## Appendix NValid molecule samples across targets and methods

For the FA\-oracle small\-molecule targets, we render representative oracle\-evaluated SMILES generated by theDDPP\-LBandVIDDfull methods\. Per \(method, target\) we display three*high\-score*picks \(top decile of all oracle calls\) and three*low\-score*picks \(bottom decile\)\.

![[Uncaptioned image]](https://arxiv.org/html/2607.02834v1/image2.png)

![[Uncaptioned image]](https://arxiv.org/html/2607.02834v1/image4.png)

![[Uncaptioned image]](https://arxiv.org/html/2607.02834v1/image3.png)

![[Uncaptioned image]](https://arxiv.org/html/2607.02834v1/image5.png)

![[Uncaptioned image]](https://arxiv.org/html/2607.02834v1/image6.png)

![[Uncaptioned image]](https://arxiv.org/html/2607.02834v1/image7.png)