ReMedI: Inference-Time Refinement and Self-Distillation for Controllable Molecular Optimization

# Inference-Time Refinement and Self-Distillationfor Controllable Molecular Optimization
Source: [https://arxiv.org/html/2610.32103](https://arxiv.org/html/2610.32103)
Shivanshu Shekhar11footnotemark:1Affiliation:University of Illinois Urbana\-ChampaignAllison McLevigeAffiliation:University of Illinois Urbana\-ChampaignTong ZhangAffiliation:University of Illinois Urbana\-Champaign

###### Abstract

In drug development, lead optimization must improve competing properties, such as increasing permeability and reducing toxicity, while limiting scaffold changes that could compromise the lead’s potency\. RL\-based post\-training, such as C\-MORAL, addresses this problem by fine\-tuning molecular language models to directly optimize molecular leads for objectives of interest, which relies on expensive task\-specific datasets and faces challenges in achieving multi\-objective optimization \(MOO\)\. We argue a pretrained model may already contain useful molecular edits that zero\-shot generation cannot efficiently reach\. Motivated by the limited search capabilities of zero\-shot generation, we develop an efficient Metropolis–Hastings search strategy to better explore the proposal distribution\. We propose ReMedI, which uses the molecular language model as a base proposer for inference\-time alignment\. Given a lead and a set of multi\-objective tasks, the model proposes edits conditioned on the current molecule, while Metropolis–Hastings refinement guides sequential edits under an objective for property improvement, preservation, and lead similarity\. On both C\-MuMOInstruct and SMDD benchmarks, refinement improves frozen base models across three backbones, reaching significantly higher multi\-objective optimization success rates compared to baselines\. These search traces can then distilled into the base model which improves zero\-shot generation and subsequent search using≈7\.8×\\approx 7\.8\\timesless training molecular leads than that of C\-MORAL\. ReMedI achieves≈2\.8×\\approx 2\.8\\timesthe strict\-success coverage of RL baseline C\-MORAL, including on out\-of\-distribution tasks, with matched lead\-relative similarity\.

## 1Introduction

Chemical language models \(CLMs\) have increasingly become popular for text\-guided molecular generation\([Edwards et al\., 2022](https://arxiv.org/html/2610.32103#bib.bib5)\), multi\-property optimization\([Dey et al\., 2025b](https://arxiv.org/html/2610.32103#bib.bib4);[Dey et al\., 2025a](https://arxiv.org/html/2610.32103#bib.bib3)\), and the design of functional, makeable molecules\([Edwards et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib6)\)\. A central application of CLMs is lead optimization, which starts with a promising molecule \(lead\) and seeks local edits that improve deficient properties, such as permeability or toxicity, while preserving satisfactory properties and maintaining lead\-relative similarity\. However, current CLMs do not reliably produce molecules that meet all property targets without drifting off\-scaffold\. RL post\-training and reference\-guided policy optimization methods try to align CLMs to property rewards to satisfy MOO\([Gao et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib7);[Li et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib12)\)\. For instance, C\-MORAL combines group\-based policy optimization with property\-specific score transformations and nonlinear reward aggregation to improve selective molecular editing\([Gao et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib7)\)\. This approach relies on task\-specific RL adapters and editing data, which are costly to develop and can limit transfer to new combinations of objectives\. Since chemists often optimize several properties together, the challenge therefore is to meet every property requirement and maintaining acceptable lead\-relative similarity without adapting a separate model for each task\.

Inference\-time alignment offers an alternative by guiding generation while holding the model weights fixed\. Existing approaches include similarity\-guided decoding\([Navratil et al\., 2025](https://arxiv.org/html/2610.32103#bib.bib20)\)and iterative editing\([Edwards et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib6)\)\. For instance, GP\-MoLFormer\-Sim guides decoding through reference similarity and uses an outer genetic algorithm for property optimization\([Navratil et al\., 2025](https://arxiv.org/html/2610.32103#bib.bib20)\), but its decoding guidance does not directly encode which properties should improve or remain unchanged\. MARS demonstrates that multi\-property molecular optimization can benefit from iterative search over intermediate molecular states\([Xie et al\., 2021](https://arxiv.org/html/2610.32103#bib.bib31)\)\. We build on this principle in ReMedI by using an instruction\-conditioned language model to propose successive edits with a search algorithm which determines which states to retain\.

We introduce ReMedI, which uses a pretrained molecular language model to propose local edits and Metropolis–Hastings \(MH\) refinement to guide the search\. Its lead\-relative objective balances property improvement, preservation, and similarity to the lead\. Because the proposal remains frozen during refinement, the improvement–similarity trade\-off can be adjusted at inference time without task\-specific RL adapters\. Thus our method allows chemists to explore pharmacological spaces that are hard to reach via zero\-shot prompting\. On C\-MuMOInstruct and SMDD, refinement improves success across GeLLMO\-Llama, GeLLMO\-Mistral, and mCLM\-4B, and ReMedI achieves higher strict\-success coverage than C\-MORAL on matched GeLLMO\-Mistral leads, including on combinations of properties held out during training\.

We show that in ReMedI a single shared proposal adapter trained from only 500 lead–task instances achieves performance comparable to C\-MORAL, which relies on five task\-specific policies trained from 3,883 lead–task prompts\. Our traces contain successful endpoints and intermediate states, so training does not require a separate specialized editing dataset\. Search improves candidates at inference time and supplies examples that strengthen later generation and refinement\.

![Refer to caption](https://arxiv.org/html/2610.32103v1/figures/graphical_abstract.drawio_9.png)Figure 1:ReMedI separates the molecular edit proposal from the inference\-time objective\. Refinement discovers successful trajectories; lead\- and state\-conditioned examples from those trajectories train a shared proposal for later generation and search\.#### Contributions\.

- •Inference\-time molecular alignment\.We use a chemical language model \(CLM\) to propose edits\. Metropolis–Hastings \(MH\) accepts or rejects each proposal using a lead\-relative objective for property improvement, preservation, and similarity, together with the corresponding LM probabilities\. These priorities can be changed at inference time without retraining the model\.
- •Learning from refinement\.We convert search\-discovered endpoints, intermediate states, and improving transitions into supervised training pairs and distill them back into the base CLM which improves zero\-shot generation and subsequent MH refinements\.
- •Success under similarity requirements\.ReMedI achieves≈2\.8×\\approx 2\.8\\timesthe strict\-success coverage of C\-MORAL on matched GeLLMO\-Mistral leads\. On C\-MuMOInstruct, MH increases frozen\-model SOR by 66–143% relative to zero\-shot generation across three backbones and IND/OOD splits; on SMDD, it improves SOR by 12–50%\.

## 2Related Works

Post\-training for molecular optimization\.Molecular LLMs support text\-guided generation, editing, and multi\-property optimization\([Wang et al\., 2025](https://arxiv.org/html/2610.32103#bib.bib29)\)\. Early molecule–text models such as MolT5 and MoleculeSTM connect molecular strings to natural language\([Edwards et al\., 2022](https://arxiv.org/html/2610.32103#bib.bib5);[Liu et al\., 2023a](https://arxiv.org/html/2610.32103#bib.bib14)\), while instruction\- and interaction\-based systems such as LlaSMol, PEIT\-LLM, ChatDrug, and DrugAssist improve chemistry task following, property\-conditioned generation, or conversational molecular editing\([Yu et al\., 2024](https://arxiv.org/html/2610.32103#bib.bib34);[Lin et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib13);[Liu et al\., 2023b](https://arxiv.org/html/2610.32103#bib.bib15);[Ye et al\., 2023](https://arxiv.org/html/2610.32103#bib.bib32)\)\. For controllable multi\-property optimization, Prompt\-MolOpt steers property\-specific transformations with prompt\-conditioned embeddings\([Wu et al\., 2024](https://arxiv.org/html/2610.32103#bib.bib30)\), while GeLLMO\-C and C\-MuMOInstruct train models to follow multi\-property instructions and prevent property regressions\([Dey et al\., 2025b](https://arxiv.org/html/2610.32103#bib.bib4);[Dey et al\., 2025a](https://arxiv.org/html/2610.32103#bib.bib3)\)\. Earlier molecular optimization methods also applied reward\-guided reinforcement learning to sequence generation and direct molecular edits\([Olivecrona et al\., 2017](https://arxiv.org/html/2610.32103#bib.bib21);[Popova et al\., 2018](https://arxiv.org/html/2610.32103#bib.bib23);[Zhou et al\., 2019](https://arxiv.org/html/2610.32103#bib.bib35)\)\. More broadly, post\-training methods modify the generative policy through supervised fine\-tuning, preference optimization, or reinforcement learning\([Ouyang et al\., 2022](https://arxiv.org/html/2610.32103#bib.bib22);[Rafailov et al\., 2024](https://arxiv.org/html/2610.32103#bib.bib24);[Schulman et al\., 2017](https://arxiv.org/html/2610.32103#bib.bib26)\)\. C\-MORAL uses GRPO/GDPO\-style optimization to align molecular LLM outputs with multi\-objective constraints\([Gao et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib7);[Shao et al\., 2024](https://arxiv.org/html/2610.32103#bib.bib27)\), while RePO uses reference\-guided policy optimization\([Li et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib12)\)\. These approaches adapt the model through specialized training\. ReMedI instead keeps the CLM fixed during refinement and specifies the improvement–similarity trade\-off through an explicit objective at inference time\.

Inference\-time alignment\.A smaller line of work shifts optimization to inference time\. MARS uses Markov\-chain search for multi\-objective molecular optimization with graph neural networks\([Xie et al\., 2021](https://arxiv.org/html/2610.32103#bib.bib31)\), while GP\-MoLFormer\-Sim provides training\-free similarity\-guided decoding for chemical language models\([Navratil et al\., 2025](https://arxiv.org/html/2610.32103#bib.bib20)\)\. ReMedI uses the CLM as a state\-dependent proposal within a Metropolis–Hastings refinement loop\([Metropolis et al\., 1953](https://arxiv.org/html/2610.32103#bib.bib17);[Hastings, 1970](https://arxiv.org/html/2610.32103#bib.bib9)\), with an explicit target that favors requested property improvements, preservation of guarded properties, and similarity to the lead\. Search can also improve the proposal itself\. Expert Iteration formalizes the search\-and\-distill pattern\([Anthony et al\., 2017](https://arxiv.org/html/2610.32103#bib.bib1)\), and we train a shared proposal adapter on successful refinement endpoints and intermediate\-state examples, reconditioning multi\-step solutions on the original lead as one\-step targets\. This combines inference\-time refinement with proposal improvement and tests whether search\-derived supervision transfers to unseen objective combinations\.

## 3Preliminary

#### Metropolis–Hastings\.

Metropolis–Hastings \(MH\) draws samples from a distribution known only up to a normalizing constant\. Given an unnormalized targetπ\\piand a proposalq\(⋅∣x\)q\(\\cdot\\mid x\), from the current statexxit draws a candidatey∼q\(⋅∣x\)y\\sim q\(\\cdot\\mid x\)and accepts it with probability

A⁡\(x→y\)=min⁡\(1,π⁡\(y\)π⁡\(x\)​q⁡\(x∣y\)q⁡\(y∣x\)\),A\(x\\rightarrow y\)=\\min\\\!\\left\(1,\\;\\frac\{\\pi\(y\)\}\{\\pi\(x\)\}\\,\\frac\{q\(x\\mid y\)\}\{q\(y\\mid x\)\}\\right\),\(1\)where the second factor, the Hastings correction, offsets the asymmetry of a state\-dependent proposal \(a forward edit likely fromxxneed not be equally likely in reverse\)\. Sinceπ\\piappears only through a ratio, its normalizing constant cancels\. The proposal controls exploration, whileπ\\pidetermines the target distribution\. The proposal can be changed or learned without changing that target, provided the Hastings correction is retained and the resulting chain remains irreducible and aperiodic over the target’s support\.

#### Nash bargaining and Pareto optimality\.

Let𝒳\\mathcal\{X\}be the feasible outcome set\. Considermmobjectives with utilitiesui​\(x\)u\_\{i\}\(x\)and a reference \(disagreement\) pointd∈ℝmd\\in\\mathbb\{R\}^\{m\}that specifies the baseline value for each objective\. An outcomexxis Pareto\-optimal if there is no feasiblex′x^\{\\prime\}such thatui​\(x′\)≥ui​\(x\)u\_\{i\}\(x^\{\\prime\}\)\\geq u\_\{i\}\(x\)for everyii, with strict inequality for at least one objective\. In other words, no objective can be improved without sacrificing another\. If the feasible utility set is compact and convex and contains a point that strictly improves every objective overdd, the Nash bargaining solution[Nash & Jr\. \(1950\)](https://arxiv.org/html/2610.32103#bib.bib19)has a unique utility vector, attained by any maximizerxNBSx^\{\\mathrm\{NBS\}\}of

xNBS∈arg​maxx∈𝒳ui​\(x\)≥di​∀i∏i=1m\(ui\(x\)−di\),x^\{\\mathrm\{NBS\}\}\\in\\argmax\_\{\\begin\{subarray\}\{c\}x\\in\\mathcal\{X\}\\\\ u\_\{i\}\(x\)\\geq d\_\{i\}\\ \\forall i\\end\{subarray\}\}\\ \\prod\_\{i=1\}^\{m\}\\big\(u\_\{i\}\(x\)\-d\_\{i\}\\big\),\(2\)which maximizes the*product*of gains over the reference point rather than their weighted sum\. Equivalently, it maximizes∑ilog⁡\(ui​\(x\)−di\)\\sum\_\{i\}\\log\\big\(u\_\{i\}\(x\)\-d\_\{i\}\\big\)\. This makes the solution invariant to positive affine rescaling of an objective when its disagreement value is transformed in the same way, and favors balanced gains across objectives\. Any outcome that leaves one objective at its reference value has a zero product, regardless of gains in the other objectives\.

## 4Method

Algorithm 1Focused refinement and trace generation\. The inner chain is given in Alg\.[2](https://arxiv.org/html/2610.32103#alg2)\.1:lead

x0x\_\{0\}, task requirements

tt, base prompt

P0​\(t\)P\_\{0\}\(t\), valid proposal

qθvalidq\_\{\\theta\}^\{\\mathrm\{valid\}\}, stages

RR, chains

CC, steps

NN, similarity target

τs\\tau\_\{s\}, weights

βS,0,βG,βL\\beta\_\{S,0\},\\beta\_\{G\},\\beta\_\{L\}, annealing rate

ρ\\rho
2:incumbent

x⋆x^\{\\star\}; transition log

𝒯\\mathcal\{T\}
3:

x⋆←x0x^\{\\star\}\\leftarrow x\_\{0\};

ϕ←\\phi\\leftarrowsimilarity;

P←Pϕ​\(t\)P\\leftarrow P\_\{\\phi\}\(t\);

Tried←∅\\textsc\{Tried\}\\leftarrow\\varnothing;

𝒯←∅\\mathcal\{T\}\\leftarrow\\varnothing
4:for

j=1,…,Rj=1,\\dots,Rdo

5:

𝒱←\{x⋆\}\\mathcal\{V\}\\leftarrow\\\{x^\{\\star\}\\\}
6:for

c=1,…,Cc=1,\\dots,Cdo

7:

𝒱c,𝒯c←MHChain​\(x⋆,x0,P,N,qθvalid,βS,0,ρ,βG,βL,τs\)\\mathcal\{V\}\_\{c\},\\mathcal\{T\}\_\{c\}\\leftarrow\\textsc\{MHChain\}\(x^\{\\star\},x\_\{0\},P,N,q\_\{\\theta\}^\{\\mathrm\{valid\}\},\\beta\_\{S,0\},\\rho,\\beta\_\{G\},\\beta\_\{L\},\\tau\_\{s\}\)
8:

𝒱←𝒱∪𝒱c\\mathcal\{V\}\\leftarrow\\mathcal\{V\}\\cup\\mathcal\{V\}\_\{c\};

𝒯←𝒯∪𝒯c\\mathcal\{T\}\\leftarrow\\mathcal\{T\}\\cup\\mathcal\{T\}\_\{c\}
9:endfor

10:

x^←\\hat\{x\}\\leftarrowbest\-ranked candidate in

𝒱\\mathcal\{V\}under the criteria above

11:if

x^\\hat\{x\}is preferred to

x⋆x^\{\\star\}under the criteria abovethen

x⋆←x^x^\{\\star\}\\leftarrow\\hat\{x\};

Tried←∅\\textsc\{Tried\}\\leftarrow\\varnothing
12:elseif

ϕ\\phiis a property indexthen

Tried←Tried∪\{ϕ\}\\textsc\{Tried\}\\leftarrow\\textsc\{Tried\}\\cup\\\{\\phi\\\}
13:endif

14:if

ℛt​\(x⋆\)=∅\\mathcal\{R\}\_\{t\}\(x^\{\\star\}\)=\\varnothingand

sim⁡\(x⋆,x0\)≥τs\\simop\(x^\{\\star\},x\_\{0\}\)\\geq\\tau\_\{s\}thenbreak

15:endif

16:

ℱ←ℛt​\(x⋆\)∖Tried\\mathcal\{F\}\\leftarrow\\mathcal\{R\}\_\{t\}\(x^\{\\star\}\)\\setminus\\textsc\{Tried\}
17:if

ℱ≠∅\\mathcal\{F\}\\neq\\varnothingthen

ϕ←arg​mini∈ℱ⁡si​\(x⋆\)\\phi\\leftarrow\\argmin\_\{i\\in\\mathcal\{F\}\}s\_\{i\}\(x^\{\\star\}\)⊳\\trianglerightworst untried improve\-property

18:else

ϕ←\\phi\\leftarrowsimilarity

19:endif

20:

P←Pϕ​\(t\)P\\leftarrow P\_\{\\phi\}\(t\)
21:endfor

22:return

x⋆,𝒯x^\{\\star\},\\mathcal\{T\}

#### Scoring and target\.

Letqθq\_\{\\theta\}denote the LM’s conditional proposal distribution,x0x\_\{0\}the lead,ttthe task requirements, andP0​\(t\)P\_\{0\}\(t\)the original prompt specifying those requirements\. Letp⁡\(x\)∈ℝmp\(x\)\\in\\mathbb\{R\}^\{m\}be the oracle’s property values andd=p⁡\(x0\)d=p\(x\_\{0\}\)the lead’s values\. The task specifies which properties to improve and which to guard\. We denote these sets byℐ\\mathcal\{I\}and𝒢\\mathcal\{G\}, and usesgni∈\{\+1,−1\}\\mathrm\{sgn\}\_\{i\}\\in\\\{\+1,\-1\\\}for the requested direction of propertyii\. We map each requested property to a lead\-centred satisfaction

si​\(x\)=clip\[ϵ,1−ϵ\]⁡\(σ⁡\(ki​sgni​\(pi​\(x\)−di\)\)\),σ⁡\(z\)=11\+e−z,s\_\{i\}\(x\)=\\operatorname\{clip\}\_\{\[\\epsilon,1\-\\epsilon\]\}\\\!\\left\(\\sigma\\\!\\big\(k\_\{i\}\\,\\mathrm\{sgn\}\_\{i\}\\,\(p\_\{i\}\(x\)\-d\_\{i\}\)\\big\)\\right\),\\qquad\\sigma\(z\)=\\tfrac\{1\}\{1\+e^\{\-z\}\},\(3\)whereki\>0k\_\{i\}\>0controls the response scale for propertyii, and0<ϵ<1/20<\\epsilon<1/2keeps each satisfaction away from zero and one so its logarithm remains bounded\. We aggregate the satisfactions of properties to improve asS⁡\(x\)=∏i∈ℐsi​\(x\)S\(x\)=\\prod\_\{i\\in\\mathcal\{I\}\}s\_\{i\}\(x\), so gains on one property cannot fully mask poor satisfaction on another\. For guarded properties, the penaltyG⁡\(x\)G\(x\)activates only when a property declines beyond its allowed margin\. The similarity penaltyL⁡\(x\)L\(x\)activates when similarity to the lead falls belowτs\\tau\_\{s\}\.

G\(x\)=−∑i∈𝒢max\(0,−τig−sgni\(pi\(x\)−di\)\),L\(x\)=−max\(0,τs−sim\(x,x0\)\)\.G\(x\)=\-\\\!\\sum\_\{i\\in\\mathcal\{G\}\}\\max\\\!\\big\(0,\-\\tau\_\{i\}^\{g\}\-\\mathrm\{sgn\}\_\{i\}\(p\_\{i\}\(x\)\-d\_\{i\}\)\\big\),\\qquad L\(x\)=\-\\max\\\!\\big\(0,\\tau\_\{s\}\-\\simop\(x,x\_\{0\}\)\\big\)\.\(4\)
Hereτig\\tau\_\{i\}^\{g\}is the allowed decline for guarded propertyii, andτs\\tau\_\{s\}is the minimum lead similarity\. Combining the three terms gives the utility score and target distribution,

U⁡\(x\)\\displaystyle U\(x\)=βS​log⁡S⁡\(x\)\+βG​G​\(x\)\+βL​L​\(x\),\\displaystyle=\\beta\_\{S\}\\log S\(x\)\+\\beta\_\{G\}\\,G\(x\)\+\\beta\_\{L\}\\,L\(x\),\(5\)π⁡\(x\)∝\\displaystyle\\pi\(x\)\\propto\{\}exp⁡\(U⁡\(x\)\),\\displaystyle\\exp\\\!\\big\(U\(x\)\\big\),Herelog⁡S⁡\(x\)=∑i∈ℐlog⁡si​\(x\)\\log S\(x\)=\\sum\_\{i\\in\\mathcal\{I\}\}\\log s\_\{i\}\(x\)is the sum of per\-property log\-satisfactions and the log of the Nash product\. The positive weightsβS\\beta\_\{S\},βG\\beta\_\{G\}, andβL\\beta\_\{L\}control improvement, guarding, and similarity, respectively\.

#### Sampling and focused refinement\.

At each refinement stage, we set the focused prompt toP=Pϕ​\(t\)=P0​\(t\)⊕clause⁡\(ϕ\)P=P\_\{\\phi\}\(t\)=P\_\{0\}\(t\)\\oplus\\mathrm\{clause\}\(\\phi\), whereϕ\\phiidentifies the property or similarity constraint to focus on\. The model proposes edits fromqθvalid\(⋅∣x,P\)q\_\{\\theta\}^\{\\mathrm\{valid\}\}\(\\cdot\\mid x,P\)\. For a proposed move fromxxtox′x^\{\\prime\}, writeΔ​F=F⁡\(x′\)−F⁡\(x\)\\Delta F=F\(x^\{\\prime\}\)\-F\(x\)for any scoreFF\. At MH iterationnn, we anneal the satisfaction weight asβS,n=βS,0​ρn−1\\beta\_\{S,n\}=\\beta\_\{S,0\}\\rho^\{n\-1\}, giving the step\-specific targetπn​\(x\)∝exp⁡\(βS,n​log⁡S⁡\(x\)\+βG​G​\(x\)\+βL​L​\(x\)\)\\pi\_\{n\}\(x\)\\propto\\exp\\\!\\big\(\\beta\_\{S,n\}\\log S\(x\)\+\\beta\_\{G\}G\(x\)\+\\beta\_\{L\}L\(x\)\\big\)\. The Metropolis–Hastings acceptance ratio is

log⁡r=βS,n​Δ​log⁡S\+βG​Δ​G\+βL​Δ​L⏟Δ​log⁡πn\+log⁡qθvalid​\(x∣x′,P\)−log⁡qθvalid​\(x′∣x,P\)⏟log⁡h,\\log r=\\underbrace\{\\beta\_\{S,n\}\\,\\Delta\\log S\+\\beta\_\{G\}\\,\\Delta G\+\\beta\_\{L\}\\,\\Delta L\}\_\{\\Delta\\log\\pi\_\{n\}\}\+\\underbrace\{\\log q\_\{\\theta\}^\{\\mathrm\{valid\}\}\(x\\mid x^\{\\prime\},P\)\-\\log q\_\{\\theta\}^\{\\mathrm\{valid\}\}\(x^\{\\prime\}\\mid x,P\)\}\_\{\\log h\},\(6\)We accept the move with probabilityα=min⁡\{1,elog⁡r\}\\alpha=\\min\\\{1,e^\{\\log r\}\\\}\. Each iteration applies the MH rule for its step\-specific targetπn\\pi\_\{n\}\. We trim and parse proposals to valid molecules and retry invalid draws\. The focus prompt guides which edits the model proposes, while the target score is determined bySS,GG, andLL\. The acceptance ratio accounts for the prompt by comparing the probabilities of proposingx′x^\{\\prime\}fromxxandxxfromx′x^\{\\prime\}\. Computing the target ratio requires oracle scores for both states\.

#### Proposal distillation\.

The search produces traces of proposed edits and their outcomes\. We use these traces as supervision to train the proposal model for future searches, curating a corpus𝒟\\mathcal\{D\}of context–molecule pairs with three types of examples\.*Seed\-conditioned*examples pair the lead and its task prompt\(x0,P0​\(t\)\)\(x\_\{0\},P\_\{0\}\(t\)\)with a satisfying endpoint, teaching the model to generate directly from the lead\.*Chain\-conditioned*examples pair an intermediate state and its focused task prompt\(x,Pϕ​\(t\)\)\(x,P\_\{\\phi\}\(t\)\)with a satisfying endpoint reached from that state\. We also include some*accepted transitions*that do not meet every requirement, so the model learns incremental edits that keep refinement moving\. We fitqθq\_\{\\theta\}by weighted maximum likelihood,

ℒ\(θ\)=−∑\(c,x′\)∈𝒟w\(x′\)logqθ\(x′∣c\),w\(x′\)∝exp\(U\(x′\)/βdist\),\\mathcal\{L\}\(\\theta\)=\-\\\!\\\!\\sum\_\{\(c,x^\{\\prime\}\)\\in\\mathcal\{D\}\}w\(x^\{\\prime\}\)\\,\\log q\_\{\\theta\}\(x^\{\\prime\}\\mid c\),\\qquad w\(x^\{\\prime\}\)\\propto\\exp\\\!\\big\(U\(x^\{\\prime\}\)/\\beta\_\{\\mathrm\{dist\}\}\\big\),\(7\)
where the positive temperatureβdist\\beta\_\{\\mathrm\{dist\}\}controls how strongly the weights favor higher\-scoring targets\. Seed\- and chain\-conditioned examples match the two proposal contexts used at inference, so one distilled model can improve both direct generation and refinement\. The process can be repeated by generating new traces with the current proposal and refitting Eq\.[7](https://arxiv.org/html/2610.32103#S4.E7)\.

Letℛt​\(x\)\\mathcal\{R\}\_\{t\}\(x\)be the unmet improvement requirements for tasktt\. We prefer candidates with fewer unmet requirements, then those meetingτs\\tau\_\{s\}, then higher utilityU⁡\(x\)U\(x\)\. The focusϕ\\phiis the least\-satisfied unmet requirement, or similarity if none remain, and its clause tells the model what to focus on\.

Algorithm 2MHChainperforms one refinement chain of lengthNN\.1:functionMHChain\(

x0,P,N,qθvalid,βS,0,ρ,βG,βL,τsx\_\{0\},P,N,q\_\{\\theta\}^\{\\mathrm\{valid\}\},\\beta\_\{S,0\},\\rho,\\beta\_\{G\},\\beta\_\{L\},\\tau\_\{s\}\)

2:

x←x0x\\leftarrow x\_\{0\};

𝒱←\{x\}\\mathcal\{V\}\\leftarrow\\\{x\\\};

𝒯←∅\\mathcal\{T\}\\leftarrow\\varnothing
3:for

n=1,…,Nn=1,\\dots,Ndo

4:

βS,n←βS,0​ρn−1\\beta\_\{S,n\}\\leftarrow\\beta\_\{S,0\}\\rho^\{n\-1\}
5:

x′∼qθvalid\(⋅∣x,P\)x^\{\\prime\}\\sim q\_\{\\theta\}^\{\\mathrm\{valid\}\}\(\\cdot\\mid x,P\)
6:

Δ​log⁡πn←βS,n​Δ​log⁡S\+βG​Δ​G\+βL​Δ​L\\Delta\\log\\pi\_\{n\}\\leftarrow\\beta\_\{S,n\}\\,\\Delta\\log S\+\\beta\_\{G\}\\,\\Delta G\+\\beta\_\{L\}\\,\\Delta L;

log⁡h←log⁡qθvalid​\(x∣x′,P\)−log⁡qθvalid​\(x′∣x,P\)\\log h\\leftarrow\\log q\_\{\\theta\}^\{\\mathrm\{valid\}\}\(x\\mid x^\{\\prime\},P\)\-\\log q\_\{\\theta\}^\{\\mathrm\{valid\}\}\(x^\{\\prime\}\\mid x,P\)
7:

log⁡r←Δ​log⁡πn\+log⁡h\\log r\\leftarrow\\Delta\\log\\pi\_\{n\}\+\\log h;

α←min⁡\{1,exp⁡\(log⁡r\)\}\\alpha\\leftarrow\\min\\\{1,\\exp\(\\log r\)\\\}
8:

xprev←xx\_\{\\mathrm\{prev\}\}\\leftarrow x;

b∼Bernoulli⁡\(α\)b\\sim\\operatorname\{Bernoulli\}\(\\alpha\)
9:if

b=1b=1then

x←x′x\\leftarrow x^\{\\prime\};

𝒱←𝒱∪\{x\}\\mathcal\{V\}\\leftarrow\\mathcal\{V\}\\cup\\\{x\\\}
10:endif

11:

𝒯←𝒯∪\{\(x0,xprev,P,x′,U⁡\(x′\),log⁡qθvalid​\(x′∣xprev,P\),log⁡h,α,b\)\}\\mathcal\{T\}\\leftarrow\\mathcal\{T\}\\cup\\\{\(x\_\{0\},x\_\{\\mathrm\{prev\}\},P,x^\{\\prime\},U\(x^\{\\prime\}\),\\log q\_\{\\theta\}^\{\\mathrm\{valid\}\}\(x^\{\\prime\}\\mid x\_\{\\mathrm\{prev\}\},P\),\\log h,\\alpha,b\)\\\}
12:endfor

13:return

𝒱,𝒯\\mathcal\{V\},\\mathcal\{T\}
14:endfunction

## 5Experiments

### 5\.1Setup

#### Models and benchmarks\.

We evaluate ReMedI primarily on C\-MuMOInstruct, which contains ten multi\-property editing tasks: five training\-seen property combinations \(IND\) and five held\-out combinations \(OOD\)\([Dey et al\., 2025a](https://arxiv.org/html/2610.32103#bib.bib3)\)\. We use GeLLMO\-C Llama\-3\.1\-8B, GeLLMO\-C Mistral\-7B, and mCLM\-4B as proposal models\([Dey et al\., 2025b](https://arxiv.org/html/2610.32103#bib.bib4);[Edwards et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib6)\), with property evaluation provided by ADMET\-AI and PyTDC\([Swanson et al\., 2024](https://arxiv.org/html/2610.32103#bib.bib28);[Huang et al\., 2022](https://arxiv.org/html/2610.32103#bib.bib10)\)\. IND and OOD follow the benchmark’s original task split; our shared proposal adapter is distilled only from trajectories spanning the five IND tasks\. We additionally evaluate ReMedI on 314 instances from SMDD\-Bench Task 4 \(Lead Optimization\)[Han et al\. \(2026\)](https://arxiv.org/html/2610.32103#bib.bib8)\. We use a non\-agentic evaluation in which both the lead and generated candidate are scored with ADMET\-AI and RDKit\. We compare direct generation \(*base*\) and MH refinement \(*MH*\) using the original proposal \(q0q\_\{0\}\) and the proposal after one distillation round \(q1q\_\{1\}\), against the RL\-trained baseline C\-MORAL\. Since C\-MORAL’s task objectives do not align with SMDD Task 4, we pool the five IND seed files and train one shared adapter per backbone\. Each seed retains its own improvement and hold\-constant keys, while the oracle scores the union of IND properties\.

#### Metrics\.

Following C\-MORAL\([Gao et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib7)\), we report Success Optimized Rate \(SOR\), the proportion of molecules that meet the requested property\-improvement criteria, and Strict Success Optimized Rate \(SSOR\), the proportion whose relevant properties all satisfy prescribed near\-optimal thresholds\. For SMDD\-Bench\([Han et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib8)\), an SOR success requires every requested objective to improve by its instance\-specific threshold\. Similarity \(Sim\) is the Morgan fingerprint\([Morgan, 1965](https://arxiv.org/html/2610.32103#bib.bib18);[Rogers & Hahn, 2010](https://arxiv.org/html/2610.32103#bib.bib25);[Landrum, 2026](https://arxiv.org/html/2610.32103#bib.bib11)\)Tanimoto similarity\([Bajusz et al\., 2015](https://arxiv.org/html/2610.32103#bib.bib2);[Maggiora et al\., 2014](https://arxiv.org/html/2610.32103#bib.bib16)\)between each generated molecule and its lead\. Clearance is the rate at which the body, mainly the liver, removes a drug from plasma\([Yim et al\., 2021](https://arxiv.org/html/2610.32103#bib.bib33)\)\. For the 144 SMDD\-Bench instances requesting reduced clearance, CLR is the fraction whose selected molecule’s predicted clearance decreases by at least the instance\-specific threshold\([Swanson et al\., 2024](https://arxiv.org/html/2610.32103#bib.bib28)\)\. We select the best of 20 candidates by SOR and emphasize SSOR because it captures simultaneous satisfaction of all property targets at near\-optimality\.

### 5\.2Main Results

Table 1:C\-MuMOInstruct results on 100 held\-out leads per task\.*q0q\_\{0\}*denotes the original proposal andq1q\_\{1\}the proposal after one distillation round;*base*denotes direct generation and*MH*refinement\. Best\-of\-2020selection prioritizes SOR, then SSOR, then similarity\. Success–similarity tradeoffs are shown in Fig\.[2](https://arxiv.org/html/2610.32103#S5.F2)\. Bold marks the best value in each row\.#### Refinement across backbones\.

Table[1](https://arxiv.org/html/2610.32103#S5.T1)shows effect of inference\-time refinement across three model families on both IND/OOD splits\. MH improvesq0q\_\{0\}SOR by≈80%\\approx 80\\%and SSOR by≈90%\\approx 90\\%on average\. The same pattern persists after distillation, withq1q\_\{1\}\-MH outperformingq1q\_\{1\}\-base on every backbone and split\. These gains show that substantial optimization capability is already present in the proposal models but is not recovered by one\-shot decoding; sequential MH refinement exposes this capability by composing and selecting edits under the target objective\.

![Refer to caption](https://arxiv.org/html/2610.32103v1/figures/ssor_cmoral.png)Figure 2:Strict\-success coverage versus minimum lead similarity on 25 matched leads per task\. Each method selects candidates by SOR success, then maximum similarity; coverage counts selected outputs that also satisfy SSOR and the indicated similarity threshold\. C\-MORAL OOD results are averaged over five task\-specific GRPO policies\.
#### Success under similarity requirements\.

Figure[2](https://arxiv.org/html/2610.32103#S5.F2)evaluates the tradeoff between strict multi\-objective success and similarity to the starting lead\. Across increasingly stringent similarity requirements, ReMedI forms empirical Pareto frontier in SSOR–similarity space, maintaining higher strict\-success coverage than C\-MORAL over the evaluated range\. This advantage holds on the pooled evaluation and extends to the held\-out OOD property combinations\. Because SSOR requires all relevant properties to reach their prescribed near\-optimal thresholds\([Gao et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib7)\), the frontier measures how often a method can satisfy all molecular objectives while remaining close to the original lead\. These results show that ReMedI improves the strict\-success–locality tradeoff rather than obtaining higher success simply through larger structural changes\.

#### Search and proposal distillation\.

Table[1](https://arxiv.org/html/2610.32103#S5.T1)shows that proposal distillation substantially strengthens the base model\. On GeLLMO\-Mistral, one distillation round improves zero\-shot base SOR by33%33\\%on IND \(→0\.4720\.354\\\!\\rightarrow\\\!0\.472\) and23%23\\%on OOD \(→0\.3320\.271\\\!\\rightarrow\\\!0\.332\)\. The trained base model also improves subsequent MH refinement by20%20\\%on IND \(→0\.7060\.590\\\!\\rightarrow\\\!0\.706\) and18%18\\%on OOD \(→0\.5520\.467\\\!\\rightarrow\\\!0\.552\)\. The same pattern holds for GeLLMO\-Llama, where distillation improves zero\-shot base by32%32\\%/28%28\\%and MH refinement by10%10\\%/8%8\\%on IND/OOD\. These results suggest not only could inference\-time MH improve multi\-objective optimization: distilling search trajectories back into the base CLM produces an improved base model for both zero\-shot generation and subsequent search\.

Table 2:SMDD\-Bench Task 4 results on 314 instances\. ReMedI\-MH uses the frozen proposal, and pooled C\-MORAL uses one shared adapter per backbone at the same 20\-candidate inference budget\.
#### Generalization to SMDD\-Bench\.

On 314 SMDD\-Bench instances, ReMedI\-MH improves SOR by0\.130\.13–0\.290\.29and clearance\-target success on every backbone compared with pooled C\-MORAL, while C\-MORAL retains higher similarity \(Table[2](https://arxiv.org/html/2610.32103#S5.T2)\)\. ReMedI’s shared adapter uses trajectories from 500 lead–task instances, about7\.8×7\.8\\timesfewer than C\-MORAL’s 3,883 RL training prompts\([Gao et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib7)\)\.

![Refer to caption](https://arxiv.org/html/2610.32103v1/figures/evobrutinib_real.png)Figure 3:Evobrutinib case study\.Starting from the same “fallen angel” drug lead[Edwards et al\. \(2026\)](https://arxiv.org/html/2610.32103#bib.bib6), ReMedI identifies a candidate that simultaneously reduces all three predicted liabilities while preserving substantial structural similarity\. This example illustrates how inference\-time refinement can incorporate multiple beneficial edits into a single molecule while retaining the lead scaffold\.
#### Evobrutinib case study\.

Figure[3](https://arxiv.org/html/2610.32103#S5.F3)compares optimization of the failed drug Evobrutinib\. ReMedI simultaneously reduces AMES \(→0\.0500\.858\\\!\\rightarrow\\\!0\.050\), DILI \(→0\.5460\.826\\\!\\rightarrow\\\!0\.546\), and PGP \(→0\.4740\.648\\\!\\rightarrow\\\!0\.474\) while retaining similarity0\.7380\.738\. By contrast, C\-MORAL improves PGP but worsens DILI and AMES, with similarity0\.7070\.707\. Whereas the mCLM example applies sequential property\-specific edits, ReMedI finds a single search\-derived candidate that improves all three liabilities simultaneously\.

## 6Ablations

![Refer to caption](https://arxiv.org/html/2610.32103v1/figures/intermediate_feedback.png)Figure 4:SOR versus candidate budget on 250 GeLLMO\-Mistral instances spanning ten C\-MuMOInstruct tasks\. At matched candidate budgets, MH achieves the highest observed SOR\. Legend values report mean lead similarity\.#### Effect of search strategy\.

Figure[4](https://arxiv.org/html/2610.32103#S6.F4)compares four search strategies under the same evaluated candidate budget: \(i\)*Independent best\-of\-KK*, which samples every candidate directly from zero\-shot generations; \(ii\)*unguided sequential editing*, which repeatedly conditions on the most recent molecule and always advances to the next proposal; \(iii\)*greedy feedback*, which edits sequentially but always accepts proposals that improve the objective; and \(iv\)*MH*, which is our method\. Sequential methods all outperform independent sampling, among those, MH achieves the highest SOR at every tested budget, reaching 0\.41 atK=20K=20compared with 0\.39 for unguided editing and 0\.38 for greedy feedback\. MH also preserves greater lead similarity than unguided editing \(0\.44 vs\. 0\.39\), indicating that its gain does not come simply from allowing larger structural changes\. Overall, the figure demonstrates the advantage of sequential edits over one\-shot sampling, and that MH gives the best success–similarity tradeoff among the sequential strategies\.

![Refer to caption](https://arxiv.org/html/2610.32103v1/figures/sor_iteration.png)

\(a\) SOR and similarity\.

![Refer to caption](https://arxiv.org/html/2610.32103v1/figures/ssor_iteration.png)

\(b\) SSOR and similarity\.

Figure 5:Effect of repeated proposal distillation\. Red dashed lines show SOR/SSOR; blue lines show mean lead similarity\.q0q\_\{0\}is the original proposal andq1q\_\{1\}–q3q\_\{3\}are successive distillation rounds\.
#### Choosing the distillation round\.

Figure[5](https://arxiv.org/html/2610.32103#S6.F5)shows that the first distillation round captures the largest useful gain from search\-derived supervision\. Fromq0q\_\{0\}toq1q\_\{1\}, SOR increases from approximately0\.450\.45to0\.630\.63and SSOR from0\.120\.12to0\.160\.16, with only a modest decrease in similarity from0\.470\.47to0\.430\.43\. Additional rounds provide diminishing returns:q2q\_\{2\}improves SOR but not SSOR, whileq3q\_\{3\}raises SSOR further only as similarity falls to approximately0\.360\.36\. We therefore useq1q\_\{1\}as our primary proposal, as it captures the largest simultaneous improvement in SOR and SSOR before later distillation rounds increasingly trade locality for additional optimization\.

![Refer to caption](https://arxiv.org/html/2610.32103v1/figures/train_source.png)Figure 6:Effect of distillation sources onq1q\_\{1\}\-Base andq1q\_\{1\}\-MH\.−\-A,−\-B, and−\-C remove successful endpoints, intermediate states, and improving transitions, respectively\. Blue shows mean similarity; red dashed shows SSOR\.
#### Effect of distillation sources\.

Figure[6](https://arxiv.org/html/2610.32103#S6.F6)ablates the three supervision sources used for proposal distillation: \(A\)*lead\-conditioned successful endpoints*, which teach complete solutions from the original lead; \(B\)*intermediate\-state examples*, which train the proposal on states encountered during refinement; and \(C\)*accepted transitions*, which teach useful local edits that do not yet satisfy SOR\. Removing \(A\) causes the largest drop in SSOR, showing that endpoint supervision is essential for learning successful optimization\. Ablation \(A\) makes the proposal much more conservative: for q1\-Base, similarity rises from0\.300\.30to0\.5050\.505while SSOR drops from0\.1150\.115to0\.0700\.070; after MH refinement, similarity rises from0\.230\.23to0\.4140\.414while SSOR falls from0\.1600\.160to0\.0900\.090\. Thus, lead\-conditioned successful endpoints appear to provide the main supervision for learning edits that are large enough to satisfy the optimization targets\. Removing \(B\) or \(C\) lowers refined SSOR while also reducing similarity\. The full mixture gives the highest refined SSOR \(0\.1600\.160\), supporting trajectory\-level distillation and providing the strongest operating point for subsequent MH refinement\.

## 7Limitations & Conclusion

ReMedI reframes molecular optimization as inference\-time refinement: a CLM proposes sequential edits, MH selects them under an explicit lead\-relative objective, and successful search trajectories are distilled back into the proposal\. Across C\-MuMOInstruct and SMDD\-Bench, this improves frozen proposals and yields a stronger strict\-success–similarity tradeoff, while one round of distillation further improves both direct generation and subsequent search\. The ablations further support the role of each component: sequential editing provides the main search gain, MH improves the success–similarity tradeoff, and the three distillation sources contribute complementary supervision for direct generation and subsequent refinement\. A limitation of our method is all discoveries remains limitedin silicopredictions, which do not yet establishin vitroorin vivoefficacy\.

### AI use statement

Generative AI tools assisted with prose editing and routine code and analysis\. The authors are responsible for verifying claims, figures, and references before submission\.

## References

- Anthony et al\. \(2017\)Thomas Anthony, Zheng Tian, and David Barber\.Thinking fast and slow with deep learning and tree search, 2017\.URL[https://arxiv\.org/abs/1705\.08439](https://arxiv.org/abs/1705.08439)\.
- Bajusz et al\. \(2015\)D Bajusz, A Rácz, and K Héberger\.Why is tanimoto index an appropriate choice for fingerprint\-based similarity calculations?*Journal of cheminformatics*, 7:20, 2015\.doi:10\.1186/s13321\-015\-0069\-3\.URL[https://pubmed\.ncbi\.nlm\.nih\.gov/26052348/](https://pubmed.ncbi.nlm.nih.gov/26052348/)\.
- Dey et al\. \(2025a\)Vishal Dey, Xiao Hu, and Xia Ning\.Large language models for controllable multi\-property multi\-objective molecule optimization, 2025a\.URL[https://arxiv\.org/abs/2505\.23987](https://arxiv.org/abs/2505.23987)\.
- Dey et al\. \(2025b\)Vishal Dey, Xiao Hu, and Xia Ning\.Gellmo: Generalizing large language models for multi\-property molecule optimization, 2025b\.URL[https://arxiv\.org/abs/2502\.13398](https://arxiv.org/abs/2502.13398)\.
- Edwards et al\. \(2022\)Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji\.Translation between molecules and natural language, 2022\.URL[https://arxiv\.org/abs/2204\.11817](https://arxiv.org/abs/2204.11817)\.
- Edwards et al\. \(2026\)Carl Edwards, Chi Han, Gawon Lee, Thao Nguyen, Sara Szymkuć, Chetan Kumar Prasad, Bowen Jin, Jiawei Han, Ying Diao, Ge Liu, Hao Peng, Bartosz A\. Grzybowski, Martin D\. Burke, and Heng Ji\.mclm: A modular chemical language model that generates functional and makeable molecules, 2026\.URL[https://arxiv\.org/abs/2505\.12565](https://arxiv.org/abs/2505.12565)\.
- Gao et al\. \(2026\)Rui Gao, Youngseung Jeon, Swastik Roy, Morteza Ziyadi, and Xiang ’Anthony’ Chen\.C\-moral: Controllable multi\-objective molecular optimization with reinforcement alignment for llms, 2026\.URL[https://arxiv\.org/abs/2604\.23061](https://arxiv.org/abs/2604.23061)\.
- Han et al\. \(2026\)Kevin Han, Renfei Zhang, Kathy Wei, Hamed Mahdavi, Niloofar Mireshghallah, and Amir Barati Farimani\.Smdd\-bench: Can llms solve real\-world small molecule drug design tasks?, 2026\.URL[https://arxiv\.org/abs/2605\.21740](https://arxiv.org/abs/2605.21740)\.
- Hastings \(1970\)W\. K\. Hastings\.Monte carlo sampling methods using markov chains and their applications\.*Biometrika*, 57\(1\):97–109, 1970\.doi:10\.1093/biomet/57\.1\.97\.
- Huang et al\. \(2022\)Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik\.Artificial intelligence foundation for therapeutic science\.*Nature Chemical Biology*, 2022\.
- Landrum \(2026\)Greg Landrum\.RDKit: Open\-source cheminformatics\.[https://www\.rdkit\.org](https://www.rdkit.org/), 2026\.Accessed: \[Insert Date Here\]\.
- Li et al\. \(2026\)Xuan Li, Zhanke Zhou, Zongze Li, Jiangchao Yao, Yu Rong, Lu Zhang, and Bo Han\.Reference\-guided policy optimization for molecular optimization via llm reasoning, 2026\.URL[https://arxiv\.org/abs/2603\.05900](https://arxiv.org/abs/2603.05900)\.
- Lin et al\. \(2026\)Xuan Lin, Long Chen, Yile Wang, Yangyang Chen, and Xiangxiang Zeng\.Property enhanced instruction tuning for multi\-task molecule generation with large language models, 2026\.URL[https://arxiv\.org/abs/2412\.18084](https://arxiv.org/abs/2412.18084)\.
- Liu et al\. \(2023a\)Shengchao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Anima Anandkumar\.Multi\-modal molecule structure–text model for text\-based retrieval and editing\.*Nature Machine Intelligence*, 5:1447–1457, 2023a\.doi:10\.1038/s42256\-023\-00759\-6\.
- Liu et al\. \(2023b\)Shengchao Liu, Jiongxiao Wang, Yijin Yang, Chengpeng Wang, Ling Liu, Hongyu Guo, and Chaowei Xiao\.Chatgpt\-powered conversational drug editing using retrieval and domain feedback, 2023b\.URL[https://arxiv\.org/abs/2305\.18090](https://arxiv.org/abs/2305.18090)\.
- Maggiora et al\. \(2014\)G Maggiora, M Vogt, D Stumpfe, and J Bajorath\.Molecular similarity in medicinal chemistry\.*Journal of medicinal chemistry*, 57\(8\):3186–204, 2014\.doi:10\.1021/jm401411z\.URL[https://pubmed\.ncbi\.nlm\.nih\.gov/24151987/](https://pubmed.ncbi.nlm.nih.gov/24151987/)\.
- Metropolis et al\. \(1953\)Nicholas Metropolis, Arianna W\. Rosenbluth, Marshall N\. Rosenbluth, Augusta H\. Teller, and Edward Teller\.Equation of state calculations by fast computing machines\.*Journal of Chemical Physics*, 21\(6\):1087–1092, 1953\.doi:10\.1063/1\.1699114\.
- Morgan \(1965\)Harry L\. Morgan\.The generation of a unique machine description for chemical structures\-a technique developed at chemical abstracts service\.*Journal of Chemical Documentation*, 5:107–113, 1965\.URL[https://api\.semanticscholar\.org/CorpusID:62164893](https://api.semanticscholar.org/CorpusID:62164893)\.
- Nash & Jr\. \(1950\)John F\. Nash and Jr\.The bargaining problem\.*Econometrica*, 18\(2\):155–162, 1950\.URL[https://onlinelibrary\.wiley\.com/doi/abs/0012\-9682\(195004\)18:2&lt;155:TBP&gt;2\.0\.CO;2\-H](https://onlinelibrary.wiley.com/doi/abs/0012-9682(195004)18:2&lt;155:TBP&gt;2.0.CO;2-H)\.
- Navratil et al\. \(2025\)Jiri Navratil, Jarret Ross, Payel Das, Youssef Mroueh, Samuel C Hoffman, Vijil Chenthamarakshan, and Brian Belgodere\.Gp\-molformer\-sim: Test time molecular optimization through contextual similarity guidance, 2025\.URL[https://arxiv\.org/abs/2506\.05628](https://arxiv.org/abs/2506.05628)\.
- Olivecrona et al\. \(2017\)Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen\.Molecular de novo design through deep reinforcement learning, 2017\.URL[https://arxiv\.org/abs/1704\.07555](https://arxiv.org/abs/1704.07555)\.
- Ouyang et al\. \(2022\)Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L\. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe\.Training language models to follow instructions with human feedback, 2022\.URL[https://arxiv\.org/abs/2203\.02155](https://arxiv.org/abs/2203.02155)\.
- Popova et al\. \(2018\)Mariya Popova, Olexandr Isayev, and Alexander Tropsha\.Deep reinforcement learning for de novo drug design\.*Science Advances*, 4\(7\), 2018\.ISSN 2375\-2548\.doi:10\.1126/sciadv\.aap7885\.URL[http://dx\.doi\.org/10\.1126/sciadv\.aap7885](http://dx.doi.org/10.1126/sciadv.aap7885)\.
- Rafailov et al\. \(2024\)Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D\. Manning, and Chelsea Finn\.Direct preference optimization: Your language model is secretly a reward model, 2024\.URL[https://arxiv\.org/abs/2305\.18290](https://arxiv.org/abs/2305.18290)\.
- Rogers & Hahn \(2010\)David Rogers and Mathew Hahn\.Extended\-connectivity fingerprints\.*Journal of Chemical Information and Modeling*, 50\(5\):742–754, 2010\.doi:10\.1021/ci100050t\.
- Schulman et al\. \(2017\)John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov\.Proximal policy optimization algorithms, 2017\.URL[https://arxiv\.org/abs/1707\.06347](https://arxiv.org/abs/1707.06347)\.
- Shao et al\. \(2024\)Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y\. K\. Li, Y\. Wu, and Daya Guo\.Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024\.URL[https://arxiv\.org/abs/2402\.03300](https://arxiv.org/abs/2402.03300)\.
- Swanson et al\. \(2024\)Kyle Swanson, Patrick Walther, Jeremy Leitz, Saurabh Mukherjee, Joseph C\. Wu, Rohan V\. Shivnaraine, and James Zou\.ADMET\-AI: A machine learning ADMET platform for evaluation of large\-scale chemical libraries\.*Bioinformatics*, 40\(7\):btae416, 2024\.doi:10\.1093/bioinformatics/btae416\.
- Wang et al\. \(2025\)Ziqing Wang, Kexin Zhang, Zihan Zhao, Yibo Wen, Abhishek Pandey, Han Liu, and Kaize Ding\.A survey of large language models for text\-guided molecular discovery: from molecule generation to optimization, 2025\.URL[https://arxiv\.org/abs/2505\.16094](https://arxiv.org/abs/2505.16094)\.
- Wu et al\. \(2024\)Zhenxing Wu, Odin Zhang, Xiaorui Wang, Li Fu, Huifeng Zhao, Jike Wang, Hongyan Du, Dejun Jiang, Yafeng Deng, Dongsheng Cao, Chang\-Yu Hsieh, and Tingjun Hou\.Leveraging language model for advanced multiproperty molecular optimization via prompt engineering\.*Nature Machine Intelligence*, 6:1359–1369, 2024\.doi:10\.1038/s42256\-024\-00916\-5\.
- Xie et al\. \(2021\)Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang, Weinan Zhang, Yong Yu, and Lei Li\.MARS: Markov molecular sampling for multi\-objective drug discovery\.In*International Conference on Learning Representations*, 2021\.URL[https://openreview\.net/forum?id=kHSu4ebxFXY](https://openreview.net/forum?id=kHSu4ebxFXY)\.
- Ye et al\. \(2023\)Geyan Ye, Xibao Cai, Houtim Lai, Xing Wang, Junhong Huang, Longyue Wang, Wei Liu, and Xiangxiang Zeng\.Drugassist: A large language model for molecule optimization, 2023\.URL[https://arxiv\.org/abs/2401\.10334](https://arxiv.org/abs/2401.10334)\.
- Yim et al\. \(2021\)DS Yim, SH Bae, and S Choi\.Predicting human pharmacokinetics from preclinical data: clearance\.*Translational and clinical pharmacology*, 29\(2\):78–87, 2021\.doi:10\.12793/tcp\.2021\.29\.e12\.URL[https://pubmed\.ncbi\.nlm\.nih\.gov/34235120/](https://pubmed.ncbi.nlm.nih.gov/34235120/)\.
- Yu et al\. \(2024\)Botao Yu, Frazier N\. Baker, Ziqi Chen, Xia Ning, and Huan Sun\.Llasmol: Advancing large language models for chemistry with a large\-scale, comprehensive, high\-quality instruction tuning dataset, 2024\.URL[https://arxiv\.org/abs/2402\.09391](https://arxiv.org/abs/2402.09391)\.
- Zhou et al\. \(2019\)Zhenpeng Zhou, Steven Kearnes, Li Li, Richard N\. Zare, and Patrick Riley\.Optimization of molecules via deep reinforcement learning\.*Scientific Reports*, 9\(1\), 2019\.ISSN 2045\-2322\.doi:10\.1038/s41598\-019\-47148\-x\.URL[http://dx\.doi\.org/10\.1038/s41598\-019\-47148\-x](http://dx.doi.org/10.1038/s41598-019-47148-x)\.

## Appendix AAppendix

### A\.1Benchmark Metrics

Letfr​\(x\)f\_\{r\}\(x\)denote propertyrrof moleculexx, and letsr∈\{\+1,−1\}s\_\{r\}\\in\\\{\+1,\-1\\\}indicate whether higher or lower values are preferred\. For leadx0,ix\_\{0,i\}and outputxix\_\{i\}, SOR requires each targeted property to improve by its prescribed marginΔr\\Delta\_\{r\}and each protected property to remain within toleranceϵr\\epsilon\_\{r\}:

SOR=1n∑i=1n∏r∈𝒫improve𝟏\{sr\(fr\(xi\)−fr\(x0,i\)\)≥Δr\}∏r∈𝒫preserve𝟏\{\|fr\(xi\)−fr\(x0,i\)\|≤ϵr\}\.\\mathrm\{SOR\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\prod\_\{r\\in\\mathcal\{P\}\_\{\\mathrm\{improve\}\}\}\\mathbf\{1\}\\\{s\_\{r\}\(f\_\{r\}\(x\_\{i\}\)\-f\_\{r\}\(x\_\{0,i\}\)\)\\geq\\Delta\_\{r\}\\\}\\prod\_\{r\\in\\mathcal\{P\}\_\{\\mathrm\{preserve\}\}\}\\mathbf\{1\}\\\{\|f\_\{r\}\(x\_\{i\}\)\-f\_\{r\}\(x\_\{0,i\}\)\|\\leq\\epsilon\_\{r\}\\\}\.\(8\)SSOR instead checks whether all relevant properties meet their prescribed near\-optimal thresholdsΘr\\Theta\_\{r\}\([Gao et al\., 2026](https://arxiv.org/html/2610.32103#bib.bib7)\):

SSOR=1n∑i=1n∏r∈𝒫\[sr\(fr\(xi\)−Θr\)≥0\]\.\\mathrm\{SSOR\}=\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\prod\_\{r\\in\\mathcal\{P\}\}\\mathbf\{1\}\\\!\\left\[s\_\{r\}\\bigl\(f\_\{r\}\(x\_\{i\}\)\-\\Theta\_\{r\}\\bigr\)\\geq 0\\right\]\.\(9\)
For example, consider hypothetical normalized permeability and toxicity scores, with targets of permeability≥0\.8\\geq 0\.8and toxicity≤0\.2\\leq 0\.2\. Suppose the lead has permeability0\.40\.4and toxicity0\.10\.1, and the request requires a permeability increase of at least0\.10\.1while preserving toxicity\. An output with permeability0\.60\.6and unchanged toxicity passes SOR but fails SSOR: permeability improves sufficiently but remains below its absolute target\. An output with permeability0\.850\.85and unchanged toxicity passes both\.

Table 3:C\-MuMOInstruct performance of C\-MORAL and ReMedI across GeLLMO\-C \(Llama, Mistral\) and mCLM\-4B backbones\.n=50n=50held\-out seeds/task, best\-of\-2020\.*q0q\_\{0\}*is the frozen proposal,*q1q\_\{1\}*one round of search distillation,*base*direct generation, and*MH*inference\-time refinement\. This 50\-lead\-per\-task summary uses the candidate selection in the source draft \(success, then strict success, then similarity\), distinct from the success\-then\-maximum\-similarity selector of the 25\-lead coverage figures\. SOR, SSOR, and Sim are reported across methods; RI is a within\-method diagnostic and is not comparable across methods \(C\-MORAL’s RI is computed on a different scale\), so RI rows are not bolded\. Bold marks the best comparable value in each row, excluding*q0q\_\{0\}\-base*\.Table 4:Effect of trajectory depth on frozen GeLLMO\-Mistral\. Every configuration uses 80 proposals per lead\. Results aggregate IND and OOD; bold marks the best value in each column\.
### A\.2Performance across backbones\.

Table[3](https://arxiv.org/html/2610.32103#A1.T3)reports the full C\-MuMOInstruct results across GeLLMO\-Llama, GeLLMO\-Mistral, and mCLM\-4B\. Inference\-time refinement consistently improves the frozen proposal:q1q\_\{1\}\-MH increases both SOR and SSOR overq1q\_\{1\}\-base for every backbone and split\. Distillation further strengthens the proposal on the GeLLMO backbones, and combining one round of distillation with refinement \(q1q\_\{1\}\-MH\) gives the highest SOR and SSOR in nearly all settings\. The gains also extend to OOD property combinations, indicating that trajectories distilled from IND tasks improve optimization beyond the combinations used for training\. Similarity generally decreases as optimization becomes more aggressive, motivating the similarity\-constrained coverage analysis in the main text rather than interpreting success and similarity independently\. RI is included only as a within\-method diagnostic because its scale is not comparable to the C\-MORAL implementation\.

### A\.3Trajectory Depth Ablation

Table[4](https://arxiv.org/html/2610.32103#A1.T4)compares trajectory depths at a fixed budget of 80 proposals per lead\. Independent proposals preserve locality but rarely reach the stringent target\. Replacing 80 independent proposals with 10 trajectories of eight steps doubles overall SSOR from0\.0700\.070to0\.1400\.140\. Because every configuration evaluates the same number of proposals, this gain cannot be attributed to additional sampling\. Instead, sequential conditioning allows multiple edits to compose into solutions satisfying the stringent multi\-property constraints\.