Are we really tilting? The mechanics of reward guidance in flow and diffusion models

# Are we really tilting? The mechanics of reward guidance in flow and diffusion models
Source: [https://arxiv.org/html/2606.02884](https://arxiv.org/html/2606.02884)
###### Abstract

Reward guidance algorithms steer a learned generative process toward the reward\-tilted measure at inference time\. While empirically powerful, these methods are prone to*reward hacking*: the guided model over\-optimizes the reward at the cost of fidelity to the learned distribution\. Prior work has attributed this to the complexity of neural reward functions or implicit biases in diffusion training, but its fundamental origins remain poorly understood\. We show that reward hacking arises from an approximation made in most practical implementations of reward\-guided diffusion—finite\-particleplug\-in estimationof the Doobhh\-function—even in the simplest non\-trivial settings of Gaussian and Gaussian mixture targets with quadratic rewards\. In closed form, we isolate two distinct failure modes of the plug\-in estimator: it leads toreward hacking within each modeand itcannot select high\-reward modes\. We propose a closed\-form reward damping schedule that corrects the within\-mode bias with no additional compute, and clarify the role of best\-of\-nnsampling in compensating for the mode selection failure\. Experiments on Gaussian mixture targets, a 2D checkerboard, and FLUX\.1 text\-to\-image generation confirm that our theoretical insights carry over to practical settings\.

![Refer to caption](https://arxiv.org/html/2606.02884v1/x1.png)Figure 1:Reward damping\.We introduce*reward damping*, a simple and principled guidance schedule to mitigate reward hacking\. Base FLUX\.1\[[1](https://arxiv.org/html/2606.02884#bib.bib1)\]samples guided with ImageReward\[[2](https://arxiv.org/html/2606.02884#bib.bib2)\]; further experimental details in[Section˜D\.4](https://arxiv.org/html/2606.02884#A4.SS4)\.## Introduction

Flow and diffusion\-based generative models have become the dominant paradigm for high\-quality sample generation across diverse domains, powering state\-of\-the\-art systems for text\-to\-image synthesis\[[3](https://arxiv.org/html/2606.02884#bib.bib3),[4](https://arxiv.org/html/2606.02884#bib.bib4),[5](https://arxiv.org/html/2606.02884#bib.bib5),[1](https://arxiv.org/html/2606.02884#bib.bib1)\], molecular design\[[6](https://arxiv.org/html/2606.02884#bib.bib6),[7](https://arxiv.org/html/2606.02884#bib.bib7),[8](https://arxiv.org/html/2606.02884#bib.bib8),[9](https://arxiv.org/html/2606.02884#bib.bib9)\], and protein structure prediction\[[10](https://arxiv.org/html/2606.02884#bib.bib10),[11](https://arxiv.org/html/2606.02884#bib.bib11),[12](https://arxiv.org/html/2606.02884#bib.bib12),[13](https://arxiv.org/html/2606.02884#bib.bib13)\]\. In many applications, however, sampling from the learned distributionρ1​\(x\)\\rho\_\{1\}\(x\)is not the end goal—we often want samples from the reward\-tilted measureρ~1​\(x\)∝ρ1​\(x\)​eλ​r​\(x\)\\tilde\{\\rho\}\_\{1\}\(x\)\\propto\\rho\_\{1\}\(x\)\\,e^\{\\lambda r\(x\)\}given a reward functionrrand inverse temperatureλ\\lambda\. For instance, practitioners may desire images that better match a text prompt\[[2](https://arxiv.org/html/2606.02884#bib.bib2),[14](https://arxiv.org/html/2606.02884#bib.bib14),[15](https://arxiv.org/html/2606.02884#bib.bib15),[16](https://arxiv.org/html/2606.02884#bib.bib16),[17](https://arxiv.org/html/2606.02884#bib.bib17)\], molecules with higher binding affinity to a target protein pocket\[[18](https://arxiv.org/html/2606.02884#bib.bib18),[19](https://arxiv.org/html/2606.02884#bib.bib19),[20](https://arxiv.org/html/2606.02884#bib.bib20)\], or proteins with improved stability\[[10](https://arxiv.org/html/2606.02884#bib.bib10),[21](https://arxiv.org/html/2606.02884#bib.bib21),[12](https://arxiv.org/html/2606.02884#bib.bib12)\]\. This is the problem of*reward\-guided generation*\. Widely used methods such as classifier guidance\[[22](https://arxiv.org/html/2606.02884#bib.bib22)\]and classifier\-free guidance\[[23](https://arxiv.org/html/2606.02884#bib.bib23)\]can be understood as instances of reward guidance for particular choices of reward function, and more general value function approaches have been developed inUehara et al\. \[[24](https://arxiv.org/html/2606.02884#bib.bib24),[25](https://arxiv.org/html/2606.02884#bib.bib25)\]\.

Despite widespread empirical use, these methods are often prone to*reward hacking*: as the guidance scale increases, generated samples over\-optimize the reward at the expense of fidelity to the learned distribution, producing unrealistic or degenerate outputs\[[26](https://arxiv.org/html/2606.02884#bib.bib26)\]\. While prior experimental work has attributed reward hacking to the complexity of neural reward functions\[[26](https://arxiv.org/html/2606.02884#bib.bib26),[27](https://arxiv.org/html/2606.02884#bib.bib27),[28](https://arxiv.org/html/2606.02884#bib.bib28)\]or implicit biases in generative model training\[[29](https://arxiv.org/html/2606.02884#bib.bib29)\], the fundamental origins of reward hacking still remain poorly understood\. Similarly, it has been empirically observed that taking the highest\-reward option out ofnnguided samples \(*best\-of\-nn*\) significantly improves the performance of guidance in generative models in many settings, but the precise mechanism underlying the improvement from best\-of\-nnis not well understood\. This motivates the primary question that we study in this work:

*Why does guidance fail to sample from the reward tilt, and how can we mitigate this bias?*

\\lxSVG@picture

Base targetAnalytic reward tiltingPractical guidanceDamping \+ best\-of\-nSamples from a mixtureof several modesSelects correct modeand optimizes rewardOverconcentratesandovershootsthe meanRecovers similar behaviorto analytic tilting\\lxSVG@picture\\endlxSVG@picturebase target\\lxSVG@picture\\endlxSVG@picturebase samples\\lxSVG@picture\\endlxSVG@picturetilted samples\\lxSVG@picture\\endlxSVG@pictureguided samples\\lxSVG@picture\\endlxSVG@pictureour proposal\\lxSVG@picture\\endlxSVG@picturereward maximizer\\endlxSVG@picture

Figure 2:Overview\.Compared to analytic reward tilting, practical guidance algorithms over\-concentrate within each mode and fail to select high\-reward modes\. We propose a damped reward scale to mitigate within\-mode reward hacking and clarify the role of best\-of\-nnin mode selection; combining these two methods often enables us to approximately recover the reward tilt\.Altogether, ourmain contributionsare \(visually summarized in[Figure˜2](https://arxiv.org/html/2606.02884#S1.F2)\):

1. 1\.We prove in Gaussian settings that significant within\-mode reward hacking arises from the*finite\-particle plug\-in estimation*in most implementations of reward\-guided diffusion\.
2. 2\.We show in Gaussian mixture settings that guidance using plug\-in estimation fails to select between modes and has no mechanism for accurately weighting distant high\-reward modes\.
3. 3\.We propose a simple closed form damped reward scheduleλt\\lambda\_\{t\}to mitigate within\-mode reward hacking and clarify the role of best\-of\-nnsampling in performing mode selection\.
4. 4\.We demonstrate the effectiveness of our damped reward schedule and showcase the role of best\-of\-nnsampling through a diverse set of experiments on Gaussian mixture targets, a 2D checkerboard, and FLUX\.1 text\-to\-image generation\.

## Background on reward guidance

We begin by reviewing the general theoretical framework for reward guidance in generative models through stochastic interpolants and the Doobhh\-transform\.

### Stochastic interpolants and flow matching

We assume access to a pre\-trained flow modelb:\[0,1\]×ℝd→ℝdb:\[0,1\]\\times\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}^\{d\}, and samples from the data distributionρ1∈𝒫​\(ℝd\)\\rho\_\{1\}\\in\\mathcal\{P\}\(\\mathbb\{R\}^\{d\}\)are drawn by numerically integrating the probability flow ODE

x˙t=bt​\(xt\)\\displaystyle\\dot\{x\}\_\{t\}=b\_\{t\}\(x\_\{t\}\)starting from noisex0∼𝒩​\(0,Id\)x\_\{0\}\\sim\\mathcal\{N\}\(0,I\_\{d\}\)untilt=1t=1\. Such a modelbtb\_\{t\}can be obtained by minimizing the flow matching objective\[[30](https://arxiv.org/html/2606.02884#bib.bib30),[31](https://arxiv.org/html/2606.02884#bib.bib31)\]\. To facilitate our analysis, we consider the casebt:=𝔼​\[I˙t\|It=x\]b\_\{t\}:=\\mathbb\{E\}\[\\dot\{I\}\_\{t\}\\,\|\\,I\_\{t\}=x\], where

It:=\(1−t\)​I0\+t​I1I\_\{t\}:=\(1\-t\)I\_\{0\}\+tI\_\{1\}\(1\)is the*linear interpolant*forI0∼𝒩​\(0,Id\)I\_\{0\}\\sim\\mathcal\{N\}\(0,I\_\{d\}\)andI1∼ρ1I\_\{1\}\\sim\\rho\_\{1\}independent\. Furthermore, ifσt∈ℝd×ℓ\\sigma\_\{t\}\\in\\mathbb\{R\}^\{d\\times\\ell\}is any fixed noise schedule andρt\\rho\_\{t\}denotes the density ofxtx\_\{t\}, the solution of the SDE

d​Xt=\(bt​\(Xt\)\+12​\(σt​σt⊤\)​∇log⁡ρt​\(Xt\)\)​d​t\+σt​d​BtdX\_\{t\}=\\left\(b\_\{t\}\(X\_\{t\}\)\+\\frac\{1\}\{2\}\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\nabla\\log\\rho\_\{t\}\(X\_\{t\}\)\\right\)dt\+\\sigma\_\{t\}\\,dB\_\{t\}\(2\)also shares the same time\-marginal distribution asxtx\_\{t\}\([Section˜A\.1](https://arxiv.org/html/2606.02884#A1.SS1)\)\.

### Reward guidance and plug\-in estimation

The problem of reward\-guided generation is often formulated as steering a trained generative model at inference\-time \(without additional fine\-tuning\) to obtain samples from the*reward\-tilted measure*

ρ~1​\(x\)∝ρ1​\(x\)​eλ​r​\(x\)\\tilde\{\\rho\}\_\{1\}\(x\)\\propto\\rho\_\{1\}\(x\)\\,e^\{\\lambda r\(x\)\}\(3\)for a reward functionr:ℝd→ℝr:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}and inverse temperatureλ\>0\\lambda\>0\. The reward\-tilted measure \([3](https://arxiv.org/html/2606.02884#S2.E3)\) commonly arises in reinforcement learning\[[25](https://arxiv.org/html/2606.02884#bib.bib25)\]as the solution to a KL\-regularized variational optimization problem \([Section˜A\.2](https://arxiv.org/html/2606.02884#A1.SS2)\)\.

##### Doobhh\-transform,

The Doobhh\-transform \(e\.g\.,\[[32](https://arxiv.org/html/2606.02884#bib.bib32)\]\) provides a principled framework to solve the reward guidance problem by modifying the drift of the unguided SDE \([2](https://arxiv.org/html/2606.02884#S2.E2)\) to steer the terminal distribution toward the reward\-tilted measure \([3](https://arxiv.org/html/2606.02884#S2.E3)\)\. Defining the*Doobhh\-function*ht​\(x\)≔𝔼​\[eλ​r​\(X1\)\|Xt=x\]h\_\{t\}\(x\)\\coloneq\\mathbb\{E\}\[e^\{\\lambda r\(X\_\{1\}\)\}\\,\|\\,X\_\{t\}=x\], the*Doobhh\-transform*follows the guided ODE

x~˙t=bt​\(x~t\)\+12​\(σt​σt⊤\)​∇log⁡ht​\(x~t\)\\dot\{\\tilde\{x\}\}\_\{t\}=b\_\{t\}\(\\tilde\{x\}\_\{t\}\)\+\\frac\{1\}\{2\}\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\nabla\\log h\_\{t\}\(\\tilde\{x\}\_\{t\}\)\(4\)fromx~0=x0\\tilde\{x\}\_\{0\}=x\_\{0\}yields a samplex~1∼ρ~1\\tilde\{x\}\_\{1\}\\sim\\tilde\{\\rho\}\_\{1\}from the reward tilt*as long asX0⟂⟂X1X\_\{0\}\\perp\\\!\\\!\\\!\\perp X\_\{1\}*\. To satisfy this constraint and ensure validity of the Doobhh\-transform,Domingo\-Enrich et al\. \[[33](https://arxiv.org/html/2606.02884#bib.bib33)\]demonstrates that choosing the*memoryless noise schedule*σt=2​\(1−t\)/t​Id\\sigma\_\{t\}=\\sqrt\{2\(1\-t\)/t\}\\,I\_\{d\}ensures thatX0⟂⟂X1X\_\{0\}\\perp\\\!\\\!\\\!\\perp X\_\{1\}\.

##### Plug\-in estimation\.

The Doobhh\-functionht​\(x\)=𝔼​\[eλ​r​\(X1\)\|Xt=x\]h\_\{t\}\(x\)=\\mathbb\{E\}\[e^\{\\lambda r\(X\_\{1\}\)\}\\,\|\\,X\_\{t\}=x\]in[Theorem˜12](https://arxiv.org/html/2606.02884#Thmtheorem12)is intractable in general, so practitioners usually approximate it with a*kk\-particle plug\-in estimator*

h^t\(k\)​\(x\)=1k​∑i=1keλ​r​\(X1\(i\)\),\\hat\{h\}^\{\(k\)\}\_\{t\}\(x\)=\\frac\{1\}\{k\}\\sum\_\{i=1\}^\{k\}e^\{\\lambda r\(X\_\{1\}^\{\(i\)\}\)\},\(5\)whereX1\(1\),…,X1\(k\)∼p1\|t\(⋅\|x\)X\_\{1\}^\{\(1\)\},\\ldots,X\_\{1\}^\{\(k\)\}\\sim p\_\{1\|t\}\(\\cdot\\,\|\\,x\)are independent samples \(p1\|tp\_\{1\|t\}denotes the law of\(X1\|Xt=x\)\(X\_\{1\}\\,\|\\,X\_\{t\}=x\)\)\. In practice,kkis often chosen to be small due to computational constraints\[[34](https://arxiv.org/html/2606.02884#bib.bib34),[35](https://arxiv.org/html/2606.02884#bib.bib35),[36](https://arxiv.org/html/2606.02884#bib.bib36)\], introducing finite\-sample bias\.

## Related work

In this section, we briefly review related work on reward guidance for generative models\.

##### Applications of reward guidance\.

Reward guidance has become a central ingredient in aligning generative models with downstream objectives across several domains\. In text\-to\-image generation, learned human\-preference reward models such as ImageReward\[[2](https://arxiv.org/html/2606.02884#bib.bib2)\], PickScore\[[14](https://arxiv.org/html/2606.02884#bib.bib14)\], and HPSv2\[[37](https://arxiv.org/html/2606.02884#bib.bib37)\]are routinely used to fine\-tune and steer state\-of\-the\-art diffusion and flow\-matching models\. In structure\-based drug design, reward guidance is used to bias diffusion\-generated ligands toward target pockets and higher predicted binding affinity\[[18](https://arxiv.org/html/2606.02884#bib.bib18),[19](https://arxiv.org/html/2606.02884#bib.bib19),[20](https://arxiv.org/html/2606.02884#bib.bib20)\]\. In protein design, classifier\-guided and fine\-tuned diffusion models are used to generate sequences and structures optimized for stability, binding, and other functional properties\[[10](https://arxiv.org/html/2606.02884#bib.bib10),[21](https://arxiv.org/html/2606.02884#bib.bib21),[12](https://arxiv.org/html/2606.02884#bib.bib12),[24](https://arxiv.org/html/2606.02884#bib.bib24)\]\.

##### Practical approaches to reward guidance\.

The*stochastic interpolant*framework\[[31](https://arxiv.org/html/2606.02884#bib.bib31),[38](https://arxiv.org/html/2606.02884#bib.bib38)\]provides a flexible formulation of diffusion and flow\-matching generative models, and underpins many modern reward\-alignment algorithms\. Within this framework,*GLASS flows*\[[34](https://arxiv.org/html/2606.02884#bib.bib34)\]implement the Doobhh\-transform by approximating the intractable value functionht​\(x\)=𝔼​\[eλ​r​\(X1\)\|Xt=x\]h\_\{t\}\(x\)=\\mathbb\{E\}\[e^\{\\lambda r\(X\_\{1\}\)\}\\,\|\\,X\_\{t\}=x\]with akk\-particle plug\-in estimator, where each particle is obtained by simulating an inner ODE starting from fresh Gaussian noise\. This inner ODE simulation is expensive, motivating the development of*stochastic flow map*frameworks that sample the posterior\(X1\|Xt\)\(X\_\{1\}\\,\|\\,X\_\{t\}\)in a single step\[[35](https://arxiv.org/html/2606.02884#bib.bib35),[36](https://arxiv.org/html/2606.02884#bib.bib36)\]\. A different deterministic approach is*flow map reward guidance*\[[39](https://arxiv.org/html/2606.02884#bib.bib39)\], which backpropagates the reward gradient through the unguided probability flow ODE fromXtX\_\{t\}toX1X\_\{1\}\. A complementary line of work\[[24](https://arxiv.org/html/2606.02884#bib.bib24),[25](https://arxiv.org/html/2606.02884#bib.bib25),[33](https://arxiv.org/html/2606.02884#bib.bib33)\]casts reward\-guided generation as a reinforcement learning or stochastic optimal control problem and fine\-tunes the generative model via policy\-gradient or value\-function learning; we discuss the connection to stochastic optimal control in[Section˜A\.3](https://arxiv.org/html/2606.02884#A1.SS3)\.

##### Theoretical results for guidance\.

Several recent works have studied guidance in simplified settings\.Chidambaram et al\. \[[40](https://arxiv.org/html/2606.02884#bib.bib40)\]analyzed classifier\-free guidance for Gaussian mixtures, showing it can amplify the signal\-to\-noise ratio but distort mode weights\.Wu et al\. \[[41](https://arxiv.org/html/2606.02884#bib.bib41)\]provided theoretical insights under classifier guidance, whilePavasovic et al\. \[[42](https://arxiv.org/html/2606.02884#bib.bib42)\]andVentura et al\. \[[43](https://arxiv.org/html/2606.02884#bib.bib43)\]study classifier\-free guidance from a high\-dimensional perspective\.Wang et al\. \[[44](https://arxiv.org/html/2606.02884#bib.bib44)\]finds that monotonic schedulers perform well for classifier\-free guidance in practice, and our reward damping schedule in[Section˜4\.1\.2](https://arxiv.org/html/2606.02884#S4.SS1.SSS2)is a principled choice for such a monotonic schedule\. These works focus on specific guidance schemes; in contrast, we study the bias introduced by practical approximations to the general reward\-guided Doobhh\-transform framework\. Closest to our setting,Potaptchik et al\. \[[35](https://arxiv.org/html/2606.02884#bib.bib35)\]prove that thekk\-particle plug\-in estimator of the Doobhh\-transform converges to exact guidance in the limit ask→∞k\\to\\infty; we complement this asymptotic guarantee by characterizing the bias induced by a finitekkand showing that it systematically produces overly aggressive guidance\. A recent line of work also proves computational hardness results for reward\-guided diffusion models\[[45](https://arxiv.org/html/2606.02884#bib.bib45),[46](https://arxiv.org/html/2606.02884#bib.bib46)\], showing that exact reward tilting is NP\-hard even with certain quadratic rewards\.

## Theoretical results

### Within\-mode reward hacking

We now show that the plug\-in approximation \([5](https://arxiv.org/html/2606.02884#S2.E5)\) leads to reward hacking within a single mode; we consider a Gaussian target𝒩​\(μ,Σ\)\\mathcal\{N\}\(\\mu,\\Sigma\)and a quadratic rewardr​\(x\)=−∥x−a∥22r\(x\)=\-\\lVert x\-a\\rVert\_\{2\}^\{2\}for a target pointa∈ℝda\\in\\mathbb\{R\}^\{d\}\. We will compare the plug\-in guidance to the analytic guidance framework described in[Section˜B\.1](https://arxiv.org/html/2606.02884#A2.SS1), which provides the benchmark for exact reward guidance\. Throughout this section, we use the notation of[Section˜2](https://arxiv.org/html/2606.02884#S2)and assume noise scheduleσt=ηt​Id\\sigma\_\{t\}=\\eta\_\{t\}I\_\{d\}for someηt\>0\\eta\_\{t\}\>0\. We letμt=t​μ\\mu\_\{t\}=t\\muandΣt=\(1−t\)2​Id\+t2​Σ\\Sigma\_\{t\}=\(1\-t\)^\{2\}I\_\{d\}\+t^\{2\}\\Sigmadenote the mean and covariance ofXtX\_\{t\}, and let

μ1\|t​\(x\)=μ\+Σ1/2​Σt−1/2​Ψt1/2​\(x−μt\),Σ1\|t=Σ​\(Id−Ψt\),\\displaystyle\\mu\_\{1\|t\}\(x\)=\\mu\+\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\(x\-\\mu\_\{t\}\),\\qquad\\Sigma\_\{1\|t\}=\\Sigma\(I\_\{d\}\-\\Psi\_\{t\}\),denote the mean and covariance of the Gaussian distribution\(X1\|Xt=x\)\(X\_\{1\}\\,\|\\,X\_\{t\}=x\), whereΨt≔exp⁡\(−∫t1ηv2​Σv−1​𝑑v\)\\Psi\_\{t\}\\coloneq\\exp\\\!\\left\(\-\\int\_\{t\}^\{1\}\\eta\_\{v\}^\{2\}\\,\\Sigma\_\{v\}^\{\-1\}\\,dv\\right\)\(see[Section˜B\.2](https://arxiv.org/html/2606.02884#A2.SS2)for the derivation\)\.

#### Plug\-in estimation bias

In this section, we analyze the bias due to thekk\-particle plug\-in estimator \([5](https://arxiv.org/html/2606.02884#S2.E5)\) in the limit as the step size in the numerical ODE solver goes to zero; we call this the*plug\-in flow*\. Definingut\(k\)​\(x\)=𝔼​\[∇log⁡h^t\(k\)​\(Xt\)\|Xt=x\]u\_\{t\}^\{\(k\)\}\(x\)=\\mathbb\{E\}\[\\nabla\\log\\hat\{h\}^\{\(k\)\}\_\{t\}\(X\_\{t\}\)\\,\|\\,X\_\{t\}=x\]\(where the expectation is taken over thekkparticle draws\), we obtain the following result\.

###### Proposition 1\(Convergence to plug\-in flow, informal\)\.

Under appropriate regularity conditions, the Euler trajectoryx¯t\\bar\{x\}\_\{t\}obtained by running the guided ODE with the plug\-in estimator converges uniformly in probability as the step size in the solver goes to zero:

supt∈\[0,1\]∥x¯t−x~t∥2​⟶𝑝​0\.\\sup\_\{t\\in\[0,1\]\}\\;\\lVert\\bar\{x\}\_\{t\}\-\\tilde\{x\}\_\{t\}\\rVert\_\{2\}\\overset\{p\}\{\\longrightarrow\}0\.where the limiting trajectory starts atx~0=X0\\tilde\{x\}\_\{0\}=X\_\{0\}and follows the plug\-in flow defined by

\\lxSVG@picture

x~˙t=bt​\(x~t\)\+12​\(σt​σt⊤\)​ut\(k\)​\(x~t\)\.\\dot\{\\tilde\{x\}\}\_\{t\}=b\_\{t\}\(\\tilde\{x\}\_\{t\}\)\+\\frac\{1\}\{2\}\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\,u\_\{t\}^\{\(k\)\}\(\\tilde\{x\}\_\{t\}\)\.\(6\)\\endlxSVG@picture

Intuitively, this result means that the Monte Carlo error of the particles in the plug\-in estimator vanishes as the step size goes to zero, so the plug\-in estimator behaves like its expectationut\(k\)​\(x\)u\_\{t\}^\{\(k\)\}\(x\)in the limit; note that eachkkproduces a*different*biased plug\-in flow as the step size vanishes \(k→∞k\\to\\inftyrecovers the analytic guidance\)\. We defer the formal statement to[Proposition˜14](https://arxiv.org/html/2606.02884#Thmtheorem14)in[Section˜C\.3](https://arxiv.org/html/2606.02884#A3.SS3), whose proof relies on Grönwall’s inequality along with Doob’sL2L^\{2\}maximal inequality, and study the bias induced by the plug\-in flow directly\. We first characterize the plug\-in flow withk=1k=1particle for a Gaussian target under the memoryless schedule; the corresponding result for an arbitrary noise schedule is deferred to[Theorem˜15](https://arxiv.org/html/2606.02884#Thmtheorem15)in[Section˜C\.4](https://arxiv.org/html/2606.02884#A3.SS4)\.

###### Theorem 2\(Plug\-in flow for Gaussian target\)\.

Under the memoryless schedule, the guidance term for thek=1k=1plug\-in flow is given by

ut\(1\)​\(x\)=−2​λ​t​Σ​Σt−1​\(μ1\|t​\(x\)−a\),\\displaystyle u\_\{t\}^\{\(1\)\}\(x\)=\-2\\lambda t\\,\\Sigma\\Sigma\_\{t\}^\{\-1\}\(\\mu\_\{1\|t\}\(x\)\-a\),and the terminal distribution of the plug\-in flow isx~1∼𝒩​\(μ\(1\),Σ\(1\)\)\\tilde\{x\}\_\{1\}\\sim\\mathcal\{N\}\(\\mu^\{\(1\)\},\\Sigma^\{\(1\)\}\), where

μ\(1\)≔μ−Tpull\(1\)​\(μ−a\),\\displaystyle\\mu^\{\(1\)\}\\coloneq\\mu\-T\_\{\\mathrm\{pull\}\}^\{\(1\)\}\(\\mu\-a\),Tpull\(1\)≔π​\(λ​Σ\)1/2​exp⁡\(−λ​Σ\)​erfi⁡\(\(λ​Σ\)1/2\),\\displaystyle\\qquad T\_\{\\mathrm\{pull\}\}^\{\(1\)\}\\coloneq\\sqrt\{\\pi\}\\,\(\\lambda\\Sigma\)^\{1/2\}\\exp\(\-\\lambda\\Sigma\)\\,\\operatorname\{erfi\}\\\!\\left\(\(\\lambda\\Sigma\)^\{1/2\}\\right\),Σ\(1\)\\displaystyle\\Sigma^\{\(1\)\}≔Σ​exp⁡\(−2​λ​Σ\)\.\\displaystyle\\coloneq\\Sigma\\exp\(\-2\\lambda\\Sigma\)\.

The proof is in[Section˜C\.5](https://arxiv.org/html/2606.02884#A3.SS5)and follows by the reparameterization trick and solving ODEs for the mean and covariance of the plug\-in flow in closed form\. SinceTpull\(1\)T\_\{\\mathrm\{pull\}\}^\{\(1\)\}andΣ\(1\)\\Sigma^\{\(1\)\}are matrix functions ofΣ\\Sigma, all operators simultaneously diagonalize\. Along an eigendirectionvvwithΣ\\Sigmahaving eigenvalueσ\>0\\sigma\>0, letx≔λ​σx\\coloneq\\lambda\\sigma\. The eigenvalues ofTpull\(1\)T\_\{\\mathrm\{pull\}\}^\{\(1\)\}andTpullT\_\{\\mathrm\{pull\}\}alongvvare

λv​\(Tpull\(1\)\)=π​x​e−x​erfi⁡\(x\),λv​\(Tpull\)=2​x1\+2​x\.\\displaystyle\\lambda\_\{v\}\(T\_\{\\mathrm\{pull\}\}^\{\(1\)\}\)=\\sqrt\{\\pi x\}\\,e^\{\-x\}\\,\\operatorname\{erfi\}\\\!\\left\(\\sqrt\{x\}\\right\),\\qquad\\lambda\_\{v\}\(T\_\{\\mathrm\{pull\}\}\)=\\frac\{2x\}\{1\+2x\}\.Note that the eigenvalues ofTpull\(1\)T\_\{\\mathrm\{pull\}\}^\{\(1\)\}overshoot 1 aroundx≈0\.854x\\approx 0\.854before decaying back down to 1, while the eigenvalue of the trueTpullT\_\{\\mathrm\{pull\}\}increases to 1 at a rational rate\. Therefore, the mean of the plug\-in target will overshoot in the direction of the reward maximizeraacompared to the analytic tilt\. Similarly, the eigenvalues ofΣ\(1\)\\Sigma^\{\(1\)\}andΣ~\\tilde\{\\Sigma\}alongvvare

λv​\(Σ\(1\)\)=σ​e−2​x,λv​\(Σ~\)=σ1\+2​x\.\\displaystyle\\lambda\_\{v\}\(\\Sigma^\{\(1\)\}\)=\\sigma e^\{\-2x\},\\qquad\\lambda\_\{v\}\(\\tilde\{\\Sigma\}\)=\\frac\{\\sigma\}\{1\+2x\}\.This implies that the plug\-in estimator leads to exponentially fast contraction in the covariance of the guided samples\. This mean overshoot and covariance contraction demonstrates that*significant*reward hacking arises from thek=1k=1plug\-in estimator, even in a simple Gaussian setting \(panel \(B\) of[Figure˜3](https://arxiv.org/html/2606.02884#S4.F3)\)\. Although this result holds fork=1k=1particle, we prove next that exponentially many particles are required to resolve reward hacking within a single mode\. Recall thatW∞​\(μ,ν\)≥Wp​\(μ,ν\)W\_\{\\infty\}\(\\mu,\\nu\)\\geq W\_\{p\}\(\\mu,\\nu\)for allp≥1p\\geq 1\.

###### Theorem 3\(∞\\infty\-Wasserstein bound\)\.

Assuming thatρ1=𝒩​\(μ,Σ\)\\rho\_\{1\}=\\mathcal\{N\}\(\\mu,\\Sigma\)andr​\(x\)=−∥x−a∥22r\(x\)=\-\\lVert x\-a\\rVert\_\{2\}^\{2\}, letρ~1\(k\)\\tilde\{\\rho\}\_\{1\}^\{\(k\)\}denote the terminal distribution of the plug\-in flow \([6](https://arxiv.org/html/2606.02884#S4.E6)\) withkkparticles\. Then, the densityρ~1\(k\)\\tilde\{\\rho\}\_\{1\}^\{\(k\)\}is close to the densityρ~1\(1\)\\tilde\{\\rho\}\_\{1\}^\{\(1\)\}in the∞\\infty\-Wasserstein distance:

W∞​\(ρ~1\(k\),ρ~1\(1\)\)≲log⁡k\.\\displaystyle W\_\{\\infty\}\(\\tilde\{\\rho\}\_\{1\}^\{\(k\)\},\\tilde\{\\rho\}\_\{1\}^\{\(1\)\}\)\\lesssim\\sqrt\{\\log k\}\.

The constant in this corollary does not depend on dimension\. The proof, given in[Section˜C\.7](https://arxiv.org/html/2606.02884#A3.SS7), follows from the reparameterization trick, Gaussian concentration inequality, and Grönwall’s inequality applied to the coupled ODE trajectories\. Thus, increasingkkcan only logarithmically mitigate the reward hacking from thek=1k=1plug\-in flow \(panel \(C\) of[Figure˜3](https://arxiv.org/html/2606.02884#S4.F3)\)\. In[Appendix˜E](https://arxiv.org/html/2606.02884#A5), we compare flow map reward guidance \(FMRG\)\[[39](https://arxiv.org/html/2606.02884#bib.bib39)\]to the plug\-in flow and show that their behavior is qualitatively similar\.

#### Reward scale damping for bias correction

In this section, we propose a simple practical method to correct the plug\-in bias\.

###### Proposition 4\(Reward damping schedule\)\.

Assuming an isotropic Gaussian targetρ1=𝒩​\(μ,σ2​Id\)\\rho\_\{1\}=\\mathcal\{N\}\(\\mu,\\sigma^\{2\}I\_\{d\}\), we can recover the correct guidance by replacing the reward scaleλ\\lambdawith the*reward damping*schedule

\\lxSVG@picture

λt≔λ1\+2​λ​σ1\|t2,σ1\|t2=σ2​\(1−t\)2\(1−t\)2\+t2​σ2\.\\lambda\_\{t\}\\coloneq\\frac\{\\lambda\}\{1\+2\\lambda\\sigma\_\{1\|t\}^\{2\}\},\\qquad\\sigma\_\{1\|t\}^\{2\}=\\frac\{\\sigma^\{2\}\(1\-t\)^\{2\}\}\{\(1\-t\)^\{2\}\+t^\{2\}\\sigma^\{2\}\}\.\(7\)\\endlxSVG@picture

In practice, although the target may be non\-isotropic with unknown covariance, we propose treatingσ\>0\\sigma\>0as a tunable hyperparameter \(σ=0\\sigma=0recovers constantλ\\lambdaand increasingσ\\sigmacorresponds to damping the guidance\)\. This method is computationally free compared to akk\-fold increase in the cost from usingk\>1k\>1particles\. We visualize the effect of damping in[Figure˜3](https://arxiv.org/html/2606.02884#S4.F3)for a symmetric isotropic 2\-component Gaussian mixture with quadratic rewardr​\(x\)=−∥x−a∥22r\(x\)=\-\\lVert x\-a\\rVert\_\{2\}^\{2\}; experimental details and additional ablations are provided in[Section˜D\.2](https://arxiv.org/html/2606.02884#A4.SS2)\. We verify our theory in[Section˜D\.1](https://arxiv.org/html/2606.02884#A4.SS1), and in[Section˜5\.1](https://arxiv.org/html/2606.02884#S5.SS1), we further demonstrate that these considerations extend to high\-dimensional text\-to\-image settings\.

![Refer to caption](https://arxiv.org/html/2606.02884v1/x2.png)Figure 3:Reward hacking for Gaussian mixtures\.Exact guidance \(A\) faithfully samples the tilted distributionρ~1\\tilde\{\\rho\}\_\{1\}\. Thek=1k=1plug\-in estimator \(B\) overshoots the mean and shrinks covariance; reward damping \(D\) corrects the collapse better thank=8k=8\(C\) with8×8\\timesless computational cost\.Two natural alternatives to the reward damping schedule fail to sample from the reward\-tilted measure, even for isotropic Gaussians\. We show in[Section˜B\.2](https://arxiv.org/html/2606.02884#A2.SS2)that non\-memoryless schedules do under\-correct the mean and inflate the variance, but no practical choice of non\-memoryless schedule can match the analytic tilt\. It also follows from[Theorem˜2](https://arxiv.org/html/2606.02884#Thmtheorem2)that using a smaller reward scaleλ′<λ\\lambda^\{\\prime\}<\\lambdasimilarly fails to recover the analytic tilt\.

### Mode selection: the role of the initial seed

Throughout this section, we consider a Gaussian mixture targetI1∼∑i=1mπi𝒩\(⋅\|μi,Σi\)I\_\{1\}\\sim\\sum\_\{i=1\}^\{m\}\\pi\_\{i\}\\,\\mathcal\{N\}\(\\cdot\\,\|\\,\\mu\_\{i\},\\Sigma\_\{i\}\)under the memoryless noise scheduleσt=2​\(1−t\)/t​Id\\sigma\_\{t\}=\\sqrt\{2\(1\-t\)/t\}\\,I\_\{d\}\. Letρi​t=𝒩\(⋅\|tμi,\(1−t\)2Id\+t2Σi\)\\rho\_\{it\}=\\mathcal\{N\}\(\\cdot\\,\|\\,t\\mu\_\{i\},\\,\(1\-t\)^\{2\}I\_\{d\}\+t^\{2\}\\Sigma\_\{i\}\)denote the unguided marginal density of theiith component,ρt=∑i=1mπi​ρi​t\\rho\_\{t\}=\\sum\_\{i=1\}^\{m\}\\pi\_\{i\}\\,\\rho\_\{it\}the unguided density, andhi​th\_\{it\}the Doobhh\-function for theiith component \(given by[Proposition˜10](https://arxiv.org/html/2606.02884#Thmtheorem10)withμi\\mu\_\{i\},Σi\\Sigma\_\{i\}in place ofμ\\mu,Σ\\Sigma\)\.

#### Plug\-in guidance cannot select modes

We now extend the analysis of[Section˜4\.1](https://arxiv.org/html/2606.02884#S4.SS1)from Gaussian targets to Gaussian mixtures\. The exact Doobhh\-transform performs long\-range mode selection, while thek=1k=1plug\-in flow does not\. In[Section˜C\.8](https://arxiv.org/html/2606.02884#A3.SS8), we show that the overall analytic guidance term for a Gaussian mixture is given by a weighted sum of the component\-wise guidance terms∑i=1mwi​t​∇log⁡hi​t\\sum\_\{i=1\}^\{m\}w\_\{it\}\\,\\nabla\\log h\_\{it\}, wherewi​t∝πi​ρi​t​\(x\)​hi​t​\(x\)w\_\{it\}\\propto\\pi\_\{i\}\\,\\rho\_\{it\}\(x\)\\,h\_\{it\}\(x\)\. Evaluating the Gaussian integral forhi​th\_\{it\}, one finds

hi​t​\(x\)∝exp⁡\(−λ​\(μi,1\|t​\(x\)−a\)⊤​\(Id\+2​λ​Σi,1\|t\)−1​\(μi,1\|t​\(x\)−a\)\),\\displaystyle h\_\{it\}\(x\)\\propto\\exp\\\!\\left\(\-\\lambda\\,\(\\mu\_\{i,1\|t\}\(x\)\-a\)^\{\\top\}\(I\_\{d\}\+2\\lambda\\Sigma\_\{i,1\|t\}\)^\{\-1\}\(\\mu\_\{i,1\|t\}\(x\)\-a\)\\right\),so a modeiifar from the current location can still exert exponentially large guidance if its reward is high\. This long\-range mode attraction is a special feature of the exacthh\-transform; we now show that the plug\-in estimator does not share this property\.

###### Theorem 5\(Plug\-in flow for Gaussian mixture target\)\.

Fixingt∈\[0,1\]t\\in\[0,1\], letX¯1​\(x\)\\bar\{X\}\_\{1\}\(x\)denote a sample fromp1\|t\(⋅\|x\)p\_\{1\|t\}\(\\cdot\\,\|\\,x\)\. Then, the guidance term for the plug\-in flow withk=1k=1particle is given by

ut\(1\)​\(x\)=−2​λ​𝔼​\[∇X¯1​\(x\)⊤​\(X¯1​\(x\)−a\)\]\.u\_\{t\}^\{\(1\)\}\(x\)=\-2\\lambda\\,\\mathbb\{E\}\\\!\\left\[\\nabla\\bar\{X\}\_\{1\}\(x\)^\{\\top\}\(\\bar\{X\}\_\{1\}\(x\)\-a\)\\right\]\.\(8\)

The proof \([Section˜C\.9](https://arxiv.org/html/2606.02884#A3.SS9)\) follows from applying the chain rule\. In[Section˜C\.9](https://arxiv.org/html/2606.02884#A3.SS9), we additionally provide a variational equation using GLASS flows \([Theorem˜9](https://arxiv.org/html/2606.02884#Thmtheorem9)\) which can be solved to obtain the Jacobian term∇X¯1​\(x\)\\nabla\\bar\{X\}\_\{1\}\(x\)\. In \([8](https://arxiv.org/html/2606.02884#S4.E8)\), the reward enters only through the term\(X¯1​\(x\)−a\)\(\\bar\{X\}\_\{1\}\(x\)\-a\)\. Since the sample fromp1\|t\(⋅\|x\)p\_\{1\|t\}\(\\cdot\\,\|\\,x\)typically remains in the same mode asxx, a far\-away mode contributes negligibly to𝔼​\[∇X¯1​\(x\)⊤​\(X¯1​\(x\)−a\)\]\\mathbb\{E\}\[\\nabla\\bar\{X\}\_\{1\}\(x\)^\{\\top\}\(\\bar\{X\}\_\{1\}\(x\)\-a\)\]regardless of how large its reward is\. This is in contrast to the exacthh\-transform, which attracts trajectories toward high\-reward modes exponentially through the weightshi​t​\(x\)h\_\{it\}\(x\)\. In this sense, thek=1k=1plug\-in flow cannot select high\-reward modes from a distance\.

#### Best\-of\-nncan select modes

We now show through a simple representative example that plug\-in guidance with any finite number of particles fundamentally cannot select between well\-separated modes with poor reward gradients, while best\-of\-nnsampling compensates for this failure\. In the following theorem, we letρ1​\(x\)=12​𝒩​\(x\|μ,σ2\)\+12​𝒩​\(x\|−μ,σ2\)\\rho\_\{1\}\(x\)=\\frac\{1\}\{2\}\\mathcal\{N\}\(x\\,\|\\,\\mu,\\sigma^\{2\}\)\+\\frac\{1\}\{2\}\\mathcal\{N\}\(x\\,\|\\,\-\\mu,\\sigma^\{2\}\)forμ\>0\\mu\>0, use the rewardr​\(x\)=−R​1x<0r\(x\)=\-R\\,\\mathbf\{1\}\_\{x<0\}forR\>0R\>0, and assume the memoryless noise schedule \([Theorem˜13](https://arxiv.org/html/2606.02884#Thmtheorem13)\)\. Define the*correct mode probability*as the probability that the final sample is nonnegative\.

###### Theorem 6\(Best\-of\-nnfor mode selection\)\.

In the above setting, we have:

1. \(i\)*\(Analytic guidance\)*Under the tilted measure, the correct mode probability isp~=\(1\+e−λ​R\)−1→1\\tilde\{p\}=\(1\+e^\{\-\\lambda R\}\)^\{\-1\}\\to 1exponentially asR→∞R\\to\\infty\.
2. \(ii\)*\(Plug\-in guidance\)*Under thekk\-particle plug\-in flow with any finitek≥1k\\geq 1, the correct mode probability isp~1\(k\)=1/2\\tilde\{p\}\_\{1\}^\{\(k\)\}=1/2\.
3. \(iii\)*\(Best\-of\-nn\)*Runningnnindependent finite\-kkplug\-in trajectories and selecting the highest\-reward output yieldsp~n\(k\)=1−2−n\\tilde\{p\}\_\{n\}^\{\(k\)\}=1\-2^\{\-n\}\.

![Refer to caption](https://arxiv.org/html/2606.02884v1/x3.png)Figure 4:Mode selection for Gaussian mixture\.Best\-of\-nnincreases the correct mode probability compared tok=1k=1guidance, better matching the analytic tilt\.The proof is given in[Section˜C\.10](https://arxiv.org/html/2606.02884#A3.SS10)\. Part \(ii\) holds because the indicator reward has zero gradient Lebesgue\-a\.e\., so the plug\-in guidance vanishes and the guided flow coincides with the unguided flow, which is symmetric\.[Figure˜4](https://arxiv.org/html/2606.02884#S4.F4)illustrates this empirically on the setting above; we defer a similar demonstration with a Gaussian reward \(deviating from the theory\) as well as further experiments to[Section˜D\.7](https://arxiv.org/html/2606.02884#A4.SS7)\. In[Section˜5\.2](https://arxiv.org/html/2606.02884#S5.SS2), we demonstrate that this phenomenon arises in a 2D checkerboard setting as well as in a realistic text\-to\-image setting with a VLM reward\.

## Experiments

All code to reproduce these experiments is available in the following GitHub repository:

[https://github\.com/sanjitdp/reward\-guidance](https://github.com/sanjitdp/reward-guidance)\.

All experiments run on a single NVIDIA RTX A6000 or L40S GPU, and each image takes less than 1\.5 minutes to generate; see[Section˜D\.5](https://arxiv.org/html/2606.02884#A4.SS5)for further descriptions of the compute used\.

### Within\-mode reward hacking \(FLUX\.1: text\-to\-image\)

![Refer to caption](https://arxiv.org/html/2606.02884v1/x4.png)Figure 5:Masked intensity reward\.The reward is the mean pixel intensity inside a top\-right circular mask minus the mean intensity outside\. Naive guidance maximizes the masked\-region brightness by removing the welder from the frame; reward damping obtains a high\-reward sample including the welder\.![Refer to caption](https://arxiv.org/html/2606.02884v1/x5.png)Figure 6:Blueness reward \(artist\)\.The reward is the mean blue channel minus the mean of the red and green channels\. Naive guidance produces blue images that forget the artist or candles, while reward damping produces a very blue image that retains the artist and shows the warm candlelight\.![Refer to caption](https://arxiv.org/html/2606.02884v1/x6.png)Figure 7:Blueness reward \(fox\)\.Naive guidance creates overwhelmingly blue outputs that wash out the fox’s orange fur; reward damping recognizes that the fox should remain orange while the sweater and background turn blue\.![Refer to caption](https://arxiv.org/html/2606.02884v1/x7.png)Figure 8:ImageReward \(archaeologist\)\.We guide using ImageReward\[[2](https://arxiv.org/html/2606.02884#bib.bib2)\], a learned human\-preference reward\. Both the unguided and naively guided images produce unnatural images or fail to show a brush, while the damped guidance produces a more natural image with a visible brush\.![Refer to caption](https://arxiv.org/html/2606.02884v1/x8.png)Figure 9:ImageReward \(market\)\.The image is scored against the prompt “a vibrant Indian outdoor market with colorful stalls and produce\.” Naive guidance brightens the lamps while leaving the rest of the image washed out; reward damping produces a more balanced improvement across brightness and color\.![Refer to caption](https://arxiv.org/html/2606.02884v1/x9.png)Figure 10:ImageReward \(miner\)\.Naive guidance hacks ImageReward by ignoring the split diorama constraint; reward damping produces dramatic images that still respect the constraint\.We apply guidance to FLUX\.1\-dev\[[1](https://arxiv.org/html/2606.02884#bib.bib1)\], a state\-of\-the\-art 12\-billion\-parameter text\-to\-image diffusion transformer using Diamond maps\[[36](https://arxiv.org/html/2606.02884#bib.bib36)\]for lookahead; further details are provided in[Section˜D\.4](https://arxiv.org/html/2606.02884#A4.SS4)\. Across our experiments, damped guidance \([Section˜4\.1](https://arxiv.org/html/2606.02884#S4.SS1)\) matches or exceeds the reward hacking mitigation of plug\-in withk=8k=8particles using8×8\\timesless compute per step\.[Figure˜5](https://arxiv.org/html/2606.02884#S5.F5)contains an experiment with a masked\-intensity reward,[Figures˜6](https://arxiv.org/html/2606.02884#S5.F6)and[7](https://arxiv.org/html/2606.02884#S5.F7)contain experiments with a blueness reward, and[Figures˜8](https://arxiv.org/html/2606.02884#S5.F8),[10](https://arxiv.org/html/2606.02884#S5.F10)and[9](https://arxiv.org/html/2606.02884#S5.F9)contain experiments with ImageReward\[[2](https://arxiv.org/html/2606.02884#bib.bib2)\], a learned human\-preference reward\.

##### Base model\.

All FLUX experiments use the pretrainedblack\-forest\-labs/FLUX\.1\-devcheckpoint at512×512512\\times 512resolution with2828inference steps, the standard FLUX guidance scalar3\.53\.5, and the FLUX flow\-matching schedule\. The posterior sampling from the law of\(X1\|Xt=x\)\(X\_\{1\}\\,\|\\,X\_\{t\}=x\)is computed with a Diamond map\[[36](https://arxiv.org/html/2606.02884#bib.bib36)\]with55inner steps\.

##### Reward functions\.

We use three rewards: the blueness rewardr​\(x\)=x¯blue−x¯red−x¯greenr\(x\)=\\bar\{x\}\_\{\\mathrm\{blue\}\}\-\\bar\{x\}\_\{\\mathrm\{red\}\}\-\\bar\{x\}\_\{\\mathrm\{green\}\}\(mean blue\-channel excess\), the masked\-brightness reward \(mean intensity inside a top\-right circular mask, minus the mean intensity outside\), ImageReward\[[2](https://arxiv.org/html/2606.02884#bib.bib2)\]\(a learned human\-preference scorer applied with the BLIP visual encoder, BLIP text encoder, and reward MLP exposed in\-graph for backpropagation\)\.

##### Results\.

For the masked\-intensity reward \([Figure˜5](https://arxiv.org/html/2606.02884#S5.F5)\), naive guidance removes the welder to brighten the masked region\. Damping reaches a high reward while keeping the welder in the scene\. For the blueness reward, naive guidance produces a flat blue image that drops the artist and the candles \([Figure˜6](https://arxiv.org/html/2606.02884#S5.F6)\), and washes the orange out of the fox’s fur \([Figure˜7](https://arxiv.org/html/2606.02884#S5.F7)\)\. Damping produces very blue images that keep the artist visible with the warm candlelight, and that leave the fox’s fur orange while turning the sweater and background blue\. For ImageReward, naive guidance hacks the reward in three different ways: it produces an unnatural archaeologist scene with no brush \([Figure˜8](https://arxiv.org/html/2606.02884#S5.F8)\), brightens the market lamps and leaves the rest of the scene washed out \([Figure˜9](https://arxiv.org/html/2606.02884#S5.F9)\), and produces a single dramatic miner scene that ignores the split\-diorama constraint \([Figure˜10](https://arxiv.org/html/2606.02884#S5.F10)\)\. With damping, the archaeologist holds a brush, the market improves in brightness and color evenly, and the miner images stay dramatic while respecting the split\-diorama constraint\.

### Mode selection

#### Checkerboard\.

Table 11:Checkerboard statistics\.Mean reward and covariance trace for each method \(λ=10\\lambda=10\)\. Uncertainties are±2\\pm 2standard errors of the mean; covariance\-trace uncertainties are bootstrapped\.![Refer to caption](https://arxiv.org/html/2606.02884v1/x10.png)Figure 12:Checkerboard guidance\.Unlike plug\-in guidance, best\-of\-nncan select modes\. With reward damping, best\-of\-nnsignificantly improves fidelity to the analytic tilt\.We first demonstrate the failure of plug\-in guidance for mode selection using a base model sampling uniformly from a 2D checkerboard, where the reward is a Gaussian bumpr​\(x\)=exp⁡\(−∥x−c∥22/\(2​σr2\)\)r\(x\)=\\exp\(\-\\lVert x\-c\\rVert\_\{2\}^\{2\}/\(2\\sigma\_\{r\}^\{2\}\)\)\.[Figure˜12](https://arxiv.org/html/2606.02884#S5.F12)and[Table˜11](https://arxiv.org/html/2606.02884#S5.T11)show that plug\-in estimators fail to concentrate samples nearcc, while a combination of best\-of\-nnand damping improves fidelity toρ~1\\tilde\{\\rho\}\_\{1\}\. We provide further details on the experimental setup and additional results in[Section˜D\.3](https://arxiv.org/html/2606.02884#A4.SS3)\.

#### FLUX\.1: text\-to\-image\.

![Refer to caption](https://arxiv.org/html/2606.02884v1/x11.png)Figure 13:FLUX mode selection \(ECLIPSE DINER\)\.The VLM reward is derived from the question “Does this image clearly show a neon sign with the word ‘ECLIPSE’ as the main readable text?” Finite\-particle guidance hacks the complex reward function, damping slightly improves the reward, and best\-of\-nnsubstantially improves the reward, confirming the importance of the initial seed for mode selection\.![Refer to caption](https://arxiv.org/html/2606.02884v1/x12.png)Figure 14:FLUX mode selection \(NEXT TRAIN MARS\)\.The VLM reward is derived from the question “Does this image clearly show a display with the text ‘NEXT TRAIN MARS’ as the main readable text?” We see the qualitative pattern of[Figure˜13](https://arxiv.org/html/2606.02884#S5.F13): plug\-in guidance hacks the reward, and best\-of\-nnsubstantially improves it\.In[Figures˜13](https://arxiv.org/html/2606.02884#S5.F13)and[14](https://arxiv.org/html/2606.02884#S5.F14), we use FLUX\.1\-dev\[[1](https://arxiv.org/html/2606.02884#bib.bib1)\]with a VLM reward using Qwen2\.5\-VL\-3B\[[47](https://arxiv.org/html/2606.02884#bib.bib47)\]to demonstrate that plug\-in guidance cannot perform mode selection in a realistic setting\. Here, the reward isr​\(x\)=log⁡\(p​\(Yes\)\)−log⁡\(p​\(No\)\)r\(x\)=\\log\(p\(\\mathrm\{Yes\}\)\)\-\\log\(p\(\\mathrm\{No\}\)\), wherep​\(⋅\)p\(\\cdot\)denotes the VLM next\-token probability\. We provide numerics in[Section˜D\.6](https://arxiv.org/html/2606.02884#A4.SS6)\.

## Conclusion

We demonstrate that reward hacking arises from the plug\-in estimator commonly used in reward\-guided generative modeling even in simple Gaussian settings, and showed that plug\-in guidance cannot select between distant high\-reward modes for Gaussian mixtures\. We proposed reward damping, a principled time\-dependent reward scale that corrects the within\-mode bias at no additional cost, and clarified the role of best\-of\-nnin compensating for the mode selection failure\. Our experiments in Gaussian mixture, checkerboard, and text\-to\-image settings demonstrate the limitations of guidance and showcase the usefulness of our methods for overcoming these limitations\.

##### Limitations and future work\.

Our closed\-form theoretical results are derived for Gaussian and Gaussian\-mixture targets with quadratic rewards under the memoryless noise schedule, and the reward damping schedule itself is obtained from the Gaussian analysis; our empirical study is limited to a 2D checkerboard and FLUX\.1 text\-to\-image generation\. Several directions remain open: an analytic study of plug\-in bias for non\-Gaussian targets and non\-quadratic rewards, an extension of our analysis to broader reward\-tilted\-sampling problems in reinforcement learning beyond diffusion, and practical applications of reward damping to settings such as preference fine\-tuning, controlled molecular design, and constrained protein generation\.

## Acknowledgements

We thank Aaditya Ramdas, Sivaraman Balakrishnan, and Arun Kuchibhotla for providing helpful feedback on earlier versions of this work\. We also thank Jerry Huang and Peter Holderrieth for useful discussions about text\-to\-image guidance experiments\.

## References

- Esser et al\. \[2024\]Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach\.Scaling rectified flow transformers for high\-resolution image synthesis\.*arXiv preprint arXiv:2403\.03206*, 2024\.
- Xu et al\. \[2023\]Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong\.ImageReward: Learning and evaluating human preferences for text\-to\-image generation\.In*Advances in Neural Information Processing Systems*, volume 36, pages 15903–15935\. Curran Associates, Inc\., 2023\.
- Ho et al\. \[2020\]Jonathan Ho, Ajay Jain, and Pieter Abbeel\.Denoising diffusion probabilistic models\.In*Advances in Neural Information Processing Systems*, 2020\.
- Rombach et al\. \[2022\]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer\.High\-resolution image synthesis with latent diffusion models\.In*IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2022\.
- Saharia et al\. \[2022\]Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L\. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J\. Fleet, and Mohammad Norouzi\.Photorealistic text\-to\-image diffusion models with deep language understanding\.In*Advances in Neural Information Processing Systems*, 2022\.
- Hoogeboom et al\. \[2022\]Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling\.Equivariant diffusion for molecule generation in 3D\.In*International Conference on Machine Learning*, 2022\.
- Xu et al\. \[2022\]Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang\.GeoDiff: A geometric diffusion model for molecular conformation generation\.In*International Conference on Learning Representations*, 2022\.
- Guan et al\. \[2023a\]Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma\.3D equivariant diffusion for target\-aware molecule generation and affinity prediction\.In*International Conference on Learning Representations*, 2023a\.
- Schneuing et al\. \[2024\]Arne Schneuing, Charles Harris, Yuanqi Du, Kieran Didi, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Liò, Carla Gomes, Max Welling, Michael Bronstein, and Bruno Correia\.Structure\-based drug design with equivariant diffusion models\.*Nature Computational Science*, 2024\.doi:10\.1038/s43588\-024\-00737\-x\.
- Watson et al\. \[2023\]Joseph L\. Watson, David Juergens, Nathaniel R\. Bennett, Brian L\. Trippe, Jason Yim, Helen E\. Eisenach, Woody Ahern, Andrew J\. Borst, Robert J\. Ragotte, Lukas F\. Milles, et al\.De novo design of protein structure and function with RFdiffusion\.*Nature*, 2023\.doi:10\.1038/s41586\-023\-06415\-8\.
- Yim et al\. \[2023\]Jason Yim, Brian L\. Trippe, Valentin De Bortoli, Emile Mathieu, Arnaud Doucet, Regina Barzilay, and Tommi Jaakkola\.SE\(3\) diffusion model with application to protein backbone generation\.In*International Conference on Machine Learning*, 2023\.
- Ingraham et al\. \[2023\]John B\. Ingraham, Max Baranov, Zak Costello, Karl W\. Barber, Wujie Wang, Ahmed Ismail, Vincent Frappier, Dana M\. Lord, Christopher Ng\-Thow\-Hing, Erik R\. Van Vlack, et al\.Illuminating protein space with a programmable generative model\.*Nature*, 2023\.doi:10\.1038/s41586\-023\-06728\-8\.
- Wu et al\. \[2024a\]Kevin E\. Wu, Kevin K\. Yang, Rianne van den Berg, Sarah Alamdari, James Y\. Zou, Alex X\. Lu, and Ava P\. Amini\.Protein structure generation via folding diffusion\.*Nature Communications*, 2024a\.doi:10\.1038/s41467\-024\-45051\-2\.
- Kirstain et al\. \[2023\]Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy\.Pick\-a\-Pic: An open dataset of user preferences for text\-to\-image generation\.*Advances in Neural Information Processing Systems*, 2023\.
- Black et al\. \[2024\]Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine\.Training diffusion models with reinforcement learning\.*International Conference on Learning Representations*, 2024\.
- Wallace et al\. \[2024\]Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik\.Diffusion model alignment using direct preference optimization\.In*IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2024\.
- Wu et al\. \[2025\]Jie Wu, Yu Gao, Zilyu Ye, Ming Li, Liang Li, Hanzhong Guo, Jie Liu, Zeyue Xue, Xiaoxia Hou, Wei Liu, Yan Zeng, and Weilin Huang\.RewardDance: Reward scaling in visual generation\.*arXiv preprint arXiv:2509\.08826*, 2025\.
- Guan et al\. \[2023b\]Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma\.3D equivariant diffusion for target\-aware molecule generation and affinity prediction\.*International Conference on Learning Representations*, 2023b\.
- Guan et al\. \[2023c\]Jiaqi Guan, Xiangxin Zhou, Yuwei Yang, Yu Bao, Jian Peng, Jianzhu Ma, Qiang Liu, Liang Wang, and Quanquan Gu\.DecompDiff: Diffusion models with decomposed priors for structure\-based drug design\.In*International Conference on Machine Learning*, 2023c\.
- Jian et al\. \[2024\]Yue Jian, Curtis Wu, Danny Reidenbach, and Aditi S\. Krishnapriyan\.General binding affinity guidance for diffusion models in structure\-based drug design\.*arXiv preprint arXiv:2406\.16821*, 2024\.
- Gruver et al\. \[2023\]Nate Gruver, Samuel Stanton, Nathan Frey, Tim G\. J\. Rudner, Isidro Hotzel, Julien Lafrance\-Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew G\. Wilson\.Protein design with guided discrete diffusion\.In*Advances in Neural Information Processing Systems*, 2023\.
- Dhariwal and Nichol \[2021\]Prafulla Dhariwal and Alexander Nichol\.Diffusion models beat GANs on image synthesis\.*Advances in Neural Information Processing Systems*, 2021\.
- Ho and Salimans \[2022\]Jonathan Ho and Tim Salimans\.Classifier\-free diffusion guidance\.*arXiv preprint arXiv:2207\.12598*, 2022\.
- Uehara et al\. \[2024\]Masatoshi Uehara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M Tseng, Tommaso Biancalani, and Sergey Levine\.Fine\-tuning of continuous\-time diffusion models as entropy\-regularized control\.*arXiv preprint arXiv:2402\.15194*, 2024\.
- Uehara et al\. \[2025\]Masatoshi Uehara, Yulai Zhao, Chenyu Wang, Xiner Li, Aviv Regev, Sergey Levine, and Tommaso Biancalani\.Inference\-time alignment in diffusion models with reward\-guided generation: Tutorial and review\.*arXiv preprint arXiv:2501\.09685*, 2025\.
- Gao et al\. \[2023\]Leo Gao, John Schulman, and Jacob Hilton\.Scaling laws for reward model overoptimization\.In*International Conference on Machine Learning*, pages 10835–10866\. PMLR, 2023\.
- Eisenstein et al\. \[2024\]Jacob Eisenstein, Chirag Nagpal, Alekh Agarwal, Ahmad Beirami, Alex D’Amour, DJ Dvijotham, Adam Fisch, Katherine Heller, Stephen Pfohl, Deepak Ramachandran, Peter Shaw, and Jonathan Berant\.Helping or herding? Reward model ensembles mitigate but do not eliminate reward hacking\.In*Conference on Language Modeling*, 2024\.
- Rafailov et al\. \[2024\]Rafael Rafailov, Yaswanth Chittepu, Ryan Park, Harshit Sikchi, Joey Hejna, W\. Bradley Knox, Chelsea Finn, and Scott Niekum\.Scaling laws for reward model overoptimization in direct alignment algorithms\.In*Advances in Neural Information Processing Systems*, volume 37, 2024\.
- Zhang et al\. \[2024\]Ziyi Zhang, Sen Zhang, Yibing Zhan, Yong Luo, Yonggang Wen, and Dacheng Tao\.Confronting reward overoptimization for diffusion models: A perspective of inductive and primacy biases\.In*International Conference on Machine Learning*, pages 60396–60413\. PMLR, 2024\.
- Lipman et al\. \[2023\]Yaron Lipman, Ricky TQ Chen, Heli Ben\-Hamu, Maximilian Nickel, and Matt Le\.Flow matching for generative modeling\.In*International Conference on Learning Representations*, 2023\.
- Albergo et al\. \[2025\]Michael Albergo, Nicholas M Boffi, and Eric Vanden\-Eijnden\.Stochastic interpolants: A unifying framework for flows and diffusions\.*Journal of Machine Learning Research*, 2025\.
- Rogers and Williams \[2000\]L\. C\. G\. Rogers and David Williams\.*Diffusions, Markov Processes and Martingales*\.Cambridge Mathematical Library\. Cambridge University Press, 2 edition, 2000\.
- Domingo\-Enrich et al\. \[2025\]Carles Domingo\-Enrich, Michal Drozdzal, Brian Karrer, and Ricky T\. Q\. Chen\.Adjoint matching: Fine\-tuning flow and diffusion generative models with memoryless stochastic optimal control\.*International Conference on Learning Representations*, 2025\.
- Holderrieth et al\. \[2026a\]Peter Holderrieth, Uriel Singer, Tommi Jaakkola, Ricky T\. Q\. Chen, Yaron Lipman, and Brian Karrer\.GLASS flows: Efficient inference for reward alignment of flow and diffusion models\.*International Conference on Learning Representations*, 2026a\.
- Potaptchik et al\. \[2026\]Peter Potaptchik, Adhi Saravanan, Abbas Mammadov, Alvaro Prat, Michael S\. Albergo, and Yee Whye Teh\.Meta Flow Maps enable scalable reward alignment\.*arXiv preprint arXiv:2601\.14430*, 2026\.
- Holderrieth et al\. \[2026b\]Peter Holderrieth, Douglas Chen, Luca Eyring, Ishin Shah, Giri Anantharaman, Yutong He, Zeynep Akata, Tommi Jaakkola, Nicholas Matthew Boffi, and Max Simchowitz\.Diamond maps: Efficient reward alignment via stochastic flow maps\.*arXiv preprint arXiv:2602\.05993*, 2026b\.
- Wu et al\. \[2023\]Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li\.Human preference score v2: A solid benchmark for evaluating human preferences of text\-to\-image synthesis\.*arXiv preprint arXiv:2306\.09341*, 2023\.
- Boffi et al\. \[2025\]Nicholas M Boffi, Michael S Albergo, and Eric Vanden\-Eijnden\.How to build a consistency model: Learning flow maps via self\-distillation\.*Advances in Neural Information Processing Systems*, 2025\.
- Huang et al\. \[2026\]Jerry Huang, Justin Lin, Sheel Shah, Kartik Nair, and Nicholas M\. Boffi\.How to guide your flow: Steering flow maps for rapid test\-time alignment\.In*International Conference on Machine Learning*, 2026\.
- Chidambaram et al\. \[2024\]Muthu Chidambaram, Khashayar Gatmiry, Sitan Chen, Holden Lee, and Jianfeng Lu\.What does guidance do? a fine\-grained analysis in a simple setting\.*Advances in Neural Information Processing Systems*, 2024\.
- Wu et al\. \[2024b\]Yuchen Wu, Minshuo Chen, Zihao Li, Mengdi Wang, and Yuting Wei\.Theoretical insights for diffusion guidance: A case study for Gaussian mixture models\.*International Conference on Machine Learning*, 2024b\.
- Pavasovic et al\. \[2025\]Krunoslav Lehman Pavasovic, Jakob Verbeek, Giulio Biroli, and Marc Mezard\.Classifier\-free guidance: From high\-dimensional analysis to generalized guidance forms\.*arXiv preprint arXiv:2502\.07849*, 2025\.
- Ventura et al\. \[2026\]Enrico Ventura, Beatrice Achilli, Luca Ambrogioni, and Carlo Lucibello\.Emergence of distortions in high\-dimensional guided diffusion models\.*arXiv preprint arXiv:2602\.00716*, 2026\.
- Wang et al\. \[2024\]Xi Wang, Nicolas Dufour, Nefeli Andreou, Marie\-Paule Cani, Victoria Fernández Abrevaya, David Picard, and Vicky Kalogeiton\.Analysis of classifier\-free guidance weight schedulers\.*Transactions on Machine Learning Research*, 2024\.
- Moitra et al\. \[2026a\]Ankur Moitra, Andrej Risteski, and Dhruv Rohatgi\.Steering diffusion models with quadratic rewards: a fine\-grained analysis\.*arXiv preprint arXiv:2602\.16570*, 2026a\.
- Moitra et al\. \[2026b\]Ankur Moitra, Andrej Risteski, and Dhruv Rohatgi\.The tractability landscape of diffusion alignment: regularization, rewards, and computational primitives\.*arXiv preprint arXiv:2605\.11361*, 2026b\.
- Bai et al\. \[2025\]Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin\.Qwen2\.5\-VL technical report\.*arXiv preprint arXiv:2502\.13923*, 2025\.
- Boucheron et al\. \[2013\]Stéphane Boucheron, Gábor Lugosi, and Pascal Massart\.*Concentration Inequalities: a Nonasymptotic Theory of Independence*\.Oxford University Press, 2013\.

## Appendix AFurther background

### Forward SDE matches the probability flow time\-marginals

###### Proposition 7\(Time\-marginals of the forward SDE\)\.

The solution of the forward SDE \([2](https://arxiv.org/html/2606.02884#S2.E2)\) has the same time\-marginal distribution as the probability flow ODEx˙t=bt​\(xt\)\\dot\{x\}\_\{t\}=b\_\{t\}\(x\_\{t\}\)\.

###### Proof\.

The Fokker–Planck equation associated with the SDE \([2](https://arxiv.org/html/2606.02884#S2.E2)\) is

∂tρt=−∇⋅\(ρt​bt\+12​∇⋅\(ρt​σt​σt⊤\)\)\+12​∇⋅\(∇⋅\(ρt​σt​σt⊤\)\)=−∇⋅\(ρt​bt\),\\displaystyle\\partial\_\{t\}\\rho\_\{t\}=\-\\nabla\\cdot\\left\(\\rho\_\{t\}b\_\{t\}\+\\frac\{1\}\{2\}\\nabla\\cdot\(\\rho\_\{t\}\\,\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\right\)\+\\frac\{1\}\{2\}\\nabla\\cdot\(\\nabla\\cdot\(\\rho\_\{t\}\\,\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\)=\-\\nabla\\cdot\(\\rho\_\{t\}b\_\{t\}\),which matches the continuity equation for the probability flow ODE\. ∎

### Reward\-tilted measure as a KL\-regularized variational problem

###### Proposition 8\(Variational characterization of the reward tilt\)\.

The reward\-tilted measure \([3](https://arxiv.org/html/2606.02884#S2.E3)\) solves the KL\-regularized variational problem

ρ~1=arg⁡maxρ⁡\{𝔼X∼ρ​\[r​\(X\)\]−1λ​KL⁡\(ρ∥ρ1\)\}\.\\displaystyle\\tilde\{\\rho\}\_\{1\}=\\operatorname\*\{\\arg\\max\}\_\{\\rho\}\\,\\left\\\{\\mathbb\{E\}\_\{X\\sim\\rho\}\[r\(X\)\]\-\\frac\{1\}\{\\lambda\}\\,\\operatorname\{KL\}\(\\rho\\,\\\|\\,\\rho\_\{1\}\)\\right\\\}\.

###### Proof outline\.

Writing the objective as a single integral againstρ1\\rho\_\{1\},

𝔼X∼ρ​\[r​\(X\)\]−1λ​KL⁡\(ρ∥ρ1\)=1λ​𝔼X∼ρ1​\[ρ​\(X\)ρ1​\(X\)​\(λ​r​\(X\)−log⁡ρ​\(X\)ρ1​\(X\)\)\]\.\\displaystyle\\mathbb\{E\}\_\{X\\sim\\rho\}\[r\(X\)\]\-\\frac\{1\}\{\\lambda\}\\,\\operatorname\{KL\}\(\\rho\\,\\\|\\,\\rho\_\{1\}\)=\\frac\{1\}\{\\lambda\}\\,\\mathbb\{E\}\_\{X\\sim\\rho\_\{1\}\}\\\!\\left\[\\frac\{\\rho\(X\)\}\{\\rho\_\{1\}\(X\)\}\\left\(\\lambda r\(X\)\-\\log\\frac\{\\rho\(X\)\}\{\\rho\_\{1\}\(X\)\}\\right\)\\right\]\.Pointwise maximization in the density ratiof​\(x\)=ρ​\(x\)/ρ1​\(x\)f\(x\)=\\rho\(x\)/\\rho\_\{1\}\(x\)subject to𝔼X∼ρ1​\[f​\(X\)\]=1\\mathbb\{E\}\_\{X\\sim\\rho\_\{1\}\}\[f\(X\)\]=1yieldsf​\(x\)∝eλ​r​\(x\)f\(x\)\\propto e^\{\\lambda r\(x\)\}via a Lagrange multiplier, so the optimalρ\\rhois exactlyρ~1​\(x\)∝ρ1​\(x\)​eλ​r​\(x\)\\tilde\{\\rho\}\_\{1\}\(x\)\\propto\\rho\_\{1\}\(x\)\\,e^\{\\lambda r\(x\)\}\. ∎

### Connections to stochastic optimal control

The Doobhh\-transform approach to reward guidance can also be interpreted through the lens of stochastic optimal control\. We start with an uncontrolled diffusion

d​Xt=μt​\(Xt\)​d​t\+σt​d​Bt\\displaystyle dX\_\{t\}=\\mu\_\{t\}\(X\_\{t\}\)\\,dt\+\\sigma\_\{t\}\\,dB\_\{t\}whereμt​\(x\)≔bt​\(x\)\+12​\(σt​σt⊤\)​∇log⁡ρt​\(x\)\\mu\_\{t\}\(x\)\\coloneq b\_\{t\}\(x\)\+\\frac\{1\}\{2\}\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\nabla\\log\\rho\_\{t\}\(x\)and define a controlled process under a new measureQuQ\_\{u\}by

d​Xt=\(μt​\(Xt\)\+σt​ut​\(Xt\)\)​d​t\+σt​d​B~t\.\\displaystyle dX\_\{t\}=\(\\mu\_\{t\}\(X\_\{t\}\)\+\\sigma\_\{t\}u\_\{t\}\(X\_\{t\}\)\)\\,dt\+\\sigma\_\{t\}\\,d\\tilde\{B\}\_\{t\}\.Now, the goal is to maximize the expected reward while minimizing the control cost, over all admissible controlsuu:

maxu⁡𝔼Qu​\[r​\(X1\)−12​λ​∫01∥ut​\(Xt\)∥22​𝑑t\]\.\\displaystyle\\max\_\{u\}\\,\\mathbb\{E\}\_\{Q\_\{u\}\}\\\!\\left\[r\(X\_\{1\}\)\-\\frac\{1\}\{2\\lambda\}\\int\_\{0\}^\{1\}\\lVert u\_\{t\}\(X\_\{t\}\)\\rVert\_\{2\}^\{2\}\\,dt\\right\]\.We then define the stochastic value function

Vt\(x\)=supu𝔼Qu\[r\(X1\)−12​λ∫t1∥us\(Xs\)∥22ds\|Xt=x\],\\displaystyle V\_\{t\}\(x\)=\\sup\_\{u\}\\,\\mathbb\{E\}\_\{Q\_\{u\}\}\\\!\\left\[r\(X\_\{1\}\)\-\\frac\{1\}\{2\\lambda\}\\int\_\{t\}^\{1\}\\lVert u\_\{s\}\(X\_\{s\}\)\\rVert\_\{2\}^\{2\}\\,ds\\;\\middle\|\\;X\_\{t\}=x\\right\],noting that it must satisfy the Hamilton–Jacobi–Bellman \(HJB\) equation

∂tVt​\(x\)\+supu\{\(μt​\(x\)\+σt​u\)⊤​∇Vt​\(x\)\+12​tr⁡\(\(σt​σt⊤\)​∇2Vt​\(x\)\)−12​λ​∥u∥22\}=0\\displaystyle\\partial\_\{t\}V\_\{t\}\(x\)\+\\sup\_\{u\}\\,\\left\\\{\\left\(\\mu\_\{t\}\(x\)\+\\sigma\_\{t\}u\\right\)^\{\\top\}\\nabla V\_\{t\}\(x\)\+\\frac\{1\}\{2\}\\operatorname\{tr\}\(\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\,\\nabla^\{2\}V\_\{t\}\(x\)\)\-\\frac\{1\}\{2\\lambda\}\\lVert u\\rVert\_\{2\}^\{2\}\\right\\\}=0with the boundary conditionV1​\(x\)=r​\(x\)V\_\{1\}\(x\)=r\(x\)\. Then, stationarity implies that the optimal control is given byut∗​\(x\)=λ​σt⊤​∇Vt​\(x\)u\_\{t\}^\{\*\}\(x\)=\\lambda\\,\\sigma\_\{t\}^\{\\top\}\\nabla V\_\{t\}\(x\), and substituting back into the HJB equation yields the nonlinear PDE

∂tVt​\(x\)\+μt​\(x\)⊤​∇Vt​\(x\)\+12​tr⁡\(\(σt​σt⊤\)​∇2Vt​\(x\)\)\+λ2​∥σt⊤​∇Vt​\(x\)∥22=0\.\\displaystyle\\partial\_\{t\}V\_\{t\}\(x\)\+\\mu\_\{t\}\(x\)^\{\\top\}\\nabla V\_\{t\}\(x\)\+\\frac\{1\}\{2\}\\operatorname\{tr\}\(\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\,\\nabla^\{2\}V\_\{t\}\(x\)\)\+\\frac\{\\lambda\}\{2\}\\lVert\\sigma\_\{t\}^\{\\top\}\\nabla V\_\{t\}\(x\)\\rVert\_\{2\}^\{2\}=0\.The only source of nonlinearity in this equation comes from the control cost termλ2​∥σt⊤​∇Vt​\(x\)∥22\\frac\{\\lambda\}\{2\}\\lVert\\sigma\_\{t\}^\{\\top\}\\nabla V\_\{t\}\(x\)\\rVert\_\{2\}^\{2\}, so we apply the Cole–Hopf transformationht​\(x\)=eλ​Vt​\(x\)h\_\{t\}\(x\)=e^\{\\lambda V\_\{t\}\(x\)\}to linearize the PDE:

∂tht​\(x\)λ​ht​\(x\)\+μt⊤​\(∇ht​\(x\)λ​ht​\(x\)\)\+λ2​‖σt⊤​\(∇ht​\(x\)λ​ht​\(x\)\)‖22\+12​tr⁡\(σt​σt⊤​\(∇2ht​\(x\)λ​ht​\(x\)−∇ht​\(x\)​\(∇ht​\(x\)\)⊤λ​ht​\(x\)2\)\)=0\.\\displaystyle\\frac\{\\partial\_\{t\}h\_\{t\}\(x\)\}\{\\lambda h\_\{t\}\(x\)\}\+\\mu\_\{t\}^\{\\top\}\\left\(\\frac\{\\nabla h\_\{t\}\(x\)\}\{\\lambda h\_\{t\}\(x\)\}\\right\)\+\\frac\{\\lambda\}\{2\}\\left\\lVert\\sigma\_\{t\}^\{\\top\}\\left\(\\frac\{\\nabla h\_\{t\}\(x\)\}\{\\lambda h\_\{t\}\(x\)\}\\right\)\\right\\rVert\_\{2\}^\{2\}\+\\frac\{1\}\{2\}\\operatorname\{tr\}\\\!\\left\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\\left\(\\frac\{\\nabla^\{2\}h\_\{t\}\(x\)\}\{\\lambda h\_\{t\}\(x\)\}\-\\frac\{\\nabla h\_\{t\}\(x\)\(\\nabla h\_\{t\}\(x\)\)^\{\\top\}\}\{\\lambda h\_\{t\}\(x\)^\{2\}\}\\right\)\\right\)=0\.Here, note that

12​λ​ht​\(x\)2​tr⁡\(σt​σt⊤​∇ht​\(x\)​\(∇ht​\(x\)\)⊤\)=12​λ​ht​\(x\)2​‖σt⊤​∇ht​\(x\)‖22\\displaystyle\\frac\{1\}\{2\\lambda h\_\{t\}\(x\)^\{2\}\}\\operatorname\{tr\}\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\\nabla h\_\{t\}\(x\)\(\\nabla h\_\{t\}\(x\)\)^\{\\top\}\)=\\frac\{1\}\{2\\lambda h\_\{t\}\(x\)^\{2\}\}\\left\\lVert\\sigma\_\{t\}^\{\\top\}\\nabla h\_\{t\}\(x\)\\right\\rVert\_\{2\}^\{2\}by the cyclic property of trace, so the nonlinear terms cancel and we are left with the linear PDE

∂tht​\(x\)\+μt​\(x\)⊤​∇ht​\(x\)\+12​tr⁡\(\(σt​σt⊤\)​∇2ht​\(x\)\)=0\.\\displaystyle\\partial\_\{t\}h\_\{t\}\(x\)\+\\mu\_\{t\}\(x\)^\{\\top\}\\nabla h\_\{t\}\(x\)\+\\frac\{1\}\{2\}\\operatorname\{tr\}\(\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\,\\nabla^\{2\}h\_\{t\}\(x\)\)=0\.Recalling the formula for the generatorℒt\\mathcal\{L\}\_\{t\}of the unguided diffusion from \([10](https://arxiv.org/html/2606.02884#A3.E10)\), it follows thathth\_\{t\}must satisfy the Kolmogorov backward equation∂tht\+ℒt​ht=0\\partial\_\{t\}h\_\{t\}\+\\mathcal\{L\}\_\{t\}h\_\{t\}=0\. Noting thatht​\(x\)=𝔼​\[eλ​r​\(X1\)\|Xt=x\]h\_\{t\}\(x\)=\\mathbb\{E\}\[e^\{\\lambda r\(X\_\{1\}\)\}\\,\|\\,X\_\{t\}=x\]also satisfies this PDE with boundary conditionh1​\(x\)=eλ​r​\(x\)h\_\{1\}\(x\)=e^\{\\lambda r\(x\)\}, we conclude that the stochastic value function is given byVt​\(x\)=1λ​log⁡ht​\(x\)V\_\{t\}\(x\)=\\frac\{1\}\{\\lambda\}\\log h\_\{t\}\(x\)and the optimal control is given byut∗​\(x\)=σt⊤​∇log⁡ht​\(x\)u\_\{t\}^\{\*\}\(x\)=\\sigma\_\{t\}^\{\\top\}\\nabla\\log h\_\{t\}\(x\), which matches the Doobhh\-transform guidance term\.

### GLASS flows for transition sampling

In this section, we describe the GLASS flow framework introduced byHolderrieth et al\. \[[34](https://arxiv.org/html/2606.02884#bib.bib34)\]for sampling from the law of\(I1\|It=x\)\(I\_\{1\}\\,\|\\,I\_\{t\}=x\)\. Instead of simulating the forward SDE, which is computationally expensive and hard to differentiate through, GLASS flows simulate a deterministic ODE starting from a Gaussian initial condition\. This way, transition sampling is much cheaper to simulate, more numerically stable, and easier to differentiate through\. The GLASS framework operates under the assumption that the initial distributionρ0\\rho\_\{0\}is Gaussian and that we have access to the optimal denoiserDt​\(x\)≔𝔼​\[I1\|It=x\]D\_\{t\}\(x\)\\coloneq\\mathbb\{E\}\[I\_\{1\}\\,\|\\,I\_\{t\}=x\]\. Although the GLASS flow is defined more generally, we specialize it here to the setting of linear stochastic interpolants\.

###### Theorem 9\(GLASS flow for transition sampling\)\.

Consider the linear interpolantIt=\(1−t\)​X0\+t​X1I\_\{t\}=\(1\-t\)X\_\{0\}\+tX\_\{1\}fort∈\[0,1\]t\\in\[0,1\]withX0∼𝒩​\(0,Id\)X\_\{0\}\\sim\\mathcal\{N\}\(0,I\_\{d\}\)\. Fors,t∈\[0,1\)s,t\\in\[0,1\), define the combined signal\-to\-noise ratio

ηs∗≔\(t1−t\)2\+\(s1−s\)2\.\\displaystyle\\eta^\{\*\}\_\{s\}\\coloneq\\sqrt\{\\left\(\\frac\{t\}\{1\-t\}\\right\)^\{2\}\+\\left\(\\frac\{s\}\{1\-s\}\\right\)^\{2\}\}\.To sample from the law of\(I1\|It=x\)\(I\_\{1\}\\,\|\\,I\_\{t\}=x\), we can simulate the ODE

d​X¯s=11−s​\(Dτs∗​\(Xs∗​\(x\)\)−X¯s\)​d​s,\\displaystyle d\\bar\{X\}\_\{s\}=\\frac\{1\}\{1\-s\}\\,\(D\_\{\\tau\_\{s\}^\{\*\}\}\(X^\{\*\}\_\{s\}\(x\)\)\-\\bar\{X\}\_\{s\}\)\\,ds,froms=0s=0tos=1s=1starting fromX¯0∼𝒩​\(0,Id\)\\bar\{X\}\_\{0\}\\sim\\mathcal\{N\}\(0,I\_\{d\}\), where

τs∗≔ηs∗1\+ηs∗,Xs∗​\(x\)≔1ηs∗​\(1\+ηs∗\)​\(t\(1−t\)2​x\+s\(1−s\)2​X¯s\)\.\\displaystyle\\tau\_\{s\}^\{\*\}\\coloneq\\frac\{\\eta^\{\*\}\_\{s\}\}\{1\+\\eta^\{\*\}\_\{s\}\},\\qquad X^\{\*\}\_\{s\}\(x\)\\coloneq\\frac\{1\}\{\\eta^\{\*\}\_\{s\}\(1\+\\eta^\{\*\}\_\{s\}\)\}\\left\(\\frac\{t\}\{\(1\-t\)^\{2\}\}\\,x\+\\frac\{s\}\{\(1\-s\)^\{2\}\}\\,\\bar\{X\}\_\{s\}\\right\)\.

LetXtX\_\{t\}denote the solution to the forward SDE \([2](https://arxiv.org/html/2606.02884#S2.E2)\) with initial conditionX0∼𝒩​\(0,Id\)X\_\{0\}\\sim\\mathcal\{N\}\(0,I\_\{d\}\)\. Then, note that the GLASS flow will work for transition sampling as long as the memoryless noise schedule is used, since the law of\(I1\|It=x\)\(I\_\{1\}\\,\|\\,I\_\{t\}=x\)then matches the law of\(X1\|Xt=x\)\(X\_\{1\}\\,\|\\,X\_\{t\}=x\)by[Theorem˜13](https://arxiv.org/html/2606.02884#Thmtheorem13)\.

###### Proof\.

Define another linear interpolantJs=\(1−s\)​ϵ\+s​I1J\_\{s\}=\(1\-s\)\\,\\epsilon\+sI\_\{1\}, whereϵ∼𝒩​\(0,Id\)\\epsilon\\sim\\mathcal\{N\}\(0,I\_\{d\}\)is independent of everything else; note that\(J0\|It=x\)∼𝒩​\(0,Id\)\(J\_\{0\}\\,\|\\,I\_\{t\}=x\)\\sim\\mathcal\{N\}\(0,I\_\{d\}\)and\(J1\|It=x\)\(J\_\{1\}\\,\|\\,I\_\{t\}=x\)follows the law of\(I1\|It=x\)\(I\_\{1\}\\,\|\\,I\_\{t\}=x\)\. Then, the conditional probability flow ODE for\(Js\|It=x\)\(J\_\{s\}\\,\|\\,I\_\{t\}=x\)is given by

d​X¯s=11−s​\(𝔼​\[I1\|Js=X¯s,It=x\]−X¯s\)​d​s\.\\displaystyle d\\bar\{X\}\_\{s\}=\\frac\{1\}\{1\-s\}\\,\(\\mathbb\{E\}\[I\_\{1\}\\,\|\\,J\_\{s\}=\\bar\{X\}\_\{s\},\\,I\_\{t\}=x\]\-\\bar\{X\}\_\{s\}\)\\,ds\.Therefore, it remains to show that conditioning onJs=X¯sJ\_\{s\}=\\bar\{X\}\_\{s\}andIt=xI\_\{t\}=xis equivalent to conditioning onIτs∗=Xs∗​\(x\)I\_\{\\tau\_\{s\}^\{\*\}\}=X^\{\*\}\_\{s\}\(x\)for an appropriate choice of effective timeτs∗\\tau\_\{s\}^\{\*\}and effective observationXs∗​\(x\)X^\{\*\}\_\{s\}\(x\)\. Since\(It\|I1\)∼𝒩​\(t​I1,\(1−t\)2​Id\)\(I\_\{t\}\\,\|\\,I\_\{1\}\)\\sim\\mathcal\{N\}\(tI\_\{1\},\(1\-t\)^\{2\}I\_\{d\}\)and\(Js\|I1\)∼𝒩​\(s​I1,\(1−s\)2​Id\)\(J\_\{s\}\\,\|\\,I\_\{1\}\)\\sim\\mathcal\{N\}\(sI\_\{1\},\(1\-s\)^\{2\}I\_\{d\}\)are independent, their joint density is given by

p​\(It=x,Js=X¯s\|I1\)\\displaystyle p\(I\_\{t\}=x,J\_\{s\}=\\bar\{X\}\_\{s\}\\,\|\\,I\_\{1\}\)∝exp⁡\(−12​\(1−t\)2​∥x−t​I1∥22−12​\(1−s\)2​∥X¯s−s​I1∥22\)\\displaystyle\\propto\\exp\\\!\\left\(\-\\frac\{1\}\{2\(1\-t\)^\{2\}\}\\lVert x\-tI\_\{1\}\\rVert\_\{2\}^\{2\}\-\\frac\{1\}\{2\(1\-s\)^\{2\}\}\\lVert\\bar\{X\}\_\{s\}\-sI\_\{1\}\\rVert\_\{2\}^\{2\}\\right\)∝exp⁡\(−12​I1⊤​\(t2\(1−t\)2\+s2\(1−s\)2\)​I1\+I1⊤​\(t\(1−t\)2​x\+s\(1−s\)2​X¯s\)\)\.\\displaystyle\\propto\\exp\\\!\\left\(\-\\frac\{1\}\{2\}I\_\{1\}^\{\\top\}\\left\(\\frac\{t^\{2\}\}\{\(1\-t\)^\{2\}\}\+\\frac\{s^\{2\}\}\{\(1\-s\)^\{2\}\}\\right\)I\_\{1\}\+I\_\{1\}^\{\\top\}\\left\(\\frac\{t\}\{\(1\-t\)^\{2\}\}x\+\\frac\{s\}\{\(1\-s\)^\{2\}\}\\bar\{X\}\_\{s\}\\right\)\\right\)\.On the other hand, the density of\(Iτs∗\|I1\)\(I\_\{\\tau\_\{s\}^\{\*\}\}\\,\|\\,I\_\{1\}\)is given by

p​\(Iτs∗=Xs∗​\(x\)\|I1\)\\displaystyle p\(I\_\{\\tau\_\{s\}^\{\*\}\}=X^\{\*\}\_\{s\}\(x\)\\,\|\\,I\_\{1\}\)∝exp⁡\(−12​\(1−τs∗\)2​∥Xs∗​\(x\)−τs∗​I1∥22\)\\displaystyle\\propto\\exp\\\!\\left\(\-\\frac\{1\}\{2\(1\-\\tau\_\{s\}^\{\*\}\)^\{2\}\}\\lVert X^\{\*\}\_\{s\}\(x\)\-\\tau\_\{s\}^\{\*\}I\_\{1\}\\rVert\_\{2\}^\{2\}\\right\)∝exp⁡\(−12​I1⊤​\(\(τs∗\)2\(1−τs∗\)2\)​I1\+I1⊤​\(τs∗\(1−τs∗\)2​Xs∗​\(x\)\)\)\.\\displaystyle\\propto\\exp\\\!\\left\(\-\\frac\{1\}\{2\}I\_\{1\}^\{\\top\}\\left\(\\frac\{\(\\tau\_\{s\}^\{\*\}\)^\{2\}\}\{\(1\-\\tau\_\{s\}^\{\*\}\)^\{2\}\}\\right\)I\_\{1\}\+I\_\{1\}^\{\\top\}\\left\(\\frac\{\\tau\_\{s\}^\{\*\}\}\{\(1\-\\tau\_\{s\}^\{\*\}\)^\{2\}\}X^\{\*\}\_\{s\}\(x\)\\right\)\\right\)\.It suffices now to show that the two densities are proportional to each other, since Bayes’ rule will then imply that\(I1\|It=x,Js=X¯s\)​=𝑑​\(I1\|Iτs∗=Xs∗​\(x\)\)\(I\_\{1\}\\,\|\\,I\_\{t\}=x,\\,J\_\{s\}=\\bar\{X\}\_\{s\}\)\\overset\{d\}\{=\}\(I\_\{1\}\\,\|\\,I\_\{\\tau\_\{s\}^\{\*\}\}=X^\{\*\}\_\{s\}\(x\)\)\. Equating the quadratic coefficients gives

τs∗1−τs∗=ηs∗⟹τs∗=ηs∗1\+ηs∗,\\displaystyle\\frac\{\\tau\_\{s\}^\{\*\}\}\{1\-\\tau\_\{s\}^\{\*\}\}=\\eta^\{\*\}\_\{s\}\\implies\\tau\_\{s\}^\{\*\}=\\frac\{\\eta^\{\*\}\_\{s\}\}\{1\+\\eta^\{\*\}\_\{s\}\},and becauseτs∗/\(1−τs∗\)2=ηs∗​\(1\+ηs∗\)\\tau\_\{s\}^\{\*\}/\(1\-\\tau\_\{s\}^\{\*\}\)^\{2\}=\\eta^\{\*\}\_\{s\}\(1\+\\eta^\{\*\}\_\{s\}\), equating the linear coefficients gives

Xs∗​\(x\)=1ηs∗​\(1\+ηs∗\)​\(t\(1−t\)2​x\+s\(1−s\)2​X¯s\)\.\\displaystyle X^\{\*\}\_\{s\}\(x\)=\\frac\{1\}\{\\eta^\{\*\}\_\{s\}\(1\+\\eta^\{\*\}\_\{s\}\)\}\\left\(\\frac\{t\}\{\(1\-t\)^\{2\}\}x\+\\frac\{s\}\{\(1\-s\)^\{2\}\}\\bar\{X\}\_\{s\}\\right\)\.∎

## Appendix BAdditional results

### Analytic guidance for the Gaussian target

In this appendix, we compute the exact Doobhh\-transform guidance for a Gaussian target distribution under the memoryless noise schedule, providing a benchmark for the biases that we study in[Section˜4\.1](https://arxiv.org/html/2606.02884#S4.SS1)\.

###### Proposition 10\(Analytic guidance for Gaussian target\)\.

SupposeI1∼𝒩​\(μ,Σ\)I\_\{1\}\\sim\\mathcal\{N\}\(\\mu,\\Sigma\)andσt=2​\(1−t\)/t​Id\\sigma\_\{t\}=\\sqrt\{2\(1\-t\)/t\}\\,I\_\{d\}is the memoryless noise schedule\. The guidance term of the Doobhh\-transform is

∇log⁡ht​\(x\)=−2​λ​t​Σ​Σt−1​\(Id\+2​λ​Σ1\|t\)−1​\(μ1\|t​\(x\)−a\),\\displaystyle\\nabla\\log h\_\{t\}\(x\)=\-2\\lambda t\\,\\Sigma\\Sigma\_\{t\}^\{\-1\}\(I\_\{d\}\+2\\lambda\\Sigma\_\{1\|t\}\)^\{\-1\}\(\\mu\_\{1\|t\}\(x\)\-a\),whereμ1\|t​\(x\)=μ\+t​Σ​Σt−1​\(x−t​μ\)\\mu\_\{1\|t\}\(x\)=\\mu\+t\\Sigma\\Sigma\_\{t\}^\{\-1\}\(x\-t\\mu\)andΣ1\|t=\(1−t\)2​Σ​Σt−1\\Sigma\_\{1\|t\}=\(1\-t\)^\{2\}\\Sigma\\Sigma\_\{t\}^\{\-1\}are the mean and covariance of\(X1\|Xt=x\)\(X\_\{1\}\\,\|\\,X\_\{t\}=x\)\. The terminal distribution of the guided ODE is the reward\-tilted measure

x~1∼𝒩​\(μ−2​λ​Σ​\(Id\+2​λ​Σ\)−1​\(μ−a\),Σ​\(Id\+2​λ​Σ\)−1\)\.\\displaystyle\\tilde\{x\}\_\{1\}\\sim\\mathcal\{N\}\\\!\\left\(\\mu\-2\\lambda\\Sigma\(I\_\{d\}\+2\\lambda\\Sigma\)^\{\-1\}\(\\mu\-a\),\\;\\Sigma\(I\_\{d\}\+2\\lambda\\Sigma\)^\{\-1\}\\right\)\.

###### Proof\.

The last part of the corollary follows immediately from[Theorem˜11](https://arxiv.org/html/2606.02884#Thmtheorem11)by noting thatΨ=0\\Psi=0for the memoryless noise schedule\. Note that for the memoryless noise scheduleηt=2​\(1−t\)/t\\eta\_\{t\}=\\sqrt\{2\(1\-t\)/t\},

dd​v​log⁡\(v2​Σ​Σv−1\)\\displaystyle\\frac\{d\}\{dv\}\\,\\log\(v^\{2\}\\,\\Sigma\\Sigma\_\{v\}^\{\-1\}\)=2v​Id−Σv−1​Σ˙v\\displaystyle=\\frac\{2\}\{v\}I\_\{d\}\-\\Sigma\_\{v\}^\{\-1\}\\dot\{\\Sigma\}\_\{v\}=Σv−1​\(2​v​Σ\+2​\(1−v\)2v​Id−2​v​Σ\+2​\(1−v\)​Id\)\\displaystyle=\\Sigma\_\{v\}^\{\-1\}\\left\(2v\\Sigma\+\\frac\{2\(1\-v\)^\{2\}\}\{v\}I\_\{d\}\-2v\\Sigma\+2\(1\-v\)I\_\{d\}\\right\)=2​\(1−v\)v​Σv−1\\displaystyle=\\frac\{2\(1\-v\)\}\{v\}\\Sigma\_\{v\}^\{\-1\}=ηv2​Σv−1\.\\displaystyle=\\eta\_\{v\}^\{2\}\\,\\Sigma\_\{v\}^\{\-1\}\.Therefore, we can compute the integral definingΨt\\Psi\_\{t\}in closed form:

Ψt=exp⁡\(−∫t1ηv2​Σv−1​𝑑v\)=exp⁡\(log⁡\(t2​Σ​Σt−1\)\)=t2​Σ​Σt−1\.\\displaystyle\\Psi\_\{t\}=\\exp\\\!\\left\(\-\\int\_\{t\}^\{1\}\\eta\_\{v\}^\{2\}\\,\\Sigma\_\{v\}^\{\-1\}\\,dv\\right\)=\\exp\(\\log\(t^\{2\}\\,\\Sigma\\Sigma\_\{t\}^\{\-1\}\)\)=t^\{2\}\\,\\Sigma\\Sigma\_\{t\}^\{\-1\}\.Plugging this back into the covariance formula, we obtainCov⁡\(Xt,X1\)=t​Σ\\operatorname\{Cov\}\(X\_\{t\},X\_\{1\}\)=t\\Sigma\. The Gaussian conditioning formula states that\(X1\|Xt=x\)∼𝒩​\(μ1\|t​\(x\),Σ1\|t\)\(X\_\{1\}\\,\|\\,X\_\{t\}=x\)\\sim\\mathcal\{N\}\(\\mu\_\{1\|t\}\(x\),\\Sigma\_\{1\|t\}\)whereμ1\|t​\(x\)=μ\+t​Σ​Σt−1​\(x−t​μ\)\\mu\_\{1\|t\}\(x\)=\\mu\+t\\Sigma\\Sigma\_\{t\}^\{\-1\}\(x\-t\\mu\)andΣ1\|t=\(1−t\)2​Σ​Σt−1\\Sigma\_\{1\|t\}=\(1\-t\)^\{2\}\\Sigma\\Sigma\_\{t\}^\{\-1\}\. Using the fact thatΣ\\SigmaandΣt\\Sigma\_\{t\}are simultaneously diagonalizable, the guidance term is given by

∇log⁡ht​\(x\)=−2​λ​t​Σ​Σt−1​\(Id\+2​λ​Σ1\|t\)−1​\(μ1\|t​\(x\)−a\)\.\\displaystyle\\nabla\\log h\_\{t\}\(x\)=\-2\\lambda t\\Sigma\\Sigma\_\{t\}^\{\-1\}\(I\_\{d\}\+2\\lambda\\Sigma\_\{1\|t\}\)^\{\-1\}\(\\mu\_\{1\|t\}\(x\)\-a\)\.∎

### Initial value function bias

The Doobhh\-transform of[Theorem˜12](https://arxiv.org/html/2606.02884#Thmtheorem12)requires a memoryless noise schedule, but the memoryless schedule of[Theorem˜13](https://arxiv.org/html/2606.02884#Thmtheorem13)hasσt=2​\(1−t\)/t→∞\\sigma\_\{t\}=\\sqrt\{2\(1\-t\)/t\}\\to\\inftyast→0t\\to 0, so it is numerically unstable neart=0t=0; in practice, practitioners use a non\-memoryless alternative\. Here we characterize the resulting bias—referred to as the*initial value function bias*\[[33](https://arxiv.org/html/2606.02884#bib.bib33), Section 4\.2\]—in closed form for Gaussian and Gaussian mixture targets, using the notation introduced in[Section˜4\.1](https://arxiv.org/html/2606.02884#S4.SS1)\. As discussed in the main text, this bias acts in the opposite direction to the plug\-in bias: it under\-corrects the mean and inflates the variance relative to the analytic tilt, so it partially mitigates plug\-in reward hacking\.

However, non\-memoryless schedules cannot recover the analytic tilt for any choice of schedule\. In particular, any non\-memoryless schedule withΨ≻0\\Psi\\succ 0strictly \(which holds for all schedules used in practice\) has both the mean pull \(strictly decreasing inψ\\psi\) and the terminal covariance \(strictly increasing inψ\\psi\) strictly missing their analytic\-tilt values2​λ​σ/\(1\+2​λ​σ\)2\\lambda\\sigma/\(1\+2\\lambda\\sigma\)andσ/\(1\+2​λ​σ\)\\sigma/\(1\+2\\lambda\\sigma\)along every eigendirection; no such schedule can recover the analytic tilt as the terminal distribution\.

###### Theorem 11\(Initial value function bias for Gaussian target\)\.

The guidance term in the Doobhh\-transform \([Theorem˜12](https://arxiv.org/html/2606.02884#Thmtheorem12)\) is

∇log⁡ht​\(x\)=−2​λ​Σ1/2​Σt−1/2​Ψt1/2​\(Id\+2​λ​Σ1\|t\)−1​\(μ1\|t​\(x\)−a\)\.\\displaystyle\\nabla\\log h\_\{t\}\(x\)=\-2\\lambda\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\(I\_\{d\}\+2\\lambda\\Sigma\_\{1\|t\}\)^\{\-1\}\(\\mu\_\{1\|t\}\(x\)\-a\)\.DefiningΨ≔Ψ0\\Psi\\coloneq\\Psi\_\{0\}, the terminal distribution of the guided ODE isx~1∼𝒩​\(μ~,Σ~\)\\tilde\{x\}\_\{1\}\\sim\\mathcal\{N\}\(\\tilde\{\\mu\},\\tilde\{\\Sigma\}\), where

μ~≔μ−Tpull​\(μ−a\),Tpull≔2​λ​Σ​\(Id−Ψ\)​\(Id\+2​λ​Σ​\(Id−Ψ\)\)−1,\\displaystyle\\tilde\{\\mu\}\\coloneq\\mu\-T\_\{\\mathrm\{pull\}\}\(\\mu\-a\),\\qquad T\_\{\\mathrm\{pull\}\}\\coloneq 2\\lambda\\Sigma\(I\_\{d\}\-\\Psi\)\(I\_\{d\}\+2\\lambda\\Sigma\(I\_\{d\}\-\\Psi\)\)^\{\-1\},and

Σ~≔Σ​\(Id\+2​λ​Σ​\(Id−Ψ\)2\)​\(Id\+2​λ​Σ​\(Id−Ψ\)\)−2\.\\displaystyle\\tilde\{\\Sigma\}\\coloneq\\Sigma\(I\_\{d\}\+2\\lambda\\Sigma\(I\_\{d\}\-\\Psi\)^\{2\}\)\(I\_\{d\}\+2\\lambda\\Sigma\(I\_\{d\}\-\\Psi\)\)^\{\-2\}\.

The proof is below; we express the SDE forXtX\_\{t\}as a linear system, derive a closed\-form ODE for the cross\-covarianceCov⁡\(Xs,Xt\)\\operatorname\{Cov\}\(X\_\{s\},X\_\{t\}\)via Fubini’s theorem, and evaluateht​\(x\)h\_\{t\}\(x\)as a Gaussian integral\. To provide some intuition for this formula, note that for any non\-memoryless schedule,0≺Ψ≺Id0\\prec\\Psi\\prec I\_\{d\}and all operators are simultaneously diagonalizable \(sinceΨ\\Psiis a matrix function ofΣ\\Sigma\)\. Along a joint eigendirectionvvwithΨ\\Psihaving eigenvalueψ∈\(0,1\)\\psi\\in\(0,1\)andΣ\\Sigmahaving eigenvalueσ\>0\\sigma\>0, the eigenvalue ofTpullT\_\{\\mathrm\{pull\}\}is

2​λ​σ​\(1−ψ\)1\+2​λ​σ​\(1−ψ\),\\displaystyle\\frac\{2\\lambda\\sigma\(1\-\\psi\)\}\{1\+2\\lambda\\sigma\(1\-\\psi\)\},a decreasing function ofψ\\psion\[0,1\]\[0,1\], equal to the memoryless tilt’s pull2​λ​σ/\(1\+2​λ​σ\)2\\lambda\\sigma/\(1\+2\\lambda\\sigma\)atψ=0\\psi=0and vanishing asψ→1\\psi\\to 1, while the eigenvalue ofΣ~\\tilde\{\\Sigma\}is

σ​\(1\+2​λ​σ​\(1−ψ\)2\)\(1\+2​λ​σ​\(1−ψ\)\)2,\\displaystyle\\frac\{\\sigma\(1\+2\\lambda\\sigma\(1\-\\psi\)^\{2\}\)\}\{\(1\+2\\lambda\\sigma\(1\-\\psi\)\)^\{2\}\},an increasing function ofψ\\psion\[0,1\]\[0,1\], equal to the memoryless tilt’s varianceσ/\(1\+2​λ​σ\)\\sigma/\(1\+2\\lambda\\sigma\)atψ=0\\psi=0and approaching the unguided varianceσ\\sigmaasψ→1\\psi\\to 1\. Thus, non\-memoryless schedules under\-correct the mean and inflate the covariance relative to the memoryless tilt\. In particular, since0≺Ψ≺Id0\\prec\\Psi\\prec I\_\{d\}strictly for any non\-memoryless schedule, both the mean pull \(strictly decreasing inψ\\psi\) and the terminal covariance \(strictly increasing inψ\\psi\) miss their analytic\-tilt values2​λ​σ/\(1\+2​λ​σ\)2\\lambda\\sigma/\(1\+2\\lambda\\sigma\)andσ/\(1\+2​λ​σ\)\\sigma/\(1\+2\\lambda\\sigma\)along every eigendirection; no choice of non\-memoryless noise schedule can recover the analytic tilt as the terminal distribution\. The corresponding analysis for a Gaussian mixture target appears as[Theorem˜17](https://arxiv.org/html/2606.02884#Thmtheorem17)and is used in[Section˜4\.2\.1](https://arxiv.org/html/2606.02884#S4.SS2.SSS1)to characterize the long\-range mode\-selection behavior of the analytichh\-transform\.

###### Proof of[Theorem˜11](https://arxiv.org/html/2606.02884#Thmtheorem11)\.

The unguided stochastic interpolantItI\_\{t\}has distribution𝒩​\(μt,Σt\)\\mathcal\{N\}\(\\mu\_\{t\},\\Sigma\_\{t\}\)whereμt=t​μ\\mu\_\{t\}=t\\muandΣt=\(1−t\)2​Id\+t2​Σ\\Sigma\_\{t\}=\(1\-t\)^\{2\}I\_\{d\}\+t^\{2\}\\Sigma\. It is clear that\(It,I˙t\)\(I\_\{t\},\\dot\{I\}\_\{t\}\)is a linear function of\(I0,I1\)\(I\_\{0\},I\_\{1\}\)and hence jointly Gaussian\. We then notice that

Cov⁡\(I˙t,It\)=Cov⁡\(I1−I0,\(1−t\)​I0\+t​I1\)=t​Σ−\(1−t\)​Id\\displaystyle\\operatorname\{Cov\}\(\\dot\{I\}\_\{t\},I\_\{t\}\)=\\operatorname\{Cov\}\(I\_\{1\}\-I\_\{0\},\\,\(1\-t\)I\_\{0\}\+tI\_\{1\}\)=t\\Sigma\-\(1\-t\)I\_\{d\}and hence the Gaussian conditional mean is given by

𝔼​\[I˙t\|It=x\]=μ\+\(t​Σ−\(1−t\)​Id\)​Σt−1​\(x−μt\)=μ\+12​Σ˙t​Σt−1​\(x−μt\)\.\\displaystyle\\mathbb\{E\}\[\\dot\{I\}\_\{t\}\\,\|\\,I\_\{t\}=x\]=\\mu\+\(t\\Sigma\-\(1\-t\)I\_\{d\}\)\\Sigma\_\{t\}^\{\-1\}\(x\-\\mu\_\{t\}\)=\\mu\+\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{t\}\\Sigma\_\{t\}^\{\-1\}\(x\-\\mu\_\{t\}\)\.Because the score function is∇log⁡ρt​\(x\)=−Σt−1​\(x−μt\)\\nabla\\log\\rho\_\{t\}\(x\)=\-\\Sigma\_\{t\}^\{\-1\}\(x\-\\mu\_\{t\}\), the forward SDE associated with the stochastic interpolant is

d​Xt=\(μ\+12​\(Σ˙t−ηt2​Id\)​Σt−1​\(Xt−μt\)\)​d​t\+ηt​d​Bt\.\\displaystyle dX\_\{t\}=\\left\(\\mu\+\\frac\{1\}\{2\}\(\\dot\{\\Sigma\}\_\{t\}\-\\eta\_\{t\}^\{2\}\\,I\_\{d\}\)\\,\\Sigma\_\{t\}^\{\-1\}\(X\_\{t\}\-\\mu\_\{t\}\)\\right\)dt\+\\eta\_\{t\}\\,dB\_\{t\}\.Here, we can use Fubini’s theorem to compute that for0≤s<t≤10\\leq s<t\\leq 1,

Cov⁡\(Xs,Xt\)\\displaystyle\\operatorname\{Cov\}\(X\_\{s\},X\_\{t\}\)=𝔼​\[\(Xs−μs\)​\(Xt−μt\)⊤\]\\displaystyle=\\mathbb\{E\}\[\(X\_\{s\}\-\\mu\_\{s\}\)\(X\_\{t\}\-\\mu\_\{t\}\)^\{\\top\}\]=𝔼​\[\(Xs−μs\)​\(\(Xs−μs\)\+∫st12​\(Σ˙v−ηv2​Id\)​Σv−1​\(Xv−μv\)​𝑑v\+∫stηv​𝑑Bv\)⊤\]\\displaystyle=\\mathbb\{E\}\\left\[\(X\_\{s\}\-\\mu\_\{s\}\)\\left\(\(X\_\{s\}\-\\mu\_\{s\}\)\+\\int\_\{s\}^\{t\}\\frac\{1\}\{2\}\(\\dot\{\\Sigma\}\_\{v\}\-\\eta\_\{v\}^\{2\}\\,I\_\{d\}\)\\,\\Sigma\_\{v\}^\{\-1\}\(X\_\{v\}\-\\mu\_\{v\}\)\\,dv\+\\int\_\{s\}^\{t\}\\eta\_\{v\}\\,dB\_\{v\}\\right\)^\{\\top\}\\right\]=Σs\+∫st12​Cov⁡\(Xs,Xv\)​\(\(Σ˙v−ηv2​Id\)​Σv−1\)⊤​𝑑v\.\\displaystyle=\\Sigma\_\{s\}\+\\int\_\{s\}^\{t\}\\frac\{1\}\{2\}\\operatorname\{Cov\}\(X\_\{s\},X\_\{v\}\)\(\(\\dot\{\\Sigma\}\_\{v\}\-\\eta\_\{v\}^\{2\}\\,I\_\{d\}\)\\,\\Sigma\_\{v\}^\{\-1\}\)^\{\\top\}\\,dv\.Differentiating both sides with respect tottyields the linear ODE

∂tCov⁡\(Xs,Xt\)=12​Cov⁡\(Xs,Xt\)​\(\(Σ˙t−ηt2​Id\)​Σt−1\)⊤\.\\displaystyle\\partial\_\{t\}\\operatorname\{Cov\}\(X\_\{s\},X\_\{t\}\)=\\frac\{1\}\{2\}\\operatorname\{Cov\}\(X\_\{s\},X\_\{t\}\)\(\(\\dot\{\\Sigma\}\_\{t\}\-\\eta\_\{t\}^\{2\}\\,I\_\{d\}\)\\,\\Sigma\_\{t\}^\{\-1\}\)^\{\\top\}\.Since all relevant terms are simultaneously diagonalizable, we can change into the eigenbasis ofΣ\\Sigmato obtain a scalar ODE for each eigenvalue\. As a result, we obtain the closed\-form solution

Cov⁡\(Xs,Xt\)=Σs1/2​Σt1/2​exp⁡\(−12​∫stηv2​Σv−1​𝑑v\)\\operatorname\{Cov\}\(X\_\{s\},X\_\{t\}\)=\\Sigma\_\{s\}^\{1/2\}\\Sigma\_\{t\}^\{1/2\}\\exp\\\!\\left\(\-\\frac\{1\}\{2\}\\int\_\{s\}^\{t\}\\eta\_\{v\}^\{2\}\\,\\Sigma\_\{v\}^\{\-1\}\\,dv\\right\)\(9\)using the boundary conditionCov⁡\(Xs,Xs\)=Σs\\operatorname\{Cov\}\(X\_\{s\},X\_\{s\}\)=\\Sigma\_\{s\}\. Instantiating this formula yields the joint distribution of the Gaussian vector\(X0,X1\)\(X\_\{0\},X\_\{1\}\)asCov⁡\(X0,X1\)=Σ1/2​Ψ1/2\\operatorname\{Cov\}\(X\_\{0\},X\_\{1\}\)=\\Sigma^\{1/2\}\\Psi^\{1/2\}where

Ψ≔exp⁡\(−∫01ηv2​Σv−1​𝑑v\)\.\\displaystyle\\Psi\\coloneq\\exp\\\!\\left\(\-\\int\_\{0\}^\{1\}\\eta\_\{v\}^\{2\}\\,\\Sigma\_\{v\}^\{\-1\}\\,dv\\right\)\.Therefore, we have

\(X1\|X0\)∼𝒩​\(μ\+Σ1/2​Ψ1/2​X0,Σ​\(Id−Ψ\)\)\\displaystyle\(X\_\{1\}\\,\|\\,X\_\{0\}\)\\sim\\mathcal\{N\}\\left\(\\mu\+\\Sigma^\{1/2\}\\Psi^\{1/2\}X\_\{0\},\\,\\Sigma\(I\_\{d\}\-\\Psi\)\\right\)and

\(X0\|X1\)∼𝒩​\(Σ−1/2​Ψ1/2​\(X1−μ\),Id−Ψ\)\.\\displaystyle\(X\_\{0\}\\,\|\\,X\_\{1\}\)\\sim\\mathcal\{N\}\\left\(\\Sigma^\{\-1/2\}\\Psi^\{1/2\}\(X\_\{1\}\-\\mu\),\\,I\_\{d\}\-\\Psi\\right\)\.Using these expressions, we computeh0h\_\{0\}as a Gaussian integral:

h0​\(x\)\\displaystyle h\_\{0\}\(x\)=𝔼​\[e−λ​∥X1−a∥22\|X0=x\]\\displaystyle=\\mathbb\{E\}\[e^\{\-\\lambda\\lVert X\_\{1\}\-a\\rVert\_\{2\}^\{2\}\}\\,\|\\,X\_\{0\}=x\]∝exp⁡\(−λ​\(μ\+Σ1/2​Ψ1/2​x−a\)⊤​\(Id\+2​λ​Σ​\(Id−Ψ\)\)−1​\(μ\+Σ1/2​Ψ1/2​x−a\)\)\.\\displaystyle\\propto\\exp\\\!\\left\(\-\\lambda\\left\(\\mu\+\\Sigma^\{1/2\}\\Psi^\{1/2\}x\-a\\right\)^\{\\top\}\(I\_\{d\}\+2\\lambda\\Sigma\(I\_\{d\}\-\\Psi\)\)^\{\-1\}\\left\(\\mu\+\\Sigma^\{1/2\}\\Psi^\{1/2\}x\-a\\right\)\\right\)\.Similarly, we can compute the initial value function bias by a Gaussian integral:

𝔼\[1h0​\(X0\)\|X1=x\]\\displaystyle\\mathbb\{E\}\\left\[\\frac\{1\}\{h\_\{0\}\(X\_\{0\}\)\}\\;\\middle\|\\;X\_\{1\}=x\\right\]∝𝔼\[exp\(λ\(μ\+Σ1/2Ψ1/2X0−a\)⊤\(Id\+2λΣ\(Id−Ψ\)\)−1\(μ\+Σ1/2Ψ1/2X0−a\)\)\|X1=x\]\\displaystyle\\quad\\propto\\mathbb\{E\}\\left\[\\exp\\\!\\left\(\\lambda\\left\(\\mu\+\\Sigma^\{1/2\}\\Psi^\{1/2\}X\_\{0\}\-a\\right\)^\{\\top\}\(I\_\{d\}\+2\\lambda\\Sigma\(I\_\{d\}\-\\Psi\)\)^\{\-1\}\\left\(\\mu\+\\Sigma^\{1/2\}\\Psi^\{1/2\}X\_\{0\}\-a\\right\)\\right\)\\;\\middle\|\\;X\_\{1\}=x\\right\]∝exp⁡\(λ​\(μ−a\+Ψ​\(x−μ\)\)⊤​\(Id\+2​λ​Σ​\(Id−Ψ\)2\)−1​\(μ−a\+Ψ​\(x−μ\)\)\)\.\\displaystyle\\quad\\propto\\exp\\\!\\left\(\\lambda\(\\mu\-a\+\\Psi\(x\-\\mu\)\)^\{\\top\}\(I\_\{d\}\+2\\lambda\\Sigma\(I\_\{d\}\-\\Psi\)^\{2\}\)^\{\-1\}\(\\mu\-a\+\\Psi\(x\-\\mu\)\)\\right\)\.At last, the distribution ofx~1\\tilde\{x\}\_\{1\}under the guided ODE is

ρ~1\(x\)∝exp\(−12\(x−μ\)⊤Σ−1\(x−μ\)−λ∥x−a∥22\)𝔼\[1h0​\(X0\)\|X1=x\],\\displaystyle\\tilde\{\\rho\}\_\{1\}\(x\)\\propto\\exp\\\!\\left\(\-\\frac\{1\}\{2\}\(x\-\\mu\)^\{\\top\}\\Sigma^\{\-1\}\(x\-\\mu\)\-\\lambda\\lVert x\-a\\rVert\_\{2\}^\{2\}\\right\)\\mathbb\{E\}\\left\[\\frac\{1\}\{h\_\{0\}\(X\_\{0\}\)\}\\;\\middle\|\\;X\_\{1\}=x\\right\],which is a Gaussian measure

𝒩\(\\displaystyle\\mathcal\{N\}\\biggl\(μ−2​λ​Σ​\(Id−Ψ\)​\(Id\+2​λ​Σ​\(Id−Ψ\)\)−1​\(μ−a\),\\displaystyle\\mu\-2\\lambda\\Sigma\(I\_\{d\}\-\\Psi\)\(I\_\{d\}\+2\\lambda\\Sigma\(I\_\{d\}\-\\Psi\)\)^\{\-1\}\(\\mu\-a\),Σ\(Id\+2λΣ\(Id−Ψ\)2\)\(Id\+2λΣ\(Id−Ψ\)\)−2\)\.\\displaystyle\\Sigma\(I\_\{d\}\+2\\lambda\\Sigma\(I\_\{d\}\-\\Psi\)^\{2\}\)\(I\_\{d\}\+2\\lambda\\Sigma\(I\_\{d\}\-\\Psi\)\)^\{\-2\}\\biggr\)\.Now, we know that\(Xt,X1\)\(X\_\{t\},X\_\{1\}\)is Gaussian \(since it is a linear transformation of the Gaussian vector\(X0,X1\)\(X\_\{0\},X\_\{1\}\)\), so we can computehth\_\{t\}in closed form\. In particular, we have from \([9](https://arxiv.org/html/2606.02884#A2.E9)\) thatCov⁡\(Xt,X1\)=Σt1/2​Σ1/2​Ψt1/2\\operatorname\{Cov\}\(X\_\{t\},X\_\{1\}\)=\\Sigma\_\{t\}^\{1/2\}\\Sigma^\{1/2\}\\Psi\_\{t\}^\{1/2\}and thus the Gaussian conditioning formula yields

\(X1\|Xt\)∼𝒩​\(μ1\|t​\(x\),Σ1\|t\)≔𝒩​\(μ\+Σ1/2​Σt−1/2​Ψt1/2​\(Xt−μt\),Σ​\(Id−Ψt\)\)\.\\displaystyle\(X\_\{1\}\\,\|\\,X\_\{t\}\)\\sim\\mathcal\{N\}\(\\mu\_\{1\|t\}\(x\),\\Sigma\_\{1\|t\}\)\\coloneq\\mathcal\{N\}\\\!\\left\(\\mu\+\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\(X\_\{t\}\-\\mu\_\{t\}\),\\,\\Sigma\(I\_\{d\}\-\\Psi\_\{t\}\)\\right\)\.Now, we can computehth\_\{t\}in closed form as a Gaussian integral:

ht​\(x\)=𝔼​\[e−λ​∥X1−a∥22\|Xt=x\]∝exp⁡\(−λ​\(μ1\|t​\(x\)−a\)⊤​\(Id\+2​λ​Σ1\|t\)−1​\(μ1\|t​\(x\)−a\)\)\.\\displaystyle h\_\{t\}\(x\)=\\mathbb\{E\}\[e^\{\-\\lambda\\lVert X\_\{1\}\-a\\rVert\_\{2\}^\{2\}\}\\,\|\\,X\_\{t\}=x\]\\propto\\exp\\\!\\left\(\-\\lambda\(\\mu\_\{1\|t\}\(x\)\-a\)^\{\\top\}\(I\_\{d\}\+2\\lambda\\Sigma\_\{1\|t\}\)^\{\-1\}\(\\mu\_\{1\|t\}\(x\)\-a\)\\right\)\.This means that the guidance term is given by

∇log⁡ht​\(x\)=−2​λ​Σ1/2​Σt−1/2​Ψt1/2​\(Id\+2​λ​Σ1\|t\)−1​\(μ1\|t​\(x\)−a\)\.\\displaystyle\\nabla\\log h\_\{t\}\(x\)=\-2\\lambda\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\(I\_\{d\}\+2\\lambda\\Sigma\_\{1\|t\}\)^\{\-1\}\(\\mu\_\{1\|t\}\(x\)\-a\)\.∎

## Appendix COmitted proofs

### Proof of[Theorem˜12](https://arxiv.org/html/2606.02884#Thmtheorem12)

###### Theorem 12\(Doobhh\-transform\)\.

SupposeXtX\_\{t\}satisfies \([2](https://arxiv.org/html/2606.02884#S2.E2)\) and define the Doobhh\-functionht​\(x\)≔𝔼​\[eλ​r​\(X1\)\|Xt=x\]h\_\{t\}\(x\)\\coloneq\\mathbb\{E\}\[e^\{\\lambda r\(X\_\{1\}\)\}\\,\|\\,X\_\{t\}=x\]\. Then, consider the guided ODE \([4](https://arxiv.org/html/2606.02884#S2.E4)\)\. IfX0⟂⟂X1X\_\{0\}\\perp\\\!\\\!\\\!\\perp X\_\{1\}, the time\-marginals ofx~t\\tilde\{x\}\_\{t\}will beρ~t​\(x\)∝ρt​\(x\)​ht​\(x\)\\tilde\{\\rho\}\_\{t\}\(x\)\\propto\\rho\_\{t\}\(x\)\\,h\_\{t\}\(x\); in particular,ρ~1​\(x\)∝ρ1​\(x\)​eλ​r​\(x\)\\tilde\{\\rho\}\_\{1\}\(x\)\\propto\\rho\_\{1\}\(x\)\\,e^\{\\lambda r\(x\)\}will be the reward\-tilted measure\.

###### Proof\.

We first define the Doobhh\-function

ht​\(x\)=𝔼​\[eλ​r​\(X1\)\|Xt=x\]\\displaystyle h\_\{t\}\(x\)=\\mathbb\{E\}\[e^\{\\lambda r\(X\_\{1\}\)\}\\,\|\\,X\_\{t\}=x\]and note thatht​\(Xt\)h\_\{t\}\(X\_\{t\}\)is a Doob martingale\. Lettingℒt\\mathcal\{L\}\_\{t\}denote the generator of the diffusion \([2](https://arxiv.org/html/2606.02884#S2.E2)\):

ℒt​f=\(bt\+12​\(σt​σt⊤\)​∇log⁡ρt\)⊤​∇f\+12​tr⁡\(\(σt​σt⊤\)​∇2f\),\\mathcal\{L\}\_\{t\}f=\\left\(b\_\{t\}\+\\frac\{1\}\{2\}\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\nabla\\log\\rho\_\{t\}\\right\)^\{\\top\}\\nabla f\+\\frac\{1\}\{2\}\\operatorname\{tr\}\\\!\\left\(\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\nabla^\{2\}f\\right\),\(10\)we know thatht​\(Xt\)h\_\{t\}\(X\_\{t\}\)must satisfy the Kolmogorov backward equation

∂tht​\(Xt\)\+ℒt​ht​\(Xt\)=0\.\\partial\_\{t\}h\_\{t\}\(X\_\{t\}\)\+\\mathcal\{L\}\_\{t\}h\_\{t\}\(X\_\{t\}\)=0\.\(11\)Next, let\(ℱt\)t≥0\(\\mathcal\{F\}\_\{t\}\)\_\{t\\geq 0\}denote the natural filtration generated by the Brownian motion\(Bt\)t≥0\(B\_\{t\}\)\_\{t\\geq 0\}\. LettingPPdenote the law of the path\(Xt\)t∈\[0,1\]\(X\_\{t\}\)\_\{t\\in\[0,1\]\}and lettingPtP\_\{t\}denote the restriction ofPPtoℱt\\mathcal\{F\}\_\{t\}, we define the*Doobhh\-transform*ofPPby its Radon–Nikodym derivative:

d​Qtd​Pt=gt≔ht​\(Xt\)h0​\(X0\)\.\\displaystyle\\frac\{dQ\_\{t\}\}\{dP\_\{t\}\}=g\_\{t\}\\coloneq\\frac\{h\_\{t\}\(X\_\{t\}\)\}\{h\_\{0\}\(X\_\{0\}\)\}\.By Itô’s formula and \([11](https://arxiv.org/html/2606.02884#A3.E11)\), we have

d​gt\\displaystyle dg\_\{t\}=1h0​\(X0\)​\(\(∂tht​\(Xt\)\+ℒt​ht​\(Xt\)\)​d​t\+∇ht​\(Xt\)⊤​σt​d​Bt\)\\displaystyle=\\frac\{1\}\{h\_\{0\}\(X\_\{0\}\)\}\\left\(\(\\partial\_\{t\}h\_\{t\}\(X\_\{t\}\)\+\\mathcal\{L\}\_\{t\}h\_\{t\}\(X\_\{t\}\)\)\\,dt\+\\nabla h\_\{t\}\(X\_\{t\}\)^\{\\top\}\\sigma\_\{t\}\\,dB\_\{t\}\\right\)=1h0​\(X0\)​∇ht​\(Xt\)⊤​σt​d​Bt\\displaystyle=\\frac\{1\}\{h\_\{0\}\(X\_\{0\}\)\}\\nabla h\_\{t\}\(X\_\{t\}\)^\{\\top\}\\sigma\_\{t\}\\,dB\_\{t\}=gt​∇log⁡ht​\(Xt\)⊤​σt​d​Bt\.\\displaystyle=g\_\{t\}\\nabla\\log h\_\{t\}\(X\_\{t\}\)^\{\\top\}\\sigma\_\{t\}\\,dB\_\{t\}\.At this point, applying Itô’s formula shows that

d​\(log⁡gt\)=∇log⁡ht​\(Xt\)⊤​σt​d​Bt−12​∥∇log⁡ht​\(Xt\)⊤​σt∥22​d​t\\displaystyle d\(\\log g\_\{t\}\)=\\nabla\\log h\_\{t\}\(X\_\{t\}\)^\{\\top\}\\sigma\_\{t\}\\,dB\_\{t\}\-\\frac\{1\}\{2\}\\lVert\\nabla\\log h\_\{t\}\(X\_\{t\}\)^\{\\top\}\\sigma\_\{t\}\\rVert\_\{2\}^\{2\}\\,dtand integrating both sides shows thatgtg\_\{t\}is the Doléans–Dade exponential \(local\) martingale

gt=exp⁡\(∫0t∇log⁡hs​\(Xs\)⊤​σs​𝑑Bs−12​∫0t∥∇log⁡hs​\(Xs\)⊤​σs∥22​𝑑s\)\.\\displaystyle g\_\{t\}=\\exp\\\!\\left\(\\int\_\{0\}^\{t\}\\nabla\\log h\_\{s\}\(X\_\{s\}\)^\{\\top\}\\sigma\_\{s\}\\,dB\_\{s\}\-\\frac\{1\}\{2\}\\int\_\{0\}^\{t\}\\lVert\\nabla\\log h\_\{s\}\(X\_\{s\}\)^\{\\top\}\\sigma\_\{s\}\\rVert\_\{2\}^\{2\}\\,ds\\right\)\.Under the standard Novikov’s condition \(for instance\):

𝔼​\[exp⁡\(12​∫01∥∇log⁡hs​\(Xs\)⊤​σs∥22​𝑑s\)\]<∞,\\displaystyle\\mathbb\{E\}\\left\[\\exp\\\!\\left\(\\frac\{1\}\{2\}\\int\_\{0\}^\{1\}\\lVert\\nabla\\log h\_\{s\}\(X\_\{s\}\)^\{\\top\}\\sigma\_\{s\}\\rVert\_\{2\}^\{2\}\\,ds\\right\)\\right\]<\\infty,gtg\_\{t\}is a true martingale\. At last, Girsanov’s theorem implies that underQ≔Q1Q\\coloneq Q\_\{1\}, the process

B~t=Bt−∫0tσs⊤​∇log⁡hs​\(Xs\)​𝑑s\\displaystyle\\tilde\{B\}\_\{t\}=B\_\{t\}\-\\int\_\{0\}^\{t\}\\sigma\_\{s\}^\{\\top\}\\nabla\\log h\_\{s\}\(X\_\{s\}\)\\,dsis a Brownian motion, and substituting back into the SDE forXtX\_\{t\}gives

d​Xt=\(bt​\(Xt\)\+12​\(σt​σt⊤\)​∇log⁡ρt​\(Xt\)\+\(σt​σt⊤\)​∇log⁡ht​\(Xt\)\)​d​t\+σt​d​B~t\.\\displaystyle dX\_\{t\}=\\left\(b\_\{t\}\(X\_\{t\}\)\+\\frac\{1\}\{2\}\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\nabla\\log\\rho\_\{t\}\(X\_\{t\}\)\+\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\nabla\\log h\_\{t\}\(X\_\{t\}\)\\right\)dt\+\\sigma\_\{t\}\\,d\\tilde\{B\}\_\{t\}\.We can then find the densityρ~t\\tilde\{\\rho\}\_\{t\}ofXtX\_\{t\}underQQby integrating a test functionf∈Cc∞​\(ℝd\)f\\in C\_\{c\}^\{\\infty\}\(\\mathbb\{R\}^\{d\}\):

𝔼Q​\[f​\(Xt\)\]=𝔼P​\[gt​f​\(Xt\)\]=𝔼P​\[f​\(Xt\)​ht​\(Xt\)h0​\(X0\)\]\.\\displaystyle\\mathbb\{E\}\_\{Q\}\[f\(X\_\{t\}\)\]=\\mathbb\{E\}\_\{P\}\[g\_\{t\}\\,f\(X\_\{t\}\)\]=\\mathbb\{E\}\_\{P\}\\left\[f\(X\_\{t\}\)\\,\\frac\{h\_\{t\}\(X\_\{t\}\)\}\{h\_\{0\}\(X\_\{0\}\)\}\\right\]\.In the specific case thatX0⟂⟂X1X\_\{0\}\\perp\\\!\\\!\\\!\\perp X\_\{1\}\(which happens using the memoryless noise schedule from[Theorem˜13](https://arxiv.org/html/2606.02884#Thmtheorem13)\), it is clear thath0​\(X0\)=𝔼​\[eλ​r​\(X1\)\]h\_\{0\}\(X\_\{0\}\)=\\mathbb\{E\}\[e^\{\\lambda r\(X\_\{1\}\)\}\]is a normalizing constant, and thusρ~t​\(x\)∝ρt​\(x\)​ht​\(x\)\\tilde\{\\rho\}\_\{t\}\(x\)\\propto\\rho\_\{t\}\(x\)\\,h\_\{t\}\(x\)\. At the endpoint, this means thatρ~1​\(x\)∝eλ​r​\(x\)​ρ1​\(x\)\\tilde\{\\rho\}\_\{1\}\(x\)\\propto e^\{\\lambda r\(x\)\}\\rho\_\{1\}\(x\)as desired, and the Doobhh\-transform of the original diffusion gives a principled way to sample from the reward\-tilted measure\. We can then convert the guided SDE back to a guided ODE by considering the associated probability flow:

d​Xt\\displaystyle dX\_\{t\}=\(bt​\(Xt\)\+12​\(σt​σt⊤\)​∇log⁡ρt​\(Xt\)\+\(σt​σt⊤\)​∇log⁡ht​\(Xt\)−12​\(σt​σt⊤\)​∇log⁡ρ~t​\(Xt\)\)​d​t\\displaystyle=\\left\(b\_\{t\}\(X\_\{t\}\)\+\\frac\{1\}\{2\}\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\nabla\\log\\rho\_\{t\}\(X\_\{t\}\)\+\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\nabla\\log h\_\{t\}\(X\_\{t\}\)\-\\frac\{1\}\{2\}\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\nabla\\log\\tilde\{\\rho\}\_\{t\}\(X\_\{t\}\)\\right\)dt=\(bt​\(Xt\)\+12​\(σt​σt⊤\)​∇log⁡ht​\(Xt\)\)​d​t\.\\displaystyle=\\left\(b\_\{t\}\(X\_\{t\}\)\+\\frac\{1\}\{2\}\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\nabla\\log h\_\{t\}\(X\_\{t\}\)\\right\)dt\.Hence, the guided probability flow ODE simply includes an additional score term12​\(σt​σt⊤\)​∇log⁡ht​\(Xt\)\\frac\{1\}\{2\}\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\nabla\\log h\_\{t\}\(X\_\{t\}\)that steers the dynamics toward high\-reward regions of the output space\. ∎

### Proof of[Theorem˜13](https://arxiv.org/html/2606.02884#Thmtheorem13)

###### Theorem 13\(Memoryless noise schedule\)\.

Consider the forward SDE \([2](https://arxiv.org/html/2606.02884#S2.E2)\) for the linear interpolant \([1](https://arxiv.org/html/2606.02884#S2.E1)\)\. Then, choosing the memoryless noise scheduleσt=2​\(1−t\)/t​Id\\sigma\_\{t\}=\\sqrt\{2\(1\-t\)/t\}\\,I\_\{d\}ensures that\(Xt\|X1\)​=𝑑​\(It\|I1\)\(X\_\{t\}\\,\|\\,X\_\{1\}\)\\overset\{d\}\{=\}\(I\_\{t\}\\,\|\\,I\_\{1\}\)\.

###### Proof\.

Note that\(It\|I1\)∼𝒩​\(t​I1,\(1−t\)2​Id\)\(I\_\{t\}\\,\|\\,I\_\{1\}\)\\sim\\mathcal\{N\}\(tI\_\{1\},\\,\(1\-t\)^\{2\}I\_\{d\}\), so by Tweedie’s formula, we have

𝔼​\[I1\|It=x\]=xt\+\(1−t\)2t​∇log⁡ρt​\(x\)\\displaystyle\\mathbb\{E\}\[I\_\{1\}\\,\|\\,I\_\{t\}=x\]=\\frac\{x\}\{t\}\+\\frac\{\(1\-t\)^\{2\}\}\{t\}\\nabla\\log\\rho\_\{t\}\(x\)and

𝔼\[X0\|It=x\]=𝔼\[It−t​I11−t\|It=x\]=−\(1−t\)∇logρt\(x\)\.\\displaystyle\\mathbb\{E\}\[X\_\{0\}\\,\|\\,I\_\{t\}=x\]=\\mathbb\{E\}\\left\[\\frac\{I\_\{t\}\-tI\_\{1\}\}\{1\-t\}\\;\\middle\|\\;I\_\{t\}=x\\right\]=\-\(1\-t\)\\nabla\\log\\rho\_\{t\}\(x\)\.As a result, we deduce that

bt​\(x\)=𝔼​\[I˙t\|It=x\]=𝔼​\[I1−I0\|It=x\]=xt\+1−tt​∇log⁡ρt​\(x\)\.\\displaystyle b\_\{t\}\(x\)=\\mathbb\{E\}\[\\dot\{I\}\_\{t\}\\,\|\\,I\_\{t\}=x\]=\\mathbb\{E\}\[I\_\{1\}\-I\_\{0\}\\,\|\\,I\_\{t\}=x\]=\\frac\{x\}\{t\}\+\\frac\{1\-t\}\{t\}\\nabla\\log\\rho\_\{t\}\(x\)\.Parameterizingσt=ηt​Id\\sigma\_\{t\}=\\eta\_\{t\}I\_\{d\}, the forward SDE for the stochastic interpolant with the memoryless noise schedule is

d​Xt=\(Xtt\+1−tt​∇log⁡ρt​\(Xt\)\+ηt22​∇log⁡ρt​\(Xt\)\)​d​t\+ηt​d​Bt\.\\displaystyle dX\_\{t\}=\\left\(\\frac\{X\_\{t\}\}\{t\}\+\\frac\{1\-t\}\{t\}\\nabla\\log\\rho\_\{t\}\(X\_\{t\}\)\+\\frac\{\\eta\_\{t\}^\{2\}\}\{2\}\\nabla\\log\\rho\_\{t\}\(X\_\{t\}\)\\right\)dt\+\\eta\_\{t\}\\,dB\_\{t\}\.The Anderson time\-reversal formula then yields the backward SDE

d​Xt←=\(−Xt←t−\(1−tt−ηt22\)​∇log⁡ρt​\(Xt←\)\)​d​t\+ηt​d​Bt←,\\displaystyle dX\_\{t\}^\{\\leftarrow\}=\\left\(\-\\frac\{X\_\{t\}^\{\\leftarrow\}\}\{t\}\-\\left\(\\frac\{1\-t\}\{t\}\-\\frac\{\\eta\_\{t\}^\{2\}\}\{2\}\\right\)\\nabla\\log\\rho\_\{t\}\(X\_\{t\}^\{\\leftarrow\}\)\\right\)dt\+\\eta\_\{t\}\\,dB\_\{t\}^\{\\leftarrow\},and settingηt=2​\(1−t\)/t\\eta\_\{t\}=\\sqrt\{2\(1\-t\)/t\}cancels the score term in the drift of the backward SDE so that

d​Xt←=−Xt←t​d​t\+2​\(1−t\)t​d​Bt←\.\\displaystyle dX\_\{t\}^\{\\leftarrow\}=\-\\frac\{X\_\{t\}^\{\\leftarrow\}\}\{t\}\\,dt\+\\sqrt\{\\frac\{2\(1\-t\)\}\{t\}\}\\,dB\_\{t\}^\{\\leftarrow\}\.This shows that\(Xt\|X1\)\(X\_\{t\}\\,\|\\,X\_\{1\}\)is Gaussian \(because the diffusion coefficient is deterministic\) with meant​X1tX\_\{1\}and covariance\(1−t\)2​Id\(1\-t\)^\{2\}I\_\{d\}\(for example, by the Itô isometry\), matching the distribution of\(It\|I1\)\(I\_\{t\}\\,\|\\,I\_\{1\}\)\. ∎

### Convergence to plug\-in flow

We state and prove the formal version of the convergence result discussed in[Section˜4\.1\.1](https://arxiv.org/html/2606.02884#S4.SS1.SSS1)\.

###### Proposition 14\(Convergence to plug\-in flow\)\.

Letx~t\\tilde\{x\}\_\{t\}be the solution to the exact continuous\-time ODEd​x~t=bt\(k\)​\(x~t\)​d​td\\tilde\{x\}\_\{t\}=b\_\{t\}^\{\(k\)\}\(\\tilde\{x\}\_\{t\}\)\\,dtfort∈\[0,1\]t\\in\[0,1\]with initial conditionx~0=x0\\tilde\{x\}\_\{0\}=x\_\{0\}\. Let\(x¯t\)t∈\[0,1\]\(\\bar\{x\}\_\{t\}\)\_\{t\\in\[0,1\]\}denote the continuous\-time Euler approximation with step sizeh=1/Nh=1/N, defined byx¯0=x0\\bar\{x\}\_\{0\}=x\_\{0\}and

d​x¯t=\(bη​\(t\)​\(x¯η​\(t\)\)\+12​\(ση​\(t\)​ση​\(t\)⊤\)​∇log⁡h^η​\(t\)\(k\)​\(x¯η​\(t\)\)\)​d​t,d\\bar\{x\}\_\{t\}=\\left\(b\_\{\\eta\(t\)\}\(\\bar\{x\}\_\{\\eta\(t\)\}\)\+\\frac\{1\}\{2\}\(\\sigma\_\{\\eta\(t\)\}\\sigma\_\{\\eta\(t\)\}^\{\\top\}\)\\nabla\\log\\hat\{h\}\_\{\\eta\(t\)\}^\{\(k\)\}\(\\bar\{x\}\_\{\\eta\(t\)\}\)\\right\)dt,whereη​\(t\)≔⌊t/h⌋​h\\eta\(t\)\\coloneq\\lfloor t/h\\rfloor his the time of the most recent grid point\. We make the following assumptions:

1. \(i\)∥bs\(k\)​\(x\)−bt\(k\)​\(y\)∥2≤L​\(∥x−y∥2\+\|s−t\|\)\\lVert b\_\{s\}^\{\(k\)\}\(x\)\-b\_\{t\}^\{\(k\)\}\(y\)\\rVert\_\{2\}\\leq L\(\\lVert x\-y\\rVert\_\{2\}\+\\lvert s\-t\\rvert\)for someL\>0L\>0and for alls,t∈\[0,1\]s,t\\in\[0,1\]andx,y∈ℝdx,y\\in\\mathbb\{R\}^\{d\}\.
2. \(ii\)Defining the guidance residualξt​\(x\)≔∇log⁡h^t\(k\)​\(x\)−ut\(k\)​\(x\)\\xi\_\{t\}\(x\)\\coloneq\\nabla\\log\\hat\{h\}^\{\(k\)\}\_\{t\}\(x\)\-u\_\{t\}^\{\(k\)\}\(x\), we have𝔼​\[‖12​\(σt​σt⊤\)​ξt​\(x\)‖22\]≤Vk\\mathbb\{E\}\\left\[\\left\\lVert\\frac\{1\}\{2\}\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)\\,\\xi\_\{t\}\(x\)\\right\\rVert\_\{2\}^\{2\}\\right\]\\leq V\_\{k\}for someVk\>0V\_\{k\}\>0and for allx∈ℝdx\\in\\mathbb\{R\}^\{d\}\.
3. \(iii\)∥bt\(k\)​\(x¯t\)∥2≤B\\lVert b\_\{t\}^\{\(k\)\}\(\\bar\{x\}\_\{t\}\)\\rVert\_\{2\}\\leq Bis uniformly bounded byB\>0B\>0\.

Then, the Euler trajectory converges uniformly over\[0,1\]\[0,1\]in probability to the ODE dynamics asN→∞N\\to\\infty:

supt∈\[0,1\]∥x¯t−x~t∥2​→𝑝​0\.\\sup\_\{t\\in\[0,1\]\}\\;\\lVert\\bar\{x\}\_\{t\}\-\\tilde\{x\}\_\{t\}\\rVert\_\{2\}\\overset\{p\}\{\\to\}0\.

The proof combines Grönwall’s inequality with Doob’sL2L^\{2\}maximal inequality for the martingale residual\.

###### Proof\.

First, substitutingbt\(k\)​\(x\)=bt​\(x\)\+12​\(σt​σt⊤\)​ut\(k\)​\(x\)b\_\{t\}^\{\(k\)\}\(x\)=b\_\{t\}\(x\)\+\\frac\{1\}\{2\}\(\\sigma\_\{t\}\\sigma\_\{t\}^\{\\top\}\)u\_\{t\}^\{\(k\)\}\(x\)and∇log⁡h^t\(k\)​\(x\)=ut\(k\)​\(x\)\+ξt​\(x\)\\nabla\\log\\hat\{h\}\_\{t\}^\{\(k\)\}\(x\)=u\_\{t\}^\{\(k\)\}\(x\)\+\\xi\_\{t\}\(x\)shows that

x¯t\\displaystyle\\bar\{x\}\_\{t\}=x0\+∫0t\(bη​\(s\)​\(x¯η​\(s\)\)\+12​\(ση​\(s\)​ση​\(s\)⊤\)​\(uη​\(s\)\(k\)​\(x¯η​\(s\)\)\+ξη​\(s\)​\(x¯η​\(s\)\)\)\)​𝑑s\\displaystyle=x\_\{0\}\+\\int\_\{0\}^\{t\}\\left\(b\_\{\\eta\(s\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\+\\frac\{1\}\{2\}\(\\sigma\_\{\\eta\(s\)\}\\sigma\_\{\\eta\(s\)\}^\{\\top\}\)\\left\(u\_\{\\eta\(s\)\}^\{\(k\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\+\\xi\_\{\\eta\(s\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\\right\)\\right\)ds=x0\+∫0t\(bη​\(s\)\(k\)​\(x¯η​\(s\)\)\+12​\(ση​\(s\)​ση​\(s\)⊤\)​ξη​\(s\)​\(x¯η​\(s\)\)\)​𝑑s\.\\displaystyle=x\_\{0\}\+\\int\_\{0\}^\{t\}\\left\(b\_\{\\eta\(s\)\}^\{\(k\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\+\\frac\{1\}\{2\}\(\\sigma\_\{\\eta\(s\)\}\\sigma\_\{\\eta\(s\)\}^\{\\top\}\)\\xi\_\{\\eta\(s\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\\right\)ds\.Thus, we have

∥x¯t−x~t∥2\\displaystyle\\lVert\\bar\{x\}\_\{t\}\-\\tilde\{x\}\_\{t\}\\rVert\_\{2\}=‖∫0t\(bη​\(s\)\(k\)​\(x¯η​\(s\)\)−bs\(k\)​\(X~s\)\)​𝑑s\+Mt‖2\\displaystyle=\\left\\lVert\\int\_\{0\}^\{t\}\\left\(b\_\{\\eta\(s\)\}^\{\(k\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\-b\_\{s\}^\{\(k\)\}\(\\tilde\{X\}\_\{s\}\)\\right\)ds\+M\_\{t\}\\right\\rVert\_\{2\}≤‖∫0t\(bs\(k\)​\(x¯s\)−bs\(k\)​\(X~s\)\)​𝑑s‖2\+‖∫0t\(bη​\(s\)\(k\)​\(x¯η​\(s\)\)−bs\(k\)​\(x¯s\)\)​𝑑s‖2\+∥Mt∥2\\displaystyle\\leq\\left\\lVert\\int\_\{0\}^\{t\}\\left\(b\_\{s\}^\{\(k\)\}\(\\bar\{x\}\_\{s\}\)\-b\_\{s\}^\{\(k\)\}\(\\tilde\{X\}\_\{s\}\)\\right\)ds\\right\\rVert\_\{2\}\+\\left\\lVert\\int\_\{0\}^\{t\}\\left\(b\_\{\\eta\(s\)\}^\{\(k\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\-b\_\{s\}^\{\(k\)\}\(\\bar\{x\}\_\{s\}\)\\right\)ds\\right\\rVert\_\{2\}\+\\lVert M\_\{t\}\\rVert\_\{2\}≤L​∫0t∥x¯s−X~s∥2​𝑑s\+‖∫0t\(bη​\(s\)\(k\)​\(x¯η​\(s\)\)−bs\(k\)​\(x¯s\)\)​𝑑s‖2\+∥Mt∥2,\\displaystyle\\leq L\\int\_\{0\}^\{t\}\\lVert\\bar\{x\}\_\{s\}\-\\tilde\{X\}\_\{s\}\\rVert\_\{2\}\\,ds\+\\left\\lVert\\int\_\{0\}^\{t\}\\left\(b\_\{\\eta\(s\)\}^\{\(k\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\-b\_\{s\}^\{\(k\)\}\(\\bar\{x\}\_\{s\}\)\\right\)ds\\right\\rVert\_\{2\}\+\\lVert M\_\{t\}\\rVert\_\{2\},whereMt≔∫0t12​\(ση​\(s\)​ση​\(s\)⊤\)​ξη​\(s\)​\(x¯η​\(s\)\)​𝑑sM\_\{t\}\\coloneq\\int\_\{0\}^\{t\}\\frac\{1\}\{2\}\(\\sigma\_\{\\eta\(s\)\}\\sigma\_\{\\eta\(s\)\}^\{\\top\}\)\\,\\xi\_\{\\eta\(s\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\\,ds\. By Grönwall’s inequality, this implies the bound

supt∈\[0,1\]∥x¯t−x~t∥2≤\(supt∈\[0,1\]‖∫0t\(bη​\(s\)\(k\)​\(x¯η​\(s\)\)−bs\(k\)​\(x¯s\)\)​𝑑s‖2\+supt∈\[0,1\]∥Mt∥2\)​eL\.\\displaystyle\\sup\_\{t\\in\[0,1\]\}\\;\\lVert\\bar\{x\}\_\{t\}\-\\tilde\{x\}\_\{t\}\\rVert\_\{2\}\\leq\\left\(\\sup\_\{t\\in\[0,1\]\}\\,\\left\\lVert\\int\_\{0\}^\{t\}\\left\(b\_\{\\eta\(s\)\}^\{\(k\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\-b\_\{s\}^\{\(k\)\}\(\\bar\{x\}\_\{s\}\)\\right\)ds\\right\\rVert\_\{2\}\+\\sup\_\{t\\in\[0,1\]\}\\lVert M\_\{t\}\\rVert\_\{2\}\\right\)e^\{L\}\.At this point, we have

supt∈\[0,1\]‖∫0t\(bη​\(s\)\(k\)​\(x¯η​\(s\)\)−bs\(k\)​\(x¯s\)\)​𝑑s‖2\\displaystyle\\sup\_\{t\\in\[0,1\]\}\\,\\left\\lVert\\int\_\{0\}^\{t\}\\left\(b\_\{\\eta\(s\)\}^\{\(k\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\-b\_\{s\}^\{\(k\)\}\(\\bar\{x\}\_\{s\}\)\\right\)ds\\right\\rVert\_\{2\}≤L​\(∫01∥x¯η​\(s\)−x¯s∥2​𝑑s\+∫01\|η​\(s\)−s\|​𝑑s\)\\displaystyle\\quad\\leq L\\left\(\\int\_\{0\}^\{1\}\\lVert\\bar\{x\}\_\{\\eta\(s\)\}\-\\bar\{x\}\_\{s\}\\rVert\_\{2\}\\,ds\+\\int\_\{0\}^\{1\}\\lvert\\eta\(s\)\-s\\rvert\\,ds\\right\)≤L​∫01\(s−η​\(s\)\)​‖bη​\(s\)\(k\)​\(x¯η​\(s\)\)\+12​\(ση​\(s\)​ση​\(s\)⊤\)​ξη​\(s\)​\(x¯η​\(s\)\)‖2​𝑑s\+L​∫01h​𝑑s\\displaystyle\\quad\\leq L\\int\_\{0\}^\{1\}\(s\-\\eta\(s\)\)\\left\\lVert b\_\{\\eta\(s\)\}^\{\(k\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\+\\frac\{1\}\{2\}\(\\sigma\_\{\\eta\(s\)\}\\sigma\_\{\\eta\(s\)\}^\{\\top\}\)\\,\\xi\_\{\\eta\(s\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\\right\\rVert\_\{2\}\\,ds\+L\\int\_\{0\}^\{1\}h\\,ds≤L​h​∫01\(B\+‖12​\(ση​\(s\)​ση​\(s\)⊤\)​ξη​\(s\)​\(x¯η​\(s\)\)‖2\)​𝑑s\+L​h\.\\displaystyle\\quad\\leq Lh\\int\_\{0\}^\{1\}\\left\(B\+\\left\\lVert\\frac\{1\}\{2\}\(\\sigma\_\{\\eta\(s\)\}\\sigma\_\{\\eta\(s\)\}^\{\\top\}\)\\,\\xi\_\{\\eta\(s\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\\right\\rVert\_\{2\}\\right\)ds\+Lh\.Taking expectations on both sides and using Jensen’s inequality, we obtain

𝔼​\[supt∈\[0,1\]‖∫0t\(bη​\(s\)\(k\)​\(x¯η​\(s\)\)−bs\(k\)​\(x¯s\)\)​𝑑s‖2\]\\displaystyle\\mathbb\{E\}\\left\[\\sup\_\{t\\in\[0,1\]\}\\,\\left\\lVert\\int\_\{0\}^\{t\}\\left\(b\_\{\\eta\(s\)\}^\{\(k\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\-b\_\{s\}^\{\(k\)\}\(\\bar\{x\}\_\{s\}\)\\right\)ds\\right\\rVert\_\{2\}\\right\]≤L​h​∫01\(B\+𝔼\[∥12\(ση​\(s\)ση​\(s\)⊤\)ξη​\(s\)\(x¯η​\(s\)\)∥22\|x¯η​\(s\)\]\)​𝑑s\+L​h\\displaystyle\\quad\\leq Lh\\int\_\{0\}^\{1\}\\left\(B\+\\sqrt\{\\mathbb\{E\}\\left\[\\left\\lVert\\frac\{1\}\{2\}\(\\sigma\_\{\\eta\(s\)\}\\sigma\_\{\\eta\(s\)\}^\{\\top\}\)\\,\\xi\_\{\\eta\(s\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\\right\\rVert\_\{2\}^\{2\}\\;\\middle\|\\;\\bar\{x\}\_\{\\eta\(s\)\}\\right\]\}\\right\)ds\+Lh≤L​h​\(B\+Vk\+1\)\.\\displaystyle\\quad\\leq Lh\(B\+\\sqrt\{V\_\{k\}\}\+1\)\.Markov’s inequality now implies that ash↓0h\\downarrow 0,

supt∈\[0,1\]‖∫0t\(bη​\(s\)\(k\)​\(x¯η​\(s\)\)−bs\(k\)​\(x¯s\)\)​𝑑s‖2​→𝑝​0\.\\displaystyle\\sup\_\{t\\in\[0,1\]\}\\,\\left\\lVert\\int\_\{0\}^\{t\}\\left\(b\_\{\\eta\(s\)\}^\{\(k\)\}\(\\bar\{x\}\_\{\\eta\(s\)\}\)\-b\_\{s\}^\{\(k\)\}\(\\bar\{x\}\_\{s\}\)\\right\)ds\\right\\rVert\_\{2\}\\overset\{p\}\{\\to\}0\.Lastly, it remains to show thatsupt∈\[0,1\]∥Mt∥2​→𝑝​0\\sup\_\{t\\in\[0,1\]\}\\,\\lVert M\_\{t\}\\rVert\_\{2\}\\overset\{p\}\{\\to\}0ash↓0h\\downarrow 0\. Definingtn=n​ht\_\{n\}=nhfor0≤n≤N0\\leq n\\leq N, we know that the supremum ofMtM\_\{t\}is achieved at sometnt\_\{n\}sinceMtM\_\{t\}is piecewise linear\. Furthermore,\(Mtn\)n=0N\(M\_\{t\_\{n\}\}\)\_\{n=0\}^\{N\}is a discrete\-time martingale with respect to the filtration\(ℱtn\)n=0N\(\\mathcal\{F\}\_\{t\_\{n\}\}\)\_\{n=0\}^\{N\}generated by the independent samples drawn in the plug\-in estimator, so by Doob’sL2L^\{2\}maximal inequality for discrete\-time martingales:

𝔼\[max0≤n≤N∥Mtn∥22\]≤4𝔼\[∥M1∥22\]\.\\displaystyle\\mathbb\{E\}\\left\[\\max\_\{0\\leq n\\leq N\}\\,\\lVert M\_\{t\_\{n\}\}\\rVert\_\{2\}^\{2\}\\right\]\\leq 4\\,\\mathbb\{E\}\[\\lVert M\_\{1\}\\rVert\_\{2\}^\{2\}\]\.Since the martingale increments are orthogonal inL2​\(ℙ\)L^\{2\}\(\\mathbb\{P\}\), we have

𝔼​\[∥M1∥22\]=∑n=0N−1h2​𝔼​\[‖12​\(σtn​σtn⊤\)​ξtn​\(x¯tn\)‖22\]≤Vk​∑n=0N−1h2=h​Vk\.\\displaystyle\\mathbb\{E\}\[\\lVert M\_\{1\}\\rVert\_\{2\}^\{2\}\]=\\sum\_\{n=0\}^\{N\-1\}h^\{2\}\\,\\mathbb\{E\}\\left\[\\left\\lVert\\frac\{1\}\{2\}\(\\sigma\_\{t\_\{n\}\}\\sigma\_\{t\_\{n\}\}^\{\\top\}\)\\,\\xi\_\{t\_\{n\}\}\(\\bar\{x\}\_\{t\_\{n\}\}\)\\right\\rVert\_\{2\}^\{2\}\\right\]\\leq V\_\{k\}\\sum\_\{n=0\}^\{N\-1\}h^\{2\}=hV\_\{k\}\.Thus, we havemax0≤n≤N∥Mtn∥2→0\\max\_\{0\\leq n\\leq N\}\\,\\lVert M\_\{t\_\{n\}\}\\rVert\_\{2\}\\to 0inL2​\(ℙ\)L^\{2\}\(\\mathbb\{P\}\)and therefore in probability by Chebyshev’s inequality, completing the proof\. ∎

### Plug\-in flow for Gaussian target \(non\-memoryless\)

The general non\-memoryless analogue of[Theorem˜2](https://arxiv.org/html/2606.02884#Thmtheorem2)is the following\.

###### Theorem 15\(Plug\-in flow for Gaussian target \(non\-memoryless\)\)\.

Consider the plug\-in flow \([6](https://arxiv.org/html/2606.02884#S4.E6)\) withk=1k=1, and letΨ≔Ψ0\\Psi\\coloneq\\Psi\_\{0\}as in[Theorem˜11](https://arxiv.org/html/2606.02884#Thmtheorem11)\. Then, the guidance term is given by

ut\(1\)​\(x\)=−2​λ​Σ1/2​Σt−1/2​Ψt1/2​\(μ1\|t​\(x\)−a\)\\displaystyle u\_\{t\}^\{\(1\)\}\(x\)=\-2\\lambda\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\(\\mu\_\{1\|t\}\(x\)\-a\)and the terminal distribution of the plug\-in flow isx~1∼𝒩​\(μ\(1\),Σ\(1\)\)\\tilde\{x\}\_\{1\}\\sim\\mathcal\{N\}\(\\mu^\{\(1\)\},\\,\\Sigma^\{\(1\)\}\), where

μ\(1\)=μ−π​\(λ​Σ\)1/2​exp⁡\(−λ​Σ\)​\(erfi⁡\(\(λ​Σ\)1/2\)−erfi⁡\(\(λ​Σ\)1/2​Ψ1/2\)\)​\(μ−a\)\\displaystyle\\mu^\{\(1\)\}=\\mu\-\\sqrt\{\\pi\}\\,\(\\lambda\\Sigma\)^\{1/2\}\\exp\(\-\\lambda\\Sigma\)\\left\(\\operatorname\{erfi\}\\\!\\left\(\(\\lambda\\Sigma\)^\{1/2\}\\right\)\-\\operatorname\{erfi\}\\\!\\left\(\(\\lambda\\Sigma\)^\{1/2\}\\Psi^\{1/2\}\\right\)\\right\)\(\\mu\-a\)and

Σ\(1\)=Σ​exp⁡\(−2​λ​Σ​\(Id−Ψ\)\)\.\\displaystyle\\Sigma^\{\(1\)\}=\\Sigma\\exp\(\-2\\lambda\\Sigma\(I\_\{d\}\-\\Psi\)\)\.

###### Proof\.

We begin by computing the guidance termut\(1\)​\(x\)u\_\{t\}^\{\(1\)\}\(x\)using the reparameterization trick

X1\(1\)​\(x\)=μ1\|t​\(x\)\+Σ1\|t1/2​ϵ\\displaystyle X\_\{1\}^\{\(1\)\}\(x\)=\\mu\_\{1\|t\}\(x\)\+\\Sigma\_\{1\|t\}^\{1/2\}\\epsilonwhereϵ∼𝒩​\(0,Id\)\\epsilon\\sim\\mathcal\{N\}\(0,I\_\{d\}\)is independent of everything else\. In particular, sinceμ1\|t​\(x\)\\mu\_\{1\|t\}\(x\)is affine inxx, we have∇X1\(1\)​\(x\)=Σ1/2​Σt−1/2​Ψt1/2\\nabla X\_\{1\}^\{\(1\)\}\(x\)=\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\. Thus, we apply the chain rule:

∇log⁡h^t\(1\)​\(x\)=∇\(−λ​∥X1\(1\)​\(x\)−a∥22\)=−2​λ​Σ1/2​Σt−1/2​Ψt1/2​\(X1\(1\)​\(x\)−a\),\\displaystyle\\nabla\\log\\hat\{h\}^\{\(1\)\}\_\{t\}\(x\)=\\nabla\\left\(\-\\lambda\\lVert X\_\{1\}^\{\(1\)\}\(x\)\-a\\rVert\_\{2\}^\{2\}\\right\)=\-2\\lambda\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\(X\_\{1\}^\{\(1\)\}\(x\)\-a\),and taking conditional expectations on both sides yields the guidance term

ut\(1\)​\(x\)=𝔼​\[∇log⁡h^t\(1\)​\(Xt\)\|Xt=x\]=−2​λ​Σ1/2​Σt−1/2​Ψt1/2​\(μ1\|t​\(x\)−a\)\.\\displaystyle u\_\{t\}^\{\(1\)\}\(x\)=\\mathbb\{E\}\[\\nabla\\log\\hat\{h\}^\{\(1\)\}\_\{t\}\(X\_\{t\}\)\\,\|\\,X\_\{t\}=x\]=\-2\\lambda\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\(\\mu\_\{1\|t\}\(x\)\-a\)\.Substituting this back into the plug\-in flow ODE, the dynamics are given by

d​x~t=\(μ\+12​Σ˙t​Σt−1​\(x~t−μt\)−λ​ηt2​Σ1/2​Σt−1/2​Ψt1/2​\(μ1\|t​\(x~t\)−a\)\)​d​t\.\\displaystyle d\\tilde\{x\}\_\{t\}=\\left\(\\mu\+\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{t\}\\Sigma\_\{t\}^\{\-1\}\(\\tilde\{x\}\_\{t\}\-\\mu\_\{t\}\)\-\\lambda\\eta\_\{t\}^\{2\}\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\(\\mu\_\{1\|t\}\(\\tilde\{x\}\_\{t\}\)\-a\)\\right\)dt\.Because the initial conditionx~0∼𝒩​\(0,Id\)\\tilde\{x\}\_\{0\}\\sim\\mathcal\{N\}\(0,I\_\{d\}\)is Gaussian and the drift is an affine function ofx~t\\tilde\{x\}\_\{t\}, the processx~t\\tilde\{x\}\_\{t\}remains exactly Gaussian for allt∈\[0,1\]t\\in\[0,1\], and we track its meanμt\(1\)=𝔼​\[x~t\]\\mu^\{\(1\)\}\_\{t\}=\\mathbb\{E\}\[\\tilde\{x\}\_\{t\}\]and covarianceΣt\(1\)=Cov⁡\(x~t\)\\Sigma^\{\(1\)\}\_\{t\}=\\operatorname\{Cov\}\(\\tilde\{x\}\_\{t\}\)\. Taking the expectation of the drift yields an ODE for the mean:

d​μt\(1\)=μ\+\(12​Σ˙t​Σt−1−λ​ηt2​Σ​Σt−1​Ψt\)​\(μt\(1\)−μt\)−λ​ηt2​Σ1/2​Σt−1/2​Ψt1/2​\(μ−a\)\.\\displaystyle d\\mu^\{\(1\)\}\_\{t\}=\\mu\+\\left\(\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{t\}\\Sigma\_\{t\}^\{\-1\}\-\\lambda\\eta\_\{t\}^\{2\}\\,\\Sigma\\Sigma\_\{t\}^\{\-1\}\\Psi\_\{t\}\\right\)\(\\mu^\{\(1\)\}\_\{t\}\-\\mu\_\{t\}\)\-\\lambda\\eta\_\{t\}^\{2\}\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\(\\mu\-a\)\.We then multiply by the integrating factorΣt−1/2​exp⁡\(λ​Σ​Ψt\)\\Sigma\_\{t\}^\{\-1/2\}\\exp\(\\lambda\\Sigma\\Psi\_\{t\}\)to get

dd​t​\(Σt−1/2​exp⁡\(λ​Σ​Ψt\)​\(μt\(1\)−μt\)\)=−2​λ​Σ1/2​exp⁡\(λ​Σ​Ψt\)​dd​t​\(Ψt1/2\)​\(μ−a\)\.\\displaystyle\\frac\{d\}\{dt\}\\left\(\\Sigma\_\{t\}^\{\-1/2\}\\exp\(\\lambda\\Sigma\\Psi\_\{t\}\)\(\\mu^\{\(1\)\}\_\{t\}\-\\mu\_\{t\}\)\\right\)=\-2\\lambda\\Sigma^\{1/2\}\\exp\(\\lambda\\Sigma\\Psi\_\{t\}\)\\,\\frac\{d\}\{dt\}\(\\Psi\_\{t\}^\{1/2\}\)\(\\mu\-a\)\.Integrating both sides from 0 to 1 then gives us

Σ−1/2​exp⁡\(λ​Σ\)​\(μ\(1\)−μ\)=−2​λ​Σ1/2​∫01exp⁡\(λ​Σ​Ψt\)​dd​t​\(Ψt1/2\)​𝑑t​\(μ−a\)\.\\displaystyle\\Sigma^\{\-1/2\}\\exp\(\\lambda\\Sigma\)\(\\mu^\{\(1\)\}\-\\mu\)=\-2\\lambda\\Sigma^\{1/2\}\\int\_\{0\}^\{1\}\\exp\(\\lambda\\Sigma\\Psi\_\{t\}\)\\,\\frac\{d\}\{dt\}\(\\Psi\_\{t\}^\{1/2\}\)\\,dt\\,\(\\mu\-a\)\.Making the substitutionMt=\(λ​Σ\)1/2​Ψt1/2M\_\{t\}=\(\\lambda\\Sigma\)^\{1/2\}\\Psi\_\{t\}^\{1/2\}and using the fundamental theorem of calculus gives us a formula for the terminal mean:

μ\(1\)=μ−π​\(λ​Σ\)1/2​exp⁡\(−λ​Σ\)​\(erfi⁡\(\(λ​Σ\)1/2\)−erfi⁡\(\(λ​Σ\)1/2​Ψ1/2\)\)​\(μ−a\)\.\\displaystyle\\mu^\{\(1\)\}=\\mu\-\\sqrt\{\\pi\}\\,\(\\lambda\\Sigma\)^\{1/2\}\\exp\(\-\\lambda\\Sigma\)\\left\(\\operatorname\{erfi\}\\\!\\left\(\(\\lambda\\Sigma\)^\{1/2\}\\right\)\-\\operatorname\{erfi\}\\\!\\left\(\(\\lambda\\Sigma\)^\{1/2\}\\Psi^\{1/2\}\\right\)\\right\)\(\\mu\-a\)\.Next, we have the linear ODE

d​\(x~t−μt\(1\)\)=\(12​Σ˙t​Σt−1−λ​ηt2​Σ​Σt−1​Ψt\)​\(x~t−μt\(1\)\)​d​t,\\displaystyle d\(\\tilde\{x\}\_\{t\}\-\\mu^\{\(1\)\}\_\{t\}\)=\\left\(\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{t\}\\Sigma\_\{t\}^\{\-1\}\-\\lambda\\eta\_\{t\}^\{2\}\\Sigma\\Sigma\_\{t\}^\{\-1\}\\Psi\_\{t\}\\right\)\(\\tilde\{x\}\_\{t\}\-\\mu^\{\(1\)\}\_\{t\}\)\\,dt,so by the chain rule, the covariance evolves according to the equation

d​Σt\(1\)=\(Σ˙t​Σt−1−2​λ​ηt2​Σ​Σt−1​Ψt\)​Σt\(1\)​d​t=\(Σ˙t​Σt−1−2​λ​Σ​Ψ˙t\)​Σt\(1\)​d​t\.\\displaystyle d\\Sigma^\{\(1\)\}\_\{t\}=\(\\dot\{\\Sigma\}\_\{t\}\\Sigma\_\{t\}^\{\-1\}\-2\\lambda\\eta\_\{t\}^\{2\}\\Sigma\\Sigma\_\{t\}^\{\-1\}\\Psi\_\{t\}\)\\,\\Sigma^\{\(1\)\}\_\{t\}\\,dt=\(\\dot\{\\Sigma\}\_\{t\}\\Sigma\_\{t\}^\{\-1\}\-2\\lambda\\Sigma\\dot\{\\Psi\}\_\{t\}\)\\,\\Sigma^\{\(1\)\}\_\{t\}\\,dt\.Because all relevant matrices commute, we can solve this ODE in closed form att=1t=1:

Σ\(1\)=exp⁡\(∫01\(Σ˙v​Σv−1−2​λ​Σ​Ψ˙v\)​𝑑v\)​Σ0\(1\)=Σ​exp⁡\(−2​λ​Σ​\(Id−Ψ\)\)\.\\displaystyle\\Sigma^\{\(1\)\}=\\exp\\\!\\left\(\\int\_\{0\}^\{1\}\(\\dot\{\\Sigma\}\_\{v\}\\Sigma\_\{v\}^\{\-1\}\-2\\lambda\\Sigma\\dot\{\\Psi\}\_\{v\}\)\\,dv\\right\)\\,\\Sigma^\{\(1\)\}\_\{0\}=\\Sigma\\exp\(\-2\\lambda\\Sigma\(I\_\{d\}\-\\Psi\)\)\.∎

### Proof of[Theorem˜2](https://arxiv.org/html/2606.02884#Thmtheorem2)

See[2](https://arxiv.org/html/2606.02884#Thmtheorem2)

###### Proof\.

Under the memoryless schedule,Ψ=0\\Psi=0andΣ1/2​Σt−1/2​Ψt1/2=t​Σ​Σt−1\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}=t\\Sigma\\Sigma\_\{t\}^\{\-1\}, so substituting into the formulas of[Theorem˜15](https://arxiv.org/html/2606.02884#Thmtheorem15)and usingerfi⁡\(0\)=0\\operatorname\{erfi\}\(0\)=0yields the stated guidance term and terminal mean and covariance\. ∎

### Plug\-in guidance is aggressive in the tails

The following uniform bound on thekk\-particle plug\-in drift underlies the∞\\infty\-Wasserstein bound of[Theorem˜3](https://arxiv.org/html/2606.02884#Thmtheorem3)\.

###### Lemma 16\(Plug\-in guidance is aggressive in the tails\)\.

In the setting of[Theorem˜2](https://arxiv.org/html/2606.02884#Thmtheorem2), consider the plug\-in flow \([6](https://arxiv.org/html/2606.02884#S4.E6)\) withkkparticles, whereut\(k\)​\(x\)=𝔼​\[∇log⁡h^t\(k\)​\(Xt\)\|Xt=x\]u\_\{t\}^\{\(k\)\}\(x\)=\\mathbb\{E\}\[\\nabla\\log\\hat\{h\}^\{\(k\)\}\_\{t\}\(X\_\{t\}\)\\,\|\\,X\_\{t\}=x\],h^t\(k\)​\(x\)=1k​∑i=1keλ​r​\(X1\(i\)​\(x\)\)\\hat\{h\}^\{\(k\)\}\_\{t\}\(x\)=\\frac\{1\}\{k\}\\sum\_\{i=1\}^\{k\}e^\{\\lambda r\(X\_\{1\}^\{\(i\)\}\(x\)\)\}, andX1\(1\)​\(x\),…,X1\(k\)​\(x\)∼law​\(X1\|Xt=x\)X\_\{1\}^\{\(1\)\}\(x\),\\ldots,X\_\{1\}^\{\(k\)\}\(x\)\\sim\\mathrm\{law\}\(X\_\{1\}\\,\|\\,X\_\{t\}=x\)are drawn independently and independent from everything else\. Then, we have the uniform bounds

supx∈ℝd∥ut\(k\)​\(x\)−ut\(1\)​\(x\)∥2≤Mt​\(k\)≔2​λ​‖Σ1/2​Σt−1/2​Ψt1/2​Σ1\|t1/2‖2​2​log⁡k\\displaystyle\\sup\_\{x\\in\\mathbb\{R\}^\{d\}\}\\;\\lVert u\_\{t\}^\{\(k\)\}\(x\)\-u\_\{t\}^\{\(1\)\}\(x\)\\rVert\_\{2\}\\leq M\_\{t\}\(k\)\\coloneq 2\\lambda\\,\\left\\lVert\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\\Sigma\_\{1\|t\}^\{1/2\}\\right\\rVert\_\{2\}\\sqrt\{2\\log k\}and

4​λ2​λmin​\(Σ1/2​Σt−1/2​Ψt1/2​Σ1\|t​\(Id\+2​λ​Σ1\|t\)−1\)​∥μ1\|t​\(x\)−a∥2−Mt​\(k\)\\displaystyle 4\\lambda^\{2\}\\,\\lambda\_\{\\min\}\\\!\\left\(\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\\Sigma\_\{1\|t\}\(I\_\{d\}\+2\\lambda\\Sigma\_\{1\|t\}\)^\{\-1\}\\right\)\\lVert\\mu\_\{1\|t\}\(x\)\-a\\rVert\_\{2\}\-M\_\{t\}\(k\)≤∥ut\(k\)​\(x\)−∇log⁡ht​\(x\)∥2\\displaystyle\\quad\\leq\\lVert u\_\{t\}^\{\(k\)\}\(x\)\-\\nabla\\log h\_\{t\}\(x\)\\rVert\_\{2\}≤4​λ2​λmax​\(Σ1/2​Σt−1/2​Ψt1/2​Σ1\|t​\(Id\+2​λ​Σ1\|t\)−1\)​∥μ1\|t​\(x\)−a∥2\+Mt​\(k\)\\displaystyle\\quad\\leq 4\\lambda^\{2\}\\,\\lambda\_\{\\max\}\\\!\\left\(\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\\Sigma\_\{1\|t\}\(I\_\{d\}\+2\\lambda\\Sigma\_\{1\|t\}\)^\{\-1\}\\right\)\\lVert\\mu\_\{1\|t\}\(x\)\-a\\rVert\_\{2\}\+M\_\{t\}\(k\)for allx∈ℝdx\\in\\mathbb\{R\}^\{d\}\.

The first inequality shows that thekk\-particle plug\-in flow is well\-approximated by thek=1k=1dynamics in the tails\. The second inequality shows that the error of the plug\-in drift grows linearly in∥x∥2\\lVert x\\rVert\_\{2\}, confirming that the plug\-in estimator is overly aggressive far from the reward center\.

###### Proof\.

We use the reparameterization trick to write

X1\(i\)​\(x\)=μ1\|t​\(x\)\+Σ1\|t1/2​ϵ\(i\)\\displaystyle X\_\{1\}^\{\(i\)\}\(x\)=\\mu\_\{1\|t\}\(x\)\+\\Sigma\_\{1\|t\}^\{1/2\}\\epsilon^\{\(i\)\}for i\.i\.d\.ϵ\(1\),…,ϵ\(k\)∼𝒩​\(0,Id\)\\epsilon^\{\(1\)\},\\ldots,\\epsilon^\{\(k\)\}\\sim\\mathcal\{N\}\(0,I\_\{d\}\)independent of everything else\. Recall from the proof of[Theorem˜15](https://arxiv.org/html/2606.02884#Thmtheorem15)that∇X1\(i\)​\(x\)=Σ1/2​Σt−1/2​Ψt1/2\\nabla X\_\{1\}^\{\(i\)\}\(x\)=\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}, so by the chain rule, we have

∇log⁡h^t\(k\)​\(x\)\\displaystyle\\nabla\\log\\hat\{h\}^\{\(k\)\}\_\{t\}\(x\)=∇log⁡\(1k​∑i=1ke−λ​∥X1\(i\)​\(x\)−a∥22\)\\displaystyle=\\nabla\\log\\left\(\\frac\{1\}\{k\}\\sum\_\{i=1\}^\{k\}e^\{\-\\lambda\\lVert X\_\{1\}^\{\(i\)\}\(x\)\-a\\rVert\_\{2\}^\{2\}\}\\right\)=−2​λ​Σ1/2​Σt−1/2​Ψt1/2​∑i=1ke−λ​∥X1\(i\)​\(x\)−a∥22​\(X1\(i\)​\(x\)−a\)∑i=1ke−λ​∥X1\(i\)​\(x\)−a∥22\\displaystyle=\-2\\lambda\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\\frac\{\\sum\_\{i=1\}^\{k\}e^\{\-\\lambda\\lVert X\_\{1\}^\{\(i\)\}\(x\)\-a\\rVert\_\{2\}^\{2\}\}\(X\_\{1\}^\{\(i\)\}\(x\)\-a\)\}\{\\sum\_\{i=1\}^\{k\}e^\{\-\\lambda\\lVert X\_\{1\}^\{\(i\)\}\(x\)\-a\\rVert\_\{2\}^\{2\}\}\}=−2​λ​Σ1/2​Σt−1/2​Ψt1/2​\(μ1\|t​\(x\)−a\)\\displaystyle=\-2\\lambda\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\(\\mu\_\{1\|t\}\(x\)\-a\)−2​λ​Σ1/2​Σt−1/2​Ψt1/2​Σ1\|t1/2​∑i=1ke−λ​∥X1\(i\)​\(x\)−a∥22​ϵ\(i\)∑i=1ke−λ​∥X1\(i\)​\(x\)−a∥22\.\\displaystyle\\quad\-2\\lambda\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\\Sigma\_\{1\|t\}^\{1/2\}\\,\\frac\{\\sum\_\{i=1\}^\{k\}e^\{\-\\lambda\\lVert X\_\{1\}^\{\(i\)\}\(x\)\-a\\rVert\_\{2\}^\{2\}\}\\,\\epsilon^\{\(i\)\}\}\{\\sum\_\{i=1\}^\{k\}e^\{\-\\lambda\\lVert X\_\{1\}^\{\(i\)\}\(x\)\-a\\rVert\_\{2\}^\{2\}\}\}\.Taking a conditional expectation and subtractingut\(1\)​\(x\)u\_\{t\}^\{\(1\)\}\(x\)from both sides, we have

ut\(k\)​\(x\)−ut\(1\)​\(x\)=−2​λ​Σ1/2​Σt−1/2​Ψt1/2​Σ1\|t1/2​𝔼​\[∑i=1ke−λ​∥X1\(i\)​\(x\)−a∥22​ϵ\(i\)∑i=1ke−λ​∥X1\(i\)​\(x\)−a∥22\]\.u\_\{t\}^\{\(k\)\}\(x\)\-u\_\{t\}^\{\(1\)\}\(x\)=\-2\\lambda\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\\Sigma\_\{1\|t\}^\{1/2\}\\,\\mathbb\{E\}\\\!\\left\[\\frac\{\\sum\_\{i=1\}^\{k\}e^\{\-\\lambda\\lVert X\_\{1\}^\{\(i\)\}\(x\)\-a\\rVert\_\{2\}^\{2\}\}\\,\\epsilon^\{\(i\)\}\}\{\\sum\_\{i=1\}^\{k\}e^\{\-\\lambda\\lVert X\_\{1\}^\{\(i\)\}\(x\)\-a\\rVert\_\{2\}^\{2\}\}\}\\right\]\.\(12\)For any unit vectorv∈ℝdv\\in\\mathbb\{R\}^\{d\}, the projections⟨v,ϵ\(i\)⟩\\langle v,\\epsilon^\{\(i\)\}\\rangleare i\.i\.d\.𝒩​\(0,1\)\\mathcal\{N\}\(0,1\), and since the softmax weights are nonnegative and sum to one,

‖𝔼​\[∑i=1ke−λ​∥X1\(i\)​\(x\)−a∥22​ϵ\(i\)∑i=1ke−λ​∥X1\(i\)​\(x\)−a∥22\]‖2\\displaystyle\\left\\lVert\\mathbb\{E\}\\\!\\left\[\\frac\{\\sum\_\{i=1\}^\{k\}e^\{\-\\lambda\\lVert X\_\{1\}^\{\(i\)\}\(x\)\-a\\rVert\_\{2\}^\{2\}\}\\,\\epsilon^\{\(i\)\}\}\{\\sum\_\{i=1\}^\{k\}e^\{\-\\lambda\\lVert X\_\{1\}^\{\(i\)\}\(x\)\-a\\rVert\_\{2\}^\{2\}\}\}\\right\]\\right\\rVert\_\{2\}=sup∥v∥2=1𝔼​\[⟨v,∑i=1ke−λ​∥X1\(i\)​\(x\)−a∥22​ϵ\(i\)∑i=1ke−λ​∥X1\(i\)​\(x\)−a∥22⟩\]\\displaystyle=\\sup\_\{\\lVert v\\rVert\_\{2\}=1\}\\;\\mathbb\{E\}\\\!\\left\[\\left\\langle v,\\,\\frac\{\\sum\_\{i=1\}^\{k\}e^\{\-\\lambda\\lVert X\_\{1\}^\{\(i\)\}\(x\)\-a\\rVert\_\{2\}^\{2\}\}\\,\\epsilon^\{\(i\)\}\}\{\\sum\_\{i=1\}^\{k\}e^\{\-\\lambda\\lVert X\_\{1\}^\{\(i\)\}\(x\)\-a\\rVert\_\{2\}^\{2\}\}\}\\right\\rangle\\right\]≤sup∥v∥2=1𝔼​\[max1≤i≤k⁡⟨v,ϵ\(i\)⟩\]\\displaystyle\\leq\\sup\_\{\\lVert v\\rVert\_\{2\}=1\}\\;\\mathbb\{E\}\\\!\\left\[\\max\_\{1\\leq i\\leq k\}\\,\\langle v,\\epsilon^\{\(i\)\}\\rangle\\right\]≤2​log⁡k\\displaystyle\\leq\\sqrt\{2\\log k\}by the standard sub\-Gaussian maximal inequality \(e\.g\.,\[[48](https://arxiv.org/html/2606.02884#bib.bib48), Theorem 2\.5\]\)\. Applying a spectral\-norm bound to the matrix factor gives the first bound\. For the second inequality, note that

ut\(k\)​\(x\)−∇log⁡ht​\(x\)=\(ut\(1\)​\(x\)−∇log⁡ht​\(x\)\)\+\(ut\(k\)​\(x\)−ut\(1\)​\(x\)\)=\(−4​λ2​Σ1/2​Σt−1/2​Ψt1/2​Σ1\|t​\(Id\+2​λ​Σ1\|t\)−1​\(μ1\|t​\(x\)−a\)\)\+\(ut\(k\)​\(x\)−ut\(1\)​\(x\)\),\\begin\{split\}&u\_\{t\}^\{\(k\)\}\(x\)\-\\nabla\\log h\_\{t\}\(x\)\\\\ &\\quad=\(u\_\{t\}^\{\(1\)\}\(x\)\-\\nabla\\log h\_\{t\}\(x\)\)\+\(u\_\{t\}^\{\(k\)\}\(x\)\-u\_\{t\}^\{\(1\)\}\(x\)\)\\\\ &\\quad=\\left\(\-4\\lambda^\{2\}\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\\Sigma\_\{1\|t\}\(I\_\{d\}\+2\\lambda\\Sigma\_\{1\|t\}\)^\{\-1\}\(\\mu\_\{1\|t\}\(x\)\-a\)\\right\)\+\(u\_\{t\}^\{\(k\)\}\(x\)\-u\_\{t\}^\{\(1\)\}\(x\)\),\\end\{split\}\(13\)so by the reverse triangle inequality and the first part of the theorem, we have

∥ut\(k\)​\(x\)−∇log⁡ht​\(x\)∥2\\displaystyle\\lVert u\_\{t\}^\{\(k\)\}\(x\)\-\\nabla\\log h\_\{t\}\(x\)\\rVert\_\{2\}≥‖4​λ2​Σ1/2​Σt−1/2​Ψt1/2​Σ1\|t​\(Id\+2​λ​Σ1\|t\)−1​\(μ1\|t​\(x\)−a\)‖2\\displaystyle\\quad\\geq\\left\\lVert 4\\lambda^\{2\}\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\\Sigma\_\{1\|t\}\(I\_\{d\}\+2\\lambda\\Sigma\_\{1\|t\}\)^\{\-1\}\(\\mu\_\{1\|t\}\(x\)\-a\)\\right\\rVert\_\{2\}−∥ut\(k\)​\(x\)−ut\(1\)​\(x\)∥2\\displaystyle\\quad\\quad\-\\lVert u\_\{t\}^\{\(k\)\}\(x\)\-u\_\{t\}^\{\(1\)\}\(x\)\\rVert\_\{2\}≥4​λ2​λmin​\(Σ1/2​Σt−1/2​Ψt1/2​Σ1\|t​\(Id\+2​λ​Σ1\|t\)−1\)​∥μ1\|t​\(x\)−a∥2−Mt​\(k\)\.\\displaystyle\\quad\\geq 4\\lambda^\{2\}\\,\\lambda\_\{\\min\}\\\!\\left\(\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\\Psi\_\{t\}^\{1/2\}\\Sigma\_\{1\|t\}\(I\_\{d\}\+2\\lambda\\Sigma\_\{1\|t\}\)^\{\-1\}\\right\)\\lVert\\mu\_\{1\|t\}\(x\)\-a\\rVert\_\{2\}\-M\_\{t\}\(k\)\.Similarly, the upper bound follows by applying the triangle inequality to \([13](https://arxiv.org/html/2606.02884#A3.E13)\) and using the first part of the theorem again\. ∎

### Proof of[Theorem˜3](https://arxiv.org/html/2606.02884#Thmtheorem3)

See[3](https://arxiv.org/html/2606.02884#Thmtheorem3)

###### Proof\.

LetXt\(k\)X\_\{t\}^\{\(k\)\}denote the solution to the plug\-in flow ODE withkkparticles and note that

Xt\(k\)−Xt\(1\)=∫0t\(bs\(k\)​\(Xs\(k\)\)−bs\(1\)​\(Xs\(1\)\)\)​𝑑s,\\displaystyle X\_\{t\}^\{\(k\)\}\-X\_\{t\}^\{\(1\)\}=\\int\_\{0\}^\{t\}\\left\(b\_\{s\}^\{\(k\)\}\(X\_\{s\}^\{\(k\)\}\)\-b\_\{s\}^\{\(1\)\}\(X\_\{s\}^\{\(1\)\}\)\\right\)ds,which implies that

∥Xt\(k\)−Xt\(1\)∥2\\displaystyle\\lVert X\_\{t\}^\{\(k\)\}\-X\_\{t\}^\{\(1\)\}\\rVert\_\{2\}≤∫0t‖bs\(k\)​\(Xs\(k\)\)−bs\(1\)​\(Xs\(1\)\)‖2​𝑑s\\displaystyle\\leq\\int\_\{0\}^\{t\}\\left\\lVert b\_\{s\}^\{\(k\)\}\(X\_\{s\}^\{\(k\)\}\)\-b\_\{s\}^\{\(1\)\}\(X\_\{s\}^\{\(1\)\}\)\\right\\rVert\_\{2\}\\,ds≤∫0t\(‖bs\(k\)​\(Xs\(k\)\)−bs\(1\)​\(Xs\(k\)\)‖2\+‖bs\(1\)​\(Xs\(k\)\)−bs\(1\)​\(Xs\(1\)\)‖2\)​𝑑s\.\\displaystyle\\leq\\int\_\{0\}^\{t\}\\left\(\\left\\lVert b\_\{s\}^\{\(k\)\}\(X\_\{s\}^\{\(k\)\}\)\-b\_\{s\}^\{\(1\)\}\(X\_\{s\}^\{\(k\)\}\)\\right\\rVert\_\{2\}\+\\left\\lVert b\_\{s\}^\{\(1\)\}\(X\_\{s\}^\{\(k\)\}\)\-b\_\{s\}^\{\(1\)\}\(X\_\{s\}^\{\(1\)\}\)\\right\\rVert\_\{2\}\\right\)ds\.The first term in the integrand is bounded above by12​ηs2​Ms​\(k\)\\frac\{1\}\{2\}\\eta\_\{s\}^\{2\}\\,M\_\{s\}\(k\)by[Lemma˜16](https://arxiv.org/html/2606.02884#Thmtheorem16)\. For the second term, note thatbs\(1\)b\_\{s\}^\{\(1\)\}is affine, so it is Lipschitz with constant

∇bs\(1\)​\(x\)=12​Σ˙s​Σs−1−λ​ηs2​Σ​Σs−1​Ψs\.\\displaystyle\\nabla b\_\{s\}^\{\(1\)\}\(x\)=\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{s\}\\Sigma\_\{s\}^\{\-1\}\-\\lambda\\eta\_\{s\}^\{2\}\\,\\Sigma\\Sigma\_\{s\}^\{\-1\}\\Psi\_\{s\}\.Combining these observations gives us

∥Xt\(k\)−Xt\(1\)∥2≤∫01\(12​ηs2​Ms​\(k\)\+‖12​Σ˙s​Σs−1−λ​ηs2​Σ​Σs−1​Ψs‖2​∥Xs\(k\)−Xs\(1\)∥2\)​𝑑s\.\\displaystyle\\lVert X\_\{t\}^\{\(k\)\}\-X\_\{t\}^\{\(1\)\}\\rVert\_\{2\}\\leq\\int\_\{0\}^\{1\}\\left\(\\frac\{1\}\{2\}\\eta\_\{s\}^\{2\}\\,M\_\{s\}\(k\)\+\\left\\lVert\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{s\}\\Sigma\_\{s\}^\{\-1\}\-\\lambda\\eta\_\{s\}^\{2\}\\,\\Sigma\\Sigma\_\{s\}^\{\-1\}\\Psi\_\{s\}\\right\\rVert\_\{2\}\\,\\lVert X\_\{s\}^\{\(k\)\}\-X\_\{s\}^\{\(1\)\}\\rVert\_\{2\}\\right\)ds\.Applying Grönwall’s inequality then gives us

∥X1\(k\)−X1\(1\)∥2\\displaystyle\\lVert X\_\{1\}^\{\(k\)\}\-X\_\{1\}^\{\(1\)\}\\rVert\_\{2\}≤∫0112​ηs2​Ms​\(k\)​exp⁡\(∫s1‖12​Σ˙v​Σv−1−λ​ηv2​Σ​Σv−1​Ψv‖2​𝑑v\)​𝑑s\\displaystyle\\quad\\leq\\int\_\{0\}^\{1\}\\frac\{1\}\{2\}\\eta\_\{s\}^\{2\}\\,M\_\{s\}\(k\)\\exp\\\!\\left\(\\int\_\{s\}^\{1\}\\left\\lVert\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{v\}\\Sigma\_\{v\}^\{\-1\}\-\\lambda\\eta\_\{v\}^\{2\}\\,\\Sigma\\Sigma\_\{v\}^\{\-1\}\\Psi\_\{v\}\\right\\rVert\_\{2\}\\,dv\\right\)ds=λ​2​log⁡k​∫01ηs2​‖Σ1/2​Σs−1/2​Ψs1/2​Σ1\|s1/2‖2​exp⁡\(∫s1‖12​Σ˙v​Σv−1−λ​ηv2​Σ​Σv−1​Ψv‖2​𝑑v\)​𝑑s\.\\displaystyle\\quad=\\lambda\\sqrt\{2\\log k\}\\int\_\{0\}^\{1\}\\eta\_\{s\}^\{2\}\\,\\left\\lVert\\Sigma^\{1/2\}\\Sigma\_\{s\}^\{\-1/2\}\\Psi\_\{s\}^\{1/2\}\\Sigma\_\{1\|s\}^\{1/2\}\\right\\rVert\_\{2\}\\exp\\\!\\left\(\\int\_\{s\}^\{1\}\\left\\lVert\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{v\}\\Sigma\_\{v\}^\{\-1\}\-\\lambda\\eta\_\{v\}^\{2\}\\,\\Sigma\\Sigma\_\{v\}^\{\-1\}\\Psi\_\{v\}\\right\\rVert\_\{2\}\\,dv\\right\)ds\.Lastly, we can choose the couplingπ=law​\(X1\(k\),X1\(1\)\)\\pi=\\mathrm\{law\}\(X\_\{1\}^\{\(k\)\},X\_\{1\}^\{\(1\)\}\)to conclude the desired bound\. ∎

### Proof of[Theorem˜17](https://arxiv.org/html/2606.02884#Thmtheorem17)

###### Theorem 17\(Analytic guidance for Gaussian mixture target\)\.

The overall guidance term is given by

∇log⁡ht​\(x\)=∑i=1mwi​t​\(x\)​\(∇log⁡ρi​t​\(x\)\+∇log⁡hi​t​\(x\)\)−∇log⁡ρt​\(x\)\\displaystyle\\nabla\\log h\_\{t\}\(x\)=\\sum\_\{i=1\}^\{m\}w\_\{it\}\(x\)\\,\(\\nabla\\log\\rho\_\{it\}\(x\)\+\\nabla\\log h\_\{it\}\(x\)\)\-\\nabla\\log\\rho\_\{t\}\(x\)where

wi​t​\(x\)≔πi​ρi​t​\(x\)​hi​t​\(x\)∑j=1mπj​ρj​t​\(x\)​hj​t​\(x\)\.\\displaystyle w\_\{it\}\(x\)\\coloneq\\frac\{\\pi\_\{i\}\\,\\rho\_\{it\}\(x\)\\,h\_\{it\}\(x\)\}\{\\sum\_\{j=1\}^\{m\}\\pi\_\{j\}\\,\\rho\_\{jt\}\(x\)\\,h\_\{jt\}\(x\)\}\.

###### Proof\.

Introducing a latent mixture indexZ∈\[m\]Z\\in\[m\]and applying the law of total expectation:

ht​\(x\)=∑i=1mℙ​\(Z=i\|Xt=x\)​𝔼​\[eλ​r​\(X1\)\|Xt=x,Z=i\]=1ρt​\(x\)​∑i=1mπi​ρi​t​\(x\)​hi​t​\(x\)\.\\displaystyle h\_\{t\}\(x\)=\\sum\_\{i=1\}^\{m\}\\mathbb\{P\}\(Z=i\\,\|\\,X\_\{t\}=x\)\\,\\mathbb\{E\}\[e^\{\\lambda r\(X\_\{1\}\)\}\\,\|\\,X\_\{t\}=x,Z=i\]=\\frac\{1\}\{\\rho\_\{t\}\(x\)\}\\sum\_\{i=1\}^\{m\}\\pi\_\{i\}\\,\\rho\_\{it\}\(x\)\\,h\_\{it\}\(x\)\.Taking∇log\\nabla\\logand applying the product rule yields the result\. ∎

### Proof of[Theorem˜5](https://arxiv.org/html/2606.02884#Thmtheorem5)

See[5](https://arxiv.org/html/2606.02884#Thmtheorem5)

###### Proof\.

The guidance formula \([8](https://arxiv.org/html/2606.02884#S4.E8)\) follows directly from the reparameterization trick and the chain rule\. To compute the Jacobian∇X¯1​\(x\)\\nabla\\bar\{X\}\_\{1\}\(x\), we differentiate the GLASS ODE with respect toxx\. Using

∇Xs∗​\(x\)=1ηs∗​\(1\+ηs∗\)​\(t\(1−t\)2​Id\+s\(1−s\)2​∇X¯s​\(x\)\),\\displaystyle\\nabla X^\{\*\}\_\{s\}\(x\)=\\frac\{1\}\{\\eta^\{\*\}\_\{s\}\(1\+\\eta^\{\*\}\_\{s\}\)\}\\left\(\\frac\{t\}\{\(1\-t\)^\{2\}\}I\_\{d\}\+\\frac\{s\}\{\(1\-s\)^\{2\}\}\\nabla\\bar\{X\}\_\{s\}\(x\)\\right\),differentiating under the integral sign yields the variational equation

d​\(∇X¯s​\(x\)\)\\displaystyle d\(\\nabla\\bar\{X\}\_\{s\}\(x\)\)=11−s​\(∇Dτs∗​\(Xs∗​\(x\)\)​\(1ηs∗​\(1\+ηs∗\)​\(t\(1−t\)2​Id\+s\(1−s\)2​∇X¯s​\(x\)\)\)−∇X¯s​\(x\)\)​d​s\\displaystyle=\\frac\{1\}\{1\-s\}\\left\(\\nabla D\_\{\\tau\_\{s\}^\{\*\}\}\(X^\{\*\}\_\{s\}\(x\)\)\\left\(\\frac\{1\}\{\\eta^\{\*\}\_\{s\}\(1\+\\eta^\{\*\}\_\{s\}\)\}\\left\(\\frac\{t\}\{\(1\-t\)^\{2\}\}I\_\{d\}\+\\frac\{s\}\{\(1\-s\)^\{2\}\}\\nabla\\bar\{X\}\_\{s\}\(x\)\\right\)\\right\)\-\\nabla\\bar\{X\}\_\{s\}\(x\)\\right\)dswith initial condition∇X¯0​\(x\)=0\\nabla\\bar\{X\}\_\{0\}\(x\)=0\. By Tweedie’s formula, the optimal denoiser is given by

Dt​\(x\)=xt\+\(1−t\)2t​∇log⁡ρt​\(x\),\\displaystyle D\_\{t\}\(x\)=\\frac\{x\}\{t\}\+\\frac\{\(1\-t\)^\{2\}\}\{t\}\\nabla\\log\\rho\_\{t\}\(x\),so taking the gradient of both sides gives us

∇Dt​\(x\)=1t​Id\+\(1−t\)2t​∇2log⁡ρt​\(x\)\.\\displaystyle\\nabla D\_\{t\}\(x\)=\\frac\{1\}\{t\}I\_\{d\}\+\\frac\{\(1\-t\)^\{2\}\}\{t\}\\nabla^\{2\}\\log\\rho\_\{t\}\(x\)\.Sinceρt=∑i=1mπi​ρi​t\\rho\_\{t\}=\\sum\_\{i=1\}^\{m\}\\pi\_\{i\}\\,\\rho\_\{it\}, applying the chain rule to compute the Hessian oflog⁡ρt\\log\\rho\_\{t\}yields

∇Dt​\(x\)\\displaystyle\\nabla D\_\{t\}\(x\)=1t​Id\+\(1−t\)2t​∑i=1mwi​t​\(x\)​\(\(∇log⁡ρi​t​\(x\)−∇log⁡ρt​\(x\)\)​\(∇log⁡ρi​t​\(x\)−∇log⁡ρt​\(x\)\)⊤\+∇2log⁡ρi​t​\(x\)\),\\displaystyle\\quad=\\frac\{1\}\{t\}I\_\{d\}\+\\frac\{\(1\-t\)^\{2\}\}\{t\}\\sum\_\{i=1\}^\{m\}w\_\{it\}\(x\)\\left\(\(\\nabla\\log\\rho\_\{it\}\(x\)\-\\nabla\\log\\rho\_\{t\}\(x\)\)\(\\nabla\\log\\rho\_\{it\}\(x\)\-\\nabla\\log\\rho\_\{t\}\(x\)\)^\{\\top\}\+\\nabla^\{2\}\\log\\rho\_\{it\}\(x\)\\right\),wherewi​t​\(x\)≔πi​ρi​t​\(x\)/∑j=1mπj​ρj​t​\(x\)w\_\{it\}\(x\)\\coloneq\\pi\_\{i\}\\,\\rho\_\{it\}\(x\)/\\sum\_\{j=1\}^\{m\}\\pi\_\{j\}\\,\\rho\_\{jt\}\(x\)\. ∎

### Proof of[Theorem˜6](https://arxiv.org/html/2606.02884#Thmtheorem6)

See[6](https://arxiv.org/html/2606.02884#Thmtheorem6)

###### Proof\.

For \(i\), we compute that

Z−≔∫−∞0eλ​r​\(x\)​ρ1​\(x\)​𝑑x=e−λ​R​∫−∞0ρ1​\(x\)​𝑑x=e−λ​R2\.\\displaystyle Z\_\{\-\}\\coloneq\\int\_\{\-\\infty\}^\{0\}e^\{\\lambda r\(x\)\}\\rho\_\{1\}\(x\)\\,dx=e^\{\-\\lambda R\}\\int\_\{\-\\infty\}^\{0\}\\rho\_\{1\}\(x\)\\,dx=\\frac\{e^\{\-\\lambda R\}\}\{2\}\.On the other hand, we have

Z\+≔∫0∞eλ​r​\(x\)​ρ1​\(x\)​𝑑x=∫0∞ρ1​\(x\)​𝑑x=12\.\\displaystyle Z\_\{\+\}\\coloneq\\int\_\{0\}^\{\\infty\}e^\{\\lambda r\(x\)\}\\rho\_\{1\}\(x\)\\,dx=\\int\_\{0\}^\{\\infty\}\\rho\_\{1\}\(x\)\\,dx=\\frac\{1\}\{2\}\.Therefore, the correct mode probability under the tilted measure is

p~=ℙ​\(X≥0\)=Z\+Z\+\+Z−=\(1\+e−λ​R\)−1\.\\displaystyle\\tilde\{p\}=\\mathbb\{P\}\(X\\geq 0\)=\\frac\{Z\_\{\+\}\}\{Z\_\{\+\}\+Z\_\{\-\}\}=\(1\+e^\{\-\\lambda R\}\)^\{\-1\}\.For \(ii\), note that the gradient of the reward is zero Lebesgue\-almost everywhere, so by the chain rule thekk\-particle plug\-in gradient

∇log⁡h^t\(k\)​\(x\)=∑i=1kwi​λ​∇X1\(i\)​\(x\)⊤​∇r​\(X1\(i\)​\(x\)\),wi≔eλ​r​\(X1\(i\)​\(x\)\)∑j=1keλ​r​\(X1\(j\)​\(x\)\),\\displaystyle\\nabla\\log\\hat\{h\}\_\{t\}^\{\(k\)\}\(x\)=\\sum\_\{i=1\}^\{k\}w\_\{i\}\\,\\lambda\\,\\nabla X\_\{1\}^\{\(i\)\}\(x\)^\{\\top\}\\nabla r\(X\_\{1\}^\{\(i\)\}\(x\)\),\\qquad w\_\{i\}\\coloneq\\frac\{e^\{\\lambda r\(X\_\{1\}^\{\(i\)\}\(x\)\)\}\}\{\\sum\_\{j=1\}^\{k\}e^\{\\lambda r\(X\_\{1\}^\{\(j\)\}\(x\)\)\}\},is zero almost everywhere for any finitek≥1k\\geq 1, since the law of\(X1\|Xt=x\)\(X\_\{1\}\\,\|\\,X\_\{t\}=x\)is absolutely continuous with respect to Lebesgue measure\. Therefore, definingσt2≔\(1−t\)2\+t2​σ2\\sigma\_\{t\}^\{2\}\\coloneq\(1\-t\)^\{2\}\+t^\{2\}\\sigma^\{2\}, thekk\-particle plug\-in flow coincides with the unguided flow:

bt​\(x\)=\(t​σ2−\(1−t\)σt2\)​x\+\(1−t\)​μσt2​tanh⁡\(t​μ​xσt2\)\.\\displaystyle b\_\{t\}\(x\)=\\left\(\\frac\{t\\sigma^\{2\}\-\(1\-t\)\}\{\\sigma\_\{t\}^\{2\}\}\\right\)x\+\\frac\{\(1\-t\)\\mu\}\{\\sigma\_\{t\}^\{2\}\}\\tanh\\\!\\left\(\\frac\{t\\mu x\}\{\\sigma\_\{t\}^\{2\}\}\\right\)\.Since the drift is an odd function, the distribution of the final sample is symmetric about zero, so the correct mode probability isp~1\(k\)=12\\tilde\{p\}\_\{1\}^\{\(k\)\}=\\frac\{1\}\{2\}\. For \(iii\), note that thennindependent finite\-kkplug\-in trajectories are i\.i\.d\., so the probability that allnntrajectories end up negative is\(1/2\)n\(1/2\)^\{n\}\. ∎

## Appendix DExperimental details and ablations

### Damping for Gaussian target

We use a single Gaussian targetρ1=𝒩​\(0,0\.5​I2\)\\rho\_\{1\}=\\mathcal\{N\}\(0,0\.5I\_\{2\}\)with quadratic rewardr​\(x\)=−∥x−a∥22r\(x\)=\-\\lVert x\-a\\rVert\_\{2\}^\{2\}centered ata=\(0,2\.5\)⊤a=\(0,2\.5\)^\{\\top\}andλ=3\.0\\lambda=3\.0\. By[Proposition˜10](https://arxiv.org/html/2606.02884#Thmtheorem10)and[Theorem˜2](https://arxiv.org/html/2606.02884#Thmtheorem2), the analytic guidance andk=1k=1plug\-in guidance both produce Gaussian terminal distributions with analytically known means and covariances\. We use a Heun integrator for the GLASS flow and draw 200 samples\.

### Gaussian\-mixture ablations: within\-mode reward hacking

We provide additional experiments verifying that the plug\-in estimator hacks the reward and that reward scale damping mitigates this in a wider variety of settings than the symmetric isotropic 2\-component mixture of[Figure˜3](https://arxiv.org/html/2606.02884#S4.F3)\. The protocol matches that of the main text: 200 samples drawn with the GLASS\-flow Heun integrator atλ=3\.0\\lambda=3\.0, background colormap shows the analytic tiltρ~1\\tilde\{\\rho\}\_\{1\}together with green level sets, and the red×\\timesmarks the reward targetaa\.

##### Single Gaussian\.

[Figure˜15](https://arxiv.org/html/2606.02884#A4.F15)verifies the within\-mode reward hacking and damping correction predicted by[Theorem˜2](https://arxiv.org/html/2606.02884#Thmtheorem2)on the simplest setting, a single isotropic Gaussian targetρ1=𝒩​\(\(0,0\)⊤,0\.5​I2\)\\rho\_\{1\}=\\mathcal\{N\}\(\(0,0\)^\{\\top\},\\,0\.5I\_\{2\}\)with quadratic rewardr​\(x\)=−∥x−a∥22r\(x\)=\-\\lVert x\-a\\rVert\_\{2\}^\{2\}ata=\(0,2\.5\)⊤a=\(0,2\.5\)^\{\\top\}\. Thek=1k=1plug\-in over\-concentrates exactly as predicted; increasingkkdoes not help; reward damping recovers the analytic tilt\.

![Refer to caption](https://arxiv.org/html/2606.02884v1/x13.png)Figure 15:Single Gaussian target\.Thek=1k=1plug\-in over\-concentrates exactly as predicted by[Theorem˜2](https://arxiv.org/html/2606.02884#Thmtheorem2); increasingkkdoes not help; reward damping recovers the true tilted distribution\.
##### Non\-quadratic reward\.

To verify that the plug\-in bias is not specific to the quadratic reward setting of our theory,[Figure˜16](https://arxiv.org/html/2606.02884#A4.F16)repeats the experiment with a double\-well rewardr​\(x\)=−0\.125​\(x14−12\.5​x12\)−\(x2−0\.4​x1\)2r\(x\)=\-0\.125\(x\_\{1\}^\{4\}\-12\.5x\_\{1\}^\{2\}\)\-\(x\_\{2\}\-0\.4x\_\{1\}\)^\{2\}, which has local maxima at\(−2\.5,−1\.0\)\(\-2\.5,\-1\.0\)and\(2\.5,1\.0\)\(2\.5,1\.0\)\. Although the double\-well reward has no closed\-form Doobhh\-transform, we approximate the exact reward\-tilt using ak=1000k=1000plug\-in that draws particles directly from this closed\-form posterior\. The qualitative pattern persists: samples spread across both reward peaks, thek=1k=1plug\-in estimator over\-concentrates near the maxima, and reward damping partially corrects this\.

![Refer to caption](https://arxiv.org/html/2606.02884v1/x14.png)Figure 16:Double\-well reward\.The plug\-in over\-concentrates at the reward maxima \(marked by red×\\times\); reward damping partially corrects this\.
##### GMM ablations\.

[Figures˜17](https://arxiv.org/html/2606.02884#A4.F17),[18](https://arxiv.org/html/2606.02884#A4.F18)and[19](https://arxiv.org/html/2606.02884#A4.F19)verify that the plug\-in bias and the effectiveness of reward damping persist across three further variations: \(i\)*non\-isotropic covariances*, where the component covariances are\[0\.5±0\.25±0\.250\.5\]\\bigl\[\\begin\{smallmatrix\}0\.5&\\pm 0\.25\\\\ \\pm 0\.25&0\.5\\end\{smallmatrix\}\\bigr\]\(tilted towardaa\); \(ii\)*unequal weights*\(π1,π2\)=\(0\.2,0\.8\)\(\\pi\_\{1\},\\pi\_\{2\}\)=\(0\.2,0\.8\); and \(iii\)*uncentered components*at\(−4\.0,0\)\(\-4\.0,0\)and\(1\.0,0\)\(1\.0,0\)\. In each case the plug\-in estimator collapses samples near the reward target while reward damping restores spread consistent with the tilted density\.

![Refer to caption](https://arxiv.org/html/2606.02884v1/x15.png)Figure 17:Non\-isotropic covariances\.Both components are tilted toward the targeta=\(0,2\.5\)⊤a=\(0,2\.5\)^\{\\top\}via off\-diagonal entries±0\.25\\pm 0\.25\.![Refer to caption](https://arxiv.org/html/2606.02884v1/x16.png)Figure 18:Unequal component weights\(π1,π2\)=\(0\.2,0\.8\)\(\\pi\_\{1\},\\pi\_\{2\}\)=\(0\.2,0\.8\)\.![Refer to caption](https://arxiv.org/html/2606.02884v1/x17.png)Figure 19:Uncentered componentsat\(−4\.0,0\)\(\-4\.0,0\)and\(1\.0,0\)\(1\.0,0\)\.

### Checkerboard mode selection: experimental details

In this section, we provide further details on the checkerboard mode\-selection experiment described in[Section˜5\.2\.1](https://arxiv.org/html/2606.02884#S5.SS2.SSS1)\.

##### Base model\.

The base velocity fieldbtb\_\{t\}is a 4\-layer MLP with 256 hidden units and GELU activations, taking the concatenated input\(x/s,t\)∈ℝ3\(x/s,t\)\\in\\mathbb\{R\}^\{3\}wheressis the empirical standard deviation of the checkerboard data\. We train it via flow matching against the linear interpolantIt=\(1−t\)​ϵ\+t​X1I\_\{t\}=\(1\-t\)\\,\\epsilon\+t\\,X\_\{1\}withϵ∼𝒩​\(0,I2\)\\epsilon\\sim\\mathcal\{N\}\(0,I\_\{2\}\)andX1X\_\{1\}drawn uniformly from the 18 filled checkerboard squares\. The model is optimized with Adam \(learning rate10−310^\{\-3\}, cosine annealing to0\) for5×1055\\times 10^\{5\}steps with batch size40964096, using an exponential moving average of the weights with decay0\.99990\.9999for inference\. Unguided samples are obtained by integrating the learned velocity ODE with a Heun integrator at200200steps, starting fromϵ∼𝒩​\(0,s2​I2\)\\epsilon\\sim\\mathcal\{N\}\(0,s^\{2\}I\_\{2\}\)\.

##### Guided sampling\.

We useλ=10\\lambda=10,k=1k=1particle by default, and the memoryless scheduleσt=2​\(1−t\)/t\\sigma\_\{t\}=\\sqrt\{2\(1\-t\)/t\}\. Guided trajectories are integrated with a Heun integrator at200200outer steps, and the GLASS lookahead\(X1\|Xt=x\)\(X\_\{1\}\\,\|\\,X\_\{t\}=x\)is computed with5050inner Euler steps\. The reward is the Gaussian bumpr​\(x\)=exp⁡\(−∥x−c∥22/\(2​σr2\)\)r\(x\)=\\exp\(\-\\lVert x\-c\\rVert\_\{2\}^\{2\}/\(2\\sigma\_\{r\}^\{2\}\)\)withc=\(0\.5,0\.5\)⊤c=\(0\.5,0\.5\)^\{\\top\}andσr=1\.5\\sigma\_\{r\}=1\.5\. We draw50005000samples per condition\.

##### Analytic samples\.

We obtain exact samples fromρ~1∝eλ​r​ρ1\\tilde\{\\rho\}\_\{1\}\\propto e^\{\\lambda r\}\\rho\_\{1\}via rejection sampling over the 18 filled squares: sample a square uniformly \(sinceρ1\\rho\_\{1\}is uniform on each square\), sample a point uniformly within that square, and accept with probabilityeλ​\(r​\(x\)−1\)e^\{\\lambda\(r\(x\)\-1\)\}\. This is valid becauser​\(x\)≤1r\(x\)\\leq 1everywhere \(the Gaussian bump peaks atr=1r=1at its center\), soeλe^\{\\lambda\}is a global upper bound oneλ​re^\{\\lambda r\}\. The worst\-case acceptance rate ise−λe^\{\-\\lambda\}, attained wherer​\(x\)→0r\(x\)\\to 0far fromcc; atccitself, wherer​\(c\)=1r\(c\)=1, the acceptance probability is 1\.

### FLUX\.1\-dev experimental details

##### Licenses\.

FLUX\.1\-dev is released under the FLUX\.1 \[dev\] Non\-Commercial License; Qwen2\.5\-VL\-3B\-Instruct is released under the Qwen Research License Agreement; ImageReward is released under the Apache\-2\.0 License\. All three checkpoints are used within the terms of their respective licenses for non\-commercial academic research\.

##### Cover photo\.

[Figure˜1](https://arxiv.org/html/2606.02884#S0.F1)compares unguided FLUX\.1 samples to guided and damped runs with ImageReward across six prompts\. We use the following parameters:

- •Guided baselines:λ=50\\lambda=50for all prompts\.
- •Damped runs:λ=100\\lambda=100withσdamp=0\.1\\sigma\_\{\\mathrm\{damp\}\}=0\.1for all prompts except the skyscraper prompt, which usesσdamp=0\.15\\sigma\_\{\\mathrm\{damp\}\}=0\.15\.

##### Main experiments\.

The within\-mode and mode\-selection experiments shown in[Section˜5](https://arxiv.org/html/2606.02884#S5)use the following parameters\.

- •[Figure˜5](https://arxiv.org/html/2606.02884#S5.F5)\(welder, masked\-brightness\): naiveλ=100\\lambda=100, lower\-λ\\lambdabaselineλ=50\\lambda=50,k=8k=8baseline atλ=50\\lambda=50, dampedλ=100\\lambda=100withσdamp=0\.1\\sigma\_\{\\mathrm\{damp\}\}=0\.1\.
- •[Figure˜6](https://arxiv.org/html/2606.02884#S5.F6)\(rococo, blueness\): naiveλ=50\\lambda=50, lower\-λ\\lambdabaselineλ=30\\lambda=30,k=8k=8baseline atλ=50\\lambda=50, dampedλ=100\\lambda=100withσdamp=0\.1\\sigma\_\{\\mathrm\{damp\}\}=0\.1\.
- •[Figure˜7](https://arxiv.org/html/2606.02884#S5.F7)\(fox, blueness\): naiveλ=100\\lambda=100, lower\-λ\\lambdabaselineλ=50\\lambda=50,k=8k=8baseline atλ=50\\lambda=50, dampedλ=100\\lambda=100withσdamp=0\.1\\sigma\_\{\\mathrm\{damp\}\}=0\.1\.
- •[Figure˜8](https://arxiv.org/html/2606.02884#S5.F8)\(archaeologist, ImageReward\): naiveλ=100\\lambda=100, lower\-λ\\lambdabaselineλ=50\\lambda=50,k=8k=8baseline atλ=50\\lambda=50, dampedλ=100\\lambda=100withσdamp=0\.15\\sigma\_\{\\mathrm\{damp\}\}=0\.15\.
- •[Figure˜10](https://arxiv.org/html/2606.02884#S5.F10)\(miner, ImageReward\): naiveλ=100\\lambda=100, lower\-λ\\lambdabaselineλ=50\\lambda=50,k=8k=8baseline atλ=50\\lambda=50, dampedλ=100\\lambda=100withσdamp=0\.1\\sigma\_\{\\mathrm\{damp\}\}=0\.1\.
- •[Figure˜9](https://arxiv.org/html/2606.02884#S5.F9)\(market, ImageReward\): naiveλ=50\\lambda=50, lower\-λ\\lambdabaselineλ=30\\lambda=30,k=8k=8baseline atλ=50\\lambda=50, dampedλ=100\\lambda=100withσdamp=0\.05\\sigma\_\{\\mathrm\{damp\}\}=0\.05\.
- •[Figure˜13](https://arxiv.org/html/2606.02884#S5.F13)\(ECLIPSE DINER, VLM\): naiveλ=100\\lambda=100,k=8k=8baseline atλ=50\\lambda=50, dampedλ=100\\lambda=100withσdamp=0\.1\\sigma\_\{\\mathrm\{damp\}\}=0\.1\. Best\-of\-nnselections taken from 20 i\.i\.d\. generations per row\.
- •[Figure˜14](https://arxiv.org/html/2606.02884#S5.F14)\(NEXT TRAIN MARS, VLM\): same parameters as[Figure˜13](https://arxiv.org/html/2606.02884#S5.F13)\.

##### Guidance\.

The plug\-in guidance gradient∇log⁡h^t\(k\)​\(x\)\\nabla\\log\\hat\{h\}\_\{t\}^\{\(k\)\}\(x\)is rescaled to unit norm before applying guidance; this is a standard approximation made for numerical stability\[[36](https://arxiv.org/html/2606.02884#bib.bib36),[39](https://arxiv.org/html/2606.02884#bib.bib39)\]\. We see empirically that this approximation corresponds to a slight damping effect, decreasing the effective guidance scaleλ\\lambdathroughout the trajectory\. FollowingHolderrieth et al\. \[[36](https://arxiv.org/html/2606.02884#bib.bib36)\], we apply guidance for55guidance steps starting from outer step11, skipping steps withσt\>0\.9\\sigma\_\{t\}\>0\.9to ensure numerical stability\. Reward damping \([Section˜4\.1\.2](https://arxiv.org/html/2606.02884#S4.SS1.SSS2)\) replacesλ\\lambdawith the time\-dependent schedule \([7](https://arxiv.org/html/2606.02884#S4.E7)\), parameterized byσdamp\\sigma\_\{\\mathrm\{damp\}\}\. Gradient checkpointing on the FLUX transformer is used to fit backpropagation in memory\. We generate44images per condition for the within\-mode reward hacking grids and2020images per condition for the mode\-selection best\-of\-nnexperiments\.

### Compute

All FLUX text\-to\-image experiments run on a single NVIDIA RTX A6000 \(48 GB\) or L40S \(48 GB\) GPU, with 8 CPUs and 64 GB RAM\. At 512 x 512 resolution with 28 inference steps, an unguided sample takes approximately 7\-15 seconds; ak=1k=1guided sample with 5 Diamond\-map guidance steps takes 11\-23 seconds; and ak=8k=8guided sample takes 37\-76 seconds\. A typical 20\-image condition therefore completes in 4\-8 minutes fork=1k=1and roughly 25 minutes fork=8k=8\. Including hyperparameter sweeps overλ\\lambda,σdamp\\sigma\_\{\\mathrm\{damp\}\}, gradient\-norm scale, prompt variants, and earlier lookahead schemes, the total FLUX compute is approximately 50\-55 GPU\-hours, of which roughly 10 GPU\-hours back the figures and tables in the paper\. Checkerboard base\-model training \(5×1055\\times 10^\{5\}steps of a 4\-layer MLP\) takes about 60 minutes on a single GPU, and the reported guided\-sampling grid adds another 1\-2 GPU\-hours\. The Gaussian\-mixture and 1D mode\-selection experiments are closed\-form analytics with small numerical integrators and complete in CPU\-minutes\.

### FLUX\.1 mode\-selection: additional experiments

[Table˜20](https://arxiv.org/html/2606.02884#A4.T20)reports the mean VLM probability of a legible “ECLIPSE” sign for the diner experiment of[Section˜5\.2\.2](https://arxiv.org/html/2606.02884#S5.SS2.SSS2)across best\-of\-nnvalues, complementing the qualitative results of[Figures˜13](https://arxiv.org/html/2606.02884#S5.F13)and[14](https://arxiv.org/html/2606.02884#S5.F14)\. We report VLM scores as probabilitiessigmoid​\(log⁡p​\(Yes\)−log⁡p​\(No\)\)\\mathrm\{sigmoid\}\(\\log p\(\\text\{Yes\}\)\-\\log p\(\\text\{No\}\)\)\.

Table 20:Mode selection on FLUX\.1 \(ECLIPSE DINER\)\.Mean VLM probability of a clear “ECLIPSE” sign with best\-of\-nnsampling \(higher is better\)\. Uncertainties are±2\\pm 2bootstrap standard errors over the 20 generations per condition\.
### Gaussian\-mixture mode\-selection experiments

We provide additional empirical results for the 1D symmetric Gaussian mixtureρ1=12​𝒩​\(μ,σ2\)\+12​𝒩​\(−μ,σ2\)\\rho\_\{1\}=\\tfrac\{1\}\{2\}\\mathcal\{N\}\(\\mu,\\sigma^\{2\}\)\+\\tfrac\{1\}\{2\}\\mathcal\{N\}\(\-\\mu,\\sigma^\{2\}\)withμ=5\\mu=5,σ=1\\sigma=1, andλ=5\.0\\lambda=5\.0, complementing the empirical illustration in[Figure˜4](https://arxiv.org/html/2606.02884#S4.F4)\.

##### Step function reward\.

With the step rewardr​\(x\)=𝟏x≥0r\(x\)=\\mathbf\{1\}\_\{x\\geq 0\}, the gradient is zero almost everywhere, so the plug\-in estimator \(run withk=1k=1\) receives no signal and achieves correct\-mode probabilityp~1\(1\)=0\.502\\tilde\{p\}\_\{1\}^\{\(1\)\}=0\.502, essentially random\. Best\-of\-nnsampling exactly recovers the predictionp~n\(1\)=1−\(1/2\)n\\tilde\{p\}\_\{n\}^\{\(1\)\}=1\-\(1/2\)^\{n\}from[Theorem˜6](https://arxiv.org/html/2606.02884#Thmtheorem6)\([Table˜21](https://arxiv.org/html/2606.02884#A4.T21)\)\.[Figure˜22](https://arxiv.org/html/2606.02884#A4.F22)shows the underlying particle trajectories: unguided and plug\-in trajectories split roughly evenly between the two modes, while best\-of\-nnselection concentrates on positive\-half trajectories\.

Table 21:Step\-reward best\-of\-nn\.Empiricalp~n\(1\)\\tilde\{p\}\_\{n\}^\{\(1\)\}for best\-of\-nnguided sampling withk=1k=1\. Theory:1−\(1/2\)n1\-\(1/2\)^\{n\}\. Uncertainties are±2\\pm 2binomial standard errors2​p^​\(1−p^\)/M2\\sqrt\{\\hat\{p\}\(1\-\\hat\{p\}\)/M\}overM=1,000M=1\{,\}000independent groups of3232trajectories\.![Refer to caption](https://arxiv.org/html/2606.02884v1/x18.png)Figure 22:Step reward trajectories\.Colored by terminal sign\. Unguided and plug\-in trajectories split evenly between positive \(green\) and negative \(coral\) modes; best\-of\-nnselection produces a strong majority of positive\-mode trajectories, increasingly so asnngrows\.
##### Gaussian reward\.

We repeat the same 1D mixture experiment with a Gaussian rewardr​\(x\)=exp⁡\(−\(x−3\)2/2\)r\(x\)=\\exp\(\-\(x\-3\)^\{2\}/2\)centered atx=3x=3in the positive half\.[Theorem˜6](https://arxiv.org/html/2606.02884#Thmtheorem6)does not apply here—the reward is smooth and the plug\-in estimator receives a nonzero gradient signal—so this case goes beyond what our theory covers\. Nevertheless, the qualitative behavior persists: plug\-in guidance skews trajectories toward the positive mode \([Figure˜23](https://arxiv.org/html/2606.02884#A4.F23), left\), and best\-of\-nnfurther concentrates them \([Figure˜23](https://arxiv.org/html/2606.02884#A4.F23), center and right\)\.

![Refer to caption](https://arxiv.org/html/2606.02884v1/x19.png)Figure 23:Gaussian reward trajectories\.Plug\-in already skews toward the positive mode; best\-of\-nnfurther concentrates it\.

## Appendix EComparison with flow map reward guidance \(FMRG\)

Huang et al\. \[[39](https://arxiv.org/html/2606.02884#bib.bib39)\]propose a deterministic guidance scheme derived directly from the probability flow ODE, without passing through a diffusion\. Given the unguided probability flowd​Xs=bs​\(Xs\)​d​sdX\_\{s\}=b\_\{s\}\(X\_\{s\}\)\\,dsand flow mapXt,s​\(x\)X\_\{t,s\}\(x\)denoting its solution at timessgivenXt=xX\_\{t\}=x, the*flow map reward guidance*for the rewardrris the feedback control

utFMRG​\(x\)≔λ​∇Xt,1​\(x\)⊤​∇r​\(Xt,1​\(x\)\),u\_\{t\}^\{\\mathrm\{FMRG\}\}\(x\)\\coloneq\\lambda\\,\\nabla X\_\{t,1\}\(x\)^\{\\top\}\\nabla r\(X\_\{t,1\}\(x\)\),and the guided trajectory solves

d​x~t=\(bt​\(x~t\)\+utFMRG​\(x~t\)\)​d​td\\tilde\{x\}\_\{t\}=\(b\_\{t\}\(\\tilde\{x\}\_\{t\}\)\+u\_\{t\}^\{\\mathrm\{FMRG\}\}\(\\tilde\{x\}\_\{t\}\)\)\\,dt\(14\)withx~0∼𝒩​\(0,Id\)\\tilde\{x\}\_\{0\}\\sim\\mathcal\{N\}\(0,I\_\{d\}\)\. For the quadratic rewardr​\(x\)=−∥x−a∥22r\(x\)=\-\\lVert x\-a\\rVert\_\{2\}^\{2\}, this simplifies to

utFMRG​\(x\)=−2​λ​∇Xt,1​\(x\)⊤​\(Xt,1​\(x\)−a\)\.u\_\{t\}^\{\\mathrm\{FMRG\}\}\(x\)=\-2\\lambda\\,\\nabla X\_\{t,1\}\(x\)^\{\\top\}\(X\_\{t,1\}\(x\)\-a\)\.\(15\)This is structurally similar to thek=1k=1plug\-in guidance of[Theorem˜2](https://arxiv.org/html/2606.02884#Thmtheorem2): both schemes are of the form−2​λ​\(Jacobian\)⊤​\(endpoint−a\)\-2\\lambda\\,\(\\text\{Jacobian\}\)^\{\\top\}\(\\text\{endpoint\}\-a\), and they differ only in which endpoint is used\. The plug\-in scheme evaluates at a random conditional sampleX1\(1\)​\(x\)∼\(X1\|Xt=x\)X\_\{1\}^\{\(1\)\}\(x\)\\sim\(X\_\{1\}\\,\|\\,X\_\{t\}=x\)and averages over the initial noise in the GLASS flow, while the flow map reward scheme evaluates at the deterministic probability flow endpointXt,1​\(x\)X\_\{t,1\}\(x\)\. Neither scheme targets the exact Doobhh\-functionht​\(x\)=𝔼​\[eλ​r​\(X1\)\|Xt=x\]h\_\{t\}\(x\)=\\mathbb\{E\}\[e^\{\\lambda r\(X\_\{1\}\)\}\\,\|\\,X\_\{t\}=x\], so both are single\-endpoint surrogates that do not capture the long\-range mode attraction of the exacthh\-transform\. In this appendix, we carry out the analysis ofHuang et al\. \[[39](https://arxiv.org/html/2606.02884#bib.bib39)\]inℝd\\mathbb\{R\}^\{d\}and extend it to Gaussian mixture targets, allowing us to compare the flow map reward guidance directly with the plug\-in formulae of[Theorems˜2](https://arxiv.org/html/2606.02884#Thmtheorem2)and[5](https://arxiv.org/html/2606.02884#Thmtheorem5)\.

### Gaussian target

Throughout this subsection,ρ1=𝒩​\(μ,Σ\)\\rho\_\{1\}=\\mathcal\{N\}\(\\mu,\\Sigma\)withΣ≻0\\Sigma\\succ 0, andΣt≔\(1−t\)2​Id\+t2​Σ\\Sigma\_\{t\}\\coloneq\(1\-t\)^\{2\}I\_\{d\}\+t^\{2\}\\Sigmadenotes the marginal covariance ofItI\_\{t\}as in[Theorem˜11](https://arxiv.org/html/2606.02884#Thmtheorem11)\. We begin by computing the flow map in closed form\.

###### Lemma 18\(Flow map for Gaussian target\)\.

Supposeρ1=𝒩​\(μ,Σ\)\\rho\_\{1\}=\\mathcal\{N\}\(\\mu,\\Sigma\)\. Then the probability flow ODEd​Xs=bs​\(Xs\)​d​sdX\_\{s\}=b\_\{s\}\(X\_\{s\}\)\\,dswithbs​\(x\)=μ\+12​Σ˙s​Σs−1​\(x−s​μ\)b\_\{s\}\(x\)=\\mu\+\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{s\}\\Sigma\_\{s\}^\{\-1\}\(x\-s\\mu\)has flow map

Xt,1​\(x\)=μ\+Σ1/2​Σt−1/2​\(x−t​μ\)\\displaystyle X\_\{t,1\}\(x\)=\\mu\+\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\(x\-t\\mu\)and state\-independent Jacobian∇Xt,1​\(x\)=Σ1/2​Σt−1/2\\nabla X\_\{t,1\}\(x\)=\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\.

###### Proof\.

Along a trajectory, defineYs≔Xs−s​μY\_\{s\}\\coloneq X\_\{s\}\-s\\muso thatd​Ys=12​Σ˙s​Σs−1​Ys​d​sdY\_\{s\}=\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{s\}\\Sigma\_\{s\}^\{\-1\}Y\_\{s\}\\,ds\. BecauseΣs\\Sigma\_\{s\}is a matrix polynomial inΣ\\Sigma, the family\{Σ˙s​Σs−1\}s\\\{\\dot\{\\Sigma\}\_\{s\}\\Sigma\_\{s\}^\{\-1\}\\\}\_\{s\}commutes with itself and withΣ\\Sigma, so integrating froms=ts=ttos=1s=1yields

Y1=exp⁡\(12​∫t1Σ˙s​Σs−1​𝑑s\)​Yt=exp⁡\(12​log⁡\(Σ​Σt−1\)\)​Yt=Σ1/2​Σt−1/2​Yt\.\\displaystyle Y\_\{1\}=\\exp\\\!\\left\(\\frac\{1\}\{2\}\\int\_\{t\}^\{1\}\\dot\{\\Sigma\}\_\{s\}\\Sigma\_\{s\}^\{\-1\}\\,ds\\right\)Y\_\{t\}=\\exp\\\!\\left\(\\frac\{1\}\{2\}\\log\(\\Sigma\\Sigma\_\{t\}^\{\-1\}\)\\right\)Y\_\{t\}=\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}Y\_\{t\}\.Addingμ\\muback gives the flow map, and the Jacobian follows by differentiating inxx\. ∎

###### Theorem 19\(Flow map reward guidance for Gaussian target\)\.

Supposeρ1=𝒩​\(μ,Σ\)\\rho\_\{1\}=\\mathcal\{N\}\(\\mu,\\Sigma\)and consider the flow map reward guided ODE \([14](https://arxiv.org/html/2606.02884#A5.E14)\) withutFMRGu\_\{t\}^\{\\mathrm\{FMRG\}\}given by \([15](https://arxiv.org/html/2606.02884#A5.E15)\)\. Then, the guidance term is

utFMRG​\(x\)=−2​λ​Σ1/2​Σt−1/2​\(Xt,1​\(x\)−a\)=−2​λ​Σ​Σt−1​\(x−t​μ\)−2​λ​Σ1/2​Σt−1/2​\(μ−a\),\\displaystyle u\_\{t\}^\{\\mathrm\{FMRG\}\}\(x\)=\-2\\lambda\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\(X\_\{t,1\}\(x\)\-a\)=\-2\\lambda\\,\\Sigma\\Sigma\_\{t\}^\{\-1\}\(x\-t\\mu\)\-2\\lambda\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\(\\mu\-a\),and the terminal distribution of the guided ODE isx~1∼𝒩​\(μ~FMRG,Σ~FMRG\)\\tilde\{x\}\_\{1\}\\sim\\mathcal\{N\}\(\\tilde\{\\mu\}^\{\\mathrm\{FMRG\}\},\\,\\tilde\{\\Sigma\}^\{\\mathrm\{FMRG\}\}\), where

μ~FMRG≔μ−TpullFMRG​\(μ−a\),TpullFMRG≔Id−exp⁡\(−π​λ​Σ1/2\),\\displaystyle\\tilde\{\\mu\}^\{\\mathrm\{FMRG\}\}\\coloneq\\mu\-T\_\{\\mathrm\{pull\}\}^\{\\mathrm\{FMRG\}\}\(\\mu\-a\),\\qquad T\_\{\\mathrm\{pull\}\}^\{\\mathrm\{FMRG\}\}\\coloneq I\_\{d\}\-\\exp\(\-\\pi\\lambda\\Sigma^\{1/2\}\),and

Σ~FMRG≔Σ​exp⁡\(−2​π​λ​Σ1/2\)\.\\displaystyle\\tilde\{\\Sigma\}^\{\\mathrm\{FMRG\}\}\\coloneq\\Sigma\\exp\(\-2\\pi\\lambda\\Sigma^\{1/2\}\)\.

###### Proof\.

We begin by computing the guidance termutFMRG​\(x\)u\_\{t\}^\{\\mathrm\{FMRG\}\}\(x\)\. From[Lemma˜18](https://arxiv.org/html/2606.02884#Thmtheorem18), the Jacobian∇Xt,1​\(x\)=Σ1/2​Σt−1/2\\nabla X\_\{t,1\}\(x\)=\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}is a matrix function ofΣ\\Sigmaand hence symmetric\. Substituting into \([15](https://arxiv.org/html/2606.02884#A5.E15)\) and expandingXt,1​\(x\)X\_\{t,1\}\(x\)using the flow map formula gives

utFMRG​\(x\)=−2​λ​Σ1/2​Σt−1/2​\(Xt,1​\(x\)−a\)=−2​λ​Σ​Σt−1​\(x−t​μ\)−2​λ​Σ1/2​Σt−1/2​\(μ−a\)\.\\displaystyle u\_\{t\}^\{\\mathrm\{FMRG\}\}\(x\)=\-2\\lambda\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\(X\_\{t,1\}\(x\)\-a\)=\-2\\lambda\\,\\Sigma\\Sigma\_\{t\}^\{\-1\}\(x\-t\\mu\)\-2\\lambda\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\(\\mu\-a\)\.Substituting back into the FMRG ODE \([14](https://arxiv.org/html/2606.02884#A5.E14)\), the dynamics are given by

d​x~t=\(μ\+12​Σ˙t​Σt−1​\(x~t−μt\)−2​λ​Σ​Σt−1​\(x~t−μt\)−2​λ​Σ1/2​Σt−1/2​\(μ−a\)\)​d​t\.\\displaystyle d\\tilde\{x\}\_\{t\}=\\left\(\\mu\+\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{t\}\\Sigma\_\{t\}^\{\-1\}\(\\tilde\{x\}\_\{t\}\-\\mu\_\{t\}\)\-2\\lambda\\,\\Sigma\\Sigma\_\{t\}^\{\-1\}\(\\tilde\{x\}\_\{t\}\-\\mu\_\{t\}\)\-2\\lambda\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\(\\mu\-a\)\\right\)dt\.Because the initial conditionx~0∼𝒩​\(0,Id\)\\tilde\{x\}\_\{0\}\\sim\\mathcal\{N\}\(0,I\_\{d\}\)is Gaussian and the drift is an affine function ofx~t\\tilde\{x\}\_\{t\}, the processx~t\\tilde\{x\}\_\{t\}remains exactly Gaussian for allt∈\[0,1\]t\\in\[0,1\], and we track its meanμtFMRG=𝔼​\[x~t\]\\mu\_\{t\}^\{\\mathrm\{FMRG\}\}=\\mathbb\{E\}\[\\tilde\{x\}\_\{t\}\]and covarianceΣtFMRG=Cov⁡\(x~t\)\\Sigma\_\{t\}^\{\\mathrm\{FMRG\}\}=\\operatorname\{Cov\}\(\\tilde\{x\}\_\{t\}\)\. Taking the expectation of the drift yields an ODE for the mean:

d​μtFMRG=μ\+\(12​Σ˙t​Σt−1−2​λ​Σ​Σt−1\)​\(μtFMRG−μt\)−2​λ​Σ1/2​Σt−1/2​\(μ−a\)\.\\displaystyle d\\mu\_\{t\}^\{\\mathrm\{FMRG\}\}=\\mu\+\\left\(\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{t\}\\Sigma\_\{t\}^\{\-1\}\-2\\lambda\\,\\Sigma\\Sigma\_\{t\}^\{\-1\}\\right\)\(\\mu\_\{t\}^\{\\mathrm\{FMRG\}\}\-\\mu\_\{t\}\)\-2\\lambda\\,\\Sigma^\{1/2\}\\Sigma\_\{t\}^\{\-1/2\}\(\\mu\-a\)\.DefiningΦt≔∫0tΣ​Σs−1​𝑑s\\Phi\_\{t\}\\coloneq\\int\_\{0\}^\{t\}\\Sigma\\Sigma\_\{s\}^\{\-1\}\\,ds, we multiply by the integrating factorΣt−1/2​exp⁡\(2​λ​Φt\)\\Sigma\_\{t\}^\{\-1/2\}\\exp\(2\\lambda\\Phi\_\{t\}\)to get

dd​t​\(Σt−1/2​exp⁡\(2​λ​Φt\)​\(μtFMRG−μt\)\)=−Σ−1/2​dd​t​exp⁡\(2​λ​Φt\)​\(μ−a\),\\displaystyle\\frac\{d\}\{dt\}\\left\(\\Sigma\_\{t\}^\{\-1/2\}\\exp\(2\\lambda\\Phi\_\{t\}\)\(\\mu\_\{t\}^\{\\mathrm\{FMRG\}\}\-\\mu\_\{t\}\)\\right\)=\-\\Sigma^\{\-1/2\}\\,\\frac\{d\}\{dt\}\\exp\(2\\lambda\\Phi\_\{t\}\)\\,\(\\mu\-a\),where we used thatΦ˙t=Σ​Σt−1\\dot\{\\Phi\}\_\{t\}=\\Sigma\\Sigma\_\{t\}^\{\-1\}\. Integrating both sides from 0 to 1 then gives us

Σ−1/2​exp⁡\(2​λ​Φ1\)​\(μ~FMRG−μ\)=−Σ−1/2​\(exp⁡\(2​λ​Φ1\)−Id\)​\(μ−a\),\\displaystyle\\Sigma^\{\-1/2\}\\exp\(2\\lambda\\Phi\_\{1\}\)\(\\tilde\{\\mu\}^\{\\mathrm\{FMRG\}\}\-\\mu\)=\-\\Sigma^\{\-1/2\}\(\\exp\(2\\lambda\\Phi\_\{1\}\)\-I\_\{d\}\)\(\\mu\-a\),soμ~FMRG=μ−\(Id−exp⁡\(−2​λ​Φ1\)\)​\(μ−a\)\\tilde\{\\mu\}^\{\\mathrm\{FMRG\}\}=\\mu\-\(I\_\{d\}\-\\exp\(\-2\\lambda\\Phi\_\{1\}\)\)\(\\mu\-a\)\. To evaluateΦ1\\Phi\_\{1\}, we note that all relevant matrices are matrix functions ofΣ\\Sigmaand hence simultaneously diagonalize\. Along an eigendirection ofΣ\\Sigmawith eigenvalueσ\>0\\sigma\>0and substitutingu=s/\(1−s\)u=s/\(1\-s\),

∫01σ\(1−s\)2\+s2​σ​𝑑s=∫0∞σ1\+u2​σ​𝑑u=π​σ2,\\displaystyle\\int\_\{0\}^\{1\}\\frac\{\\sigma\}\{\(1\-s\)^\{2\}\+s^\{2\}\\sigma\}\\,ds=\\int\_\{0\}^\{\\infty\}\\frac\{\\sigma\}\{1\+u^\{2\}\\sigma\}\\,du=\\frac\{\\pi\\sqrt\{\\sigma\}\}\{2\},soΦ1=π2​Σ1/2\\Phi\_\{1\}=\\frac\{\\pi\}\{2\}\\,\\Sigma^\{1/2\}andμ~FMRG=μ−\(Id−exp⁡\(−π​λ​Σ1/2\)\)​\(μ−a\)\\tilde\{\\mu\}^\{\\mathrm\{FMRG\}\}=\\mu\-\(I\_\{d\}\-\\exp\(\-\\pi\\lambda\\Sigma^\{1/2\}\)\)\(\\mu\-a\), which is the claimed form\. Next, we have the linear ODE

d​\(x~t−μtFMRG\)=\(12​Σ˙t​Σt−1−2​λ​Σ​Σt−1\)​\(x~t−μtFMRG\)​d​t,\\displaystyle d\(\\tilde\{x\}\_\{t\}\-\\mu\_\{t\}^\{\\mathrm\{FMRG\}\}\)=\\left\(\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{t\}\\Sigma\_\{t\}^\{\-1\}\-2\\lambda\\,\\Sigma\\Sigma\_\{t\}^\{\-1\}\\right\)\(\\tilde\{x\}\_\{t\}\-\\mu\_\{t\}^\{\\mathrm\{FMRG\}\}\)\\,dt,so by the chain rule, the covariance evolves according to the equation

d​ΣtFMRG=\(Σ˙t​Σt−1−4​λ​Σ​Σt−1\)​ΣtFMRG​d​t\.\\displaystyle d\\Sigma\_\{t\}^\{\\mathrm\{FMRG\}\}=\\left\(\\dot\{\\Sigma\}\_\{t\}\\Sigma\_\{t\}^\{\-1\}\-4\\lambda\\,\\Sigma\\Sigma\_\{t\}^\{\-1\}\\right\)\\Sigma\_\{t\}^\{\\mathrm\{FMRG\}\}\\,dt\.Because all relevant matrices commute, we can solve this ODE in closed form att=1t=1:

Σ~FMRG=exp⁡\(∫01\(Σ˙s​Σs−1−4​λ​Σ​Σs−1\)​𝑑s\)​Σ0FMRG=Σ​exp⁡\(−2​π​λ​Σ1/2\),\\displaystyle\\tilde\{\\Sigma\}^\{\\mathrm\{FMRG\}\}=\\exp\\\!\\left\(\\int\_\{0\}^\{1\}\\left\(\\dot\{\\Sigma\}\_\{s\}\\Sigma\_\{s\}^\{\-1\}\-4\\lambda\\,\\Sigma\\Sigma\_\{s\}^\{\-1\}\\right\)ds\\right\)\\Sigma\_\{0\}^\{\\mathrm\{FMRG\}\}=\\Sigma\\exp\(\-2\\pi\\lambda\\Sigma^\{1/2\}\),using∫01Σ˙s​Σs−1​𝑑s=log⁡Σ\\int\_\{0\}^\{1\}\\dot\{\\Sigma\}\_\{s\}\\Sigma\_\{s\}^\{\-1\}\\,ds=\\log\\Sigmaand∫01Σ​Σs−1​𝑑s=π2​Σ1/2\\int\_\{0\}^\{1\}\\Sigma\\Sigma\_\{s\}^\{\-1\}\\,ds=\\frac\{\\pi\}\{2\}\\,\\Sigma^\{1/2\}\. ∎

The proof follows by solving linear ODEs for the mean and covariance of the FMRG flow in closed form\. SinceTpullFMRGT\_\{\\mathrm\{pull\}\}^\{\\mathrm\{FMRG\}\}andΣ~FMRG\\tilde\{\\Sigma\}^\{\\mathrm\{FMRG\}\}are matrix functions ofΣ\\Sigma, all operators simultaneously diagonalize\. Along an eigendirectionvvwithΣ\\Sigmahaving eigenvalueσ\>0\\sigma\>0, the eigenvalues ofTpullFMRGT\_\{\\mathrm\{pull\}\}^\{\\mathrm\{FMRG\}\}andTpull\(1\)T\_\{\\mathrm\{pull\}\}^\{\(1\)\}from[Theorem˜2](https://arxiv.org/html/2606.02884#Thmtheorem2)alongvvare

λv​\(TpullFMRG\)=1−e−π​λ​σ,λv​\(Tpull\(1\)\)=π​λ​σ​e−λ​σ​erfi⁡\(λ​σ\)\.\\displaystyle\\lambda\_\{v\}\(T\_\{\\mathrm\{pull\}\}^\{\\mathrm\{FMRG\}\}\)=1\-e^\{\-\\pi\\lambda\\sqrt\{\\sigma\}\},\\qquad\\lambda\_\{v\}\(T\_\{\\mathrm\{pull\}\}^\{\(1\)\}\)=\\sqrt\{\\pi\\lambda\\sigma\}\\,e^\{\-\\lambda\\sigma\}\\,\\operatorname\{erfi\}\\\!\\left\(\\sqrt\{\\lambda\\sigma\}\\right\)\.The FMRG eigenvalue is monotonically increasing inλ\\lambdaand bounded above by 1, so the FMRG mean contracts towardaawithout ever overshooting; in contrast, the plug\-in eigenvalue crosses 1 aroundλ​σ≈0\.854\\lambda\\sigma\\approx 0\.854and exhibits mean overshoot\. Similarly, the eigenvalues ofΣ~FMRG\\tilde\{\\Sigma\}^\{\\mathrm\{FMRG\}\}andΣ\(1\)\\Sigma^\{\(1\)\}alongvvare

λv​\(Σ~FMRG\)=σ​e−2​π​λ​σ,λv​\(Σ\(1\)\)=σ​e−2​λ​σ\.\\displaystyle\\lambda\_\{v\}\(\\tilde\{\\Sigma\}^\{\\mathrm\{FMRG\}\}\)=\\sigma\\,e^\{\-2\\pi\\lambda\\sqrt\{\\sigma\}\},\\qquad\\lambda\_\{v\}\(\\Sigma^\{\(1\)\}\)=\\sigma\\,e^\{\-2\\lambda\\sigma\}\.Both covariances shrink exponentially inλ\\lambda, but with different scalings: the plug\-in exponent grows likeλ​σ\\lambda\\sigmawhile the FMRG exponent grows likeλ​σ\\lambda\\sqrt\{\\sigma\}, with a crossover atσ=π2\\sigma=\\pi^\{2\}\. Although flow map reward guidance avoids mean overshoot, it still exponentially contracts the covariance, so flow map reward guidance also exhibits reward hacking on a Gaussian target\.

### Gaussian mixture target

We now extend the analysis toρ1=∑i=1mπi𝒩\(⋅\|μi,Σi\)\\rho\_\{1\}=\\sum\_\{i=1\}^\{m\}\\pi\_\{i\}\\,\\mathcal\{N\}\(\\cdot\\,\|\\,\\mu\_\{i\},\\Sigma\_\{i\}\)\. LetΣi​t≔\(1−t\)2​Id\+t2​Σi\\Sigma\_\{it\}\\coloneq\(1\-t\)^\{2\}I\_\{d\}\+t^\{2\}\\Sigma\_\{i\}, letρi​t\\rho\_\{it\}denote the unguided marginal density for theiith component, and letwi​t​\(x\)≔πi​ρi​t​\(x\)/ρt​\(x\)w\_\{it\}\(x\)\\coloneq\\pi\_\{i\}\\rho\_\{it\}\(x\)/\\rho\_\{t\}\(x\)withρt=∑i=1mπi​ρi​t\\rho\_\{t\}=\\sum\_\{i=1\}^\{m\}\\pi\_\{i\}\\rho\_\{it\}\. The probability flow drift is the weighted combination

bt​\(x\)=∑i=1mwi​t​\(x\)​bi​t​\(x\),bi​t​\(x\)≔μi\+12​Σ˙i​t​Σi​t−1​\(x−t​μi\),b\_\{t\}\(x\)=\\sum\_\{i=1\}^\{m\}w\_\{it\}\(x\)\\,b\_\{it\}\(x\),\\qquad b\_\{it\}\(x\)\\coloneq\\mu\_\{i\}\+\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{it\}\\Sigma\_\{it\}^\{\-1\}\(x\-t\\mu\_\{i\}\),\(16\)which is nonlinear inxxthrough the mixture weightswi​t​\(x\)w\_\{it\}\(x\)\. Unlike the single\-Gaussian case, the flow mapXt,1X\_\{t,1\}has no closed form and must be obtained by integrating the ODE numerically\.

###### Theorem 20\(Flow map reward guidance for Gaussian mixture target\)\.

In the above setting, fixt∈\[0,1\)t\\in\[0,1\)and letXt,s​\(x\)X\_\{t,s\}\(x\)denote the solution of the probability flow ODEd​Xs=bs​\(Xs\)​d​sdX\_\{s\}=b\_\{s\}\(X\_\{s\}\)\\,dswith drift \([16](https://arxiv.org/html/2606.02884#A5.E16)\) and initial conditionXt=xX\_\{t\}=x\. Then, the flow map reward guidance is

utFMRG​\(x\)=−2​λ​∇Xt,1​\(x\)⊤​\(Xt,1​\(x\)−a\),\\displaystyle u\_\{t\}^\{\\mathrm\{FMRG\}\}\(x\)=\-2\\lambda\\,\\nabla X\_\{t,1\}\(x\)^\{\\top\}\(X\_\{t,1\}\(x\)\-a\),where the Jacobian∇Xt,s​\(x\)\\nabla X\_\{t,s\}\(x\)can be computed by integrating the variational equation

dd​s​∇Xt,s​\(x\)=∇bs​\(Xt,s​\(x\)\)​∇Xt,s​\(x\),∇Xt,t​\(x\)=Id\.\\frac\{d\}\{ds\}\\nabla X\_\{t,s\}\(x\)=\\nabla b\_\{s\}\(X\_\{t,s\}\(x\)\)\\,\\nabla X\_\{t,s\}\(x\),\\qquad\\nabla X\_\{t,t\}\(x\)=I\_\{d\}\.\(17\)In this formula, the spatial Jacobian of the mixture drift is

∇bs​\(y\)=∑i=1mwi​s​\(y\)​\[12​Σ˙i​s​Σi​s−1\+\(bi​s​\(y\)−bs​\(y\)\)​\(∇log⁡ρi​s​\(y\)−∇log⁡ρs​\(y\)\)⊤\]\.\\displaystyle\\nabla b\_\{s\}\(y\)=\\sum\_\{i=1\}^\{m\}w\_\{is\}\(y\)\\left\[\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{is\}\\Sigma\_\{is\}^\{\-1\}\+\(b\_\{is\}\(y\)\-b\_\{s\}\(y\)\)\(\\nabla\\log\\rho\_\{is\}\(y\)\-\\nabla\\log\\rho\_\{s\}\(y\)\)^\{\\top\}\\right\]\.

###### Proof\.

The guidance formula follows from \([15](https://arxiv.org/html/2606.02884#A5.E15)\), and the variational equation \([17](https://arxiv.org/html/2606.02884#A5.E17)\) follows from differentiatingd​Xs=bs​\(Xs\)​d​sdX\_\{s\}=b\_\{s\}\(X\_\{s\}\)\\,dswith respect to the initial condition and applying the chain rule\. For the expression for∇bs\\nabla b\_\{s\}, the product rule applied to \([16](https://arxiv.org/html/2606.02884#A5.E16)\) gives

∇bs​\(y\)=∑i=1m\(wi​s​\(y\)​∇bi​s​\(y\)\+bi​s​\(y\)​∇wi​s​\(y\)⊤\)\.\\displaystyle\\nabla b\_\{s\}\(y\)=\\sum\_\{i=1\}^\{m\}\\left\(w\_\{is\}\(y\)\\,\\nabla b\_\{is\}\(y\)\+b\_\{is\}\(y\)\\,\\nabla w\_\{is\}\(y\)^\{\\top\}\\right\)\.Sincebi​sb\_\{is\}is affine inyy, we have∇bi​s​\(y\)=12​Σ˙i​s​Σi​s−1\\nabla b\_\{is\}\(y\)=\\frac\{1\}\{2\}\\dot\{\\Sigma\}\_\{is\}\\Sigma\_\{is\}^\{\-1\}, and the standard softmax gradient gives∇wi​s​\(y\)=wi​s​\(y\)​\(∇log⁡ρi​s​\(y\)−∇log⁡ρs​\(y\)\)\\nabla w\_\{is\}\(y\)=w\_\{is\}\(y\)\(\\nabla\\log\\rho\_\{is\}\(y\)\-\\nabla\\log\\rho\_\{s\}\(y\)\)\. Because the weights sum to 1, we have∑i∇wi​s​\(y\)=0\\sum\_\{i\}\\nabla w\_\{is\}\(y\)=0, and we can use this to recenterbi​s​\(y\)−bs​\(y\)b\_\{is\}\(y\)\-b\_\{s\}\(y\)inside the second sum, yielding the claimed formula\. ∎

Comparing[Theorem˜20](https://arxiv.org/html/2606.02884#Thmtheorem20)with[Theorem˜5](https://arxiv.org/html/2606.02884#Thmtheorem5), the FMRG guidance has the same algebraic form as thek=1k=1plug\-in guidance: the random GLASS endpointX¯1​\(x\)\\bar\{X\}\_\{1\}\(x\)is replaced by the deterministic probability flow endpointXt,1​\(x\)X\_\{t,1\}\(x\), and the GLASS Jacobian by the unguided flow map Jacobian∇Xt,1​\(x\)\\nabla X\_\{t,1\}\(x\)\. In both cases, the reward enters only through the displacement \(endpoint−a\\mathrm\{endpoint\}\-a\), and the Jacobian is determined entirely by the unguided dynamics\.

Since the probability flow starting fromxxtypically transports mass to its own mode, a far\-away mode contributes negligibly to∇Xt,1​\(x\)⊤​\(Xt,1​\(x\)−a\)\\nabla X\_\{t,1\}\(x\)^\{\\top\}\(X\_\{t,1\}\(x\)\-a\)regardless of how large its reward is\. The mode selection failure of[Theorem˜6](https://arxiv.org/html/2606.02884#Thmtheorem6)therefore applies equally to flow map reward guidance, and best\-of\-nnsampling compensates for it verbatim withut\(1\)u\_\{t\}^\{\(1\)\}replaced byutFMRGu\_\{t\}^\{\\mathrm\{FMRG\}\}\. Reward damping \([Section˜4\.1\.2](https://arxiv.org/html/2606.02884#S4.SS1.SSS2)\) does not lift to FMRG, however: from[Theorem˜19](https://arxiv.org/html/2606.02884#Thmtheorem19),utFMRG​\(x\)u\_\{t\}^\{\\mathrm\{FMRG\}\}\(x\)has mismatched coefficients on\(x−t​μ\)\(x\-t\\mu\)and\(μ−a\)\(\\mu\-a\)that no scalar scheduleλt\\lambda\_\{t\}can simultaneously match, so the FMRG bias is not simply multiplicative, even for an isotropic Gaussian target\.

### Empirical comparison

We empirically verify two claims from the analysis above\. First, we show that flow map reward guidance and thek=1k=1plug\-in flow exhibit qualitatively similar reward hacking on FLUX\.1\-dev with a blueness reward; the two schemes drive the model in the same direction and produce visually comparable reward\-hacked outputs\. Second, on the single\-Gaussian target of[Theorem˜19](https://arxiv.org/html/2606.02884#Thmtheorem19), we verify theσ\\sigma\-regime crossover predicted by the analysis: the eigenvalues ofΣ~FMRG\\tilde\{\\Sigma\}^\{\\mathrm\{FMRG\}\}are smaller than those ofΣ\(1\)\\Sigma^\{\(1\)\}whenσ<π2\\sigma<\\pi^\{2\}\(FMRG hacks more aggressively than the plug\-in flow\), and larger whenσ\>π2\\sigma\>\\pi^\{2\}\(FMRG is less aggressive\)\.

##### FLUX\.1\-dev\.

For the FLUX run in[Figure˜24](https://arxiv.org/html/2606.02884#A5.F24), we use the implementation of flow map reward guidance defined by \([14](https://arxiv.org/html/2606.02884#A5.E14)\) and \([15](https://arxiv.org/html/2606.02884#A5.E15)\)\. The flow map reward guidance and thek=1k=1plug\-in are evaluated at matched gradient\-norm scale on a blue\-minus\-red\-green reward, and produce visually similar reward\-hacked outputs\. This is consistent with the structural analysis of[Theorems˜19](https://arxiv.org/html/2606.02884#Thmtheorem19)and[20](https://arxiv.org/html/2606.02884#Thmtheorem20): both schemes are single\-endpoint surrogates and exhibit the same characteristic reward hacking failure mode\.

![Refer to caption](https://arxiv.org/html/2606.02884v1/images/flux/fmrg_dragon_brg.png)Figure 24:FMRG vs\. plug\-in on FLUX\.1\.Top: unguided samples\. Middle: thek=1k=1plug\-in flow\. Bottom: flow map reward guidance\. Both guided schemes drive the entire image onto a blue tint, sharing the same reward hacking failure mode\.
##### Crossover atσ=π2\\sigma=\\pi^\{2\}\.

Recall from[Theorems˜19](https://arxiv.org/html/2606.02884#Thmtheorem19)and[2](https://arxiv.org/html/2606.02884#Thmtheorem2)that for an isotropic Gaussian targetρ1=𝒩​\(μ,σ​Id\)\\rho\_\{1\}=\\mathcal\{N\}\(\\mu,\\sigma I\_\{d\}\)under the memoryless schedule, the eigenvalues of the terminal covariances areΣ~FMRG=σ​e−2​π​λ​σ\\tilde\{\\Sigma\}^\{\\mathrm\{FMRG\}\}=\\sigma e^\{\-2\\pi\\lambda\\sqrt\{\\sigma\}\}andΣ\(1\)=σ​e−2​λ​σ\\Sigma^\{\(1\)\}=\\sigma e^\{\-2\\lambda\\sigma\}\. The plug\-in covariance shrinks faster than the FMRG covariance precisely when2​λ​σ\>2​π​λ​σ⇔σ\>π22\\lambda\\sigma\>2\\pi\\lambda\\sqrt\{\\sigma\}\\iff\\sigma\>\\pi^\{2\}\.[Figure˜25](https://arxiv.org/html/2606.02884#A5.F25)verifies this regime crossover empirically: with the narrow target \(σ=0\.5<π2\\sigma=0\.5<\\pi^\{2\}\) atλ=1\\lambda=1, FMRG produces a much sharper terminal distribution than thek=1k=1plug\-in flow, while with the wide target \(σ=16\>π2\\sigma=16\>\\pi^\{2\}\) atλ=0\.1\\lambda=0\.1, the same comparison reverses and FMRG samples are visibly more spread out than the plug\-in samples\. Both regimes exhibit aggressive reward hacking, but the relative aggressiveness flips withσ\\sigmaexactly as predicted\.

![Refer to caption](https://arxiv.org/html/2606.02884v1/x20.png)Figure 25:σ\\sigma\-regime crossover\.Empirical comparison of FMRG vs\. thek=1k=1plug\-in flow on an isotropic Gaussian target\. Panels A,B \(narrow,σ=0\.5<π2\\sigma=0\.5<\\pi^\{2\},λ=1\\lambda=1\): FMRG concentrates much more sharply at the reward target than the plug\-in\. Panels C,D \(wide,σ=16\>π2\\sigma=16\>\\pi^\{2\},λ=0\.1\\lambda=0\.1\): the relative aggressiveness reverses, with FMRG samples visibly more spread out, exactly as predicted by[Theorem˜19](https://arxiv.org/html/2606.02884#Thmtheorem19)\.