LLM-OSDA: An Optimal-Stopping Dynamic Auction for Native Advertising in Multi-Turn LLM Conversations

# LLM-OSDA: An Optimal-Stopping Dynamic Auction for Native Advertising in Multi-Turn LLM Conversations
Source: [https://arxiv.org/html/2608.00123](https://arxiv.org/html/2608.00123)
###### Abstract

LLM\-native advertising embeds sponsored content directly into model\-generated responses, shifting the unit of sale from a fixed slot to a moment within an evolving conversation\. Existing LLM ad\-auction mechanisms primarily operate within a single response, settling the winner but not the timing\. The extension is nontrivial: with one native insertion opportunity per session, the stopping time depends on bids, coupling timing with allocation, so static truthfulness arguments no longer apply\. We propose theLLM\-basedOptimalStoppingDynamicAuction \(LLM\-OSDA\), a dynamic cost\-per\-click auction that integrates Bellman optimal stopping, winner allocation, and envelope pricing\. A bid\-independent LLM layer estimates contextual click quality and seamlessly renders the winning ad, while bids enter only the committed auction mechanism\. Under an exact Bellman oracle, the expected discounted\-click allocation is monotone in each advertiser’s bid, and the corresponding envelope payment makes truthful bidding weakly dominant in expectation\. For practical deployment, a learned StopNet approximates the Bellman action values\. We show that its decisions differ from the optimal policy only near the stopping boundary and bound the resulting incentive loss in terms of its approximation error\. Experiments on a simulated conversational advertising corpus show that LLM\-OSDA improves net revenue by 11% over the strongest fixed\-timing baseline while maintaining comparable user retention\. Code is athttps://github\.com/2025Fang2025/llm\-osda\.

## Introduction

Large language models \(LLMs\)\(Brownet al\.[2020](https://arxiv.org/html/2608.00123#bib.bib19); Zhaoet al\.[2023](https://arxiv.org/html/2608.00123#bib.bib41)\)are transforming information retrieval from one\-shot search into multi\-turn dialogue\(Metzleret al\.[2021](https://arxiv.org/html/2608.00123#bib.bib1)\)and enabling user behavior to be simulated at scale\(Parket al\.[2023](https://arxiv.org/html/2608.00123#bib.bib40)\)\. Yet serving these interactions is costly\(Popeet al\.[2023](https://arxiv.org/html/2608.00123#bib.bib3)\), making advertising a natural means of supporting them\(Feiziet al\.[2026](https://arxiv.org/html/2608.00123#bib.bib16)\)\. As Figure[1](https://arxiv.org/html/2608.00123#Sx1.F1)illustrates, monetization has progressed from search slots and feed impressions to insertion moments within agentic conversations\. Because an LLM\-native ad is embedded directly into a generated response, the mechanism must determine not only what to show but also when to show it\.

![Refer to caption](https://arxiv.org/html/2608.00123v1/Figures/ad_monetization_evolution.png)Figure 1:Advertising monetization from search slots to feed impressions to agentic conversations, where timing becomes the decision\.Existing auctions for generative models either enforce incentive compatibility token by token\(Düttinget al\.[2024b](https://arxiv.org/html/2608.00123#bib.bib10)\), use Retrieval\-Augmented Generation\(Lewiset al\.[2020](https://arxiv.org/html/2608.00123#bib.bib23)\)to allocate and integrate ads\(Hajiaghayiet al\.[2024](https://arxiv.org/html/2608.00123#bib.bib13)\)or support data\-marketplace primitives\(Hanet al\.[2025](https://arxiv.org/html/2608.00123#bib.bib14)\), or keep the auction outside the LLM and use the model only to summarize the winning bundle\(Dubeyet al\.[2024](https://arxiv.org/html/2608.00123#bib.bib12)\)\. By separating auction decisions from generation, these designs can overlook allocation externalities or require repeated forward passes that are costly at industrial scale\(Liuet al\.[2025](https://arxiv.org/html/2608.00123#bib.bib15)\)\.

More recent work couples allocation more closely with generation through preference aggregation\(Soumaliaset al\.[2024](https://arxiv.org/html/2608.00123#bib.bib11)\)or end\-to\-end optimization\(Zhaoet al\.[2026](https://arxiv.org/html/2608.00123#bib.bib17)\), yet remains single\-turn: it neither tracks commercial intent across a conversation nor determines*when*the platform should intervene\. Multi\-turn LLM advertising therefore introduces a joint timing problem\. With a single native insertion opportunity, the platform must decide when to intervene, which advertiser wins, how to render the sponsored response, and how to price the allocation while user intent is still evolving\. Inserting too early wastes the opportunity on ambiguous intent, whereas waiting risks user departure; moreover, bid\-dependent stopping couples the insertion time with the expected\-click allocation, so a turn\-local truthfulness or pricing argument no longer suffices\.

![Refer to caption](https://arxiv.org/html/2608.00123v1/x1.png)Figure 2:Framework of LLM\-OSDA\. Online, a bid\-independent language layer feeds click\-quality signals to the bid\-aware mechanism, which sets stopping, allocation, and pricing before the renderer produces the native response\. Offline, iterative refinement jointly aligns the learned components\.With timing endogenous to the bid, monotonicity and truthful pricing must be re\-established over the stopping\-allocation policy rather than a fixed slot\. We therefore propose the LLM\-based Optimal Stopping Dynamic Auction \(LLM\-OSDA\), a dynamic CPC \(cost\-per\-click\) auction\. It delegates contextual click\-quality estimation to a bid\-independent LLM layer and assigns timing, allocation, and payment to a committed mechanism that observes the bids\. Building on optimal\-stopping theory\(Peskir and Shiryaev[2006](https://arxiv.org/html/2608.00123#bib.bib31); Board[2007](https://arxiv.org/html/2608.00123#bib.bib8)\)and single\-parameter envelope methods\(Myerson[1981](https://arxiv.org/html/2608.00123#bib.bib6); Milgrom and Segal[2002](https://arxiv.org/html/2608.00123#bib.bib20)\), it differs from prior timing models\(Banchioet al\.[2024](https://arxiv.org/html/2608.00123#bib.bib9); Alaeiet al\.[2026](https://arxiv.org/html/2608.00123#bib.bib42)\)by coupling bid\-dependent stopping with LLM\-estimated click quality and native generation\. Theoretically, this single\-parameter construction keeps truthful bidding dominant under bid\-dependent timing; empirically, LLM\-OSDA lifts net revenue by11%11\\%over the strongest fixed\-timing baseline\. Our contributions are as follows:

- •We introduce a learning\-based framework that jointly decides when to insert an ad, which advertiser wins, how to render the sponsored response, and how much to charge per click\. To our knowledge, LLM\-OSDA is the first CPC auction for multi\-turn LLM\-native advertising to combine conversational intent estimation and native response generation with endogenous stopping, allocation, and click\-contingent envelope pricing\.
- •Our mechanism prices the option value that bid\-dependent stopping creates\. Because a higher bid can change not just who wins but which turn the ad lands on, the payment is an envelope CPC integrated over the whole bid range and computed from the expected discounted clicks, which a single\-turn critical price cannot capture\.
- •With an exact Bellman oracle, the mechanism is truthful in expectation and never charges a winner above its value, thanks to the single\-parameter allocation the bid\-independent layer preserves\. A learned StopNet cannot match this exactly, so we bound its misreport gain, showing it departs from the ideal only near the stopping boundary\. It recovers the classical single\-turn Myerson auction as a special case\.
- •On a simulated conversational advertising corpus, LLM\-OSDA raises net revenue by11%11\\%over the strongest fixed\-timing baseline without hurting user retention, and diagnostics confirm that its gains come from timing exercised for the intended reason\.

## Related Work

#### Static auction design\.

Myerson’s optimal\-auction framework, based on virtual values, laid the foundation for revenue\-optimal auction design\(Myerson[1981](https://arxiv.org/html/2608.00123#bib.bib6)\)\. In search advertising, subsequent work characterized the Generalized Second\-Price mechanism and truthful position auctions in relation to VCG\(Edelmanet al\.[2007](https://arxiv.org/html/2608.00123#bib.bib4); Varian[2007](https://arxiv.org/html/2608.00123#bib.bib5); Aggarwalet al\.[2006](https://arxiv.org/html/2608.00123#bib.bib21)\), while Balseiro and Gur\(Balseiro and Gur[2019](https://arxiv.org/html/2608.00123#bib.bib22)\)studied budget\-constrained learning agents\. Learning\-based auction design has also employed permutation\-equivariant architectures to encode bidder symmetry\(Qinet al\.[2022](https://arxiv.org/html/2608.00123#bib.bib46)\)\.All assume exogenous selling times\.

#### Dynamic mechanisms with endogenous timing\.

Bergemann and Välimäki\(Bergemann and Välimäki[2019](https://arxiv.org/html/2608.00123#bib.bib7)\)extend VCG to settings with privately evolving types through the dynamic pivot mechanism; Athey and Segal\(Athey and Segal[2013](https://arxiv.org/html/2608.00123#bib.bib32)\)attain efficiency with budget balance; and Pavan, Segal, and Toikka\(Pavanet al\.[2014](https://arxiv.org/html/2608.00123#bib.bib33)\)provide a Myersonian envelope characterization of dynamic incentive compatibility\. Milgrom and Segal’s envelope theorem\(Milgrom and Segal[2002](https://arxiv.org/html/2608.00123#bib.bib20)\)provides the identity underlying our dominant\-strategy incentive compatibility \(DSIC\) proof, which we adapt to endogenous stopping\. Board’s work on “selling options”\(Board[2007](https://arxiv.org/html/2608.00123#bib.bib8); Peskir and Shiryaev[2006](https://arxiv.org/html/2608.00123#bib.bib31)\)motivates the option\-value interpretation of timing\. Our payment rule, however, is based on the global expected\-click allocation rather than a single local continuation value\.

#### Ad auctions coupled with LLM generation\.

In the LLM era, AI\-driven advertising raises intertwined questions in market design, generative modeling, and ethics\(Heet al\.[2024](https://arxiv.org/html/2608.00123#bib.bib47)\)\. Ad\-auction designs place the auction at different points in the generation pipeline\. Dütting et al\.\(Düttinget al\.[2024b](https://arxiv.org/html/2608.00123#bib.bib10)\)designed a token\-level auction in which incentive compatibility follows from aggregation monotonicity, in the differentiable\-economics tradition\(Düttinget al\.[2024a](https://arxiv.org/html/2608.00123#bib.bib34)\)\. Soumalias et al\.\(Soumaliaset al\.[2024](https://arxiv.org/html/2608.00123#bib.bib11)\)aggregated advertiser preferences over responses without modifying model weights, while Dubey et al\.\(Dubeyet al\.[2024](https://arxiv.org/html/2608.00123#bib.bib12)\)kept the auction outside the LLM and used the model only to summarize the winning bundle\. Hajiaghayi et al\.\(Hajiaghayiet al\.[2024](https://arxiv.org/html/2608.00123#bib.bib13)\)embedded segment auctions in a Retrieval\-Augmented Generation pipeline\(Lewiset al\.[2020](https://arxiv.org/html/2608.00123#bib.bib23)\), whereas Zhao et al\.\(Zhaoet al\.[2026](https://arxiv.org/html/2608.00123#bib.bib17)\)fused allocation with generation end\-to\-end\. LERA\(Sunet al\.[2026](https://arxiv.org/html/2608.00123#bib.bib43)\)selects ads at each segment, whereas we optimize a single bid\-dependent insertion time over the session\. Related work also studies sponsored questions\(Bhawalkaret al\.[2025](https://arxiv.org/html/2608.00123#bib.bib27)\), LLM\-judged relevance\(Deyet al\.[2025](https://arxiv.org/html/2608.00123#bib.bib29)\), and data auctions for RAG\(Hanet al\.[2025](https://arxiv.org/html/2608.00123#bib.bib14)\)\. However, all of these approaches operate within a single response; none determines*when*to intervene during a conversation\.

#### Timing and intent across turns\.

The evolution of user intent across dialogue turns has been studied in LLM\-based conversational recommendation\(Friedmanet al\.[2023](https://arxiv.org/html/2608.00123#bib.bib24); Heet al\.[2023](https://arxiv.org/html/2608.00123#bib.bib25); Wanget al\.[2023](https://arxiv.org/html/2608.00123#bib.bib26)\)\. However, this literature focuses on recommendation without considering incentives or payments\. On the incentive side, Banchio et al\.\(Banchioet al\.[2024](https://arxiv.org/html/2608.00123#bib.bib9)\)compared auction*formats*for conversational advertising; their real\-options framework characterizes the seller’s stopping decision, whereas we construct a committed stopping\-and\-allocation policy and derive its payments\. Closest to our work, Alaei et al\.\(Alaeiet al\.[2026](https://arxiv.org/html/2608.00123#bib.bib42)\)independently study the same optimal\-stopping problem, but under a distributional model of user beliefs rather than the LLM\-estimated signals we use\.

## Problem Formulation

### Setting and Information Structure

A session lasts at mostTTturns and contains at most one native ad\. Conditional on reaching turntt, the user continues to turnt\+1t\+1with probabilityγ∈\(0,1\]\\gamma\\in\(0,1\], soγt−1\\gamma^\{t\-1\}is the survival weight at turntt\. There arennadvertisers indexed by𝒩=\{1,…,n\}\\mathcal\{N\}=\\\{1,\\ldots,n\\\}\. Advertiseriihas a private per\-click valueθi∈Θi=\[θ¯i,θ¯i\]\\theta\_\{i\}\\in\\Theta\_\{i\}=\[\\underline\{\\theta\}\_\{i\},\\overline\{\\theta\}\_\{i\}\], drawn independently from a platform\-known distributionFiF\_\{i\}\. Before the session, each advertiser submits a fixed CPC bidbi∈Θib\_\{i\}\\in\\Theta\_\{i\}, forming the bid profileb=\(b1,…,bn\)b=\(b\_\{1\},\\ldots,b\_\{n\}\)\.

Let𝒟\\mathcal\{D\}denote the distribution over bid\-independent potential dialogue paths, assumed independent of bids\. A path is written asω=\(h1,…,hT\)\\omega=\(h\_\{1\},\\ldots,h\_\{T\}\), wherehth\_\{t\}is the history at turntt\. A direct mechanism maps\(b,ω\)\(b,\\omega\)to⟨τ​\(b;ω\),I​\(b;ω\),P​\(b;ω\)⟩\\langle\\tau\(b;\\omega\),I\(b;\\omega\),P\(b;\\omega\)\\rangle, whereτ∈\{1,…,T\}∪\{∞\}\\tau\\in\\\{1,\\ldots,T\\\}\\cup\\\{\\infty\\\},I∈𝒩∪\{0\}I\\in\\mathcal\{N\}\\cup\\\{0\\\}, andP∈ℝ\+P\\in\\mathbb\{R\}\_\{\+\}\. At turntt, the mechanism observes\(b,ht\)\(b,h\_\{t\}\)but not future histories, soτ\\tauis a stopping time\. Ifτ=∞\\tau=\\infty, we setI=P=0I=P=0\. For advertiserii,qi​\(ht\)q\_\{i\}\(h\_\{t\}\)is its true click probability if shown athth\_\{t\}, andGηG\_\{\\eta\}predictsq^η,i​\(ht\)\\widehat\{q\}\_\{\\eta,i\}\(h\_\{t\}\)\.

### Allocation and Objective

For a fixed bid profileb=\(bi,b−i\)b=\(b\_\{i\},b\_\{\-i\}\)and dialogue pathω\\omega, define advertiserii’s pathwise display indicator as

Xi​\(b;ω\):=𝟏​\{τ​\(b;ω\)<∞∧I​\(b;ω\)=i\}\.X\_\{i\}\(b;\\omega\):=\\mathbf\{1\}\\\!\\left\\\{\\tau\(b;\\omega\)<\\infty\\ \\land\\ I\(b;\\omega\)=i\\right\\\}\.ThusXi=1X\_\{i\}=1exactly when the mechanism inserts advertiserii’s ad on pathω\\omega\. Because advertiser values are per click and the click probability depends on the insertion turn, we weight the indicator by the discounted click probability:

xi​\(bi,b−i\):=𝔼ω∼𝒟​\[γτ​\(b;ω\)−1​qi​\(hτ​\(b;ω\)\)​Xi​\(b;ω\)\],x\_\{i\}\(b\_\{i\},b\_\{\-i\}\):=\\mathbb\{E\}\_\{\\omega\\sim\\mathcal\{D\}\}\\\!\\left\[\\gamma^\{\\tau\(b;\\omega\)\-1\}q\_\{i\}\\\!\\left\(h\_\{\\tau\(b;\\omega\)\}\\right\)X\_\{i\}\(b;\\omega\)\\right\],advertiserii’s expected number of discounted clicks, valued in\[0,1\]\[0,1\]\. With per\-session expected paymentmi​\(bi,b−i\)=Pi​\(bi,b−i\)​xi​\(bi,b−i\)m\_\{i\}\(b\_\{i\},b\_\{\-i\}\)=P\_\{i\}\(b\_\{i\},b\_\{\-i\}\)\\,x\_\{i\}\(b\_\{i\},b\_\{\-i\}\)under click\-contingent settlement at CPCPiP\_\{i\}, a typeθi\\theta\_\{i\}reportingbib\_\{i\}has expected utilityUi​\(θi;bi,b−i\)=θi​xi​\(bi,b−i\)−mi​\(bi,b−i\)U\_\{i\}\(\\theta\_\{i\};b\_\{i\},b\_\{\-i\}\)=\\theta\_\{i\}x\_\{i\}\(b\_\{i\},b\_\{\-i\}\)\-m\_\{i\}\(b\_\{i\},b\_\{\-i\}\)\.

The platform’s primary objective is social welfare, the expected match value delivered across sessions, which equals∑iθi​xi\\sum\_\{i\}\\theta\_\{i\}x\_\{i\}under truthful bidding\. We require the mechanism to be DSIC in expectation: for every advertiserii, true valueθi\\theta\_\{i\}, rival bid profileb−ib\_\{\-i\}, and alternative reportri∈Θir\_\{i\}\\in\\Theta\_\{i\},

Ui​\(θi;θi,b−i\)≥Ui​\(θi;ri,b−i\)\.U\_\{i\}\(\\theta\_\{i\};\\theta\_\{i\},b\_\{\-i\}\)\\geq U\_\{i\}\(\\theta\_\{i\};r\_\{i\},b\_\{\-i\}\)\.The DSIC requirement is structural, depending on allocation monotonicity rather than the timing rule’s objective, so the stopping rule can target platform goals like net revenue and user retention\.

## Methodology

### Overview

As illustrated in Figure[2](https://arxiv.org/html/2608.00123#Sx1.F2), a platform auctions the single opportunity to insert an ad into an LLM–user conversation\. A bid\-independent language layer scores each ad’s click quality, and a bid\-aware mechanism then decides whether to stop and insert, who wins, and how much to charge, before the renderer writes the winning ad into the response\. Iterative refinement aligns these learned components offline\.

### Turn\-level click quality estimator

Because click quality varies across turns as intent evolves, the estimator conditions on the full dialogue historyhth\_\{t\}\. The dialogue state is encoded aset=ELLM​\(ht\)e\_\{t\}=E\_\{\\mathrm\{LLM\}\}\(h\_\{t\}\), and the quality model outputsq^η,i​\(ht\)=Gη​\(et,ai\)\\widehat\{q\}\_\{\\eta,i\}\(h\_\{t\}\)=G\_\{\\eta\}\(e\_\{t\},a\_\{i\}\)\. Whether a click occurs also depends on how the winning ad is written into the current response, which the LLM rendererπR\\pi\_\{R\}generates, so we writeqi​\(ht;πR\)q\_\{i\}\(h\_\{t\};\\pi\_\{R\}\), abbreviatedqi​\(ht\)q\_\{i\}\(h\_\{t\}\)within a fixed round\. NeitherπR\\pi\_\{R\}norGηG\_\{\\eta\}takes a bid as input; this LLM layer captures context and relevance but does not adjudicate the auction\. The click law therefore stays bid\-independent, preserving the single\-parameter structure that envelope pricing requires\.

### Optimal stopping

#### Ideal Bellman rule\.

The ideal mechanism ranks advertisers byyi​qi​\(ht\)y\_\{i\}q\_\{i\}\(h\_\{t\}\), whereyi=ψi​\(bi\)y\_\{i\}=\\psi\_\{i\}\(b\_\{i\}\)for nondecreasingψi\\psi\_\{i\}; experiments use the identity mapψi​\(bi\)=bi\\psi\_\{i\}\(b\_\{i\}\)=b\_\{i\}, while a virtual\-value score targets revenue\. At turntt, its immediate exercise value, continuation value, and Bellman recursion are

Wt​\(ht;b\)\\displaystyle W\_\{t\}\(h\_\{t\};b\):=max⁡\{0,maxi∈𝒩⁡ψi​\(bi\)​qi​\(ht\)\},\\displaystyle:=\\max\\bigl\\\{0,\\max\_\{i\\in\\mathcal\{N\}\}\\psi\_\{i\}\(b\_\{i\}\)q\_\{i\}\(h\_\{t\}\)\\bigr\\\},C​Vt​\(ht;b\)\\displaystyle CV\_\{t\}\(h\_\{t\};b\):=𝔼​\[Vt\+1​\(Ht\+1;b\)∣Ht=ht\],\\displaystyle:=\\mathbb\{E\}\\bigl\[V\_\{t\+1\}\(H\_\{t\+1\};b\)\\mid H\_\{t\}=h\_\{t\}\\bigr\],Vt​\(ht;b\)\\displaystyle V\_\{t\}\(h\_\{t\};b\):=max⁡\{Wt​\(ht;b\),γ​C​Vt​\(ht;b\)\},VT\+1≡0\.\\displaystyle:=\\max\\\{W\_\{t\}\(h\_\{t\};b\),\\gamma\\,CV\_\{t\}\(h\_\{t\};b\)\\\},\\quad V\_\{T\+1\}\\equiv 0\.\(1\)Equivalently, the exact stopping and waiting action values are

QtS​\(ht;b\)=Wt​\(ht;b\),QtW​\(ht;b\)=γ​C​Vt​\(ht;b\)\.Q\_\{t\}^\{S\}\(h\_\{t\};b\)=W\_\{t\}\(h\_\{t\};b\),\\qquad Q\_\{t\}^\{W\}\(h\_\{t\};b\)=\\gamma CV\_\{t\}\(h\_\{t\};b\)\.The ideal mechanism stops at the first turn whose immediate value is positive and at least the continuation value,

τ⋆​\(b\):=inf\{t:Wt\>0,QtS≥QtW\},\\tau^\{\\star\}\(b\):=\\inf\\\{t:W\_\{t\}\>0,\\;Q\_\{t\}^\{S\}\\geq Q\_\{t\}^\{W\}\\\},with the conventioninf∅=∞\\inf\\varnothing=\\infty\.

#### Learned StopNet\.

The ideal recursion above needs the true click probabilityqiq\_\{i\}and the true history transition insideC​VtCV\_\{t\}, neither available at deployment\. A single head that learns one value function does not suffice: evaluating the immediate valueWtW\_\{t\}still requires the trueqiq\_\{i\}\. Our StopNet therefore uses a shared trunk feeding a stop head and a wait head that regress both action values*directly*fromsts\_\{t\}, a scalar summary of the current\-turn auction:Q^ϕ​\(st\)=\(Q^ϕ,tS,Q^ϕ,tW\)\\widehat\{Q\}\_\{\\phi\}\(s\_\{t\}\)=\(\\widehat\{Q\}\_\{\\phi,t\}^\{S\},\\widehat\{Q\}\_\{\\phi,t\}^\{W\}\)\. The learned stopping time applies the ideal test with these regressed values,

τ^​\(b\):=inf\{t:Q^ϕ,tS≥Q^ϕ,tW\>0\}\.\\widehat\{\\tau\}\(b\):=\\inf\\\{t:\\widehat\{Q\}\_\{\\phi,t\}^\{S\}\\geq\\widehat\{Q\}\_\{\\phi,t\}^\{W\}\>0\\\}\.Stopping inherently perceives the bid, so bid\-independence constrains onlyGηG\_\{\\eta\}andπR\\pi\_\{R\}, not the StopNet\.

### Winner allocation\.

The winner maximizes the ranking score times the click probability at the stopping turn\. The ideal policy takesI⋆​\(b\)∈arg⁡maxi⁡ψi​\(bi\)​qi​\(Hτ⋆\)I^\{\\star\}\(b\)\\in\\arg\\max\_\{i\}\\psi\_\{i\}\(b\_\{i\}\)q\_\{i\}\(H\_\{\\tau^\{\\star\}\}\)with the trueqiq\_\{i\}; the deployed policy uses the estimate,

I^​\(b\)∈arg⁡maxi⁡ψi​\(bi\)​q^η,i​\(Hτ^\)\.\\widehat\{I\}\(b\)\\in\\arg\\max\_\{i\}\\psi\_\{i\}\(b\_\{i\}\)\\widehat\{q\}\_\{\\eta,i\}\(H\_\{\\widehat\{\\tau\}\}\)\.Ties follow the same fixed deterministic rule\. Because the winner is selected on the historyHτ⋆H\_\{\\tau^\{\\star\}\}at the stopping turn, who wins and when to insert are coupled: raising a bid can change both the winner and the trigger turn, the source of option value that envelope pricing must handle\. Because the deployed policy ranks by the estimateq^η,i\\widehat\{q\}\_\{\\eta,i\}, estimation error can shift both the trigger turn and the selected winner\.

### Envelope pricing\.

We normalize the lowest type’s truthful utility to zero\. Fixingb−ib\_\{\-i\}and writingxi​\(bi\)x\_\{i\}\(b\_\{i\}\)forxi​\(bi,b−i\)x\_\{i\}\(b\_\{i\},b\_\{\-i\}\), the expected payment and equivalent CPC are

mi​\(bi\)\\displaystyle m\_\{i\}\(b\_\{i\}\):=bi​xi​\(bi\)−∫θ¯ibixi​\(z\)​𝑑z,\\displaystyle:=b\_\{i\}x\_\{i\}\(b\_\{i\}\)\-\\int\_\{\\underline\{\\theta\}\_\{i\}\}^\{b\_\{i\}\}x\_\{i\}\(z\)\\,dz,PiENV​\(bi\)\\displaystyle P\_\{i\}^\{\\mathrm\{ENV\}\}\(b\_\{i\}\):=\{mi​\(bi\)/xi​\(bi\),xi​\(bi\)\>0,0,xi​\(bi\)=0\.\\displaystyle:=\\begin\{cases\}m\_\{i\}\(b\_\{i\}\)/x\_\{i\}\(b\_\{i\}\),&x\_\{i\}\(b\_\{i\}\)\>0,\\\\ 0,&x\_\{i\}\(b\_\{i\}\)=0\.\\end\{cases\}\(2\)This is the standard single\-parameter envelope construction\(Myerson[1981](https://arxiv.org/html/2608.00123#bib.bib6)\); no click\-contingent charge is collected whenxi=0x\_\{i\}=0\. Here it acts not on a win probability but on the LLM\-driven allocationxix\_\{i\}, an expectation of discounted clicksγτ−1​qi​\(hτ\)\\gamma^\{\\tau\-1\}q\_\{i\}\(h\_\{\\tau\}\); because a higher bid can move the trigger turn, the integral∫xi​\(z\)​𝑑z\\int x\_\{i\}\(z\)\\,dzprices the option value of insertion timing\.

Whenxix\_\{i\}is a monotone step function with jumpsΔ​xi​k\\Delta x\_\{ik\}at thresholdszi​k∈\(θ¯i,bi\]z\_\{ik\}\\in\(\\underline\{\\theta\}\_\{i\},b\_\{i\}\], direct integration yields, forxi​\(bi\)\>0x\_\{i\}\(b\_\{i\}\)\>0,

PiENV​\(bi\)=θ¯i​xi​\(θ¯i\)\+∑zi​k≤bizi​k​Δ​xi​kxi​\(bi\)\.P\_\{i\}^\{\\mathrm\{ENV\}\}\(b\_\{i\}\)=\\frac\{\\underline\{\\theta\}\_\{i\}x\_\{i\}\(\\underline\{\\theta\}\_\{i\}\)\+\\sum\_\{z\_\{ik\}\\leq b\_\{i\}\}z\_\{ik\}\\Delta x\_\{ik\}\}\{x\_\{i\}\(b\_\{i\}\)\}\.Each thresholdzi​kz\_\{ik\}marks a global change in the stopping\-allocation plan, so the dynamic CPC generally does not reduce to any single\-turn local threshold\.

For the learned policy, letx^i​\(z,b−i\)\\widehat\{x\}\_\{i\}\(z,b\_\{\-i\}\)denote its theoretical expected\-click allocation, evaluated using the trueqiq\_\{i\}\. WithRRbid\-independent dialogue rollouts, the operational estimator instead uses predicted click quality:

x^i\(R\)​\(z,b−i\):=1R​∑r=1Rγτ^r​\(z\)−1​q^η,i​\(Hτ^r​\(z\)\(r\)\)​𝟏​\{I^r​\(z\)=i\}\.\\widehat\{x\}\_\{i\}^\{\(R\)\}\(z,b\_\{\-i\}\):=\\frac\{1\}\{R\}\\sum\_\{r=1\}^\{R\}\\gamma^\{\\widehat\{\\tau\}\_\{r\}\(z\)\-1\}\\widehat\{q\}\_\{\\eta,i\}\\\!\\left\(H\_\{\\widehat\{\\tau\}\_\{r\}\(z\)\}^\{\(r\)\}\\right\)\\mathbf\{1\}\\\{\\widehat\{I\}\_\{r\}\(z\)=i\\\}\.A summand is defined as zero if no ad is inserted\. Numerical integration ofx^i\(R\)\\widehat\{x\}\_\{i\}^\{\(R\)\}yields an estimated envelope paymentm^iENV,\(R\)\\widehat\{m\}\_\{i\}^\{\\mathrm\{ENV\},\(R\)\}and CPCP^iENV,\(R\)\\widehat\{P\}\_\{i\}^\{\\mathrm\{ENV\},\(R\)\}\. Finite rollouts, predicted click quality, and numerical bid\-grid integration make this estimator differ from both the ideal allocationxix\_\{i\}and the learned policy’s theoretical expectationx^i\\widehat\{x\}\_\{i\}; we bound the resulting payment error in the analysis below\.

Platform objectivesDiagnosticsMethodNet Rev\.RewardCTRTriggerBidPayInfo\. RentGross Rev\.Always\-Round\-1 \(base\)0\.693±\.0220\.654±\.0240\.810±\.0161\.001\.679±\.0250\.878±\.0180\.632±\.0181\.325±\.007Always\-Round\-1 \(refined\)0\.733±\.0210\.700±\.0220\.849±\.0111\.001\.679±\.0250\.878±\.0180\.670±\.0261\.403±\.010Always\-Last\-Round0\.625±\.0170\.586±\.0190\.779±\.0242\.191\.595±\.0250\.846±\.0070\.540±\.0091\.165±\.018Random\-Round0\.659±\.0190\.620±\.0210\.796±\.0161\.591\.634±\.0190\.857±\.0160\.586±\.0111\.246±\.009LLM\-OSDA \(base\)0\.769±\.0170\.729±\.0180\.796±\.0141\.391\.693±\.0230\.999±\.0200\.537±\.0161\.306±\.009LLM\-OSDA \(refined\)0\.818±\.0130\.784±\.0140\.841±\.0061\.391\.693±\.0230\.999±\.0200\.572±\.0251\.390±\.012Table 1:Main results in Round 2 \(mean±\\pmstd over three seeds\)\. Pay is the envelope CPC; Net Rev\. and Info\. Rent are the expected\-click\-weighted revenue and rent\. Bold and underline mark the best and second\-best on the platform\-objective columns \(CTR, Net Rev\., Reward\); the remaining columns are diagnostic\.
### Iterative mechanism\-aware refinement\.

Because updating the renderer makes the labels used by the quality model and StopNet stale, we align the three learned components iteratively \(Algorithm 1; see the appendix\): the quality modelGηG\_\{\\eta\}, the StopNetQ^ϕ=\(Q^ϕS,Q^ϕW\)\\widehat\{Q\}\_\{\\phi\}=\(\\widehat\{Q\}\_\{\\phi\}^\{S\},\\widehat\{Q\}\_\{\\phi\}^\{W\}\), and the renderer LLMπR\\pi\_\{R\}\. Each round regenerates responses with the current renderer, re\-scores them with a user\-simulation judge, refitsGηG\_\{\\eta\}andQ^ϕ\\widehat\{Q\}\_\{\\phi\}on updated labels, and refines the renderer by best\-of\-NNsampling, keeping the response with the highest expected click valuebi​qib\_\{i\}q\_\{i\}\. Each round freezes a committed tuple\(Gη\(k\),Q^ϕ\(k\),πR\(k\)\)\(G\_\{\\eta\}^\{\(k\)\},\\widehat\{Q\}\_\{\\phi\}^\{\(k\)\},\\pi\_\{R\}^\{\(k\)\}\)\.

## Theoretical Guarantees

We establish guarantees separately for the exact benchmark and the learned implementation\. The layered design assumes: \(A1\) advertisers communicate only through bids; \(A2\)GηG\_\{\\eta\}is bid\-independent; \(A3\) stopping, allocation, and pricing follow committed deterministic formulas; \(A4\) the rendererπR\\pi\_\{R\}takes no bid as input, so the induced click lawqi​\(ht;πR\)q\_\{i\}\(h\_\{t\};\\pi\_\{R\}\)is bid\-independent within a committed round\.

###### Lemma 1\(Monotonicity of the ideal expected\-click allocation\)\.

For every advertiseriiand every fixedb−ib\_\{\-i\}, the ideal Bellman allocationxi​\(bi,b−i\)x\_\{i\}\(b\_\{i\},b\_\{\-i\}\)is weakly increasing inyi=ψi​\(bi\)y\_\{i\}=\\psi\_\{i\}\(b\_\{i\}\), and therefore inbib\_\{i\}\.

###### Proof sketch\.

For any feasible planσ\\sigma, bid\-independent transitions makeJσ​\(yi\)=Aσ\+yi​xiσJ^\{\\sigma\}\(y\_\{i\}\)=A^\{\\sigma\}\+y\_\{i\}x\_\{i\}^\{\\sigma\}affine inyiy\_\{i\}\. Optimality atyiL<yiHy\_\{i\}^\{L\}<y\_\{i\}^\{H\}gives two inequalities whose sum yields\(yiH−yiL\)​\(xiσH−xiσL\)≥0\(y\_\{i\}^\{H\}\-y\_\{i\}^\{L\}\)\(x\_\{i\}^\{\\sigma\_\{H\}\}\-x\_\{i\}^\{\\sigma\_\{L\}\}\)\\geq 0\. A fixed tie\-breaking rule resolves score ties\. ∎

###### Theorem 2\(DSIC in expectation and click\-contingent IR\)\.

A monotone expected\-click allocation, when paired with the envelope payment, yields a mechanism that is DSIC in expectation and satisfies click\-contingent IR\.

###### Proof sketch\.

Under the envelope payment,U​\(θ;θ\)−U​\(θ;r\)=∫rθ\[x​\(z\)−x​\(r\)\]​𝑑z≥0U\(\\theta;\\theta\)\-U\(\\theta;r\)=\\int\_\{r\}^\{\\theta\}\[x\(z\)\-x\(r\)\]\\,dz\\geq 0forr<θr<\\thetaby monotonicity, and symmetrically forr\>θr\>\\theta\. IR follows fromPiENV≤θiP\_\{i\}^\{\\mathrm\{ENV\}\}\\leq\\theta\_\{i\}\. ∎

For the learned policy, Theorem[2](https://arxiv.org/html/2608.00123#Thmtheorem2)requiresx^i​\(⋅,b−i\)\\widehat\{x\}\_\{i\}\(\\cdot,b\_\{\-i\}\)to be monotone and paired with its exact envelope payment\.

###### Lemma 3\(Stopping stability\)\.

Supposesuph,b\|Q^ϕ,tS−QtS\|≤εtS\\sup\_\{h,b\}\|\\widehat\{Q\}\_\{\\phi,t\}^\{S\}\-Q\_\{t\}^\{S\}\|\\leq\\varepsilon\_\{t\}^\{S\}andsuph,b\|Q^ϕ,tW−QtW\|≤εtW\\sup\_\{h,b\}\|\\widehat\{Q\}\_\{\\phi,t\}^\{W\}\-Q\_\{t\}^\{W\}\|\\leq\\varepsilon\_\{t\}^\{W\}\. The ideal and learned decisions can differ only when

\|QtS−QtW\|≤εtS\+εtW\.\|Q\_\{t\}^\{S\}\-Q\_\{t\}^\{W\}\|\\leq\\varepsilon\_\{t\}^\{S\}\+\\varepsilon\_\{t\}^\{W\}\.Writinget=εtS\+εtWe\_\{t\}=\\varepsilon\_\{t\}^\{S\}\+\\varepsilon\_\{t\}^\{W\}, the value lost by followingτ^\\widehat\{\\tau\}instead ofτ⋆\\tau^\{\\star\}is at most∑s=tTγs−t​es\\sum\_\{s=t\}^\{T\}\\gamma^\{s\-t\}e\_\{s\}\.

###### Proposition 4\(Conditional approximate\-IC transfer\)\.

If the implemented pair\(xiimp,miimp\)\(x\_\{i\}^\{\\mathrm\{imp\}\},m\_\{i\}^\{\\mathrm\{imp\}\}\)satisfies‖xiimp−xi‖∞≤δx\\\|x\_\{i\}^\{\\mathrm\{imp\}\}\-x\_\{i\}\\\|\_\{\\infty\}\\leq\\delta\_\{x\}and‖miimp−mi‖∞≤δm\\\|m\_\{i\}^\{\\mathrm\{imp\}\}\-m\_\{i\}\\\|\_\{\\infty\}\\leq\\delta\_\{m\}for an ideal DSIC pair\(xi,mi\)\(x\_\{i\},m\_\{i\}\), then

supri\[Uiimp​\(θi;ri,b−i\)−Uiimp​\(θi;θi,b−i\)\]≤2​θ¯i​δx\+2​δm\.\\sup\_\{r\_\{i\}\}\\bigl\[U\_\{i\}^\{\\mathrm\{imp\}\}\(\\theta\_\{i\};r\_\{i\},b\_\{\-i\}\)\-U\_\{i\}^\{\\mathrm\{imp\}\}\(\\theta\_\{i\};\\theta\_\{i\},b\_\{\-i\}\)\\bigr\]\\leq 2\\overline\{\\theta\}\_\{i\}\\delta\_\{x\}\+2\\delta\_\{m\}\.

Hereδx\\delta\_\{x\}covers allocation and click\-probability errors, whileδm\\delta\_\{m\}also covers rollout and integration errors\. The reported finite\-grid regret does not estimate this uniform bound\.

## Experiments

We evaluate LLM\-OSDA on a simulated conversational\-advertising corpus, testing whether the incentive and revenue properties established in theory hold for the learned mechanism in practice\.

### Experimental Setup

#### Data\.

Our simulated corpus contains 14,918 three\-turn shopping dialogues over 3,000 user profiles and 100 advertisements from 50 product categories drawn from Amazon Reviews 2023, with 2,983 held out for testing\. Click labels are produced by a user\-simulation judge in the LLM\-as\-a\-judge paradigm\(Zhenget al\.[2023](https://arxiv.org/html/2608.00123#bib.bib39)\)\. All users, dialogues, clicks, and bids are simulated; no personally identifiable information or proprietary auction data is used\.

#### Models and training\.

The quality modelGηG\_\{\\eta\}is a Qwen3\-Embedding\-0\.6B encoder\(Yang and others[2025](https://arxiv.org/html/2608.00123#bib.bib45)\)followed by a two\-layer MLP, trained by binary cross\-entropy on\(ht,ai,click\)\(h\_\{t\},a\_\{i\},\\text\{click\}\)tuples to produceq^η,i\\widehat\{q\}\_\{\\eta,i\}\. StopNetQ^ϕ\\widehat\{Q\}\_\{\\phi\}is an MLP with a shared trunk and stop and wait action\-value heads, trained by MSE against the corresponding Bellman action\-value targets\. The rendererπR\\pi\_\{R\}is Qwen3\-4B fine\-tuned on chat\-formatted trajectories\. Bids are stripped at data\-loading time so neitherGηG\_\{\\eta\}norπR\\pi\_\{R\}sees them\. The experiments use the identity scoreψ​\(b\)=b\\psi\(b\)=b\. Detailed hyperparameters are provided in the Appendix\.

![Refer to caption](https://arxiv.org/html/2608.00123v1/x2.png)Figure 3:Net revenue before and after iterative refinement\.
#### Baselines\.

All methods share the sameGηG\_\{\\eta\}, allocation rule, and envelope payment, differing only in the trigger turn: LLM\-OSDA uses learned stopping, while the three baselines use a fixed choice:Always\-Round\-1,Always\-Last\-Round, andRandom\-Round\. Always\-Round\-1 is a single\-turn baseline representative of prior LLM\-ad designs, so the gap to it isolates the value of multi\-turn timing\. Each method is evaluated under Rounds 1 and 2 with base and refined renderers\.

#### Metrics\.

Net revenue and reward are the headline platform metrics; the other columns are diagnostic \(see the Appendix for all definitions\)\. Click and retention are scored by an independent User\-LLM judge, not by the pCTR modelGηG\_\{\\eta\}that guides the mechanism, so the reported revenue is not a self\-evaluation\.

### Main Results

Table[1](https://arxiv.org/html/2608.00123#Sx4.T1)compares LLM\-OSDA against three timing baselines under the Round\-2 mechanism, and Figure[3](https://arxiv.org/html/2608.00123#Sx6.F3)isolates the effect of iterative refinement\.

#### Payment computation\.

In the per\-session replay used here, each fixed\-timing allocation curve has one observed jump, so its envelope payment coincides with the critical\-report threshold up to numerical error\. For LLM\-OSDA, where the learned stopping rule couples with the bid, we compute the envelope CPC by numerical integration over a 64\-point bid grid\. The resulting payments are reported in the Pay column\.

#### Revenue comparison\.

LLM\-OSDA \(refined\) attains the highest Net Revenue at 0\.818 and Reward at 0\.784 \(net revenue minusλ​\(1−γ¯\)\\lambda\(1\-\\bar\{\\gamma\}\),λ=0\.25\\lambda=0\.25\), an11%11\\%net\-revenue gain over the single\-turn Always\-Round\-1 \(0\.733\)\. A stronger adaptive baseline that stops on a pCTR\-revenue threshold also trails LLM\-OSDA \(appendix\), indicating the gain comes from anticipating later turns rather than adaptivity alone\.

The Trigger column reports the average insertion round\. LLM\-OSDA’s value 1\.39 lies strictly between Always\-Round\-1 at 1\.00 and Always\-Last\-Round at 2\.19, confirming that the mechanism exercises timing rather than defaulting to either extreme\.

Predicted CTR is0\.8410\.841for LLM\-OSDA, slightly below Always\-Round\-1 refined at0\.8490\.849, since a turn\-1 insertion faces a fresh conversation with naturally high click rate\. Yet Always\-Round\-1 trails on Net Revenue because fixing insertion at turn 1 removes the option to wait for a stronger competitor; its payment0\.8780\.878lies12%12\\%below LLM\-OSDA’s envelope CPC\.

Two orthogonal effects drive these numbers\. Iterative refinement lifts revenue by 14 to 19% across every timing policy \(Figure[3](https://arxiv.org/html/2608.00123#Sx6.F3)\)\. At fixed timing, replacing the base renderer with the refined one adds a further 6 to 7% revenue\.

### Mechanism Diagnostics

We diagnose how closely the learned system approaches the ideal mechanism and whether its behavior is attributable to the stopping rule rather than sampling artifacts\.

#### Expected\-click monotonicity\.

Incentive compatibility under the envelope payment requires the theoretical learned allocationx^i​\(z\)\\widehat\{x\}\_\{i\}\(z\)to be nondecreasing inzz\. We test its finite\-rollout estimatex^i\(R\)​\(z\)\\widehat\{x\}\_\{i\}^\{\(R\)\}\(z\)by replaying the full mechanism at 64 bid probes per session\. For fixed\-timing policies \(Always\-Round\-1, Always\-Last\-Round, Random\-Round\), the estimated curve is a step function and monotonicity holds in100%100\\%of sessions, since stopping is bid\-independent\. For LLM\-OSDA,88\.5%88\.5\\%of sessions are monotone on the probed grid\. The remaining11\.5%11\.5\\%exhibit violations concentrated at later trigger turns \(turn\-2:77\.1%77\.1\\%; turn\-3:63\.0%63\.0\\%\)\. Figure[4](https://arxiv.org/html/2608.00123#Sx6.F4)shows representative curves\.

![Refer to caption](https://arxiv.org/html/2608.00123v1/x3.png)Figure 4:Finite\-rollout expected\-click estimatex^i\(R\)​\(z\)\\widehat\{x\}\_\{i\}^\{\(R\)\}\(z\)vs\. bid for eight sessions under LLM\-OSDA\. Dashed red: winner’s true bid\.
#### Approximate incentive compatibility\.

Under the envelope payment, we measure finite\-grid empirical regretmaxz⁡u​\(θ,z\)−u​\(θ,θ\)\\max\_\{z\}u\(\\theta,z\)\-u\(\\theta,\\theta\)for each session\. Overall,89%89\\%of sessions have zero observed regret, mean regret is0\.0200\.020, and the 95th percentile is0\.0800\.080\. Figure[5](https://arxiv.org/html/2608.00123#Sx6.F5)decomposes this by trigger turn\. Sessions stopped at turn 1 have low non\-monotonicity \(4\.6%4\.6\\%\) and small regret, whereas later turns accumulate more StopNet error near stopping boundaries\. Fixed\-timing baselines have zero regret on the evaluated grid\. These measurements do not establish a uniform theoretical approximation bound\.

![Refer to caption](https://arxiv.org/html/2608.00123v1/x4.png)Figure 5:Approximate\-IC diagnostics: non\-monotone rate and mean regret by trigger turn\.
#### Envelope vs\. critical\-bid payment\.

For the observed single\-jump fixed\-timing curves, the envelope CPC and critical\-report threshold coincide up to numerical precision \(gap<0\.001<0\.001\)\. For LLM\-OSDA the envelope CPC is on average5\.1%5\.1\\%higher than the critical\-bid value \(Figure[6](https://arxiv.org/html/2608.00123#Sx6.F6)\)\. This is why LLM\-OSDA’s Pay in Table[1](https://arxiv.org/html/2608.00123#Sx4.T1)exceeds the fixed\-timing baselines: because bid\-dependent stopping makes the allocation curve jump more than once, the envelope integrates the global curve and prices the option value of timing, which a single pointwise threshold cannot capture\. Both payment rules satisfy empirical individual rationality on the evaluated profiles: no session hasPiENV\>θiP\_\{i\}^\{\\mathrm\{ENV\}\}\>\\theta\_\{i\}\.

![Refer to caption](https://arxiv.org/html/2608.00123v1/x5.png)Figure 6:Envelope CPC vs\. critical\-bid CPC for LLM\-OSDA sessions\. \(a\) Scatter; diagonal = equality\. \(b\) Gap distribution \(mean\+0\.049\+0\.049\)\.
#### Timing rationality\.

An arbitrary stopping policy could in principle match Always\-Round\-1 in expectation\. Two observations rule this out\. First, LLM\-OSDA’s trigger\-turn distribution is non\-degenerate, with 65% of sessions stopped at turn 1, 32% at turn 2, and 4% at turn 3, for an average trigger turn of1\.391\.39, strictly between Always\-Round\-1 \(1\.00\) and Always\-Last\-Round \(2\.19\)\. Second, the winner’s contextual quality at the exercised turn is elevated relative to a first\-turn counterfactual on the same session, indicating that stopping is driven by rising contextual quality rather than stochastic variation\.

#### User\-side experience\.

Deferring insertion to a later turn could in principle degrade the user experience by prolonging the conversation, but the data show no such effect\. LLM\-OSDA refined attains an average retentionγ¯=0\.865\\bar\{\\gamma\}=0\.865, statistically indistinguishable from Always\-Round\-1 refined \(γ¯=0\.866\\bar\{\\gamma\}=0\.866\) despite triggering 0\.39 turns later on average, and measurably above Always\-Last\-Round \(γ¯=0\.842\\bar\{\\gamma\}=0\.842\)\. Deferring past turn 1 is therefore essentially costless in user retention, resolving the early\-versus\-late trade\-off that motivates this work\.

### Ablation Study

We ablate two design decisions: the mechanism\-side layering assumptions \(Theorem[2](https://arxiv.org/html/2608.00123#Thmtheorem2)\) and the LLM\-side turn\-level click quality estimator\.

#### Layering ablation\.

The monotonicity guarantee of Theorem[2](https://arxiv.org/html/2608.00123#Thmtheorem2)relies on the layering assumptions A2–A4, each keeping one channel bid\-independent; violating any of them lets the bid leak into a channel that should be bid\-free\. We break one assumption at a time and read the metric it predicts \(Table[2](https://arxiv.org/html/2608.00123#Sx6.T2)\)\. bid\-in\-pCTR appendsbib\_\{i\}toGηG\_\{\\eta\}, breaking A2: the bid–pCTR correlation\|r\|\|r\|rises to0\.580\.58\. llm\-stop replaces the committed action\-value comparison with a Qwen3\-4B STOP/WAIT model, breaking A3: only18%18\\%of its stop decisions agree with the committed rule\. bid\-in\-renderer injects the bid into the renderer prompt, breaking A4: renderer\|r\|=0\.42\|r\|=0\.42\. All three exhibit clear bid leakage, confirming that the layering assumptions are necessary for monotonicity and hence for DSIC\.

Table 2:Layering ablation\.\|r\|\|r\|: Pearson\|r\|​\(log⁡b,q\)\|r\|\(\\log b,q\); Rule agree\.: fraction of stop decisions matching the committed action\-value rule\. Dashes coincide with the baseline by design\.
#### Quality estimator ablation\.

Removing dialogue\-history and turn encoding fromGηG\_\{\\eta\}collapses the quality estimator to a static, session\-level pCTR\. Comparing three ablations against the full model isolates how much of the CTR gain in Table[1](https://arxiv.org/html/2608.00123#Sx4.T1)depends on dynamic intent perception\. The full model reaches validation AUC 0\.932; keeping only current\-turn text drops it to 0\.908; removing all dialogue context \(ad features only\) drops it to 0\.689, a 24\-point AUC gap, confirming that most of the quality signal comes from dialogue context rather than ad features alone\. Turn\-number embedding alone contributes marginally \(0\.935 vs\. 0\.932\), so the intent signal comes primarily from the dialogue history rather than the turn index\.

## Conclusion

We introduced LLM\-OSDA, a dynamic mechanism for multi\-turn native advertising with three committed components: a Bellman stopping rule, a bid\-independent LLM quality layer, and an envelope CPC computed from the expected\-click allocation\. The ideal Bellman mechanism is DSIC in expectation and click\-contingently IR; the learned StopNet admits stopping\-stability and error\-to\-IC guarantees, with the single\-turn limit separating the welfare and Myerson auctions\.

Empirically, LLM\-OSDA lifts net revenue by11%11\\%over fixed\-timing baselines with no retention drop, and finite\-grid diagnostics find low empirical regret concentrated near learned stopping boundaries\. Ablations further attribute the gain to bid\-dependent timing rather than adaptivity alone, confirming that the learned mechanism exercises the timing option its theory prices\. We leave to future work multi\-insertion sessions, monotonic StopNet architectures, uniform approximation certificates, and validation on production data\.

## References

- G\. Aggarwal, A\. Goel, and R\. Motwani \(2006\)Truthful auctions for pricing search keywords\.InProceedings of the 7th ACM Conference on Electronic Commerce \(EC\),pp\. 1–7\.Cited by:[Static auction design\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px1.p1.1)\.
- S\. Alaei, A\. Makhdoumi, and A\. Malekian \(2026\)Dynamic learning and optimal advertising mechanism for llm platforms\.Available at SSRN 6212838\.Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p4.1),[Timing and intent across turns\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px4.p1.1)\.
- S\. Athey and I\. Segal \(2013\)An efficient dynamic mechanism\.Econometrica81\(6\),pp\. 2463–2485\.Cited by:[Dynamic mechanisms with endogenous timing\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px2.p1.1)\.
- S\. R\. Balseiro and Y\. Gur \(2019\)Learning in repeated auctions with budgets: regret minimization and equilibrium\.Management Science65\(9\),pp\. 3952–3968\.Cited by:[Static auction design\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px1.p1.1)\.
- M\. Banchio, A\. Mehta, and A\. Perlroth \(2024\)Ads in conversations\.External Links:2403\.11022Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p4.1),[Timing and intent across turns\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px4.p1.1)\.
- D\. Bergemann and J\. Välimäki \(2019\)Dynamic mechanism design: an introduction\.Journal of Economic Literature57\(2\),pp\. 235–274\.Cited by:[Dynamic mechanisms with endogenous timing\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px2.p1.1)\.
- K\. Bhawalkar, A\. Psomas, and D\. Wang \(2025\)Sponsored questions and how to auction them\.External Links:2512\.03975Cited by:[Ad auctions coupled with LLM generation\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px3.p1.1)\.
- S\. Board \(2007\)Selling options\.Journal of Economic Theory136\(1\),pp\. 324–340\.Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p4.1),[Dynamic mechanisms with endogenous timing\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px2.p1.1)\.
- T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.\(2020\)Language models are few\-shot learners\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p1.1)\.
- S\. Dey, H\. Wu, and B\. Li \(2025\)To judge or not to judge: using llm judgements for advertiser keyphrase relevance at ebay\.External Links:2505\.04209Cited by:[Ad auctions coupled with LLM generation\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px3.p1.1)\.
- A\. Dubey, Z\. Feng, R\. Kidambi, A\. Mehta, and D\. Wang \(2024\)Auctions with LLM summaries\.InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 713–722\.Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p2.1),[Ad auctions coupled with LLM generation\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px3.p1.1)\.
- P\. Dütting, Z\. Feng, H\. Narasimhan, D\. C\. Parkes, and S\. S\. Ravindranath \(2024a\)Optimal auctions through deep learning: advances in differentiable economics\.Journal of the ACM71\(1\),pp\. 1–53\.External Links:[Document](https://dx.doi.org/10.1145/3630749)Cited by:[Ad auctions coupled with LLM generation\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px3.p1.1)\.
- P\. Dütting, V\. Mirrokni, R\. Paes Leme, H\. Xu, and S\. Zuo \(2024b\)Mechanism design for large language models\.InProceedings of the ACM Web Conference 2024,pp\. 144–155\.Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p2.1),[Ad auctions coupled with LLM generation\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px3.p1.1)\.
- B\. Edelman, M\. Ostrovsky, and M\. Schwarz \(2007\)Internet advertising and the generalized second\-price auction: selling billions of dollars worth of keywords\.American Economic Review97\(1\),pp\. 242–259\.Cited by:[Static auction design\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px1.p1.1)\.
- S\. Feizi, M\. Hajiaghayi, K\. Rezaei, and S\. Shin \(2026\)Online advertisements with LLMs: opportunities and challenges\.ACM SIGecom Exchanges22\(2\),pp\. 66–81\.Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p1.1)\.
- L\. Friedman, S\. Ahuja, D\. Allen, Z\. Tan, H\. Sidahmed, C\. Long, J\. Xie, G\. Schubiner, A\. Patel, H\. Lara,et al\.\(2023\)Leveraging large language models in conversational recommender systems\.External Links:2305\.07961Cited by:[Timing and intent across turns\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px4.p1.1)\.
- M\. Hajiaghayi, S\. Lahaie, K\. Rezaei, and S\. Shin \(2024\)Ad auctions for LLMs via retrieval\-augmented generation\.InProceedings of the 38th International Conference on Neural Information Processing Systems,Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p2.1),[Ad auctions coupled with LLM generation\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px3.p1.1)\.
- M\. Han, S\. A\. Esmaeili, M\. Albert, and H\. Xu \(2025\)Data auctions for retrieval augmented generation\.External Links:2508\.16007Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p2.1),[Ad auctions coupled with LLM generation\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px3.p1.1)\.
- F\. He, M\. Du, A\. Filos\-Ratsikas, L\. Cheng, Q\. Song, M\. Lin, and J\. Vines \(2024\)AI driven online advertising: market design, generative ai, and ethics\.InCompanion Proceedings of the ACM Web Conference 2024,WWW ’24,New York, NY, USA,pp\. 1407–1409\.External Links:ISBN 9798400701726,[Link](https://doi.org/10.1145/3589335.3641295),[Document](https://dx.doi.org/10.1145/3589335.3641295)Cited by:[Ad auctions coupled with LLM generation\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px3.p1.1)\.
- Z\. He, Z\. Xie, R\. Jha, H\. Steck, D\. Liang, Y\. Feng, B\. P\. Majumder, N\. Kallus, and J\. McAuley \(2023\)Large language models as zero\-shot conversational recommenders\.InProceedings of the 32nd ACM International Conference on Information and Knowledge Management \(CIKM\),pp\. 720–730\.Cited by:[Timing and intent across turns\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px4.p1.1)\.
- Y\. Hou, J\. Li, Z\. He, A\. Yan, X\. Chen, and J\. McAuley \(2024\)Bridging language and items for retrieval and recommendation\.External Links:2403\.03952Cited by:[Appendix C](https://arxiv.org/html/2608.00123#A3.SSx2.p1.9)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.\(2020\)Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in Neural Information Processing Systems \(NeurIPS\)33,pp\. 9459–9474\.Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p2.1),[Ad auctions coupled with LLM generation\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px3.p1.1)\.
- T\. Liu, Z\. Wang, M\. Qin, Z\. Lu, X\. Chen, Y\. Yang, and P\. Shu \(2025\)Real\-time ad retrieval via LLM\-generative commercial intention for sponsored search advertising\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 28936–28948\.Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p2.1)\.
- D\. Metzler, Y\. Tay, D\. Bahri, and M\. Najork \(2021\)Rethinking search: making domain experts out of dilettantes\.ACM SIGIR Forum55\(1\),pp\. 1–27\.Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p1.1)\.
- P\. Milgrom and I\. Segal \(2002\)Envelope theorems for arbitrary choice sets\.Econometrica70\(2\),pp\. 583–601\.Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p4.1),[Dynamic mechanisms with endogenous timing\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px2.p1.1)\.
- R\. B\. Myerson \(1981\)Optimal auction design\.Mathematics of Operations Research6\(1\),pp\. 58–73\.Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p4.1),[Static auction design\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px1.p1.1),[Envelope pricing\.](https://arxiv.org/html/2608.00123#Sx4.SSx5.p1.7)\.
- J\. S\. Park, J\. C\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein \(2023\)Generative agents: interactive simulacra of human behavior\.InProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology \(UIST\),Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p1.1)\.
- A\. Pavan, I\. Segal, and J\. Toikka \(2014\)Dynamic mechanism design: a myersonian approach\.Econometrica82\(2\),pp\. 601–653\.Cited by:[Dynamic mechanisms with endogenous timing\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px2.p1.1)\.
- G\. Peskir and A\. Shiryaev \(2006\)Optimal stopping and free\-boundary problems\.Birkhäuser\.Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p4.1),[Dynamic mechanisms with endogenous timing\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px2.p1.1)\.
- R\. Pope, S\. Douglas, A\. Chowdhery, J\. Devlin, J\. Bradbury, J\. Heek, K\. Xiao, S\. Agrawal, and J\. Dean \(2023\)Efficiently scaling transformer inference\.Proceedings of Machine Learning and Systems \(MLSys\)5\.Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p1.1)\.
- T\. Qin, F\. He, D\. Shi, W\. Huang, and D\. Tao \(2022\)Benefits of permutation\-equivariance in auction mechanisms\.InProceedings of the 36th International Conference on Neural Information Processing Systems,NIPS ’22,Red Hook, NY, USA\.External Links:ISBN 9781713871088Cited by:[Static auction design\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px1.p1.1)\.
- E\. Soumalias, M\. J\. Curry, and S\. Seuken \(2024\)Truthful aggregation of LLMs with an application to online advertising\.InAgentic Markets Workshop at ICML 2024,External Links:[Link](https://openreview.net/forum?id=Pp6483Ma1m)Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p3.1),[Ad auctions coupled with LLM generation\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px3.p1.1)\.
- H\. Sun, X\. Song, X\. Zhang, Z\. Chen, X\. Chu, Z\. Zhang, C\. Yu, J\. Xu, B\. Zheng, and X\. Deng \(2026\)LERA: llm\-enhanced rag for ad auction in generative chatbots\.External Links:2605\.16474Cited by:[Ad auctions coupled with LLM generation\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px3.p1.1)\.
- H\. R\. Varian \(2007\)Position auctions\.International Journal of Industrial Organization25\(6\),pp\. 1163–1178\.Cited by:[Static auction design\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px1.p1.1)\.
- X\. Wang, X\. Tang, X\. Zhao, J\. Wang, and J\. Wen \(2023\)Rethinking the evaluation for conversational recommendation in the era of large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 10052–10065\.Cited by:[Timing and intent across turns\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px4.p1.1)\.
- A\. Yanget al\.\(2025\)Qwen3 technical report\.External Links:2505\.09388Cited by:[Models and training\.](https://arxiv.org/html/2608.00123#Sx6.SSx1.SSS0.Px2.p1.8)\.
- C\. Zhao, Q\. Hu, S\. Song, D\. Chen, H\. Zhu, J\. Xu, and B\. Zheng \(2026\)LLM\-auction: generative auction towards llm\-native advertising\.External Links:2512\.10551Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p3.1),[Ad auctions coupled with LLM generation\.](https://arxiv.org/html/2608.00123#Sx2.SS0.SSS0.Px3.p1.1)\.
- W\. X\. Zhao, K\. Zhou, J\. Li, T\. Tang, X\. Wang, Y\. Hou, Y\. Min, B\. Zhang, J\. Zhang, Z\. Dong,et al\.\(2023\)A survey of large language models\.External Links:2303\.18223Cited by:[Introduction](https://arxiv.org/html/2608.00123#Sx1.p1.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging LLM\-as\-a\-judge with MT\-bench and chatbot arena\.InAdvances in Neural Information Processing Systems \(NeurIPS\) Datasets and Benchmarks Track,Cited by:[Appendix C](https://arxiv.org/html/2608.00123#A3.SSx2.p1.9),[Data\.](https://arxiv.org/html/2608.00123#Sx6.SSx1.SSS0.Px1.p1.1)\.

## Appendix ANotation

Table 3:Notation\. Hats denote learned or estimated quantities; a star denotes the ideal \(exact\-Bellman\) benchmark\.
## Appendix BProofs of Theoretical Guarantees

This appendix proves the theoretical guarantees stated in the main text\. The exact monotonicity and DSIC results apply to the ideal Bellman mechanism\. The learned StopNet is covered by a stopping\-stability result and a conditional error\-to\-IC transfer\. Its finite\-grid monotonicity and regret measurements are empirical diagnostics, not exact guarantees\.

### Proof of Lemma 1 \(Expected\-Click Monotonicity\)

###### Proof\.

Fix advertiseriiandb−ib\_\{\-i\}\. LetΣ\\Sigmabe the set of feasible history\-contingent stopping\-allocation plans\. Eachσ∈Σ\\sigma\\in\\Sigmais independent of the counterfactual scoreyi=ψi​\(bi\)y\_\{i\}=\\psi\_\{i\}\(b\_\{i\}\)because transitions are bid\-independent\. Forσ∈Σ\\sigma\\in\\Sigma, define

xiσ\\displaystyle x\_\{i\}^\{\\sigma\}:=𝔼​\[γτσ−1​qi​\(Hτσ\)​𝟏​\{Iσ=i\}\],\\displaystyle=\\mathbb\{E\}\\\!\\left\[\\gamma^\{\\tau^\{\\sigma\}\-1\}q\_\{i\}\(H\_\{\\tau^\{\\sigma\}\}\)\\mathbf\{1\}\\\{I^\{\\sigma\}=i\\\}\\right\],Aiσ\\displaystyle A\_\{i\}^\{\\sigma\}:=𝔼​\[∑j≠iγτσ−1​ψj​\(bj\)​qj​\(Hτσ\)​𝟏​\{Iσ=j\}\]\.\\displaystyle=\\mathbb\{E\}\\\!\\left\[\\sum\_\{j\\neq i\}\\gamma^\{\\tau^\{\\sigma\}\-1\}\\psi\_\{j\}\(b\_\{j\}\)q\_\{j\}\(H\_\{\\tau^\{\\sigma\}\}\)\\mathbf\{1\}\\\{I^\{\\sigma\}=j\\\}\\right\]\.For fixedσ\\sigma, both terms are independent ofyiy\_\{i\}, and the objective isJσ​\(yi\)=Aiσ\+yi​xiσJ^\{\\sigma\}\(y\_\{i\}\)=A\_\{i\}^\{\\sigma\}\+y\_\{i\}x\_\{i\}^\{\\sigma\}\.

TakeyiL<yiHy\_\{i\}^\{L\}<y\_\{i\}^\{H\}, and letσL,σH\\sigma\_\{L\},\\sigma\_\{H\}be optimal at the two scores\. Optimality gives

AiσL\+yiL​xiσL\\displaystyle A\_\{i\}^\{\\sigma\_\{L\}\}\+y\_\{i\}^\{L\}x\_\{i\}^\{\\sigma\_\{L\}\}≥AiσH\+yiL​xiσH,\\displaystyle\\geq A\_\{i\}^\{\\sigma\_\{H\}\}\+y\_\{i\}^\{L\}x\_\{i\}^\{\\sigma\_\{H\}\},AiσH\+yiH​xiσH\\displaystyle A\_\{i\}^\{\\sigma\_\{H\}\}\+y\_\{i\}^\{H\}x\_\{i\}^\{\\sigma\_\{H\}\}≥AiσL\+yiH​xiσL\.\\displaystyle\\geq A\_\{i\}^\{\\sigma\_\{L\}\}\+y\_\{i\}^\{H\}x\_\{i\}^\{\\sigma\_\{L\}\}\.Adding yields\(yiH−yiL\)​\(xiσH−xiσL\)≥0\(y\_\{i\}^\{H\}\-y\_\{i\}^\{L\}\)\(x\_\{i\}^\{\\sigma\_\{H\}\}\-x\_\{i\}^\{\\sigma\_\{L\}\}\)\\geq 0\. Thus allocation is nondecreasing inyiy\_\{i\}\. Sinceψi\\psi\_\{i\}is nondecreasing and ties use a fixed rule, allocation is also nondecreasing inbib\_\{i\}\. ∎

### Proof of Theorem 2 \(Envelope DSIC and IR\)

###### Proof\.

Fixi,b−ii,b\_\{\-i\}, and writex​\(r\)x\(r\)for the expected\-click allocation at reportrr\. Under the envelope paymentm​\(r\)=r​x​\(r\)−∫θ¯irx​\(z\)​𝑑zm\(r\)=rx\(r\)\-\\int\_\{\\underline\{\\theta\}\_\{i\}\}^\{r\}x\(z\)\\,dz, typeθ\\thetahas

U​\(θ;θ\)−U​\(θ;r\)=\{∫rθ\[x​\(z\)−x​\(r\)\]​𝑑z,r<θ,∫θr\[x​\(r\)−x​\(z\)\]​𝑑z,r\>θ\.U\(\\theta;\\theta\)\-U\(\\theta;r\)=\\begin\{cases\}\\displaystyle\\int\_\{r\}^\{\\theta\}\[x\(z\)\-x\(r\)\]\\,dz,&r<\\theta,\\\\\[5\.69054pt\] \\displaystyle\\int\_\{\\theta\}^\{r\}\[x\(r\)\-x\(z\)\]\\,dz,&r\>\\theta\.\\end\{cases\}Monotonicity ofxxmakes both expressions nonnegative, so truthful reporting is weakly dominant in expectation over the bid\-independent dialogue uncertainty\. Ifx​\(θ\)\>0x\(\\theta\)\>0, thenPENV​\(θ\)=θ−\[∫θ¯iθx​\(z\)​𝑑z\]/x​\(θ\)≤θP^\{\\mathrm\{ENV\}\}\(\\theta\)=\\theta\-\[\\int\_\{\\underline\{\\theta\}\_\{i\}\}^\{\\theta\}x\(z\)\\,dz\]/x\(\\theta\)\\leq\\theta\. A realized click therefore gives nonnegative utility, while no click incurs no charge\. Ifx​\(θ\)=0x\(\\theta\)=0, the CPC is defined as zero\. ∎

### Proof of Lemma 3 \(Stopping Stability\)

###### Proof\.

LetΔt=QtS−QtW\\Delta\_\{t\}=Q\_\{t\}^\{S\}\-Q\_\{t\}^\{W\}andΔ^t=Q^ϕ,tS−Q^ϕ,tW\\widehat\{\\Delta\}\_\{t\}=\\widehat\{Q\}\_\{\\phi,t\}^\{S\}\-\\widehat\{Q\}\_\{\\phi,t\}^\{W\}\. The two uniform error assumptions imply

\|Δ^t−Δt\|≤εtS\+εtW=:et\.\|\\widehat\{\\Delta\}\_\{t\}\-\\Delta\_\{t\}\|\\leq\\varepsilon\_\{t\}^\{S\}\+\\varepsilon\_\{t\}^\{W\}=:e\_\{t\}\.Opposite action choices require the two margins to have opposite signs, which is possible only if\|Δt\|≤et\|\\Delta\_\{t\}\|\\leq e\_\{t\}\. LetDtD\_\{t\}be the worst\-case loss from following the learned policy from turntt, withDT\+1=0D\_\{T\+1\}=0\. Greedy selection using the approximate action values loses at mostete\_\{t\}against the better exact action\. If the learned policy waits, its future decisions add at mostγ​Dt\+1\\gamma D\_\{t\+1\}\. HenceDt≤et\+γ​Dt\+1D\_\{t\}\\leq e\_\{t\}\+\\gamma D\_\{t\+1\}, and backward induction givesDt≤∑s=tTγs−t​esD\_\{t\}\\leq\\sum\_\{s=t\}^\{T\}\\gamma^\{s\-t\}e\_\{s\}\. ∎

### Proof of Proposition 4 \(Approximate Incentive Compatibility\)

###### Proof\.

LetU​\(θ;r\)=θ​x​\(r\)−m​\(r\)U\(\\theta;r\)=\\theta x\(r\)\-m\(r\)andUimp​\(θ;r\)=θ​ximp​\(r\)−mimp​\(r\)U^\{\\mathrm\{imp\}\}\(\\theta;r\)=\\theta x^\{\\mathrm\{imp\}\}\(r\)\-m^\{\\mathrm\{imp\}\}\(r\)\. For any reportrr, ideal DSIC and the triangle inequality give

Uimp​\(θ;r\)\\displaystyle U^\{\\mathrm\{imp\}\}\(\\theta;r\)−Uimp​\(θ;θ\)≤θ​\|ximp​\(r\)−x​\(r\)\|\\displaystyle\-U^\{\\mathrm\{imp\}\}\(\\theta;\\theta\)\\leq\\theta\|x^\{\\mathrm\{imp\}\}\(r\)\-x\(r\)\|\+θ​\|ximp​\(θ\)−x​\(θ\)\|\+\|mimp​\(r\)−m​\(r\)\|\\displaystyle\+\\theta\|x^\{\\mathrm\{imp\}\}\(\\theta\)\-x\(\\theta\)\|\+\|m^\{\\mathrm\{imp\}\}\(r\)\-m\(r\)\|\+\|mimp​\(θ\)−m​\(θ\)\|≤2​θ¯i​δx\+2​δm\.\\displaystyle\+\|m^\{\\mathrm\{imp\}\}\(\\theta\)\-m\(\\theta\)\|\\leq 2\\overline\{\\theta\}\_\{i\}\\delta\_\{x\}\+2\\delta\_\{m\}\.∎

### Marginal\-Threshold Interpretation

###### Lemma 5\(Global marginal\-threshold decomposition\)\.

Supposexix\_\{i\}is a nondecreasing step function with jumpsΔ​xi​k\\Delta x\_\{ik\}at report thresholdszi​k∈\(θ¯i,bi\]z\_\{ik\}\\in\(\\underline\{\\theta\}\_\{i\},b\_\{i\}\]\. For anybib\_\{i\}withxi​\(bi\)\>0x\_\{i\}\(b\_\{i\}\)\>0, its envelope CPC is

PiENV​\(bi\)=θ¯i​xi​\(θ¯i\)\+∑zi​k≤bizi​k​Δ​xi​kxi​\(bi\)\.P\_\{i\}^\{\\mathrm\{ENV\}\}\(b\_\{i\}\)=\\frac\{\\underline\{\\theta\}\_\{i\}x\_\{i\}\(\\underline\{\\theta\}\_\{i\}\)\+\\sum\_\{z\_\{ik\}\\leq b\_\{i\}\}z\_\{ik\}\\Delta x\_\{ik\}\}\{x\_\{i\}\(b\_\{i\}\)\}\.Thus each threshold is weighted by the expected\-click increment that it creates across the entire history\-contingent mechanism\.

###### Proof\.

Writexi​\(bi\)=xi​\(θ¯i\)\+∑zi​k≤biΔ​xi​kx\_\{i\}\(b\_\{i\}\)=x\_\{i\}\(\\underline\{\\theta\}\_\{i\}\)\+\\sum\_\{z\_\{ik\}\\leq b\_\{i\}\}\\Delta x\_\{ik\}\. Integrating this step function and substituting it into the envelope identity gives

mi​\(bi\)\\displaystyle m\_\{i\}\(b\_\{i\}\)=bi​xi​\(bi\)−∫θ¯ibixi​\(z\)​𝑑z\\displaystyle=b\_\{i\}x\_\{i\}\(b\_\{i\}\)\-\\int\_\{\\underline\{\\theta\}\_\{i\}\}^\{b\_\{i\}\}x\_\{i\}\(z\)\\,dz=θ¯i​xi​\(θ¯i\)\+∑zi​k≤bizi​k​Δ​xi​k\.\\displaystyle=\\underline\{\\theta\}\_\{i\}x\_\{i\}\(\\underline\{\\theta\}\_\{i\}\)\+\\sum\_\{z\_\{ik\}\\leq b\_\{i\}\}z\_\{ik\}\\Delta x\_\{ik\}\.Dividing byxi​\(bi\)x\_\{i\}\(b\_\{i\}\)proves the result\. The thresholds are global because changing a report may alter both the stopping time and the winner across dialogue histories\. ∎

Ifxi​\(θ¯i\)=0x\_\{i\}\(\\underline\{\\theta\}\_\{i\}\)=0and the allocation curve has one jump atzicz\_\{i\}^\{c\}, the lemma reduces toPiENV=zicP\_\{i\}^\{\\mathrm\{ENV\}\}=z\_\{i\}^\{c\}\. This single\-jump case justifies a critical\-report interpretation\. Multiple timing\-induced jumps generally do not admit one local critical\-value formula\.

### Proof of Corollary[6](https://arxiv.org/html/2608.00123#Thmtheorem6)\(T=1T=1Special Case\)

###### Corollary 6\(T=1T=1special case\)\.

WhenT=1T=1, identity scoring yields a quality\-weighted welfare auction, and a Myerson virtual\-value score yields the corresponding Myerson auction\.

###### Proof\.

WhenT=1T=1,C​V1=0CV\_\{1\}=0, so no intertemporal stopping decision remains\. Conditional onh1h\_\{1\}, identity scoring choosesarg⁡maxi⁡bi​qi​\(h1\)\\arg\\max\_\{i\}b\_\{i\}q\_\{i\}\(h\_\{1\}\), which maximizes reported quality\-weighted welfare\. The state\-specific allocation is a single step, so its envelope CPC equals the corresponding critical report\. IfH1H\_\{1\}remains random at bidding time, the ex\-ante allocation averages these state\-specific curves and follows the global decomposition in Lemma[5](https://arxiv.org/html/2608.00123#Thmtheorem5)\. Under a regular Myerson scoreψi​\(bi\)\\psi\_\{i\}\(b\_\{i\}\), the mechanism instead maximizes nonnegative quality\-weighted virtual surplus\. This is the quality\-weighted Myerson auction, with the envelope payment determined in the original bid space\. ∎

## Appendix CAdditional Experiments

### Iterative Refinement

Algorithm[1](https://arxiv.org/html/2608.00123#alg1)details the round\-based training of the three learned components\(Gη,Q^ϕ,πR\)\(G\_\{\\eta\},\\widehat\{Q\}\_\{\\phi\},\\pi\_\{R\}\)summarized in the main text\. StopNetQ^ϕ=\(Q^ϕS,Q^ϕW\)\\widehat\{Q\}\_\{\\phi\}=\(\\widehat\{Q\}\_\{\\phi\}^\{S\},\\widehat\{Q\}\_\{\\phi\}^\{W\}\)predicts both Bellman action values\.

Algorithm 1Iterative Mechanism\-Aware Refinement1:Fit

Gη\(1\),Q^ϕS,\(1\),Q^ϕW,\(1\),πR\(1\)G\_\{\\eta\}^\{\(1\)\},\\widehat\{Q\}\_\{\\phi\}^\{S,\(1\)\},\\widehat\{Q\}\_\{\\phi\}^\{W,\(1\)\},\\pi\_\{R\}^\{\(1\)\}from initial rollouts

𝒟0\\mathcal\{D\}\_\{0\}\.

2:forround

k=2,3,…,Kk=2,3,\\ldots,Kdo

3:Initialize

𝒟k←∅\\mathcal\{D\}\_\{k\}\\leftarrow\\emptyset\.

4:foreach dialogue

d∈𝒟k−1d\\in\\mathcal\{D\}\_\{k\-1\}do

5:

y~d←πR\(k−1\)​\(d\)\\tilde\{y\}\_\{d\}\\leftarrow\\pi\_\{R\}^\{\(k\-1\)\}\(d\)\{regenerate response\}

6:

\(c~d,γ~d\)←Judge​\(d,y~d\)\(\\tilde\{c\}\_\{d\},\\tilde\{\\gamma\}\_\{d\}\)\\leftarrow\\text\{Judge\}\(d,\\tilde\{y\}\_\{d\}\)\{re\-score click/retention\}

7:

\{yd\(1\),…,yd\(N\)\}←πR\(k−1\)​\(d;N\)\\\{y\_\{d\}^\{\(1\)\},\\ldots,y\_\{d\}^\{\(N\)\}\\\}\\leftarrow\\pi\_\{R\}^\{\(k\-1\)\}\(d;N\)\{best\-of\-NNsampling\}

8:

yd⋆←arg⁡maxn⁡r​\(yd\(n\)\)y\_\{d\}^\{\\star\}\\leftarrow\\arg\\max\_\{n\}\\ r\(y\_\{d\}^\{\(n\)\}\)with

r=bi​q^η,ir=b\_\{i\}\\,\\widehat\{q\}\_\{\\eta,i\}\{expected click value\}

9:

𝒟k←𝒟k∪\{\(d,y~d,c~d,γ~d,yd⋆\)\}\\mathcal\{D\}\_\{k\}\\leftarrow\\mathcal\{D\}\_\{k\}\\cup\\\{\(d,\\tilde\{y\}\_\{d\},\\tilde\{c\}\_\{d\},\\tilde\{\\gamma\}\_\{d\},y\_\{d\}^\{\\star\}\)\\\}
10:endfor

11:Fit

Gη\(k\)G\_\{\\eta\}^\{\(k\)\}with

ℒBCE\\mathcal\{L\}\_\{\\mathrm\{BCE\}\}and both

Q^ϕS,\(k\),Q^ϕW,\(k\)\\widehat\{Q\}\_\{\\phi\}^\{S,\(k\)\},\\widehat\{Q\}\_\{\\phi\}^\{W,\(k\)\}with Bellman\-target

ℒMSE\\mathcal\{L\}\_\{\\mathrm\{MSE\}\}on

𝒟k\\mathcal\{D\}\_\{k\}\.

12:

πR\(k\)←SFT​\(πR\(k−1\);\{yd⋆\}d∈𝒟k−1\)\\pi\_\{R\}^\{\(k\)\}\\leftarrow\\text\{SFT\}\(\\pi\_\{R\}^\{\(k\-1\)\};\\\{y\_\{d\}^\{\\star\}\\\}\_\{d\\in\\mathcal\{D\}\_\{k\-1\}\}\)\.

13:endfor

14:return

\(Gη\(K\),Q^ϕ\(K\),πR\(K\)\)\(G\_\{\\eta\}^\{\(K\)\},\\widehat\{Q\}\_\{\\phi\}^\{\(K\)\},\\pi\_\{R\}^\{\(K\)\}\)

![Refer to caption](https://arxiv.org/html/2608.00123v1/x6.png)Figure 7:Running example\. Dialogue turns and the inserted response \(left\); per\-turn mechanism diagnosis \(right\)\.You are simulating a realistic e\-commerce user in a multi\-turn AI shopping conversation\. You must judge whether the assistant’s latest response would make the user click the inserted sponsored product and whether the response preserves user trust\. Use only the user’s stated needs and the dialogue context\. Do not reward a product merely because it has a high bid\. Penalize ads that are premature, irrelevant, forced, or distracting\. Reward ads that directly answer the user’s current need and are integrated naturally\. Return strict JSON only: \{clicked, click\_probability, gamma\_score, reason\}, where gamma\_score is the retention/trust score after seeing the response \(1 = satisfied and willing to continue, 0 = likely to leave\)\. User’s original query: \{query\} Dialogue history before the assistant’s current response: \{history\} Sponsored product inserted by the assistant: title \{title\}, ad id \{id\} Assistant’s current response: \{response\} Judge the response from the user’s perspective and return strict JSON only\.

Figure 8:User\-simulation judge system prompt and input template\. The advertiser bid is not included in the input\.
### Data Generation

The corpus is built from3,0003\{,\}000user profiles with up to four turns each\. The ad pool has100100products across5050categories \(22per category\), retrieved from the Amazon Reviews 2023 corpus\(Houet al\.[2024](https://arxiv.org/html/2608.00123#bib.bib44)\)\. Each session draws three candidate ads: two retrieved for relevance to the profile and one random\. Dialogues are synthesized with proprietary GPT\-family and Claude\-family chat models \(a user model, an assistant model, and a counterfactual generator\), and click and retention labels come from a proprietary GPT\-family judge under the LLM\-as\-a\-judge paradigm\(Zhenget al\.[2023](https://arxiv.org/html/2608.00123#bib.bib39)\)\. During synthesis the assistant may see candidate bids, reflecting a platform’s tilt toward higher\-paying ads, while the judge that assigns click and retention labels does not see the bid\. Advertiser private values lie inΘ=\[0\.1,4\.1\]\\Theta=\[0\.1,4\.1\]; base bids stored with the ad pool range over\[0\.14,4\.05\]\[0\.14,4\.05\]\(mean1\.301\.30, right\-skewed\)\. At evaluation each bid is perturbed multiplicatively,bi∼Unif​\[bibase​\(1−ρ\),bibase​\(1\+ρ\)\]b\_\{i\}\\sim\\mathrm\{Unif\}\[\\,b\_\{i\}^\{\\mathrm\{base\}\}\(1\-\\rho\),\\,b\_\{i\}^\{\\mathrm\{base\}\}\(1\+\\rho\)\\,\]withρ=0\.5\\rho=0\.5, so the StopNet cannot memorize ad\-specific bids\.

### Metric Definitions

The pCTR modelGηG\_\{\\eta\}sets the envelope CPCP^isENV,\(R\)=m^isENV,\(R\)/x^is\(R\)\\widehat\{P\}\_\{i\_\{s\}\}^\{\\mathrm\{ENV\},\(R\)\}=\\widehat\{m\}\_\{i\_\{s\}\}^\{\\mathrm\{ENV\},\(R\)\}/\\widehat\{x\}\_\{i\_\{s\}\}^\{\(R\)\}through the allocation curve, but the reported outcomes are scored by the independent judge: letcsc\_\{s\}be the judge’s click for sessionss\(zero when no ad is inserted\) andγ¯\\bar\{\\gamma\}the average judged retention\. Writingωs:=γτs−1​cs\\omega\_\{s\}:=\\gamma^\{\\tau\_\{s\}\-1\}c\_\{s\}for the discounted judged click, overSStest sessions

NetRev\\displaystyle\\mathrm\{NetRev\}=1S​∑s=1SP^isENV,\(R\)​ωs,\\displaystyle=\\frac\{1\}\{S\}\\sum\_\{s=1\}^\{S\}\\widehat\{P\}\_\{i\_\{s\}\}^\{\\mathrm\{ENV\},\(R\)\}\\omega\_\{s\},InfoRent\\displaystyle\\mathrm\{InfoRent\}=1S​∑s=1S\(bis−P^isENV,\(R\)\)​ωs,\\displaystyle=\\frac\{1\}\{S\}\\sum\_\{s=1\}^\{S\}\\bigl\(b\_\{i\_\{s\}\}\-\\widehat\{P\}\_\{i\_\{s\}\}^\{\\mathrm\{ENV\},\(R\)\}\\bigr\)\\omega\_\{s\},GrossRev\\displaystyle\\mathrm\{GrossRev\}=1S​∑s=1Sbis​ωs=NetRev\+InfoRent,\\displaystyle=\\frac\{1\}\{S\}\\sum\_\{s=1\}^\{S\}b\_\{i\_\{s\}\}\\omega\_\{s\}=\\mathrm\{NetRev\}\+\\mathrm\{InfoRent\},Reward\\displaystyle\\mathrm\{Reward\}=NetRev−λ​\(1−γ¯\),λ=0\.25\.\\displaystyle=\\mathrm\{NetRev\}\-\\lambda\(1\-\\bar\{\\gamma\}\),\\quad\\lambda=0\.25\.ThusGηG\_\{\\eta\}enters only the price, while the click that scales revenue is judged independently\.

### Training Hyperparameters

All components are trained with seed 42 over two refinement rounds, with discountγ=0\.95\\gamma=0\.95and retention weightλ=0\.25\\lambda=0\.25\. Sessions are split by session id rather than by turn, preventing leakage across turns of the same dialogue: with seed 42,20%20\\%of sessions are held out for test and12\.5%12\.5\\%of the remainder form the validation set, and the split is persisted for reuse\. The quality modelGηG\_\{\\eta\}\(hidden dim 256\) and the user\-simulation judge \(hidden dim 256\) are each trained for 5 epochs by binary cross\-entropy with learning rate10−310^\{\-3\}, batch size 256, and early\-stopping patience 1\. StopNetQ^ϕ\\widehat\{Q\}\_\{\\phi\}\(hidden dim 128\) is trained for 30 epochs by MSE against Bellman action\-value targets under the same learning rate, batch size, and patience\. The rendererπR\\pi\_\{R\}is Qwen3\-4B fully fine\-tuned for 1 epoch with AdamW at learning rate2×10−52\\times 10^\{\-5\}, per\-device batch size 4 and gradient accumulation 32 \(effective batch 128\), maximum sequence length 8192, warmup ratio 0\.05, gradient clipping at 1\.0, and bf16 precision; best\-of\-NNsampling uses temperature 0\.2 and generates at most 128 new tokens\. All training runs on two NVIDIA H200 GPUs in PyTorch\.

### Payment Computation

The envelope CPC is computed offline per session\. We build a6464\-point bid grid bylinspaceoverΘ\\Thetaaugmented with the winner’s bid, and replay the committed mechanism at each grid point\. Replays reuse a frozen evaluation schedule \(the same dialogue outcomes across grid points\), so the allocation curvex^i\(R\)​\(z\)\\widehat\{x\}\_\{i\}^\{\(R\)\}\(z\)is deterministic given the session rather than averaged over fresh random rollouts\. The envelope transfer∫xi​\(z\)​𝑑z\\int x\_\{i\}\(z\)\\,dzis evaluated by piecewise\-constant \(left\-Riemann\) integration, which is exact for the step\-shaped allocation\. All payment runs use seed 42\.

You are a helpful AI shopping assistant with advertising capabilities\. \#\# Your Role You help users with shopping questions while naturally integrating a sponsored product that the platform has selected through an auction\. Your goal is to maximize user satisfaction while seamlessly mentioning the sponsored product\. \#\# Sponsored Product Product: <title\> Ad ID: <id\> \#\# Rules 1\. First and foremost, answer the user’s question helpfully and accurately\. 2\. Integrate the sponsored product exactly ONCE in your response\. 3\. Use the strict format: @<title\>@\[<id\>\] \-\-\- do not modify the ad title\. Correct: @<title\>@\[<id\>\] Wrong: @<title in lowercase\>@\[<id\>\] Wrong: <title\> \[<id\>\] Wrong: @<title\>@\(<id\>\) 4\. The ad insertion should feel natural \-\-\- as if you genuinely recommend the product\. 5\. If the product is directly relevant to the user’s question, weave it into your main recommendation naturally\. 6\. If the product is not directly relevant, find a natural transition point: \- ‘‘You might also consider\.\.\.’’ \- ‘‘On a related note\.\.\.’’ \- ‘‘While we are on the topic of \[category\]\.\.\.’’ 7\. Do NOT mention auctions, bids, pCTR, payments, advertising, sponsorship, or any internal mechanism\. 8\. Do NOT insert multiple products or use the @\.\.\.@\[\.\.\.\] format more than once\. 9\. Keep your response concise and helpful \-\-\- typically 2\-\-4 sentences\. \#\# Good Example User: What is a good desk lamp for studying? Assistant: For long study sessions, you want adjustable brightness and a wide light bar to reduce eye strain\. @<title\>@\[<id\>\] is a solid option worth checking out\. Pair it with a warm\-tone setting at night to reduce blue light exposure\. \#\# Bad Examples Bad \(too forced\): You should definitely buy @<title\>@\[<id\>\] right now\! It is the best product ever\! Bad \(no ad\): For studying, get a lamp with adjustable brightness\. Any LED desk lamp will work\. Bad \(wrong format\): Check out <title\> \[<id\>\]\. The dialogue history in chat format \(alternating user and assistant turns\) followed by the current user message; the model generates the assistant reply, and the reply that inserts the ad is the training target\.

Figure 9:Renderer SFT training system prompt\. <title\> and <id\> are the selected ad’s title and id\.
### Case Study

Figure[7](https://arxiv.org/html/2608.00123#A3.F7)illustrates the mechanism on a representative test session\. The winner’s contextual pCTR rises monotonically across the three turns \(0\.1220\.122,0\.2910\.291,0\.6680\.668\)\. The learned comparisonQ^ϕ,tS≥Q^ϕ,tW\\widehat\{Q\}\_\{\\phi,t\}^\{S\}\\geq\\widehat\{Q\}\_\{\\phi,t\}^\{W\}holds only att=3t=3\(Q^ϕ,3S=0\.310\\widehat\{Q\}\_\{\\phi,3\}^\{S\}=0\.310,Q^ϕ,3W=0\.037\\widehat\{Q\}\_\{\\phi,3\}^\{W\}=0\.037\), so the mechanism defers at earlier turns\. Because the stopping time is robust to bid perturbations in this session, the allocation curve has one observed jump and the envelope CPC coincides with its critical\-report threshold\. The winner is charged$​0\.164\\mathdollar 0\.164against a$​0\.49\\mathdollar 0\.49bid, yielding an information rent of$​0\.196\\mathdollar 0\.196\.

### Prompts

Both the click\-labeling judge and the renderer are kept bid\-independent: neither is shown the advertiser bid\.

#### User\-simulation judge\.

The judge produces click labels and a retention score from the dialogue and the inserted response, with the system and input prompts in Figure[8](https://arxiv.org/html/2608.00123#A3.F8)\. The judge never sees the advertiser bid\.

#### Renderer \(SFT training template\)\.

The renderer is fine\-tuned with the system prompt in Figure[9](https://arxiv.org/html/2608.00123#A3.F9), where<title\>and<id\>are the selected ad’s title and id\. The candidate list and bids are stripped from the training text, leaving only the selected product’s title and id; only turns that insert an ad are kept as supervised targets\.

### Adaptive Timing Baseline

To check that the gain comes from look\-ahead rather than adaptivity alone, we add a myopic greedy baseline that inserts at the first turn whose winner expected click valuebi​qib\_\{i\}q\_\{i\}exceeds a threshold, with no Bellman look\-ahead\. It reuses the sameGηG\_\{\\eta\}, allocation, and envelope payment; only the stopping rule differs\. Table[4](https://arxiv.org/html/2608.00123#A3.T4)sweeps the threshold from0\.20\.2to0\.60\.6: the myopic net revenue stays in0\.6970\.697–0\.7170\.717, always below LLM\-OSDA’s0\.7770\.777, while its trigger turn rises from1\.021\.02to1\.261\.26as the threshold tightens\. Thus learned Bellman stopping \(look\-ahead\) beats the greedy threshold, which in turn beats fixed timing—the gain is driven by look\-ahead, not by adaptivity alone\.

Table 4:Myopic threshold baseline across thresholds, under the same evaluation\. Always\-Round\-1 and LLM\-OSDA are invariant to the myopic threshold and shown for reference\.

## Appendix DLimitations

Our evaluation uses a simulated conversational corpus: dialogues, clicks, and retention are LLM\-generated rather than drawn from real traffic, and it is restricted to three\-turn shopping dialogues from a single dataset, leaving longer conversations and other domains untested\. The exact DSIC and IR guarantees hold for the ideal Bellman mechanism; the deployed StopNet is an approximation whose monotonicity and incentive loss are reported as empirical diagnostics rather than a proved uniform bound\. The mechanism also commits to at most one insertion per session, so multi\-insertion settings are out of scope\.