Optimal Top-$k$ Identification from Pairwise Comparisons

# Optimal Top-𝑘 Identification from Pairwise Comparisons
Source: [https://arxiv.org/html/2607.08979](https://arxiv.org/html/2607.08979)
###### Abstract

We study the active learning problem of fixed\-confidence top\-kkidentification from noisy pairwise comparisons\. In this problem, an algorithm sequentially chooses pairs of items to compare, observes the outcomes, and stops when it can return the set of top\-kkitems with error probability at mostδ\\delta\. The objective is to design such a*δ\\delta\-correct*procedure that minimizes the expected number of comparisons \(the sample complexity\)\. This problem falls within the broader literature on fixed\-confidence pure exploration in bandit models, where a common target is asymptotic optimality: the algorithm’s expected sample complexity matches the information theoretic lower bound asδ→0\\delta\\to 0\. Asymptotically optimal procedures have been developed for a range of fixed\-confidence pure\-exploration problems, however to the best of our knowledge, for top\-11, or more generally top\-kkidentification from pairwise comparisons under latent utility models an asymptotically optimal algorithm has not been established\. In this setting, we develop such an algorithm\. We characterize the structure of the lower bound and formulate it as a saddle\-point problem\. This structure enables a computationally efficient primal–dual procedure that learns the asymptotically optimal comparison allocation online\. We then construct an adaptive comparison\-allocation algorithm that tracks the allocation learned by the primal–dual procedure and prove it is asymptotically optimal\.

## 1Introduction

The objective of identifying the top\-kkitems from a set of candidates using noisy pairwise comparisons arises in many settings\. Consider a practitioner who wants to determine which large language model is best for their task\. They open Arena\(Chiang et al\.,[2024](https://arxiv.org/html/2607.08979#bib.bib10)\)\(formerly Chatbot Arena\), a public platform for evaluating LLMs through crowdsourced pairwise human preferences, and scan the leaderboard\. The leaderboard ranks all relevant models, but the practitioner only wants to consider the top few on the list\.

Additional settings that use pairwise comparisons for top\-kkidentification include: crowdsourced pairwise comparisons to identify the best photographs, translations, or annotations\(Narimanzadeh et al\.,[2023](https://arxiv.org/html/2607.08979#bib.bib27); Kou et al\.,[2017](https://arxiv.org/html/2607.08979#bib.bib24); Chen et al\.,[2013](https://arxiv.org/html/2607.08979#bib.bib6)\); recommender systems that elicit pairwise feedback to identify a small set of items to display to a user\(Kalloori et al\.,[2018](https://arxiv.org/html/2607.08979#bib.bib20)\); and sports tournaments\.

Top\-kkidentification serves as the natural objective in two distinct settings\. In the first setting, the top\-kkset is the desired output itself\. For example, the single best LLM is selected\(k=1\)\(k=1\), a funding program supports the top five proposals, or a streaming service displays a top ten list\. In the second setting, top\-kkidentification is the first stage of a two\-stage process, with the second stage using a different and/or more expensive signal\. For example, Arena could be used to narrow the field of candidate models tokk, after which a more careful application\-specific evaluation is conducted on the finalists\. The top\-kkidentification in the first stage can be essential when the time and resources required for a second stage with all items is prohibitively large\.

In both settings, comparisons cost money, time, and/or scarce human attention, which makes adaptivity valuable\. Rather than fixing a static sampling plan up front, an experiment designer can adaptively choose which pair to compare next based on outcomes observed so far, allocating effort on comparisons that are most informative for distinguishing the top\-kkitems from the rest\. We study this problem in the fixed\-confidence setting: an experiment designer adaptively chooses which pair to compare next, observes the outcome, and stops only once they can return the correct top\-kkset with probability at least1−δ1\-\\delta\. The objective is to design such a procedure that minimizes the expected number of comparisons\.

A key modeling decision is what to assume about the relationship between the relative ordering of items and their pairwise win probabilities\. Some work makes minimal assumptions, allowing situations wherei≻ji\\succ jwhileℙ​\(i​beats​j\)<12\\mathbb\{P\}\(i\\ \\text\{beats\}\\ j\)<\\tfrac\{1\}\{2\}; see, e\.g\.,Haddenhorst et al\. \([2021](https://arxiv.org/html/2607.08979#bib.bib14)\); Xu et al\. \([2019](https://arxiv.org/html/2607.08979#bib.bib36)\)\. Other work assumes some form of stochastic transitivity, which rules out such reversals; see, e\.g\.,Braverman et al\. \([2019](https://arxiv.org/html/2607.08979#bib.bib4)\); Shah et al\. \([2017](https://arxiv.org/html/2607.08979#bib.bib33)\)\. Finally, latent\-utility models assume that items are ordered by unobserved utilities𝜽=\(θ1,…,θn\)\\bm\{\\theta\}=\(\\theta\_\{1\},\\ldots,\\theta\_\{n\}\)\. Under this model there is a nondecreasing link functionffsuch that, for every pair\(i,j\)\(i,j\),ℙ​\(i​beats​j\)=f​\(θi−θj\)\\mathbb\{P\}\(i\\ \\text\{beats\}\\ j\)=f\(\\theta\_\{i\}\-\\theta\_\{j\}\); see, e\.g\.,Saha & Gopalan \([2020](https://arxiv.org/html/2607.08979#bib.bib32),[2019](https://arxiv.org/html/2607.08979#bib.bib31)\)\.

In this paper, we assume a latent\-utility model\. Unlike much of the pairwise\-comparison and dueling bandits literature which only considers binary outcomes, our algorithm also applies when comparison outcomes are cardinal\. In this case, a comparison returns a numerical value indicating how strongly one item is preferred, for example, a score difference or a difference in measured performance\.

Prior work on fixed\-confidence best\-arm \(item\) identification establishes information\-theoretic lower bounds on the asymptotic sample complexity\(Kaufmann et al\.,[2016](https://arxiv.org/html/2607.08979#bib.bib22); Garivier & Kaufmann,[2016](https://arxiv.org/html/2607.08979#bib.bib13)\), which is easily extended to our setting\. Algorithms that match this lower bound asδ→0\\delta\\to 0are called asymptotically optimal\. The lower bound is governed by a max–min oracle allocation problem\. If the true parameter vector𝜽\\bm\{\\theta\}were known, one could allocate pairwise comparisons so that the resulting data would rule out all parameter vectors whose top\-kkset differs from the true top\-kkset—so\-called “alternative” parameters\. The optimal oracle allocation maximizes the rate at which the most difficult alternative can be ruled out\.

Contributions\.For top\-kkidentification with pairwise comparisons, we characterize the structure of this oracle problem\. Any alternative parameter vector𝜽′\\bm\{\\theta\}^\{\\prime\}must reverse the ordering of at least one pair\(i,j\)\(i,j\)withiiin the true top\-kkandjjoutside it\. We call such a pair a*boundary pair*\. We show that the oracle problem can be formulated as a two\-player game\. A designer allocates comparisons across all\(n2\)\\binom\{n\}\{2\}pairs, while an adversary chooses which of thek​\(n−k\)k\(n\-k\)boundary pairs to “target”\. The equilibrium characterizes both the asymptotically optimal sampling strategy and the sample complexity\. We construct a comparison allocation procedure based on a primal–dual algorithm that learns this equilibrium online, and prove it is asymptotically optimal\.

Organization\.Section[2](https://arxiv.org/html/2607.08979#S2)reviews the literature on top\-kkidentification from pairwise comparisons as well as related work on optimal pure exploration in bandit models\. Section[3](https://arxiv.org/html/2607.08979#S3)formalizes the model and objective, and states the information\-theoretic lower bound\. Section[4](https://arxiv.org/html/2607.08979#S4)analyzes the oracle allocation problem\. Section[5](https://arxiv.org/html/2607.08979#S5)presents the online comparison allocation algorithm, and Section[6](https://arxiv.org/html/2607.08979#S6)proves its guarantees\. Section[7](https://arxiv.org/html/2607.08979#S7)demonstrates the algorithm’s performance on simulated instances\.

## 2Related Work

### 2\.1Top\-kkIdentification from Pairwise Comparisons

There is a large body of work on sequential learning from noisy pairwise comparisons; seeBengs et al\. \([2021](https://arxiv.org/html/2607.08979#bib.bib3)\)for a survey\. This includes the extensive literature on top\-11identification, an important special case of fixed confidence top\-kkidentification, which is our focus\. We will refer to two standard notions of fixed\-confidence guarantees:*δ\\delta\-correct*, meaning the exact target is returned with probability at least1−δ1\-\\delta, and*probably approximately correct*\(PAC\), meaning that with probability at least1−δ1\-\\deltathe algorithm returns a near\-optimal target\.

Early work on adaptive top\-kkselection from comparisons includesBusa\-Fekete et al\. \([2013](https://arxiv.org/html/2607.08979#bib.bib5)\), who propose a preference\-based racing algorithm for selecting the top\-kkitems using pairwise comparisons\. They do not make assumptions about the structure of pairwise preferences, and the “top\-kk” set is defined through a chosen ranking rule \(e\.g\., Copeland’s ranking\)\.

In contrast, and closer to our work,Ren et al\. \([2020](https://arxiv.org/html/2607.08979#bib.bib30)\)studies*δ\\delta\-correct*top\-kkselection assuming strong stochastic transitivity \(SST\) together with a stochastic triangle inequality \(STI\); seeRen et al\. \([2020](https://arxiv.org/html/2607.08979#bib.bib30)\)for formal definitions\. They develop an algorithm, SEEKS, that uses an elimination scheme for top\-kkselection\. In each round, it first chooses a pivot item that is “close” to the currentkk\-th best item, compares items to that pivot, and then assigns them into three groups: clearly above the pivot, ambiguous, and clearly below it\. Items in the first group are “accepted” into the top\-kk, and items in the third group are eliminated\. Items in both of these groups are no longer considered for future comparisons\. The algorithm then repeats on the unresolved items in the second group and stops oncekkitems have been accepted, orn−kn\-kitems have been eliminated\. They prove that SEEKS has a sample complexity ofO​\(∑i=1nΔi−2​\(log⁡\(n/δ\)\+log⁡log⁡Δi−1\)\)O\\\!\\left\(\\sum\_\{i=1\}^\{n\}\\Delta\_\{i\}^\{\-2\}\\bigl\(\\log\(n/\\delta\)\+\\log\\log\\Delta\_\{i\}^\{\-1\}\\bigr\)\\right\)whereΔi\\Delta\_\{i\}is a measure of distance of itemiito thekk\-th \(or\(k\+1\)\(k\+1\)\-th\) best item\. They also derive a lower bound on the expected number of samples required for any algorithm to beδ\\delta\-correct,Ω​\(∑i=1nΔi−2​log⁡\(1/δ\)\+log⁡log⁡Δrk−1\)\\Omega\\\!\\left\(\\sum\_\{i=1\}^\{n\}\\Delta\_\{i\}^\{\-2\}\\log\(1/\\delta\)\+\\log\\log\\Delta\_\{r\_\{k\}\}^\{\-1\}\\right\)\. However, while their model is formulated under SST and STI, the lower bound is proven under the following assumptions: \(i\) comparisons are generated from a Thurstone model i\.e\.,ℙ​\(i​beats​j\)=ℙ​\(θi\+Z1\>θj\+Z2\)\\mathbb\{P\}\(i\\ \\text\{beats\}\\ j\)=\\mathbb\{P\}\(\\theta\_\{i\}\+Z\_\{1\}\>\\theta\_\{j\}\+Z\_\{2\}\)withZ1Z\_\{1\}andZ2Z\_\{2\}independent Gaussian noise with variance11, \(ii\)δ∈\(0,1/100\)\\delta\\in\(0,1/100\), \(iii\)θ1,…,θn∈\[0,1\]\\theta\_\{1\},\\ldots,\\theta\_\{n\}\\in\[0,1\]\. Thus, under these conditions, they show SEEKS is optimal up to alog⁡n\\log nfactor, but under more general conditions, the optimality gap may be larger\. In contrast, our optimality guarantee is only in the asymptotic regimeδ→0\\delta\\to 0, but we match the lower bound exactly \(including multiplicative constants\)\.

Top\-kkidentification has also been studied under parametric latent\-utility models, specifically the multinomial logit \(MNL\) model, where queries are listwise\. At each round, the learner selects a set ofℓ≥2\\ell\\geq 2items and observes which item is most preferred in that set\.Chen et al\. \([2018](https://arxiv.org/html/2607.08979#bib.bib7)\)give an algorithm that is optimal up to polylog factors, however, their algorithm guarantees correctness with “high probability”, but the risk levelδ\\deltais not an input parameter\.Ren et al\. \([2018](https://arxiv.org/html/2607.08979#bib.bib29)\)study PAC top\-kkselection with both pairwise and listwise queries\.

Beyond top\-kkselection, there is also work that studies adaptive comparison strategies for learning rankings, with top\-kkidentification as a special case; see, e\.g\.,Heckel et al\. \([2019](https://arxiv.org/html/2607.08979#bib.bib17)\); Mohajer et al\. \([2017](https://arxiv.org/html/2607.08979#bib.bib26)\)\. Additionally, there is a large literature on non\-adaptive top\-kkrecovery from pairwise data under Bradley–Terry / Plackett–Luce models \(see, e\.g\.,Chen et al\. \([2019](https://arxiv.org/html/2607.08979#bib.bib9)\); Negahban et al\. \([2017](https://arxiv.org/html/2607.08979#bib.bib28)\); Jang et al\. \([2016](https://arxiv.org/html/2607.08979#bib.bib18)\); Khetan & Oh \([2016](https://arxiv.org/html/2607.08979#bib.bib23)\); Chen & Suh \([2015](https://arxiv.org/html/2607.08979#bib.bib8)\); Hajek et al\. \([2014](https://arxiv.org/html/2607.08979#bib.bib15)\)\)\. These works study the accuracy of computationally efficient estimators of the top\-kkitems under prespecified sampling designs\.

### 2\.2Pure Exploration and Oracle\-Tracking Methods

In the bandit literature,*pure exploration*problems are those in which samples are collected to inform a terminal decision, rather than to maximize the rewards obtained during sampling\. For the most studied pure exploration task, best\-arm identification,Garivier & Kaufmann \([2016](https://arxiv.org/html/2607.08979#bib.bib13)\)propose the Track\-and\-Stop framework: a sampling rule that tracks the optimal allocation that governs the lower bound on the sample complexity, together with a stopping rule, and prove optimality asδ→0\\delta\\to 0\.Degenne et al\. \([2019](https://arxiv.org/html/2607.08979#bib.bib11)\)formulate this same max–min lower\-bound program as a two\-player game between an experiment designer and nature, and develop online tracking schemes that learn the saddle point without solving the full max–min program at every round\.

These ideas have been extended to structured bandit models\. For linear bandits,Degenne et al\. \([2020](https://arxiv.org/html/2607.08979#bib.bib12)\)obtain asymptotically optimal fixed\-confidence algorithms by this same “gamification” idea, andJedra & Proutière \([2020](https://arxiv.org/html/2607.08979#bib.bib19)\)show that a Track\-and\-Stop approach can be made computationally scalable via lazy updates of the optimal allocation\.

Wang et al\. \([2021](https://arxiv.org/html/2607.08979#bib.bib35)\)propose a general Frank\-Wolfe oracle\-tracking method that can be applied to several pure\-exploration settings\. Their appendix includes an application to top\-kkidentification for dueling bandits under a nonparametric preference\-matrix model\. However, instantiating the Frank–Wolfe updates requires identifying the \(near\-\)most confusing alternative parameters, which in their case is combinatorial in the number of items\. In our latent\-utility model, the alternative parameters reduce to parameters that reverse the ranking of boundary pairs, yielding an online learner with computationally efficient updates\.

## 3Model and Lower Bound

In this section, we formalize the sequential comparison model and state an instance\-dependent lower bound on the expected sample complexity, which follows fromKaufmann et al\. \([2016](https://arxiv.org/html/2607.08979#bib.bib22)\); Garivier & Kaufmann \([2016](https://arxiv.org/html/2607.08979#bib.bib13)\)\.

### 3\.1Pairwise Comparison Model

Considernnitems indexedi∈\{1,…,n\}i\\in\\\{1,\\ldots,n\\\}each with an unknown utilityθi\\theta\_\{i\}\. Comparison outcomes depend on utility differences, so𝜽\\bm\{\\theta\}is identifiable only up to an additive constant\. We assume that the set of parameters is bounded by some constantRRand define the parameter space

Θ:=\{𝜽∈ℝn:∑i=1nθi=0,‖𝜽‖∞≤R\}\.\\Theta:=\\\{\\bm\{\\theta\}\\in\\mathbb\{R\}^\{n\}:\\ \\textstyle\\sum\_\{i=1\}^\{n\}\\theta\_\{i\}=0,\\\|\\bm\{\\theta\}\\\|\_\{\\infty\}\\leq R\\\}\.Let𝒫:=\{\(i,j\):1≤i<j≤n\}\\mathcal\{P\}:=\\\{\(i,j\):1\\leq i<j\\leq n\\\}denote the set of pairs\. For each\(i,j\)∈𝒫\(i,j\)\\in\\mathcal\{P\}, define𝒙i​j:=𝒆i−𝒆j\\bm\{x\}\_\{ij\}:=\\bm\{e\}\_\{i\}\-\\bm\{e\}\_\{j\}where𝒆i\\bm\{e\}\_\{i\}is theii\-th standard basis vector and the natural parameterηi​j​\(𝜽\):=𝒙i​j⊤​𝜽=θi−θj\.\\eta\_\{ij\}\(\\bm\{\\theta\}\):=\\bm\{x\}\_\{ij\}^\{\\top\}\\bm\{\\theta\}=\\theta\_\{i\}\-\\theta\_\{j\}\.Comparison outcomes follow a one\-parameter exponential family with parameterηi​j​\(𝜽\)\\eta\_\{ij\}\(\\bm\{\\theta\}\): there exist functionsT:𝒴→ℝT:\\mathcal\{Y\}\\to\\mathbb\{R\},A:ℝ→ℝA:\\mathbb\{R\}\\to\\mathbb\{R\}, andB:𝒴→ℝB:\\mathcal\{Y\}\\to\\mathbb\{R\}such that, for any comparison outcomey∈𝒴y\\in\\mathcal\{Y\},

p​\(y∣\(i,j\),𝜽\)=exp⁡\{ηi​j​\(𝜽\)​T​\(y\)−A​\(ηi​j​\(𝜽\)\)\+B​\(y\)\}\.p\(y\\mid\(i,j\),\\bm\{\\theta\}\)=\\exp\\bigl\\\{\\eta\_\{ij\}\(\\bm\{\\theta\}\)\\,T\(y\)\-A\(\\eta\_\{ij\}\(\\bm\{\\theta\}\)\)\+B\(y\)\\bigr\\\}\.
###### Assumption 1\.

AAis twice continuously differentiable onℝ\\mathbb\{R\}, and

0<A′′​\(η\)≤σ¯2for all​η∈ℝ0<A^\{\\prime\\prime\}\(\\eta\)\\ \\leq\\ \\overline\{\\sigma\}^\{2\}\\quad\\text\{for all \}\\eta\\in\\mathbb\{R\}whereσ¯2<∞\\overline\{\\sigma\}^\{2\}<\\infty\. Definea¯:=inf\|η\|≤2​RA′′​\(η\)\>0\\underline\{a\}\\ :=\\ \\inf\_\{\|\\eta\|\\leq 2R\}A^\{\\prime\\prime\}\(\\eta\)\\ \>\\ 0\.

###### Assumption 2\.

There existsσ2<∞\\sigma^\{2\}<\\inftysuch that for allη∈\[−2​R,2​R\]\\eta\\in\[\-2R,2R\]and allλ∈ℝ\\lambda\\in\\mathbb\{R\},

𝔼η​\[exp⁡\{λ​\(T​\(Y\)−A′​\(η\)\)\}\]≤exp⁡\(σ2​λ22\)\.\\mathbb\{E\}\_\{\\eta\}\\\!\\left\[\\exp\\\{\\lambda\(T\(Y\)\-A^\{\\prime\}\(\\eta\)\)\\\}\\right\]\\leq\\exp\\\!\\left\(\\frac\{\\sigma^\{2\}\\lambda^\{2\}\}\{2\}\\right\)\.

We highlight two standard models that satisfy our assumptions onΘ\\Thetaand illustrate the binary and cardinal observation regimes\.

###### Example 3\.1\(Bradley–Terry\)\.

LetY∈\{0,1\}Y\\in\\\{0,1\\\}be the random variable indicating whether itemiibeats itemjj, then given𝜽\\bm\{\\theta\},

Y∼Bernoulli​\(11\+e−\(θi−θj\)\)\.Y\\sim\\text\{Bernoulli\}\\left\(\\frac\{1\}\{1\+e^\{\-\(\\theta\_\{i\}\-\\theta\_\{j\}\)\}\}\\right\)\.

###### Example 3\.2\(Gaussian differences\)\.

LetY∈ℝY\\in\\mathbb\{R\}be the random variable denoting the observed difference betweeniiandjj, then given𝜽\\bm\{\\theta\},

Y∼N​\(θi−θj,σ02\)Y\\sim N\(\\theta\_\{i\}\-\\theta\_\{j\},\\sigma\_\{0\}^\{2\}\)whereσ02\\sigma\_\{0\}^\{2\}is known\.

### 3\.2Fixed\-Confidence top\-kkIdentification

Fixk∈\{1,…,n−1\}k\\in\\\{1,\\ldots,n\-1\\\}and letθ\(1\)≥⋯≥θ\(n\)\\theta\_\{\(1\)\}\\geq\\cdots\\geq\\theta\_\{\(n\)\}denote the order statistics of𝜽\\bm\{\\theta\}\. Assume a positive top\-kkgap,θ\(k\)\>θ\(k\+1\)\\theta\_\{\(k\)\}\>\\theta\_\{\(k\+1\)\}and defineΘgap:=\{𝜽∈Θ:θ\(k\)\>θ\(k\+1\)\}\.\\Theta^\{\\mathrm\{gap\}\}:=\\\{\\bm\{\\theta\}\\in\\Theta:\\ \\theta\_\{\(k\)\}\>\\theta\_\{\(k\+1\)\}\\\}\.OnΘgap\\Theta^\{\\mathrm\{gap\}\}, the top\-kksetS∗​\(𝜽\):=\{i:θi≥θ\(k\)\}S^\{\*\}\(\\bm\{\\theta\}\):=\\\{i:\\theta\_\{i\}\\geq\\theta\_\{\(k\)\}\\\}is unique\.

###### Assumption 3\.

The true utility parameter satisfies𝛉∈int​\(Θ\)∩Θgap\\bm\{\\theta\}\\in\\mathrm\{int\}\(\\Theta\)\\cap\\Theta^\{\\mathrm\{gap\}\}\.

At each roundt=1,2,…t=1,2,\\ldotsthe algorithm selects a pair\(it,jt\)∈𝒫\(i\_\{t\},j\_\{t\}\)\\in\\mathcal\{P\}and observesYt∼p\(⋅∣\(it,jt\),𝜽\)Y\_\{t\}\\sim p\(\\cdot\\mid\(i\_\{t\},j\_\{t\}\),\\bm\{\\theta\}\)\. Conditioned on the selected pairs, outcomes are independent\. Formally, a sequential strategy consists of:

- •a*sampling rule*: a sequence\{\(it,jt\)\}t≥1\\\{\(i\_\{t\},j\_\{t\}\)\\\}\_\{t\\geq 1\}where\(it,jt\)\(i\_\{t\},j\_\{t\}\)isℱt−1\\mathcal\{F\}\_\{t\-1\}\-measurable andℱt:=σ\(\(is,js,Ys\):s≤t\)\\mathcal\{F\}\_\{t\}:=\\sigma\(\(i\_\{s\},j\_\{s\},Y\_\{s\}\):s\\leq t\);
- •a*stopping rule*: a stopping timeτ\\tauwith respect to\(ℱt\)\(\\mathcal\{F\}\_\{t\}\);
- •a*decision rule*: anℱτ\\mathcal\{F\}\_\{\\tau\}\-measurable subsetS^τ⊂\[n\]\\hat\{S\}\_\{\\tau\}\\subset\[n\]with\|S^τ\|=k\|\\hat\{S\}\_\{\\tau\}\|=k\.

###### Definition 3\.3\(δ\\delta\-correct\)\.

A strategy isδ\\delta\-correct if for every𝜽∈Θgap\\bm\{\\theta\}\\in\\Theta^\{\\mathrm\{gap\}\},

ℙ𝜽​\(S^τ≠S∗​\(𝜽\)\)≤δandℙ𝜽​\(τ<∞\)=1\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\hat\{S\}\_\{\\tau\}\\neq S^\{\*\}\(\\bm\{\\theta\}\)\)\\leq\\delta\\quad\\text\{and\}\\quad\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\tau<\\infty\)=1\.

### 3\.3Information\-Theoretic Lower Bound

For a single comparison on pair\(i,j\)\(i,j\), letdi​j​\(𝜽,𝜽′\):=KL⁡\(Pηi​j​\(𝜽\)∥Pηi​j​\(𝜽′\)\)d\_\{ij\}\(\\bm\{\\theta\},\\bm\{\\theta\}^\{\\prime\}\):=\\operatorname\{KL\}\(P\_\{\\eta\_\{ij\}\(\\bm\{\\theta\}\)\}\\\|P\_\{\\eta\_\{ij\}\(\\bm\{\\theta\}^\{\\prime\}\)\}\)denote the KL divergence between outcome distributions under𝜽\\bm\{\\theta\}and𝜽′\\bm\{\\theta\}^\{\\prime\}\. A*sampling design*is a distribution𝒘∈Δ𝒫\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}over pairs, whereΔ𝒫:=\{𝒘∈ℝ\+\|𝒫\|:∑\(i,j\)wi​j=1\}\\Delta\_\{\\mathcal\{P\}\}:=\\\{\\bm\{w\}\\in\\mathbb\{R\}\_\{\+\}^\{\|\\mathcal\{P\}\|\}:\\sum\_\{\(i,j\)\}w\_\{ij\}=1\\\}\(wi​jw\_\{ij\}is the proportion of comparisons allocated to\(i,j\)\(i,j\)\)\. Under design𝒘\\bm\{w\}, define

D𝒘​\(𝜽∥𝜽′\):=∑\(i,j\)∈𝒫wi​j​di​j​\(𝜽,𝜽′\)\.D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\):=\\sum\_\{\(i,j\)\\in\\mathcal\{P\}\}w\_\{ij\}\\,d\_\{ij\}\(\\bm\{\\theta\},\\bm\{\\theta\}^\{\\prime\}\)\.Ultimately, our goal is to distinguish𝜽\\bm\{\\theta\}from alternative parameters𝜽′\\bm\{\\theta\}^\{\\prime\}whose top\-kkset is different\. Accordingly, for𝜽∈Θgap\\bm\{\\theta\}\\in\\Theta^\{\\mathrm\{gap\}\}define the alternative region

Alt⁡\(𝜽\):=\{𝜽′∈Θ:∃u∈S∗​\(𝜽\),v∉S∗​\(𝜽\)​s\.t\.​θv′≥θu′\}\\operatorname\{Alt\}\(\\bm\{\\theta\}\):=\\bigl\\\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta:\\ \\exists\\,u\\in S^\{\*\}\(\\bm\{\\theta\}\),\\,v\\notin S^\{\*\}\(\\bm\{\\theta\}\)\\ \\text\{s\.t\.\}\\ \\theta^\{\\prime\}\_\{v\}\\geq\\theta^\{\\prime\}\_\{u\}\\bigr\\\}and the information rate

Γ∗​\(𝜽\):=sup𝒘∈Δ𝒫inf𝜽′∈Alt⁡\(𝜽\)D𝒘​\(𝜽∥𝜽′\)\.\\Gamma^\{\*\}\(\\bm\{\\theta\}\):=\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\inf\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\operatorname\{Alt\}\(\\bm\{\\theta\}\)\}D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)\.Intuitively,Γ∗​\(𝜽\)\\Gamma^\{\*\}\(\\bm\{\\theta\}\)is the most per\-sample information we can learn for distinguishing𝜽\\bm\{\\theta\}from the hardest alternatives\. We now present the lower bound on the expected stopping time given byGarivier & Kaufmann \([2016](https://arxiv.org/html/2607.08979#bib.bib13)\)\.

###### Theorem 3\.4\(Garivier & Kaufmann \([2016](https://arxiv.org/html/2607.08979#bib.bib13)\)\)\.

Letδ∈\(0,1\)\\delta\\in\(0,1\)and let\(\{\(it,jt\)\},τδ,S^τ\)\(\\\{\(i\_\{t\},j\_\{t\}\)\\\},\\tau\_\{\\delta\},\\hat\{S\}\_\{\\tau\}\)be aδ\\delta\-correct strategy\. For any𝛉∈Θgap\\bm\{\\theta\}\\in\\Theta^\{\\mathrm\{gap\}\},

lim infδ→0𝔼𝜽​\[τδ\]log⁡\(1/δ\)≥1Γ∗​\(𝜽\)\.\\liminf\_\{\\delta\\to 0\}\\frac\{\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\[\\tau\_\{\\delta\}\]\}\{\\log\(1/\\delta\)\}\\geq\\frac\{1\}\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}\.

The Track\-and\-Stop algorithm ofGarivier & Kaufmann \([2016](https://arxiv.org/html/2607.08979#bib.bib13)\)gives a framework for sequential algorithms to track the optimal oracle allocation𝒘∗​\(𝜽\)\\bm\{w\}^\{\*\}\(\\bm\{\\theta\}\)\. In the next section, we will characterize this oracle allocation for the pairwise top\-kksetting\. Then, building on the Track\-and\-Stop framework, we will develop a sequential algorithm that tracks the oracle allocation and prove that it is asymptotically optimal\.

## 4Structure of the Oracle Problem

The oracle’s allocation problem is, given𝜽\\bm\{\\theta\}, to find a design𝒘∗∈argmax𝒘∈Δ𝒫inf𝜽′∈Alt⁡\(𝜽\)D𝒘​\(𝜽∥𝜽′\)\\bm\{w\}^\{\*\}\\in\\mathop\{\\mathrm\{argmax\}\}\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\inf\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\operatorname\{Alt\}\(\\bm\{\\theta\}\)\}D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)\. In this section, we \(i\) formulate this problem as a two\-player game, \(ii\) show that a saddle\-point exists, and \(iii\) calculate the gradients that will be used in the online algorithm, when, in contrast to the oracle problem,𝜽\\bm\{\\theta\}is not known\.

### 4\.1Boundary Reduction

We start by characterizing the alternative regionAlt⁡\(𝜽\)\\operatorname\{Alt\}\(\\bm\{\\theta\}\)for top\-kkidentification\. Define the set of boundary pairs

B​\(𝜽\):=\{\(i,j\)∈𝒫:𝟏​\{i∈S∗​\(𝜽\)\}≠𝟏​\{j∈S∗​\(𝜽\)\}\},B\(\\bm\{\\theta\}\):=\\\{\(i,j\)\\in\\mathcal\{P\}:\\mathbf\{1\}\\\{i\\in S^\{\*\}\(\\bm\{\\theta\}\)\\\}\\neq\\mathbf\{1\}\\\{j\\in S^\{\*\}\(\\bm\{\\theta\}\)\\\}\\\},so\|B​\(𝜽\)\|=k​\(n−k\)\|B\(\\bm\{\\theta\}\)\|=k\(n\-k\)\. For each\(i,j\)∈B​\(𝜽\)\(i,j\)\\in B\(\\bm\{\\theta\}\), letui​j∈S∗​\(𝜽\)u\_\{ij\}\\in S^\{\*\}\(\\bm\{\\theta\}\)andvi​j∉S∗​\(𝜽\)v\_\{ij\}\\notin S^\{\*\}\(\\bm\{\\theta\}\)denote the endpoints lying inside and outside the true top\-kkset, respectively\. Define the set of parameters that invert the true ordering,Θi​j:=\{𝜽′∈Θ:θvi​j′≥θui​j′\}\\Theta\_\{ij\}:=\\\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta:\\theta^\{\\prime\}\_\{v\_\{ij\}\}\\geq\\theta^\{\\prime\}\_\{u\_\{ij\}\}\\\}\. It follows from the definition ofAlt⁡\(𝜽\)\\operatorname\{Alt\}\(\\bm\{\\theta\}\)that if𝜽′∈Alt⁡\(𝜽\)\\bm\{\\theta\}^\{\\prime\}\\in\\operatorname\{Alt\}\(\\bm\{\\theta\}\), there is some boundary pair\(i,j\)∈B​\(𝜽\)\(i,j\)\\in B\(\\bm\{\\theta\}\)such that𝜽′∈Θi​j\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}\. Thus,Alt⁡\(𝜽\)=⋃\(i,j\)∈B​\(𝜽\)Θi​j\\operatorname\{Alt\}\(\\bm\{\\theta\}\)=\\bigcup\_\{\(i,j\)\\in B\(\\bm\{\\theta\}\)\}\\Theta\_\{ij\}, and the oracle objective can be written as

Γ∗​\(𝜽\)=sup𝒘∈Δ𝒫min\(i,j\)∈B​\(𝜽\)​inf𝜽′∈Θi​jD𝒘​\(𝜽∥𝜽′\)\.\\Gamma^\{\*\}\(\\bm\{\\theta\}\)=\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\min\_\{\(i,j\)\\in B\(\\bm\{\\theta\}\)\}\\inf\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}\}D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)\.

### 4\.2Saddle Point Formulation

Letγi​j​\(𝒘;𝜽\):=inf𝜽′∈Θi​jD𝒘​\(𝜽∥𝜽′\)\\gamma\_\{ij\}\(\\bm\{w\};\\bm\{\\theta\}\):=\\inf\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}\}D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)and define the information rate under𝒘\\bm\{w\},Γ​\(𝒘;𝜽\):=min\(i,j\)∈B​\(𝜽\)⁡γi​j​\(𝒘;𝜽\)\\Gamma\(\\bm\{w\};\\bm\{\\theta\}\)\\ :=\\ \\min\_\{\(i,j\)\\in B\(\\bm\{\\theta\}\)\}\\gamma\_\{ij\}\(\\bm\{w\};\\bm\{\\theta\}\)\. The minimum over boundary pairs is nonsmooth in general, so we writeΓ​\(𝒘;𝜽\)=min𝒒∈ΔB​\(𝜽\)⁡F​\(𝒘,𝒒;𝜽\)\\Gamma\(\\bm\{w\};\\bm\{\\theta\}\)=\\min\_\{\\bm\{q\}\\in\\Delta\_\{B\(\\bm\{\\theta\}\)\}\}F\(\\bm\{w\},\\bm\{q\};\\bm\{\\theta\}\), whereF​\(𝒘,𝒒;𝜽\):=∑\(i,j\)∈B​\(𝜽\)qi​j​γi​j​\(𝒘;𝜽\)F\(\\bm\{w\},\\bm\{q\};\\bm\{\\theta\}\):=\\sum\_\{\(i,j\)\\in B\(\\bm\{\\theta\}\)\}q\_\{ij\}\\,\\gamma\_\{ij\}\(\\bm\{w\};\\bm\{\\theta\}\)\. Consequently,

Γ∗​\(𝜽\)=max𝒘∈Δ𝒫⁡min𝒒∈ΔB​\(𝜽\)⁡F​\(𝒘,𝒒;𝜽\)\.\\Gamma^\{\*\}\(\\bm\{\\theta\}\)=\\max\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\ \\min\_\{\\bm\{q\}\\in\\Delta\_\{B\(\\bm\{\\theta\}\)\}\}F\(\\bm\{w\},\\bm\{q\};\\bm\{\\theta\}\)\.We view this as a two\-player game\. The designer chooses allocation proportions𝒘\\bm\{w\}, and an adversary chooses𝒒∈ΔB​\(𝜽\)\\bm\{q\}\\in\\Delta\_\{B\(\\bm\{\\theta\}\)\}, a distribution over boundary pairs\. For any fixed𝒘\\bm\{w\}, the inner minimization over𝒒\\bm\{q\}is equivalent to taking the minimum over\(i,j\)∈B​\(𝜽\)\(i,j\)\\in B\(\\bm\{\\theta\}\): an optimal𝒒\\bm\{q\}allocates its mass on the boundary pair\(s\) attainingmin\(i,j\)∈B​\(𝜽\)⁡γi​j​\(𝒘;𝜽\)\\min\_\{\(i,j\)\\in B\(\\bm\{\\theta\}\)\}\\gamma\_\{ij\}\(\\bm\{w\};\\bm\{\\theta\}\)\. Informally,𝒒\\bm\{q\}identifies the boundary pairs under design𝒘\\bm\{w\}that are most “vulnerable” to being misordered\.

###### Lemma 4\.1\(Basic properties ofγi​j\\gamma\_\{ij\}\)\.

Fix𝛉∈Θ\\bm\{\\theta\}\\in\\Thetaand\(i,j\)∈B​\(𝛉\)\(i,j\)\\in B\(\\bm\{\\theta\}\)\. The map𝐰↦γi​j​\(𝐰;𝛉\)\\bm\{w\}\\mapsto\\gamma\_\{ij\}\(\\bm\{w\};\\bm\{\\theta\}\)is concave and continuous onΔ𝒫\\Delta\_\{\\mathcal\{P\}\}\.

By Lemma[4\.1](https://arxiv.org/html/2607.08979#S4.Thmtheorem1),F​\(⋅,𝒒;𝜽\)=∑\(i,j\)∈B​\(𝜽\)qi​j​γi​j​\(⋅;𝜽\)F\(\\cdot,\\bm\{q\};\\bm\{\\theta\}\)=\\sum\_\{\(i,j\)\\in B\(\\bm\{\\theta\}\)\}q\_\{ij\}\\gamma\_\{ij\}\(\\cdot;\\bm\{\\theta\}\)is concave and continuous in𝒘\\bm\{w\}for each𝒒\\bm\{q\}, andF​\(𝒘,⋅;𝜽\)F\(\\bm\{w\},\\cdot;\\bm\{\\theta\}\)is linear \(hence convex and continuous\) in𝒒\\bm\{q\}for each𝒘\\bm\{w\}\. SinceΔ𝒫\\Delta\_\{\\mathcal\{P\}\}andΔB​\(𝜽\)\\Delta\_\{B\(\\bm\{\\theta\}\)\}are nonempty compact convex sets, Sion’s minimax theorem gives

max𝒘∈Δ𝒫⁡min𝒒∈ΔB​\(𝜽\)⁡F​\(𝒘,𝒒;𝜽\)=min𝒒∈ΔB​\(𝜽\)⁡max𝒘∈Δ𝒫⁡F​\(𝒘,𝒒;𝜽\)\.\\max\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\ \\min\_\{\\bm\{q\}\\in\\Delta\_\{B\(\\bm\{\\theta\}\)\}\}F\(\\bm\{w\},\\bm\{q\};\\bm\{\\theta\}\)\\;=\\;\\min\_\{\\bm\{q\}\\in\\Delta\_\{B\(\\bm\{\\theta\}\)\}\}\\ \\max\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}F\(\\bm\{w\},\\bm\{q\};\\bm\{\\theta\}\)\.Moreover, by compactness and continuity the max and min are attained, so there exists\(𝒘∗,𝒒∗\)∈Δ𝒫×ΔB​\(𝜽\)\(\\bm\{w\}^\{\*\},\\bm\{q\}^\{\*\}\)\\in\\Delta\_\{\\mathcal\{P\}\}\\times\\Delta\_\{B\(\\bm\{\\theta\}\)\}such thatΓ∗​\(𝜽\)=F​\(𝒘∗,𝒒∗;𝜽\)\\Gamma^\{\*\}\(\\bm\{\\theta\}\)=F\(\\bm\{w\}^\{\*\},\\bm\{q\}^\{\*\};\\bm\{\\theta\}\)and

F​\(𝒘,𝒒∗;𝜽\)\\displaystyle F\(\\bm\{w\},\\bm\{q\}^\{\*\};\\bm\{\\theta\}\)≤F​\(𝒘∗,𝒒∗;𝜽\)≤F​\(𝒘∗,𝒒;𝜽\)\\displaystyle\\leq F\(\\bm\{w\}^\{\*\},\\bm\{q\}^\{\*\};\\bm\{\\theta\}\)\\leq F\(\\bm\{w\}^\{\*\},\\bm\{q\};\\bm\{\\theta\}\)∀\(𝒘,𝒒\)∈Δ𝒫×ΔB​\(𝜽\)\.\\displaystyle\\quad\\forall\(\\bm\{w\},\\bm\{q\}\)\\in\\Delta\_\{\\mathcal\{P\}\}\\times\\Delta\_\{B\(\\bm\{\\theta\}\)\}\.To compute gradients ofFFwe need the KL projectioninf𝜽′∈Θi​jD𝒘​\(𝜽∥𝜽′\)\\inf\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}\}D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)to exist and be unique\. This is ensured by strict convexity ofD𝒘​\(𝜽∥⋅\)D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\cdot\)overΘ\\Theta, which is guaranteed when the sampling design has connected support\. Define the support graph

G​\(𝒘\):=\(\[n\],\{\{i,j\}:wi​j\>0\}\)\.G\(\\bm\{w\}\):=\(\[n\],\\\{\\\{i,j\\\}:w\_\{ij\}\>0\\\}\)\.G​\(𝒘\)G\(\\bm\{w\}\)is connected if for anyu,v∈\[n\]u,v\\in\[n\]there exists a pathu=v0,…,vm=vu=v\_\{0\},\\ldots,v\_\{m\}=vinG​\(𝒘\)G\(\\bm\{w\}\)\.

###### Lemma 4\.2\(Well\-posedness of KL projection onΘ\\Theta\)\.

Fix𝛉∈Θ\\bm\{\\theta\}\\in\\Thetaand𝐰∈Δ𝒫\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}withG​\(𝐰\)G\(\\bm\{w\}\)connected\. For each\(i,j\)∈B​\(𝛉\)\(i,j\)\\in B\(\\bm\{\\theta\}\):

1. \(a\)The map𝜽′↦D𝒘​\(𝜽∥𝜽′\)\\bm\{\\theta\}^\{\\prime\}\\mapsto D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)is strictly convex onΘ\\Theta
2. \(b\)𝜽i​j∗​\(𝒘;𝜽\):=argmin𝜽′∈Θi​jD𝒘​\(𝜽∥𝜽′\)\\bm\{\\theta\}^\{\*\}\_\{ij\}\(\\bm\{w\};\\bm\{\\theta\}\):=\\mathop\{\\mathrm\{argmin\}\}\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}\}D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)is unique
3. \(c\)Either\(𝜽i​j∗\)ui​j=\(𝜽i​j∗\)vi​j\(\\bm\{\\theta\}^\{\*\}\_\{ij\}\)\_\{u\_\{ij\}\}=\(\\bm\{\\theta\}^\{\*\}\_\{ij\}\)\_\{v\_\{ij\}\}, or𝜽i​j∗​\(𝒘;𝜽\)∈∂Θ\\bm\{\\theta\}^\{\*\}\_\{ij\}\(\\bm\{w\};\\bm\{\\theta\}\)\\in\\partial\\Theta\.

It follows by Danskin’s theorem thatγi​j​\(⋅;𝜽\)\\gamma\_\{ij\}\(\\cdot;\\bm\{\\theta\}\)is differentiable at𝒘\\bm\{w\}such thatG​\(𝒘\)G\(\\bm\{w\}\)is connected and

∂γi​j∂wa​b​\(𝒘;𝜽\)=da​b​\(𝜽,𝜽i​j∗​\(𝒘;𝜽\)\),\(a,b\)∈𝒫\.\\frac\{\\partial\\gamma\_\{ij\}\}\{\\partial w\_\{ab\}\}\(\\bm\{w\};\\bm\{\\theta\}\)=d\_\{ab\}\\\!\\big\(\\bm\{\\theta\},\\bm\{\\theta\}^\{\*\}\_\{ij\}\(\\bm\{w\};\\bm\{\\theta\}\)\\big\),\\qquad\(a,b\)\\in\\mathcal\{P\}\.hence, the partial derivatives ofFFare

∂F∂wa​b​\(𝒘,𝒒;𝜽\)=∑\(i,j\)∈B​\(𝜽\)qi​j​da​b​\(𝜽,𝜽i​j∗​\(𝒘;𝜽\)\),∂F∂qi​j​\(𝒘,𝒒;𝜽\)=γi​j​\(𝒘;𝜽\)\.\\begin\{split\}\\frac\{\\partial F\}\{\\partial w\_\{ab\}\}\(\\bm\{w\},\\bm\{q\};\\bm\{\\theta\}\)&=\\sum\_\{\(i,j\)\\in B\(\\bm\{\\theta\}\)\}q\_\{ij\}\\,d\_\{ab\}\\\!\\big\(\\bm\{\\theta\},\\bm\{\\theta\}^\{\*\}\_\{ij\}\(\\bm\{w\};\\bm\{\\theta\}\)\\big\),\\\\ \\frac\{\\partial F\}\{\\partial q\_\{ij\}\}\(\\bm\{w\},\\bm\{q\};\\bm\{\\theta\}\)&=\\gamma\_\{ij\}\(\\bm\{w\};\\bm\{\\theta\}\)\.\\end\{split\}\(1\)The KL projection𝜽i​j∗​\(𝒘;𝜽\)\\bm\{\\theta\}^\{\*\}\_\{ij\}\(\\bm\{w\};\\bm\{\\theta\}\)minimizesD𝒘​\(𝜽∥𝜽′\)D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)overΘi​j\\Theta\_\{ij\}\. Lemma[4\.2](https://arxiv.org/html/2607.08979#S4.Thmtheorem2)\(c\) says that, unless this minimizer is forced onto∂Θ\\partial\\Thetaby the constraint‖𝜽′‖∞≤R\\\|\\bm\{\\theta\}^\{\\prime\}\\\|\_\{\\infty\}\\leq R, the halfspace constraint\(𝜽i​j∗\)vi​j=\(𝜽i​j∗\)ui​j\(\\bm\{\\theta\}^\{\*\}\_\{ij\}\)\_\{v\_\{ij\}\}=\(\\bm\{\\theta\}^\{\*\}\_\{ij\}\)\_\{u\_\{ij\}\}is active; in other words, the most confusing alternative lies on the hyperplaneθvi​j′=θui​j′\\theta^\{\\prime\}\_\{v\_\{ij\}\}=\\theta^\{\\prime\}\_\{u\_\{ij\}\}\.

## 5Algorithm

We now construct an adaptive fixed\-confidence procedure when𝜽\\bm\{\\theta\}is not known\. The sampling rule is driven by the solution to the oracle saddle\-point problem\. At roundtt, we compute the current MLE𝜽^t\\hat\{\\bm\{\\theta\}\}\_\{t\}and its induced boundary setB^t:=B​\(𝜽^t\)\\hat\{B\}\_\{t\}:=B\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\), and evaluate the oracle objective at this estimate, that is we work withF​\(⋅,⋅;𝜽^t\)F\(\\cdot,\\cdot;\\hat\{\\bm\{\\theta\}\}\_\{t\}\)onΔ𝒫×ΔB^t\\Delta\_\{\\mathcal\{P\}\}\\times\\Delta\_\{\\hat\{B\}\_\{t\}\}\. Recomputing a saddle point at everyttis expensive and early on potentially unnecessary when the MLE has high variance\. Instead, we learn the saddle point online, maintaining iterates\(𝒘t,𝒒t\)\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\)and performing one primal–dual update step per round\.

Given\(𝒘t,𝒒t,𝜽^t−1\)\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\},\\hat\{\\bm\{\\theta\}\}\_\{t\-1\}\), a round proceeds as follows: \(i\) select the next pair to sampleAt=\(it,jt\)A\_\{t\}=\(i\_\{t\},j\_\{t\}\)based on\{𝒘s\}s=1t\\\{\\bm\{w\}\_\{s\}\\\}\_\{s=1\}^\{t\}; \(ii\) observe outcomeYtY\_\{t\}and update𝜽^t\\hat\{\\bm\{\\theta\}\}\_\{t\},S^t=S∗​\(𝜽^t\)\\hat\{S\}\_\{t\}=S^\{\*\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\), andB^t=B​\(𝜽^t\)\\hat\{B\}\_\{t\}=B\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\); \(iii\) check a stopping test to see if we are confident enough to terminate\. \(iv\) If we did not stop, perform one entropic FTRL primal–dual update using the estimated gradients ofF​\(⋅,⋅;𝜽^t\)F\(\\cdot,\\cdot;\\hat\{\\bm\{\\theta\}\}\_\{t\}\)at the current iterates\(𝒘t,𝒒t\)\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\), producing the next iterates\(𝒘t\+1,𝒒t\+1\)\(\\bm\{w\}\_\{t\+1\},\\bm\{q\}\_\{t\+1\}\)\. A high\-level summary of the procedure is given below; the full algorithm pseudocode with notation is provided at the end of this section in Algorithm[2](https://arxiv.org/html/2607.08979#alg2)\.

Algorithm 1\*High\-level Outline of Algorithm[2](https://arxiv.org/html/2607.08979#alg2)

1:for

t=1,2,…t=1,2,\\ldotsdo

2:Select a pair by tracking

\{𝒘s\}s=1t\\\{\\bm\{w\}\_\{s\}\\\}\_\{s=1\}^\{t\}\.\(Section[5\.4](https://arxiv.org/html/2607.08979#S5.SS4)\)

3:Observe outcome

YtY\_\{t\}; update

𝜽^t\\hat\{\\bm\{\\theta\}\}\_\{t\}\.\(Section[5\.1](https://arxiv.org/html/2607.08979#S5.SS1)\)

4:Check the stopping condition; if it is satisfied output

S^t\\hat\{S\}\_\{t\}and stop\.\(Section[5\.5](https://arxiv.org/html/2607.08979#S5.SS5)\)

5:Estimate the gradient of the current oracle game

F​\(𝒘t,𝒒t;𝜽^t\)F\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\.\(Section[5\.3](https://arxiv.org/html/2607.08979#S5.SS3)\)

6:Update

𝒘t\\bm\{w\}\_\{t\}and

𝒒t\\bm\{q\}\_\{t\}to

𝒘t\+1\\bm\{w\}\_\{t\+1\}and

𝒒t\+1\\bm\{q\}\_\{t\+1\}via entropic FTRL\.\(Section[5\.2](https://arxiv.org/html/2607.08979#S5.SS2)\)

7:endfor

### 5\.1Maximum Likelihood Estimate

Given observations\(A1,Y1\),…,\(At,Yt\)\(A\_\{1\},Y\_\{1\}\),\\ldots,\(A\_\{t\},Y\_\{t\}\), define the log\-likelihood \(up to additive constants\)

ℓt​\(𝜽\):=∑s=1t\[ηAs​\(𝜽\)​T​\(Ys\)−A​\(ηAs​\(𝜽\)\)\]\.\\ell\_\{t\}\(\\bm\{\\theta\}\):=\\sum\_\{s=1\}^\{t\}\\bigl\[\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}\)\\,T\(Y\_\{s\}\)\-A\(\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}\)\)\\bigr\]\.Let𝜽^t∈argmax𝜽∈Θℓt​\(𝜽\)\\hat\{\\bm\{\\theta\}\}\_\{t\}\\in\\mathop\{\\mathrm\{argmax\}\}\_\{\\bm\{\\theta\}\\in\\Theta\}\\ell\_\{t\}\(\\bm\{\\theta\}\)denote the MLE\.

### 5\.2Entropic FTRL Updates

We update the primal and dual iterates from\(𝒘t,𝒒t\)\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\)to\(𝒘t\+1,𝒒t\+1\)\(\\bm\{w\}\_\{t\+1\},\\bm\{q\}\_\{t\+1\}\)using entropic FTRL with gradient estimates ofF​\(⋅,⋅;𝜽^t\)F\(\\cdot,\\cdot;\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\. The primal player maximizes over𝒘∈Δ𝒫\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}; the dual player minimizes over𝒒∈ΔB^t\\bm\{q\}\\in\\Delta\_\{\\hat\{B\}\_\{t\}\}\. At roundtt, we construct unbiased gradient estimates𝒈^t\(w\)\\hat\{\\bm\{g\}\}\_\{t\}^\{\(w\)\}and𝒈^t\(q\)\\hat\{\\bm\{g\}\}\_\{t\}^\{\(q\)\}for the partial derivatives in equation \([1](https://arxiv.org/html/2607.08979#S4.E1)\) evaluated at\(𝒘t,𝒒t;𝜽^t\)\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\. Given these estimates, define cumulative scores

𝚿t\(w\):=𝚿t−1\(w\)\+𝒈^t\(w\),𝚿t\(q\):=𝚿t−1\(q\)\+𝒈^t\(q\)\\bm\{\\Psi\}^\{\(w\)\}\_\{t\}:=\\bm\{\\Psi\}^\{\(w\)\}\_\{t\-1\}\+\\hat\{\\bm\{g\}\}^\{\(w\)\}\_\{t\},\\qquad\\bm\{\\Psi\}^\{\(q\)\}\_\{t\}:=\\bm\{\\Psi\}^\{\(q\)\}\_\{t\-1\}\+\\hat\{\\bm\{g\}\}^\{\(q\)\}\_\{t\}with𝚿0\(w\)=𝚿0\(q\)=𝟎\\bm\{\\Psi\}^\{\(w\)\}\_\{0\}=\\bm\{\\Psi\}^\{\(q\)\}\_\{0\}=\\mathbf\{0\}\. The entropic FTRL updates are defined by

𝒘t\+1\\displaystyle\\bm\{w\}\_\{t\+1\}=argmax𝒘∈Δ𝒫\{⟨𝒘,𝚿t\(w\)⟩−1μt\+1​KL⁡\(𝒘∥𝒘1\)\},\\displaystyle=\\mathop\{\\mathrm\{argmax\}\}\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\Big\\\{\\langle\\bm\{w\},\\bm\{\\Psi\}^\{\(w\)\}\_\{t\}\\rangle\-\\tfrac\{1\}\{\\mu\_\{t\+1\}\}\\operatorname\{KL\}\(\\bm\{w\}\\\|\\bm\{w\}\_\{1\}\)\\Big\\\},𝒒t\+1\\displaystyle\\bm\{q\}\_\{t\+1\}=argmin𝒒∈ΔB^t\{⟨𝒒,𝚿t\(q\)⟩\+1μt\+1​KL⁡\(𝒒∥𝒒1\)\}\.\\displaystyle=\\mathop\{\\mathrm\{argmin\}\}\_\{\\bm\{q\}\\in\\Delta\_\{\\hat\{B\}\_\{t\}\}\}\\Big\\\{\\langle\\bm\{q\},\\bm\{\\Psi\}^\{\(q\)\}\_\{t\}\\rangle\+\\tfrac\{1\}\{\\mu\_\{t\+1\}\}\\operatorname\{KL\}\(\\bm\{q\}\\\|\\bm\{q\}\_\{1\}\)\\Big\\\}\.For each\(a,b\)∈𝒫\(a,b\)\\in\\mathcal\{P\}it follows that the primal updates are

wt\+1,a​b=w1,a​b​exp⁡\(μt\+1​Ψt,a​b\(w\)\)∑\(c,d\)∈𝒫w1,c​d​exp⁡\(μt\+1​Ψt,c​d\(w\)\)w\_\{t\+1,ab\}=\\frac\{w\_\{1,ab\}\\exp\\\!\\big\(\\mu\_\{t\+1\}\\Psi^\{\(w\)\}\_\{t,ab\}\\big\)\}\{\\sum\_\{\(c,d\)\\in\\mathcal\{P\}\}w\_\{1,cd\}\\exp\\\!\\big\(\\mu\_\{t\+1\}\\Psi^\{\(w\)\}\_\{t,cd\}\\big\)\}\(2\)For the dual variable, the feasible setΔB^t\\Delta\_\{\\hat\{B\}\_\{t\}\}depends on𝜽^t\\hat\{\\bm\{\\theta\}\}\_\{t\}throughB^t\\hat\{B\}\_\{t\}, so it is convenient to implement the update by maintaining an iterate over all pairs and then restricting to the current boundary set\. Define the unnormalized weights

rt\+1,i​j:=q1,i​j​exp⁡\(−μt\+1​Ψt,i​j\(q\)\),\(i,j\)∈𝒫\.r\_\{t\+1,ij\}:=q\_\{1,ij\}\\exp\\\!\\big\(\-\\mu\_\{t\+1\}\\Psi^\{\(q\)\}\_\{t,ij\}\\big\),\\qquad\(i,j\)\\in\\mathcal\{P\}\.Then the solution of the dual FTRL problem above is obtained by restricting and renormalizing on the current boundary set:

qt\+1,i​j:=rt\+1,i​j∑\(u,v\)∈B^trt\+1,u​v⋅𝟏​\{\(i,j\)∈B^t\}\.q\_\{t\+1,ij\}:=\\frac\{r\_\{t\+1,ij\}\}\{\\sum\_\{\(u,v\)\\in\\hat\{B\}\_\{t\}\}r\_\{t\+1,uv\}\}\\cdot\\mathbf\{1\}\\\{\(i,j\)\\in\\hat\{B\}\_\{t\}\\\}\.Initializing𝒘1\\bm\{w\}\_\{1\}to be uniform over𝒫\\mathcal\{P\}and updating via FTRL ensureswt,a​b\>0w\_\{t,ab\}\>0for every pair and everytt, soG​\(𝒘t\)G\(\\bm\{w\}\_\{t\}\)is complete \(hence connected\) for alltt\. By Lemma[4\.2](https://arxiv.org/html/2607.08979#S4.Thmtheorem2), each KL projection𝜽i​j∗​\(𝒘t;𝜽^t\)\\bm\{\\theta\}^\{\*\}\_\{ij\}\(\\bm\{w\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\)exists and is unique, so the derivatives in \([1](https://arxiv.org/html/2607.08979#S4.E1)\) are well\-defined at\(𝒘t,𝒒t;𝜽^t\)\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\.

The learning rateμt\\mu\_\{t\}controls how strongly each new gradient increment changes the weights\. To obtain standard no\-regret guarantees we takeμt=t−α\\mu\_\{t\}=t^\{\-\\alpha\}, whereα∈\(0,1\)\\alpha\\in\(0,1\), so the updates become progressively less aggressive asttgrows\.

### 5\.3Stochastic Gradients

Implementing the updates above requires evaluating the partial derivatives in \([1](https://arxiv.org/html/2607.08979#S4.E1)\)\. This requires finding𝜽i​j∗​\(𝒘t;𝜽^t\)∈argmin𝜽′∈Θi​j​\(t\)D𝒘t​\(𝜽^t∥𝜽′\)\\bm\{\\theta\}^\{\*\}\_\{ij\}\(\\bm\{w\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\\in\\mathop\{\\mathrm\{argmin\}\}\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}\(t\)\}D\_\{\\bm\{w\}\_\{t\}\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\\bm\{\\theta\}^\{\\prime\}\)for each boundary pair\(i,j\)∈B^t\(i,j\)\\in\\hat\{B\}\_\{t\}\. Each projection can be solved inO​\(n2\)O\(n^\{2\}\)time, so solving allk​\(n−k\)k\(n\-k\)projections can make the algorithm unusable in many cases\. Instead, we use an unbiased gradient estimate obtained by sampling a single boundary pair\. SampleIt∼Unif​\(B^t\)I\_\{t\}\\sim\\mathrm\{Unif\}\(\\hat\{B\}\_\{t\}\)and compute

𝜽t∗∈argmin𝜽′∈ΘIt​\(t\)D𝒘t​\(𝜽^t∥𝜽′\),γt:=D𝒘t​\(𝜽^t∥𝜽t∗\)\.\\bm\{\\theta\}\_\{t\}^\{\*\}\\in\\mathop\{\\mathrm\{argmin\}\}\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{I\_\{t\}\}\(t\)\}D\_\{\\bm\{w\}\_\{t\}\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\\bm\{\\theta\}^\{\\prime\}\),\\quad\\gamma\_\{t\}:=D\_\{\\bm\{w\}\_\{t\}\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\\bm\{\\theta\}\_\{t\}^\{\*\}\)\.Define the gradient estimates

g^t,a​b\(w\):=m​qt,It​da​b​\(𝜽^t,𝜽t∗\),\(a,b\)∈𝒫,g^t,i​j\(q\):=m​γt​1​\{\(i,j\)=It\},\(i,j\)∈B^t,\\begin\{split\}\\hat\{g\}^\{\(w\)\}\_\{t,ab\}&:=m\\,q\_\{t,I\_\{t\}\}\\,d\_\{ab\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\},\\bm\{\\theta\}\_\{t\}^\{\*\}\),\\quad\(a,b\)\\in\\mathcal\{P\},\\\\ \\hat\{g\}^\{\(q\)\}\_\{t,ij\}&:=m\\,\\gamma\_\{t\}\\,\\mathbf\{1\}\\\{\(i,j\)=I\_\{t\}\\\},\\quad\(i,j\)\\in\\hat\{B\}\_\{t\},\\end\{split\}\(3\)wherem=k​\(n−k\)m=k\(n\-k\)\. Conditioned on the past iterates,𝔼​\[𝒈^t\(w\)\]=∇𝒘F​\(𝒘t,𝒒t;𝜽^t\)\\mathbb\{E\}\[\\hat\{\\bm\{g\}\}^\{\(w\)\}\_\{t\}\]=\\nabla\_\{\\bm\{w\}\}F\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\)and𝔼​\[𝒈^t\(q\)\]=∇𝒒F​\(𝒘t,𝒒t;𝜽^t\)\\mathbb\{E\}\[\\hat\{\\bm\{g\}\}^\{\(q\)\}\_\{t\}\]=\\nabla\_\{\\bm\{q\}\}F\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\.

### 5\.4Sampling via C\-Tracking

To select the pairAtA\_\{t\}, we use the C\-tracking procedure fromGarivier & Kaufmann \([2016](https://arxiv.org/html/2607.08979#bib.bib13)\)\. Fix a mixing scheduleρt\\rho\_\{t\}and form the mixed target𝒘~t:=\(1−ρt\)​𝒘t\+ρt​𝒖,\\tilde\{\\bm\{w\}\}\_\{t\}:=\(1\-\\rho\_\{t\}\)\\bm\{w\}\_\{t\}\+\\rho\_\{t\}\\bm\{u\},where𝒖\\bm\{u\}is uniform weights over all pairs\. DefinePi​j​\(t\):=∑s=1tw~s,i​jP\_\{ij\}\(t\):=\\sum\_\{s=1\}^\{t\}\\tilde\{w\}\_\{s,ij\}and select

At∈argmax\(i,j\)∈𝒫\(Pi​j​\(t\)−Ni​j​\(t−1\)\),A\_\{t\}\\in\\mathop\{\\mathrm\{argmax\}\}\_\{\(i,j\)\\in\\mathcal\{P\}\}\\bigl\(P\_\{ij\}\(t\)\-N\_\{ij\}\(t\-1\)\\bigr\),whereNi​j​\(t\):=∑s=1t𝟏​\{As=\(i,j\)\}N\_\{ij\}\(t\):=\\sum\_\{s=1\}^\{t\}\\mathbf\{1\}\\\{A\_\{s\}=\(i,j\)\\\}are the empirical counts\.

C\-tracking ensures the empirical proportionsw^t,i​jemp:=Ni​j​\(t\)/t\\hat\{w\}^\{\\mathrm\{emp\}\}\_\{t,ij\}:=N\_\{ij\}\(t\)/ttrack the running average\(1/t\)​∑s=1tw~s,i​j\(1/t\)\\sum\_\{s=1\}^\{t\}\\tilde\{w\}\_\{s,ij\}, which converges to an optimal design\. The uniform mixtureρt​𝒖\\rho\_\{t\}\\bm\{u\}forces exploration of all pairs and takingρt=t−γ\\rho\_\{t\}=t^\{\-\\gamma\}, whereγ∈\(0,1\)\\gamma\\in\(0,1\), ensures that this exploration does not bias the allocations asymptotically\.

### 5\.5Stopping Rule

The threshold \([4](https://arxiv.org/html/2607.08979#S5.E4)\) and stopping time \([5](https://arxiv.org/html/2607.08979#S5.E5)\) are derived using a standard likelihood\-ratio martingale argument\(Kaufmann & Koolen,[2021](https://arxiv.org/html/2607.08979#bib.bib21)\)combined with a self\-normalized concentration bound used commonly in linear bandits\(Abbasi\-Yadkori et al\.,[2011](https://arxiv.org/html/2607.08979#bib.bib1); Lattimore & Szepesvári,[2020](https://arxiv.org/html/2607.08979#bib.bib25)\)\. For each estimated boundary pair\(i,j\)∈B^t\(i,j\)\\in\\hat\{B\}\_\{t\}\(w\.l\.o\.g\.θ^t,i≥θ^t,\(k\)\>θ^t,j\\hat\{\\theta\}\_\{t,i\}\\geq\\hat\{\\theta\}\_\{t,\(k\)\}\>\\hat\{\\theta\}\_\{t,j\}\), the statisticZi​j​\(t\)Z\_\{ij\}\(t\)compares the likelihood at the current MLE𝜽^t\\hat\{\\bm\{\\theta\}\}\_\{t\}to the largest likelihood under alternative parameters satisfyingθi′≤θj′\\theta^\{\\prime\}\_\{i\}\\leq\\theta^\{\\prime\}\_\{j\}\. Larger values indicate that all parameter vectors that swap this boundary pair explain the data significantly worse than the MLE\. The stopping condition given in Equation \([5](https://arxiv.org/html/2607.08979#S5.E5)\) is satisfied when the minimum of these boundary\-pair statistics exceeds the threshold in \([4](https://arxiv.org/html/2607.08979#S5.E4)\)\.

For each boundary pair\(i,j\)∈B^t\(i,j\)\\in\\hat\{B\}\_\{t\}, compute theZi​j​\(t\):=ℓt​\(𝜽^t\)−sup𝜽′∈Θi​j​\(t\)ℓt​\(𝜽′\)\.Z\_\{ij\}\(t\):=\\ell\_\{t\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\-\\sup\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}\(t\)\}\\ell\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\.For anyλ\>0\\lambda\>0define the threshold

β​\(t,δ\):=log⁡1δ\+λ2​‖𝜽^t‖22\+12​log​det\(In\+σ¯2λ​ℒt\),\\beta\(t,\\delta\):=\\log\\frac\{1\}\{\\delta\}\+\\frac\{\\lambda\}\{2\}\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\_\{2\}^\{2\}\+\\frac\{1\}\{2\}\\log\\det\\\!\\Big\(I\_\{n\}\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\mathcal\{L\}\_\{t\}\\Big\),\(4\)whereℒt:=∑s=1t𝒙As​𝒙As⊤\\mathcal\{L\}\_\{t\}:=\\sum\_\{s=1\}^\{t\}\\bm\{x\}\_\{A\_\{s\}\}\\bm\{x\}\_\{A\_\{s\}\}^\{\\top\}andσ¯2\\overline\{\\sigma\}^\{2\}is from Assumption[1](https://arxiv.org/html/2607.08979#Thmassumption1)\. We define the stopping time

τδ:=inf\{t≥1:min\(i,j\)∈B^t⁡Zi​j​\(t\)≥β​\(t,δ\)\}\.\\tau\_\{\\delta\}:=\\inf\\Bigl\\\{t\\geq 1:\\ \\min\_\{\(i,j\)\\in\\hat\{B\}\_\{t\}\}Z\_\{ij\}\(t\)\\ \\geq\\ \\beta\(t,\\delta\)\\Bigr\\\}\.\(5\)Upon stopping, outputS^τδ=S∗​\(𝜽^τδ\)\\hat\{S\}\_\{\\tau\_\{\\delta\}\}=S^\{\*\}\(\\hat\{\\bm\{\\theta\}\}\_\{\\tau\_\{\\delta\}\}\)\.

Computing the stopping condition can be costly\. It requires computingZi​j​\(t\)Z\_\{ij\}\(t\)for every boundary pair, which scales withO​\(k​\(n−k\)​n2\)O\\\!\\big\(k\(n\-k\)n^\{2\}\\big\)\. In practice, for large problems, the stopping check can be performed on a sparse schedule to reduce computational cost, with only a small impact on the expected stopping time, and no impact on the asymptotic optimality guarantee\.

Algorithm 2Oracle\-Tracking Top\-kkIdentification0:Risk

δ\\delta, exponents

α,γ∈\(0,1\)\\alpha,\\gamma\\in\(0,1\)
1:Initialize

𝒘1=𝒒1=𝒓1=𝒖\\bm\{w\}\_\{1\}=\\bm\{q\}\_\{1\}=\\bm\{r\}\_\{1\}=\\bm\{u\};

𝚿0\(w\)=𝚿0\(q\)=𝟎\\bm\{\\Psi\}^\{\(w\)\}\_\{0\}=\\bm\{\\Psi\}^\{\(q\)\}\_\{0\}=\\mathbf\{0\};

Ni​j​\(0\)=Pi​j​\(0\)=0N\_\{ij\}\(0\)=P\_\{ij\}\(0\)=0
2:for

t=1,2,…t=1,2,\\ldotsdo

3:Set

μt\+1=\(t\+1\)−α\\mu\_\{t\+1\}=\(t\+1\)^\{\-\\alpha\},

ρt=t−γ\\rho\_\{t\}=t^\{\-\\gamma\},

𝒘~t=\(1−ρt\)​𝒘t\+ρt​𝒖\\tilde\{\\bm\{w\}\}\_\{t\}=\(1\-\\rho\_\{t\}\)\\bm\{w\}\_\{t\}\+\\rho\_\{t\}\\bm\{u\}
4:Select

At∈argmax\(i,j\)∈𝒫\{Pi​j​\(t−1\)\+w~t,i​j−Ni​j​\(t−1\)\}A\_\{t\}\\in\\mathop\{\\mathrm\{argmax\}\}\_\{\(i,j\)\\in\\mathcal\{P\}\}\\\{P\_\{ij\}\(t\-1\)\+\\tilde\{w\}\_\{t,ij\}\-N\_\{ij\}\(t\-1\)\\\}
5:Observe

YtY\_\{t\}; update MLE

𝜽^t\\hat\{\\bm\{\\theta\}\}\_\{t\}; set

B^t=B​\(𝜽^t\)\\hat\{B\}\_\{t\}=B\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\)
6:If

min\(i,j\)∈B^t⁡Zi​j​\(t\)≥β​\(t,δ\)\\min\_\{\(i,j\)\\in\\hat\{B\}\_\{t\}\}Z\_\{ij\}\(t\)\\geq\\beta\(t,\\delta\): output

S∗​\(𝜽^t\)S^\{\*\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\)and stop

7:

qt,i​j←rt,i​j∑\(u,v\)∈B^trt,u​v​𝟏​\{\(i,j\)∈B^t\}q\_\{t,ij\}\\leftarrow\\frac\{r\_\{t,ij\}\}\{\\sum\_\{\(u,v\)\\in\\hat\{B\}\_\{t\}\}r\_\{t,uv\}\}\\mathbf\{1\}\\\{\(i,j\)\\in\\hat\{B\}\_\{t\}\\\}
8:Sample

It∼Unif​\(B^t\)I\_\{t\}\\sim\\mathrm\{Unif\}\(\\hat\{B\}\_\{t\}\); compute

𝜽t∗=argmin𝜽′∈ΘItD𝒘t​\(𝜽^t∥𝜽′\)\\bm\{\\theta\}\_\{t\}^\{\*\}=\\mathop\{\\mathrm\{argmin\}\}\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{I\_\{t\}\}\}D\_\{\\bm\{w\}\_\{t\}\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\\bm\{\\theta\}^\{\\prime\}\)
9:

g^t,a​b\(w\)←m​qt,It​da​b​\(𝜽^t,𝜽t∗\)\\hat\{g\}^\{\(w\)\}\_\{t,ab\}\\leftarrow m\\,q\_\{t,I\_\{t\}\}\\,d\_\{ab\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\},\\bm\{\\theta\}\_\{t\}^\{\*\}\);

g^t,i​j\(q\)←m​γIt​\(𝒘t;𝜽^t\)​1​\{\(i,j\)=It\}\\hat\{g\}^\{\(q\)\}\_\{t,ij\}\\leftarrow m\\,\\gamma\_\{I\_\{t\}\}\(\\bm\{w\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\\,\\mathbf\{1\}\\\{\(i,j\)=I\_\{t\}\\\}
10:

𝚿t\(w\)←𝚿t−1\(w\)\+𝒈^t\(w\)\\bm\{\\Psi\}^\{\(w\)\}\_\{t\}\\leftarrow\\bm\{\\Psi\}^\{\(w\)\}\_\{t\-1\}\+\\hat\{\\bm\{g\}\}^\{\(w\)\}\_\{t\};

𝚿t\(q\)←𝚿t−1\(q\)\+𝒈^t\(q\)\\bm\{\\Psi\}^\{\(q\)\}\_\{t\}\\leftarrow\\bm\{\\Psi\}^\{\(q\)\}\_\{t\-1\}\+\\hat\{\\bm\{g\}\}^\{\(q\)\}\_\{t\}
11:Update

\(𝒘t\+1,𝒓t\+1\)\(\\bm\{w\}\_\{t\+1\},\\bm\{r\}\_\{t\+1\}\)by FTRL on

F​\(⋅,⋅;𝜽^t\)F\(\\cdot,\\cdot;\\hat\{\\bm\{\\theta\}\}\_\{t\}\)
12:endfor

## 6Theoretical Guarantees of Algorithm[2](https://arxiv.org/html/2607.08979#alg2)

In this section, we establish theoretical guarantees for Algorithm[2](https://arxiv.org/html/2607.08979#alg2)\. We show: \(i\) the procedure isδ\\delta\-correct \(Proposition[6\.4](https://arxiv.org/html/2607.08979#S6.Thmtheorem4)\), and \(ii\) its expected stopping time asymptotically matches the information\-theoretic lower bound of Theorem[3\.4](https://arxiv.org/html/2607.08979#S3.Thmtheorem4)\(Theorem[6\.5](https://arxiv.org/html/2607.08979#S6.Thmtheorem5)\)\. The main technical step is Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)showing that for sufficiently largettwith high probability the empirical information rateΓ​\(𝒘^temp;𝜽\)\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)is close to the oracle valueΓ∗​\(𝜽\)\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\. We start by showing the MLE is consistent\.

###### Lemma 6\.1\.

DefineNmin​\(t\):=min\(i,j\)∈𝒫⁡Ni​j​\(t\)N\_\{\\min\}\(t\):=\\min\_\{\(i,j\)\\in\\mathcal\{P\}\}N\_\{ij\}\(t\)\. There is a finite timetwarmt\_\{\\mathrm\{warm\}\}such that for everyt≥twarmt\\geq t\_\{\\mathrm\{warm\}\}and everyδ∈\(0,1\)\\delta\\in\(0,1\), with probability at least1−δ1\-\\delta,

‖𝜽^t−𝜽‖2≤2​σa¯​n​log⁡\(2​t​\|𝒫\|δ\)Nmin​\(t\)\.\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\-\\bm\{\\theta\}\\\|\_\{2\}\\ \\leq\\ \\frac\{2\\sigma\}\{\\underline\{a\}\}\\sqrt\{\\frac\{n\\log\\\!\\big\(\\frac\{2t\|\\mathcal\{P\}\|\}\{\\delta\}\\big\)\}\{N\_\{\\min\}\(t\)\}\}\.

###### Corollary 6\.2\.

𝜽^t→𝜽\\hat\{\\bm\{\\theta\}\}\_\{t\}\\to\\bm\{\\theta\}almost surely\. Consequently, there exists almost surely a finite timetstabt\_\{\\mathrm\{stab\}\}such that for allt≥tstabt\\geq t\_\{\\mathrm\{stab\}\},S^t=S∗​\(𝛉\),and​B^t=B​\(𝛉\)\.\\hat\{S\}\_\{t\}=S^\{\*\}\(\\bm\{\\theta\}\),\\text\{ and \}\\hat\{B\}\_\{t\}=B\(\\bm\{\\theta\}\)\.

For the analysis in Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)we work on a truncated window\. Fixttand define the burn\-in indexbt:=⌈t1/4⌉b\_\{t\}:=\\lceil t^\{1/4\}\\rceil\. A union bound applied to Lemma[6\.1](https://arxiv.org/html/2607.08979#S6.Thmtheorem1)overs=bt,…,ts=b\_\{t\},\\ldots,tyields an event of probability at least1−t−p1\-t^\{\-p\}on whichB^s=B​\(𝜽\)\\hat\{B\}\_\{s\}=B\(\\bm\{\\theta\}\)for alls∈\{bt,…,t\}s\\in\\\{b\_\{t\},\\ldots,t\\\}\. This is the regime in which the oracle\-tracking regret analysis applies\.

###### Proposition 6\.3\(High\-probability oracle\-rate lower bound\)\.

Fix anyp\>1p\>1\. There exist constantsCp<∞C\_\{p\}<\\inftyandtp<∞t\_\{p\}<\\inftysuch that for allt≥tpt\\geq t\_\{p\},

ℙ𝜽\(Γ\(𝒘^temp;𝜽\)≤Γ∗\(𝜽\)−Cp​\(t−\(1−α\)\+t−α\+log⁡tt\)⏟oracle\-tracking regret\\displaystyle\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\Bigg\(\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\leq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\underbrace\{C\_\{p\}\\big\(t^\{\-\(1\-\\alpha\)\}\+t^\{\-\\alpha\}\+\\sqrt\{\\tfrac\{\\log t\}\{t\}\}\\big\)\}\_\{\\text\{oracle\-tracking regret\}\}−Cp​t−\(1−γ\)/2​log⁡t⏟estimation−Dmax​\|𝒫\|​\(\|𝒫\|−1t\+t−γ1−γ\)⏟mixing\\displaystyle\\qquad\-\\underbrace\{C\_\{p\}\\,t^\{\-\(1\-\\gamma\)/2\}\\sqrt\{\\log t\}\}\_\{\\text\{estimation\}\}\-\\underbrace\{D\_\{\\max\}\|\\mathcal\{P\}\|\\big\(\\tfrac\{\|\\mathcal\{P\}\|\-1\}\{t\}\+\\tfrac\{t^\{\-\\gamma\}\}\{1\-\\gamma\}\\big\)\}\_\{\\text\{mixing\}\}−2​Dmax​t−3/4⏟burn\-in\)≤t−p\.\\displaystyle\\qquad\-\\underbrace\{2D\_\{\\max\}\\,t^\{\-3/4\}\}\_\{\\text\{burn\-in\}\}\\Bigg\)\\leq t^\{\-p\}\.

Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)links the information rate our algorithm achieves empirically to the optimal rate as a function oftt\. There are four terms contributing to finite time non\-optimality which all vanish asymptotically\. The first error term is from the oracle tracking regret from using FTRL\. The second and third terms represent deviations from optimal allocation arising from estimation error and mixing bias\. Here, there is a trade\-off controlled byγ\\gamma\. A largerγ\\gammameans faster decay of the mixing weight, thereby decreasing mixing bias; however, estimation bias increases\. Ignoring polylog factors,γ=1/3\\gamma=1/3maximizes the minimum decay rate of these two terms\. Finally, the last term corresponds to the burn\-in phase\. In this phase, the estimated boundary may not be the correct boundary, but we can still bound the overall loss accumulated in this phase\.

Next, we show that the procedure isδ\\delta\-correct\.

###### Proposition 6\.4\.

Fixδ∈\(0,1\)\\delta\\in\(0,1\)\. Run Algorithm[2](https://arxiv.org/html/2607.08979#alg2)with stopping timeτδ\\tau\_\{\\delta\}defined in \([5](https://arxiv.org/html/2607.08979#S5.E5)\)\. Then the procedure isδ\\delta\-correct, i\.e\.

- \(i\)ℙ𝜽​\(S^τδ≠S∗​\(𝜽\)\)≤δ\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\big\(\\hat\{S\}\_\{\\tau\_\{\\delta\}\}\\neq S^\{\*\}\(\\bm\{\\theta\}\)\\big\)\\leq\\delta\.
- \(ii\)ℙ𝜽​\(τδ<∞\)=1\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\tau\_\{\\delta\}<\\infty\)=1\.

For part \(ii\), we show that the thresholdβ​\(t,δ\)\\beta\(t,\\delta\)grows at most logarithmically intt, whereasmin\(i,j\)∈B^t⁡Zi​j​\(t\)\\min\_\{\(i,j\)\\in\\hat\{B\}\_\{t\}\}Z\_\{ij\}\(t\)eventually grows linearly intt\. Consequently, almost surely there exists a finite time at which the stopping condition is met\.

Finally, we show that the expected stopping time matches the information\-theoretic lower bound\.

###### Theorem 6\.5\(Asymptotic optimality in expectation\)\.

Fix𝛉∈Θgap\\bm\{\\theta\}\\in\\Theta^\{\\mathrm\{gap\}\}and letτδ\\tau\_\{\\delta\}be the stopping time of Algorithm[2](https://arxiv.org/html/2607.08979#alg2)\. Then

limδ→0𝔼𝜽​\[τδ\]log⁡\(1/δ\)=1Γ∗​\(𝜽\)\.\\lim\_\{\\delta\\to 0\}\\ \\frac\{\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\[\\tau\_\{\\delta\}\]\}\{\\log\(1/\\delta\)\}\\;=\\;\\frac\{1\}\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}\.

On a high level this follows from Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3): for largettthe algorithm collects information at a rate arbitrarily close toΓ∗​\(𝜽\)\\Gamma^\{\*\}\(\\bm\{\\theta\}\), so the stopping rule triggers after at mostlog⁡\(1/δ\)/Γ∗​\(𝜽\)\\log\(1/\\delta\)/\\Gamma^\{\*\}\(\\bm\{\\theta\}\)samples \(up to lower\-order terms\)\. On the other hand, Theorem[3\.4](https://arxiv.org/html/2607.08979#S3.Thmtheorem4)shows that noδ\\delta\-correct procedure can asymptotically do better in expectation\.

![Refer to caption](https://arxiv.org/html/2607.08979v1/fig_combined.png)Figure 1:Mean stopping times over 100 simulations withδ=0\.01\\delta=0\.01\. Columns correspond to the three instances; rows correspond to \(a\) varyingnnwithk=5k=5fixed, and \(b\) varyingkkwithn=100n=100fixed\.
## 7Numerical Illustrations

We evaluate Algorithm[2](https://arxiv.org/html/2607.08979#alg2)on three synthetic instances\. For each, we vary the number of itemsnn\(withk=5k=5fixed\) and varykk\(withn=100n=100fixed\) fixingδ=0\.01\\delta=0\.01\.

Following Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3), we set the mixing exponent toγ=1/3\\gamma=1/3\. For the learning\-rate scheduleμt=t−α\\mu\_\{t\}=t^\{\-\\alpha\}, we setα=0\.2\\alpha=0\.2\. With stochastic gradients, the early gradient directions can be noisy111See Appendix[G\.3](https://arxiv.org/html/2607.08979#A7.SS3)for further discussion on this and tuningα\\alpha\., and if the decay rate is too fast, these will dominate the cumulative gradient scores𝚿t\(w\),𝚿t\(q\)\\bm\{\\Psi\}^\{\(w\)\}\_\{t\},\\bm\{\\Psi\}^\{\(q\)\}\_\{t\}for finitett\.α\\alphacan be tuned by running the primal–dual algorithm on the oracle problem \(with𝜽\\bm\{\\theta\}known\) over a collection of𝜽\\bm\{\\theta\}instances, and selecting the value that converges both quickly and stably to𝒘∗​\(𝜽\)\\bm\{w\}^\{\*\}\(\\bm\{\\theta\}\)\.

Baselines\.We compare against the most competitive fixed\-confidence baselines forδ\\delta\-correct top\-kkidentification from pairwise comparisons: \(i\)*SEEKS*and*SEEKS\-v2*\(Ren et al\.,[2020](https://arxiv.org/html/2607.08979#bib.bib30)\), \(ii\)*Active Ranking*fromHeckel et al\. \([2019](https://arxiv.org/html/2607.08979#bib.bib17)\)\.

We consider three data\-generating processes; in all experiments, the comparison feedback is binary, and Algorithm[2](https://arxiv.org/html/2607.08979#alg2)is implemented using the Bradley–Terry likelihood\.222We are not aware of fixed\-confidence top\-kkidentification baselines for cardinal pairwise feedback that would allow a direct comparison\.

Random utilities\.Drawθ~1,…,θ~n∼i\.i\.d\.Unif​\[−5,5\]\\tilde\{\\theta\}\_\{1\},\\ldots,\\tilde\{\\theta\}\_\{n\}\\stackrel\{\{\\scriptstyle\\text\{i\.i\.d\.\}\}\}\{\{\\sim\}\}\\mathrm\{Unif\}\[\-5,5\]and center by𝜽:=𝜽~−1n​\(∑i=1nθ~i\)​𝟏\\bm\{\\theta\}:=\\tilde\{\\bm\{\\theta\}\}\-\\frac\{1\}\{n\}\\big\(\\sum\_\{i=1\}^\{n\}\\tilde\{\\theta\}\_\{i\}\\big\)\\mathbf\{1\}\. For computational purposes, we reject the draw unlessθk−θk\+1≥0\.02\\theta\_\{k\}\-\\theta\_\{k\+1\}\\geq 0\.02\. Data are generated under the Bradley–Terry model\.

Equally spaced utilities\.Fixgap=0\.1\\mathrm\{gap\}=0\.1and setθi=n\+1−2​i2⋅gap\\theta\_\{i\}=\\frac\{n\+1\-2i\}\{2\}\\cdot\\mathrm\{gap\}fori∈\[n\]i\\in\[n\]\. Comparisons are generated under the Bradley–Terry model as above\.

SST preference matrix\.To test the algorithm when the model is misspecified, we generate outcomes from a pairwise probability matrixPPthat satisfies SST but is not induced by latent utilities\. The construction closely follows the “independent bands” generative process ofShah et al\. \([2017](https://arxiv.org/html/2607.08979#bib.bib33)\)\. FixΔ=0\.05\\Delta=0\.05and constructPPdiagonal\-by\-diagonal\. Setpi​i=12p\_\{ii\}=\\tfrac\{1\}\{2\}and drawpi,i\+1∼Unif​\[pi​i,pi​i\+Δ\]p\_\{i,i\+1\}\\sim\\mathrm\{Unif\}\[p\_\{ii\},p\_\{ii\}\+\\Delta\]fori=1,…,n−1i=1,\\ldots,n\-1\. Then, for each subsequent diagonal band, define the lower boundLi​j:=max⁡\{pi,j−1,pi\+1,j\}L\_\{ij\}:=\\max\\\{p\_\{i,j\-1\},\\,p\_\{i\+1,j\}\\\}and drawpi​j∼Unif​\[Li​j,min⁡\{Li​j\+Δ,1\}\]p\_\{ij\}\\sim\\mathrm\{Unif\}\\bigl\[L\_\{ij\},\\,\\min\\\{L\_\{ij\}\+\\Delta,\\,1\\\}\\bigr\]\. This ensurespi​j≥max⁡\{pi​ℓ,pℓ​j\}p\_\{ij\}\\geq\\max\\\{p\_\{i\\ell\},p\_\{\\ell j\}\\\}for alli<ℓ<ji<\\ell<j, satisfying SST\. Similar to the random utilities instance, we enforcepk,k\+1≥0\.51p\_\{k,k\+1\}\\geq 0\.51for computational reasons\.

### 7\.1Results

Figure[1](https://arxiv.org/html/2607.08979#S6.F1)presents the mean stopping time across 100 simulations on a log\-log scale\. All algorithms returned the correct top\-kkset across each instance\.

Random utilities\(left column\)\. Algorithm[2](https://arxiv.org/html/2607.08979#alg2)outperforms the baselines as bothnnandkkvary\. Asnnincreases, all algorithms’ stopping times increase because the instances become progressively harder\. The range of𝜽\\bm\{\\theta\}is fixed, so on average, there are boundary pairs with smaller utility gaps\.

Equally spaced\(center column\)\. In this experiment, we include an oracle benchmark\. The oracle computes𝒘∗​\(𝜽\)\\bm\{w\}^\{\*\}\(\\bm\{\\theta\}\)offline, allocates online according to those proportions, and uses the stopping rule in section[5\.5](https://arxiv.org/html/2607.08979#S5.SS5)\. We see that the stopping time of Algorithm[2](https://arxiv.org/html/2607.08979#alg2)closely tracks the oracle, demonstrating that even atδ=\.01\\delta=\.01our performance is close to the oracle performance\. However, asnngrows, the stopping time increases, but does not increase for SEEKS and SEEKS\-v2\. The primary driver is that our stopping threshold grows with‖𝜽^t‖22\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\_\{2\}^\{2\}, which grows withnnin this equally spaced instance\. Theorem[6\.5](https://arxiv.org/html/2607.08979#S6.Thmtheorem5)guarantees that for sufficiently smallδ\\deltaour expected stopping time will be smaller, and this is demonstrated in Figure[2](https://arxiv.org/html/2607.08979#S7.F2)\. However as‖𝜽‖22\\\|\\bm\{\\theta\}\\\|\_\{2\}^\{2\}grows the stopping threshold is only tight whenδ\\deltais small, solog⁡\(1δ\)\\log\(\\frac\{1\}\{\\delta\}\)is a dominant term inβ​\(t,δ\)\\beta\(t,\\delta\)\.

![Refer to caption](https://arxiv.org/html/2607.08979v1/delta_sweep_combined_vertical_v2.png)Figure 2:Mean stopping time over 100 simulations for the Equally Spaced instance withk=5k=5\.SST\(right column\)\. Even under misspecification, Algorithm[2](https://arxiv.org/html/2607.08979#alg2)remains competitive acrossnn\. We observe a similar pattern to the equally spaced instance\. Asnngrows,‖𝜽^t‖22\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\_\{2\}^\{2\}grows so the stopping threshold is loose atδ=\.01\\delta=\.01\.

## 8Conclusions

For top\-kkidentification from pairwise comparisons, we construct an online algorithm that is optimal asδ→0\\delta\\to 0\. Reducing the alternative set to parameters that invert a boundary pair yields a structured saddle\-point problem that naturally supports maintaining both primal and dual iterates\. This lowers the per\-round computational burden relative to approaches that rely on computing a best\-response oracle each iteration\. As a result, the method scales to moderately largenn\. In simulations, our procedure is competitive with existing baselines, and typically performs best when utilities are confined to a fixed range \(so many boundary gaps are small whennnis large\) and askkgrows\.

The main weakness of the algorithm is the stopping threshold at moderateδ\\delta, which in some instances can even cause the oracle to be outperformed by existing baselines\. Future work is required to investigate if a tighter stopping threshold can be constructed for moderateδ\\deltawhile maintaining optimality asδ→0\\delta\\to 0\.

## Impact Statement

This paper presents work whose goal is to advance the field of machine learning\. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here\.

## References

- Abbasi\-Yadkori et al\. \(2011\)Abbasi\-Yadkori, Y\., Pál, D\., and Szepesvári, C\.Improved algorithms for linear stochastic bandits\.In*Advances in Neural Information Processing Systems 24*, pp\. 2312–2320, 2011\.
- Arora et al\. \(2012\)Arora, S\., Hazan, E\., and Kale, S\.The multiplicative weights update method: a meta\-algorithm and applications\.*Theory of Computing*, 8:121–164, 2012\.
- Bengs et al\. \(2021\)Bengs, V\., Busa\-Fekete, R\., El Mesaoudi\-Paul, A\., and Hüllermeier, E\.Preference\-based online learning with dueling bandits: A survey\.*Journal of Machine Learning Research*, 22:1–108, 2021\.
- Braverman et al\. \(2019\)Braverman, M\., Mao, J\., and Peres, Y\.Sorted top\-k in rounds\.In*Proceedings of the Thirty\-Second Conference on Learning Theory*, pp\. 342–382, 2019\.
- Busa\-Fekete et al\. \(2013\)Busa\-Fekete, R\., Szörényi, B\., Weng, P\., Cheng, W\., and Hüllermeier, E\.Top\-kkselection based on adaptive sampling of noisy preferences\.In*Proceedings of the 30th International Conference on Machine Learning*, pp\. 1094–1102, 2013\.
- Chen et al\. \(2013\)Chen, X\., Bennett, P\. N\., Collins\-Thompson, K\., and Horvitz, E\.Pairwise ranking aggregation in a crowdsourced setting\.In*Proceedings of the Sixth ACM International Conference on Web Search and Data Mining*, pp\. 193–202, Rome, Italy, 2013\.
- Chen et al\. \(2018\)Chen, X\., Li, Y\., and Mao, J\.A nearly instance optimal algorithm for top\-kkranking under the multinomial logit model\.In*Proceedings of the Twenty\-Ninth Annual ACM\-SIAM Symposium on Discrete Algorithms*, pp\. 2504–2522, 2018\.
- Chen & Suh \(2015\)Chen, Y\. and Suh, C\.Spectral MLE: Top\-K rank aggregation from pairwise comparisons\.In*Proceedings of the 32nd International Conference on Machine Learning*, pp\. 371–380, 2015\.
- Chen et al\. \(2019\)Chen, Y\., Fan, J\., Ma, C\., and Wang, K\.Spectral method and regularized MLE are both optimal for Top\-K ranking\.*The Annals of Statistics*, 47\(4\):2204–2235, 2019\.
- Chiang et al\. \(2024\)Chiang, W\.\-L\., Zheng, L\., Sheng, Y\., Angelopoulos, A\. N\., Li, T\., Li, D\., Zhu, B\., Zhang, H\., Jordan, M\. I\., Gonzalez, J\. E\., and Stoica, I\.Chatbot arena: An open platform for evaluating LLMs by human preference\.In*Proceedings of the 41st International Conference on Machine Learning*, pp\. 8359–8388, 2024\.
- Degenne et al\. \(2019\)Degenne, R\., Koolen, W\. M\., and Ménard, P\.Non\-asymptotic pure exploration by solving games\.In*Advances in Neural Information Processing Systems 32*, pp\. 14492–14501, 2019\.
- Degenne et al\. \(2020\)Degenne, R\., Ménard, P\., Shang, X\., and Valko, M\.Gamification of pure exploration for linear bandits\.In*Proceedings of the 37th International Conference on Machine Learning*, pp\. 2432–2442, 2020\.
- Garivier & Kaufmann \(2016\)Garivier, A\. and Kaufmann, E\.Optimal best arm identification with fixed confidence\.In*Proceedings of the 29th Annual Conference on Learning Theory*, pp\. 998–1027, 2016\.
- Haddenhorst et al\. \(2021\)Haddenhorst, B\., Bengs, V\., and Hüllermeier, E\.Identification of the generalized condorcet winner in multi\-dueling bandits\.In*Advances in Neural Information Processing Systems 34*, pp\. 25904–25916, 2021\.
- Hajek et al\. \(2014\)Hajek, B\., Oh, S\., and Xu, J\.Minimax\-optimal inference from partial rankings\.In*Advances in Neural Information Processing Systems 27*, pp\. 1475–1483, 2014\.
- Hazan \(2016\)Hazan, E\.Introduction to online convex optimization\.*Foundations and Trends in Optimization*, 2\(3\-4\):157–325, 2016\.
- Heckel et al\. \(2019\)Heckel, R\., Shah, N\. B\., Ramchandran, K\., and Wainwright, M\. J\.Active ranking from pairwise comparisons and when parametric assumptions don’t help\.*The Annals of Statistics*, 47\(6\):3099–3126, 2019\.
- Jang et al\. \(2016\)Jang, M\., Kim, S\., Suh, C\., and Oh, S\.Top\-K ranking from pairwise comparisons: When spectral ranking is optimal\.*arXiv preprint arXiv:1603\.04153*, 2016\.
- Jedra & Proutière \(2020\)Jedra, Y\. and Proutière, A\.Optimal best\-arm identification in linear bandits\.In*Advances in Neural Information Processing Systems 33*, pp\. 10007–10017, 2020\.
- Kalloori et al\. \(2018\)Kalloori, S\., Ricci, F\., and Gennari, R\.Eliciting pairwise preferences in recommender systems\.In*Proceedings of the 12th ACM Conference on Recommender Systems*, pp\. 329–337, Vancouver, BC, Canada, 2018\.
- Kaufmann & Koolen \(2021\)Kaufmann, E\. and Koolen, W\. M\.Mixture martingales revisited with applications to sequential tests and confidence intervals\.*Journal of Machine Learning Research*, 22:1–44, 2021\.
- Kaufmann et al\. \(2016\)Kaufmann, E\., Cappé, O\., and Garivier, A\.On the complexity of best\-arm identification in multi\-armed bandit models\.*Journal of Machine Learning Research*, 17:1–42, 2016\.
- Khetan & Oh \(2016\)Khetan, A\. and Oh, S\.Data\-driven rank breaking for efficient rank aggregation\.In*Proceedings of the 33rd International Conference on Machine Learning*, pp\. 89–98, 2016\.
- Kou et al\. \(2017\)Kou, N\. M\., Li, Y\., Wang, H\., U, L\. H\., and Gong, Z\.Crowdsourced top\-k queries by confidence\-aware pairwise judgments\.In*Proceedings of the 2017 ACM International Conference on Management of Data*, pp\. 1415–1430, Chicago, IL, USA, 2017\.
- Lattimore & Szepesvári \(2020\)Lattimore, T\. and Szepesvári, C\.*Bandit Algorithms*\.Cambridge University Press, 2020\.
- Mohajer et al\. \(2017\)Mohajer, S\., Suh, C\., and Elmahdy, A\.Active learning for top\-kkrank aggregation from noisy comparisons\.In*Proceedings of the 34th International Conference on Machine Learning*, pp\. 2488–2497, 2017\.
- Narimanzadeh et al\. \(2023\)Narimanzadeh, H\., Badie\-Modiri, A\., Smirnova, I\. G\., and Chen, T\. H\. Y\.Crowdsourcing subjective annotations using pairwise comparisons reduces bias and error compared to the majority\-vote method\.*Proceedings of the ACM on Human\-Computer Interaction*, 7\(CSCW2\):1–29, 2023\.
- Negahban et al\. \(2017\)Negahban, S\., Oh, S\., and Shah, D\.Rank centrality: Ranking from pairwise comparisons\.*Operations Research*, 65\(1\):266–287, 2017\.
- Ren et al\. \(2018\)Ren, W\., Liu, J\., and Shroff, N\. B\.PAC ranking from pairwise and listwise queries: Lower bounds and upper bounds\.*arXiv preprint arXiv:1806\.02970*, 2018\.
- Ren et al\. \(2020\)Ren, W\., Liu, J\., and Shroff, N\. B\.The sample complexity of best\-kkitems selection from pairwise comparisons\.In*Proceedings of the 37th International Conference on Machine Learning*, pp\. 8051–8072, 2020\.
- Saha & Gopalan \(2019\)Saha, A\. and Gopalan, A\.PAC battling bandits in the plackett\-luce model\.In*Proceedings of the 30th International Conference on Algorithmic Learning Theory*, pp\. 700–737, 2019\.
- Saha & Gopalan \(2020\)Saha, A\. and Gopalan, A\.From PAC to instance\-optimal sample complexity in the plackett\-luce model\.In*Proceedings of the 37th International Conference on Machine Learning*, pp\. 8367–8376, 2020\.
- Shah et al\. \(2017\)Shah, N\. B\., Balakrishnan, S\., Guntuboyina, A\., and Wainwright, M\. J\.Stochastically transitive models for pairwise comparisons: Statistical and computational issues\.*IEEE Transactions on Information Theory*, 63\(2\):934–959, 2017\.
- Shalev\-Shwartz \(2012\)Shalev\-Shwartz, S\.Online learning and online convex optimization\.*Foundations and Trends in Machine Learning*, 4\(2\):107–194, 2012\.
- Wang et al\. \(2021\)Wang, P\.\-A\., Tzeng, R\.\-C\., and Proutière, A\.Fast pure exploration via frank\-wolfe\.In*Advances in Neural Information Processing Systems 34*, pp\. 5810–5821, 2021\.
- Xu et al\. \(2019\)Xu, L\., Honda, J\., and Sugiyama, M\.Dueling bandits with qualitative feedback\.In*Proceedings of the Thirty\-Third AAAI Conference on Artificial Intelligence*, pp\. 5549–5556, 2019\.

## Notation

This table lists notation from the main text that is used in the appendix without being redefined there\. Additional notation introduced inside a proof is local to that proof\.

## Appendix AProofs for Section[4](https://arxiv.org/html/2607.08979#S4)

### A\.1Proof of Lemma[4\.1](https://arxiv.org/html/2607.08979#S4.Thmtheorem1)

###### Proof\.

For any𝜽′∈Θi​j\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\},𝒘↦D𝒘​\(𝜽∥𝜽′\)\\bm\{w\}\\mapsto D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)is linear\. Taking the infimum over𝜽′∈Θi​j\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}is a pointwise infimum of linear functions, thus is concave\.

For continuity:\(𝒘,𝜽′\)↦D𝒘​\(𝜽∥𝜽′\)\(\\bm\{w\},\\bm\{\\theta\}^\{\\prime\}\)\\mapsto D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)is continuous andΘi​j\\Theta\_\{ij\}is compact, soγi​j​\(𝒘;𝜽\)=min𝜽′∈Θi​j⁡D𝒘​\(𝜽∥𝜽′\)\\gamma\_\{ij\}\(\\bm\{w\};\\bm\{\\theta\}\)=\\min\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}\}D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)is continuous as a minimum of a continuous function over a compact set\. ∎

### A\.2Proof of Lemma[4\.2](https://arxiv.org/html/2607.08979#S4.Thmtheorem2)

###### Proof\.

*\(a\)*

∇𝜽′2D𝒘​\(𝜽∥𝜽′\)=∑\(a,b\)∈𝒫wa​b​A′′​\(ηa​b​\(𝜽′\)\)​𝒙a​b​𝒙a​b⊤\.\\nabla^\{2\}\_\{\\bm\{\\theta\}^\{\\prime\}\}D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)=\\sum\_\{\(a,b\)\\in\\mathcal\{P\}\}w\_\{ab\}\\,A^\{\\prime\\prime\}\(\\eta\_\{ab\}\(\\bm\{\\theta\}^\{\\prime\}\)\)\\,\\bm\{x\}\_\{ab\}\\bm\{x\}\_\{ab\}^\{\\top\}\.So for any𝒗∈ℝn\\bm\{v\}\\in\\mathbb\{R\}^\{n\},

𝒗⊤​∇𝜽′2D𝒘​\(𝜽∥𝜽′\)​𝒗\\displaystyle\\bm\{v\}^\{\\top\}\\nabla^\{2\}\_\{\\bm\{\\theta\}^\{\\prime\}\}D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)\\bm\{v\}=∑\(a,b\)∈𝒫wa​b​A′′​\(ηa​b​\(𝜽′\)\)​\(va−vb\)2≥0,\\displaystyle\\quad=\\sum\_\{\(a,b\)\\in\\mathcal\{P\}\}w\_\{ab\}\\,A^\{\\prime\\prime\}\(\\eta\_\{ab\}\(\\bm\{\\theta\}^\{\\prime\}\)\)\\,\(v\_\{a\}\-v\_\{b\}\)^\{2\}\\geq 0,with equality iffva=vbv\_\{a\}=v\_\{b\}on every edge inG​\(𝒘\)G\(\\bm\{w\}\)\. SinceG​\(𝒘\)G\(\\bm\{w\}\)is connected this implies𝒗=c​𝟏\\bm\{v\}=c\\mathbf\{1\}\. Restricting to𝒗∈Θ0\\bm\{v\}\\in\\Theta\_\{0\}forcesc=0c=0, so the Hessian is positive definite onΘ0\\Theta\_\{0\}\.

*\(b\)*This follows from \(a\) and the fact thatΘi​j\\Theta\_\{ij\}is compact\.

*\(c\)*Let𝜽∗=𝜽i​j∗​\(𝒘;𝜽\)\\bm\{\\theta\}^\{\*\}=\\bm\{\\theta\}^\{\*\}\_\{ij\}\(\\bm\{w\};\\bm\{\\theta\}\)be the unique minimizer overΘi​j\\Theta\_\{ij\}\. If𝜽∗∈∂Θ\\bm\{\\theta\}^\{\*\}\\in\\partial\\Thetawe are done\. Otherwise𝜽∗∈int​\(Θ\)\\bm\{\\theta\}^\{\*\}\\in\\mathrm\{int\}\(\\Theta\)\. Ifθvi​j∗\>θui​j∗\\theta^\{\*\}\_\{v\_\{ij\}\}\>\\theta^\{\*\}\_\{u\_\{ij\}\}, then there existsλ∈\(0,1\)\\lambda\\in\(0,1\)such that𝜽¯:=\(1−λ\)​𝜽∗\+λ​𝜽\\bar\{\\bm\{\\theta\}\}:=\(1\-\\lambda\)\\bm\{\\theta\}^\{\*\}\+\\lambda\\bm\{\\theta\}satisfiesθ¯vi​j=θ¯ui​j\\bar\{\\theta\}\_\{v\_\{ij\}\}=\\bar\{\\theta\}\_\{u\_\{ij\}\}\. And𝜽¯∈Θ\\bar\{\\bm\{\\theta\}\}\\in\\Theta\. Also𝜽¯∈Θi​j\\bar\{\\bm\{\\theta\}\}\\in\\Theta\_\{ij\}by construction\. By convexity ofD𝒘​\(𝜽∥⋅\)D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\cdot\)andD𝒘​\(𝜽∥𝜽\)=0D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}\)=0,

D𝒘​\(𝜽∥𝜽¯\)≤\(1−λ\)​D𝒘​\(𝜽∥𝜽∗\)\+λ​D𝒘​\(𝜽∥𝜽\)<D𝒘​\(𝜽∥𝜽∗\),D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bar\{\\bm\{\\theta\}\}\)\\leq\(1\-\\lambda\)D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\*\}\)\+\\lambda D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}\)<D\_\{\\bm\{w\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\*\}\),contradiction\. So if𝜽∗∈int​\(Θ\)\\bm\{\\theta\}^\{\*\}\\in\\mathrm\{int\}\(\\Theta\)the halfspace must bind\. ∎

## Appendix BProof of Lemma[6\.1](https://arxiv.org/html/2607.08979#S6.Thmtheorem1)

Define

Ξi​j​\(t\):=∑s=1t𝟏\{As=\(i,j\)\}​\(T​\(Ys\)−A′​\(ηi​j​\(𝜽\)\)\),\\Xi\_\{ij\}\(t\)\\ :=\\ \\sum\_\{s=1\}^\{t\}\\mathbf\{1\}\_\{\\\{A\_\{s\}=\(i,j\)\\\}\}\\Big\(T\(Y\_\{s\}\)\-A^\{\\prime\}\(\\eta\_\{ij\}\(\\bm\{\\theta\}\)\)\\Big\),
###### Proof\.

Let𝚫:=𝜽^t−𝜽∈Θ0\\bm\{\\Delta\}:=\\hat\{\\bm\{\\theta\}\}\_\{t\}\-\\bm\{\\theta\}\\in\\Theta\_\{0\}and for each\(i,j\)∈𝒫\(i,j\)\\in\\mathcal\{P\}setzi​j:=𝒙i​j⊤​𝚫=Δi−Δjz\_\{ij\}:=\\bm\{x\}\_\{ij\}^\{\\top\}\\bm\{\\Delta\}=\\Delta\_\{i\}\-\\Delta\_\{j\}\. By definition ofℓt\\ell\_\{t\}andΞi​j​\(t\)\\Xi\_\{ij\}\(t\),

ℓt​\(𝜽\+𝚫\)−ℓt​\(𝜽\)\\displaystyle\\ell\_\{t\}\(\\bm\{\\theta\}\+\\bm\{\\Delta\}\)\-\\ell\_\{t\}\(\\bm\{\\theta\}\)=∑\(i,j\)∈𝒫\(zi​j​Ξi​j​\(t\)−Ni​j​\(t\)​di​j​\(𝜽,𝜽\+𝚫\)\)\.\\displaystyle\\quad=\\sum\_\{\(i,j\)\\in\\mathcal\{P\}\}\\Big\(z\_\{ij\}\\,\\Xi\_\{ij\}\(t\)\-N\_\{ij\}\(t\)\\,d\_\{ij\}\(\\bm\{\\theta\},\\bm\{\\theta\}\+\\bm\{\\Delta\}\)\\Big\)\.Since𝜽^t\\hat\{\\bm\{\\theta\}\}\_\{t\}maximizesℓt\\ell\_\{t\}over the constraint set and𝜽\\bm\{\\theta\}is feasible,ℓt​\(𝜽\+𝚫\)−ℓt​\(𝜽\)≥0\\ell\_\{t\}\(\\bm\{\\theta\}\+\\bm\{\\Delta\}\)\-\\ell\_\{t\}\(\\bm\{\\theta\}\)\\geq 0\.

By assumptionA′′≥a¯A^\{\\prime\\prime\}\\geq\\underline\{a\}on\[−2​R,2​R\]\[\-2R,2R\], so strong convexity givesdi​j​\(𝜽,𝜽\+𝚫\)=A​\(ηi​j​\(𝜽\)\+zi​j\)−A​\(ηi​j​\(𝜽\)\)−A′​\(ηi​j​\(𝜽\)\)​zi​j≥a¯2​zi​j2d\_\{ij\}\(\\bm\{\\theta\},\\bm\{\\theta\}\+\\bm\{\\Delta\}\)=A\(\\eta\_\{ij\}\(\\bm\{\\theta\}\)\+z\_\{ij\}\)\-A\(\\eta\_\{ij\}\(\\bm\{\\theta\}\)\)\-A^\{\\prime\}\(\\eta\_\{ij\}\(\\bm\{\\theta\}\)\)\\,z\_\{ij\}\\geq\\frac\{\\underline\{a\}\}\{2\}\\,z\_\{ij\}^\{2\}, so

0≤∑\(i,j\)∈𝒫\(zi​j​Ξi​j​\(t\)−a¯2​Ni​j​\(t\)​zi​j2\)\.0\\leq\\sum\_\{\(i,j\)\\in\\mathcal\{P\}\}\\Big\(z\_\{ij\}\\,\\Xi\_\{ij\}\(t\)\-\\frac\{\\underline\{a\}\}\{2\}N\_\{ij\}\(t\)\\,z\_\{ij\}^\{2\}\\Big\)\.\(6\)Rearranging gives

a¯2​∑\(i,j\)∈𝒫Ni​j​\(t\)​zi​j2≤∑\(i,j\)∈𝒫zi​j​Ξi​j​\(t\)\.\\frac\{\\underline\{a\}\}\{2\}\\sum\_\{\(i,j\)\\in\\mathcal\{P\}\}N\_\{ij\}\(t\)\\,z\_\{ij\}^\{2\}\\leq\\sum\_\{\(i,j\)\\in\\mathcal\{P\}\}z\_\{ij\}\\,\\Xi\_\{ij\}\(t\)\.\(7\)Apply Cauchy–Schwarz:

∑\(i,j\)zi​j​Ξi​j​\(t\)\\displaystyle\\sum\_\{\(i,j\)\}z\_\{ij\}\\Xi\_\{ij\}\(t\)=∑\(i,j\)\(Ni​j​\(t\)​zi​j\)⋅\(Ξi​j​\(t\)Ni​j​\(t\)\)\\displaystyle\\quad=\\sum\_\{\(i,j\)\}\\big\(\\sqrt\{N\_\{ij\}\(t\)\}\\,z\_\{ij\}\\big\)\\cdot\\Big\(\\frac\{\\Xi\_\{ij\}\(t\)\}\{\\sqrt\{N\_\{ij\}\(t\)\}\}\\Big\)≤∑\(i,j\)Ni​j​\(t\)​zi​j2​∑\(i,j\)Ξi​j​\(t\)2Ni​j​\(t\)\.\\displaystyle\\quad\\leq\\sqrt\{\\sum\_\{\(i,j\)\}N\_\{ij\}\(t\)\\,z\_\{ij\}^\{2\}\}\\;\\sqrt\{\\sum\_\{\(i,j\)\}\\frac\{\\Xi\_\{ij\}\(t\)^\{2\}\}\{N\_\{ij\}\(t\)\}\}\.Let

A:=∑\(i,j\)∈𝒫Ni​j​\(t\)​zi​j2,B:=∑\(i,j\)∈𝒫Ξi​j​\(t\)2Ni​j​\(t\)\.A:=\\sum\_\{\(i,j\)\\in\\mathcal\{P\}\}N\_\{ij\}\(t\)\\,z\_\{ij\}^\{2\},\\qquad B:=\\sum\_\{\(i,j\)\\in\\mathcal\{P\}\}\\frac\{\\Xi\_\{ij\}\(t\)^\{2\}\}\{N\_\{ij\}\(t\)\}\.Then \([7](https://arxiv.org/html/2607.08979#A2.E7)\) and the above imply

a¯2​A≤A​B⟹A≤4a¯2​B,\\frac\{\\underline\{a\}\}\{2\}A\\leq\\sqrt\{A\}\\sqrt\{B\}\\quad\\Longrightarrow\\quad A\\leq\\frac\{4\}\{\\underline\{a\}^\{2\}\}B,i\.e\.

∑\(i,j\)∈𝒫Ni​j​\(t\)​\(𝒙i​j⊤​𝚫\)2≤4a¯2​∑\(i,j\)∈𝒫Ξi​j​\(t\)2Ni​j​\(t\)\.\\sum\_\{\(i,j\)\\in\\mathcal\{P\}\}N\_\{ij\}\(t\)\\,\(\\bm\{x\}\_\{ij\}^\{\\top\}\\bm\{\\Delta\}\)^\{2\}\\leq\\frac\{4\}\{\\underline\{a\}^\{2\}\}\\sum\_\{\(i,j\)\\in\\mathcal\{P\}\}\\frac\{\\Xi\_\{ij\}\(t\)^\{2\}\}\{N\_\{ij\}\(t\)\}\.\(8\)Define the weighted Laplacian

Lt:=∑\(i,j\)∈𝒫Ni​j​\(t\)​𝒙i​j​𝒙i​j⊤,L\_\{t\}:=\\sum\_\{\(i,j\)\\in\\mathcal\{P\}\}N\_\{ij\}\(t\)\\,\\bm\{x\}\_\{ij\}\\bm\{x\}\_\{ij\}^\{\\top\},so𝚫⊤​Lt​𝚫=∑\(i,j\)Ni​j​\(t\)​\(𝒙i​j⊤​𝚫\)2\\bm\{\\Delta\}^\{\\top\}L\_\{t\}\\bm\{\\Delta\}=\\sum\_\{\(i,j\)\}N\_\{ij\}\(t\)\(\\bm\{x\}\_\{ij\}^\{\\top\}\\bm\{\\Delta\}\)^\{2\}\. SinceNi​j​\(t\)≥Nmin​\(t\)N\_\{ij\}\(t\)\\geq N\_\{\\min\}\(t\)for all pairs,

Lt⪰Nmin​\(t\)​∑1≤i<j≤n𝒙i​j​𝒙i​j⊤=Nmin​\(t\)​\(n​In−𝟏𝟏⊤\)\.L\_\{t\}\\succeq N\_\{\\min\}\(t\)\\sum\_\{1\\leq i<j\\leq n\}\\bm\{x\}\_\{ij\}\\bm\{x\}\_\{ij\}^\{\\top\}=N\_\{\\min\}\(t\)\\,\(nI\_\{n\}\-\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}\)\.𝚫∈Θ0\\bm\{\\Delta\}\\in\\Theta\_\{0\}implies𝟏⊤​𝚫=0\\mathbf\{1\}^\{\\top\}\\bm\{\\Delta\}=0, so

𝚫⊤​Lt​𝚫≥n​Nmin​\(t\)​‖𝚫‖22\.\\bm\{\\Delta\}^\{\\top\}L\_\{t\}\\bm\{\\Delta\}\\geq n\\,N\_\{\\min\}\(t\)\\,\\\|\\bm\{\\Delta\}\\\|\_\{2\}^\{2\}\.Combining with \([8](https://arxiv.org/html/2607.08979#A2.E8)\) gives

‖𝚫‖22≤4a¯2​n​Nmin​\(t\)​∑\(i,j\)∈𝒫Ξi​j​\(t\)2Ni​j​\(t\)\.\\\|\\bm\{\\Delta\}\\\|\_\{2\}^\{2\}\\leq\\frac\{4\}\{\\underline\{a\}^\{2\}\\,n\\,N\_\{\\min\}\(t\)\}\\sum\_\{\(i,j\)\\in\\mathcal\{P\}\}\\frac\{\\Xi\_\{ij\}\(t\)^\{2\}\}\{N\_\{ij\}\(t\)\}\.\(9\)It remains to bound∑\(i,j\)Ξi​j​\(t\)2/Ni​j​\(t\)\\sum\_\{\(i,j\)\}\\Xi\_\{ij\}\(t\)^\{2\}/N\_\{ij\}\(t\)with high probability\. Fix a pair\(i,j\)∈𝒫\(i,j\)\\in\\mathcal\{P\}and define martingale differences

ξsi​j:=𝟏\{As=\(i,j\)\}​\(T​\(Ys\)−A′​\(ηi​j​\(𝜽\)\)\),s≥1,\\xi\_\{s\}^\{ij\}:=\\mathbf\{1\}\_\{\\\{A\_\{s\}=\(i,j\)\\\}\}\\Big\(T\(Y\_\{s\}\)\-A^\{\\prime\}\(\\eta\_\{ij\}\(\\bm\{\\theta\}\)\)\\Big\),\\qquad s\\geq 1,so thatΞi​j​\(t\)=∑s=1tξsi​j\\Xi\_\{ij\}\(t\)=\\sum\_\{s=1\}^\{t\}\\xi\_\{s\}^\{ij\}andNi​j​\(t\)=∑s=1t𝟏\{As=\(i,j\)\}N\_\{ij\}\(t\)=\\sum\_\{s=1\}^\{t\}\\mathbf\{1\}\_\{\\\{A\_\{s\}=\(i,j\)\\\}\}\. By Assumption[2](https://arxiv.org/html/2607.08979#Thmassumption2), for allλ∈ℝ\\lambda\\in\\mathbb\{R\},

𝔼𝜽​\[exp⁡\(λ​ξsi​j\)∣ℱs−1\]≤exp⁡\(σ2​λ22​𝟏\{As=\(i,j\)\}\)\.\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\\\!\\left\[\\exp\\\!\\big\(\\lambda\\xi\_\{s\}^\{ij\}\\big\)\\mid\\mathcal\{F\}\_\{s\-1\}\\right\]\\leq\\exp\\\!\\left\(\\frac\{\\sigma^\{2\}\\lambda^\{2\}\}\{2\}\\mathbf\{1\}\_\{\\\{A\_\{s\}=\(i,j\)\\\}\}\\right\)\.Therefore for eachλ\\lambdathe process

Ms​\(λ\):=exp⁡\(λ​Ξi​j​\(s\)−σ2​λ22​Ni​j​\(s\)\)M\_\{s\}\(\\lambda\):=\\exp\\\!\\left\(\\lambda\\Xi\_\{ij\}\(s\)\-\\frac\{\\sigma^\{2\}\\lambda^\{2\}\}\{2\}N\_\{ij\}\(s\)\\right\)is a nonnegative supermartingale, so𝔼𝜽​\[Mt​\(λ\)\]≤1\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\[M\_\{t\}\(\\lambda\)\]\\leq 1\. Fixm∈\{1,…,t\}m\\in\\\{1,\\ldots,t\\\}andu\>0u\>0, and setλ:=2​u/\(σ​m\)\\lambda:=\\sqrt\{2u\}/\(\\sigma\\sqrt\{m\}\)\. On the event\{Ni​j​\(t\)=m\}\\\{N\_\{ij\}\(t\)=m\\\}we have

\{Ξi​j​\(t\)≥σ​2​m​u\}⊆\{Mt​\(λ\)≥eu\},\\Big\\\{\\Xi\_\{ij\}\(t\)\\geq\\sigma\\sqrt\{2mu\}\\Big\\\}\\ \\subseteq\\ \\Big\\\{M\_\{t\}\(\\lambda\)\\geq e^\{u\}\\Big\\\},sinceλ​σ​2​m​u=2​u\\lambda\\sigma\\sqrt\{2mu\}=2uand\(σ2​λ2/2\)​m=u\(\\sigma^\{2\}\\lambda^\{2\}/2\)m=u\. Thus by Markov’s inequality,

ℙ𝜽​\(Ξi​j​\(t\)≥σ​2​m​u,Ni​j​\(t\)=m\)≤e−u\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\\Xi\_\{ij\}\(t\)\\geq\\sigma\\sqrt\{2mu\},\\,N\_\{ij\}\(t\)=m\\right\)\\leq e^\{\-u\}\.Applying the same argument to−Ξi​j​\(t\)\-\\Xi\_\{ij\}\(t\)gives

ℙ𝜽​\(\|Ξi​j​\(t\)\|≥σ​2​m​u,Ni​j​\(t\)=m\)≤2​e−u\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\|\\Xi\_\{ij\}\(t\)\|\\geq\\sigma\\sqrt\{2mu\},\\,N\_\{ij\}\(t\)=m\\right\)\\leq 2e^\{\-u\}\.Sum overm=1,…,tm=1,\\ldots,tto get

ℙ𝜽​\(Ξi​j​\(t\)2Ni​j​\(t\)≥2​σ2​u,Ni​j​\(t\)≥1\)≤2​t​e−u\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\\frac\{\\Xi\_\{ij\}\(t\)^\{2\}\}\{N\_\{ij\}\(t\)\}\\geq 2\\sigma^\{2\}u,\\ N\_\{ij\}\(t\)\\geq 1\\right\)\\leq 2t\\,e^\{\-u\}\.Chooseu=log⁡\(2​t​\|𝒫\|δ\)u=\\log\\\!\\big\(\\frac\{2t\|\\mathcal\{P\}\|\}\{\\delta\}\\big\)and apply a union bound over\(i,j\)∈𝒫\(i,j\)\\in\\mathcal\{P\}: with probability at least1−δ1\-\\delta,

max\(i,j\)∈𝒫⁡Ξi​j​\(t\)2Ni​j​\(t\)≤2​σ2​log⁡\(2​t​\|𝒫\|δ\),\\max\_\{\(i,j\)\\in\\mathcal\{P\}\}\\frac\{\\Xi\_\{ij\}\(t\)^\{2\}\}\{N\_\{ij\}\(t\)\}\\leq 2\\sigma^\{2\}\\log\\\!\\Big\(\\frac\{2t\|\\mathcal\{P\}\|\}\{\\delta\}\\Big\),hence

∑\(i,j\)∈𝒫Ξi​j​\(t\)2Ni​j​\(t\)≤2​σ2​\|𝒫\|​log⁡\(2​t​\|𝒫\|δ\)\.\\sum\_\{\(i,j\)\\in\\mathcal\{P\}\}\\frac\{\\Xi\_\{ij\}\(t\)^\{2\}\}\{N\_\{ij\}\(t\)\}\\leq 2\\sigma^\{2\}\|\\mathcal\{P\}\|\\log\\\!\\Big\(\\frac\{2t\|\\mathcal\{P\}\|\}\{\\delta\}\\Big\)\.Plugging into \([9](https://arxiv.org/html/2607.08979#A2.E9)\) gives

‖𝜽^t−𝜽‖22≤8​σ2​\|𝒫\|a¯2​n​Nmin​\(t\)​log⁡\(2​t​\|𝒫\|δ\)\.\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\-\\bm\{\\theta\}\\\|\_\{2\}^\{2\}\\leq\\frac\{8\\sigma^\{2\}\|\\mathcal\{P\}\|\}\{\\underline\{a\}^\{2\}\\,n\\,N\_\{\\min\}\(t\)\}\\log\\\!\\Big\(\\frac\{2t\|\\mathcal\{P\}\|\}\{\\delta\}\\Big\)\.\|𝒫\|=n​\(n−1\)2≤n22\|\\mathcal\{P\}\|=\\frac\{n\(n\-1\)\}\{2\}\\leq\\frac\{n^\{2\}\}\{2\}so

‖𝜽^t−𝜽‖2≤2​σa¯​n​log⁡\(2​t​\|𝒫\|δ\)Nmin​\(t\),\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\-\\bm\{\\theta\}\\\|\_\{2\}\\leq\\frac\{2\\sigma\}\{\\underline\{a\}\}\\sqrt\{\\frac\{n\\log\\\!\\big\(\\frac\{2t\|\\mathcal\{P\}\|\}\{\\delta\}\\big\)\}\{N\_\{\\min\}\(t\)\}\},∎

### B\.1Proof of Corollary[6\.2](https://arxiv.org/html/2607.08979#S6.Thmtheorem2)

###### Proof\.

Apply Lemma[6\.1](https://arxiv.org/html/2607.08979#S6.Thmtheorem1)withδt=t−2\\delta\_\{t\}=t^\{\-2\}\. Since∑t≥1δt<∞\\sum\_\{t\\geq 1\}\\delta\_\{t\}<\\infty, by Borel–Cantelli almost surely there existst1<∞t\_\{1\}<\\inftysuch that for allt≥t1t\\geq t\_\{1\},

‖𝜽^t−𝜽‖2≤2​σa¯​n​log⁡\(2​t​\|𝒫\|δt\)Nmin​\(t\)\.\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\-\\bm\{\\theta\}\\\|\_\{2\}\\leq\\frac\{2\\sigma\}\{\\underline\{a\}\}\\sqrt\{\\frac\{n\\log\\\!\\big\(\\frac\{2t\|\\mathcal\{P\}\|\}\{\\delta\_\{t\}\}\\big\)\}\{N\_\{\\min\}\(t\)\}\}\.By Lemma[F\.1](https://arxiv.org/html/2607.08979#A6.Thmtheorem1)*\(e\)*,Nmin​\(t\)/log⁡t→∞N\_\{\\min\}\(t\)/\\log t\\to\\inftyalmost surely, whilelog⁡\(2​t​\|𝒫\|δt\)=log⁡\(2​t​\|𝒫\|​t2\)=O​\(log⁡t\)\\log\\\!\\big\(\\frac\{2t\|\\mathcal\{P\}\|\}\{\\delta\_\{t\}\}\\big\)=\\log\(2t\|\\mathcal\{P\}\|t^\{2\}\)=O\(\\log t\)\. Hence the right\-hand side tends to0almost surely, so‖𝜽^t−𝜽‖2→0\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\-\\bm\{\\theta\}\\\|\_\{2\}\\to 0almost surely\.

Since𝜽∈Θgap\\bm\{\\theta\}\\in\\Theta^\{\\mathrm\{gap\}\}, defineΔk:=θ\(k\)−θ\(k\+1\)\>0\\Delta\_\{k\}:=\\theta\_\{\(k\)\}\-\\theta\_\{\(k\+1\)\}\>0\. Because‖𝜽^t−𝜽‖∞≤‖𝜽^t−𝜽‖2\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\-\\bm\{\\theta\}\\\|\_\{\\infty\}\\leq\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\-\\bm\{\\theta\}\\\|\_\{2\}, almost surely there existststab<∞t\_\{\\mathrm\{stab\}\}<\\inftysuch that‖𝜽^t−𝜽‖∞≤Δk/4\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\-\\bm\{\\theta\}\\\|\_\{\\infty\}\\leq\\Delta\_\{k\}/4for allt≥tstabt\\geq t\_\{\\mathrm\{stab\}\}\. For suchtt, ifu∈S∗​\(𝜽\)u\\in S^\{\*\}\(\\bm\{\\theta\}\)andv∉S∗​\(𝜽\)v\\notin S^\{\*\}\(\\bm\{\\theta\}\)thenθu−θv≥Δk\\theta\_\{u\}\-\\theta\_\{v\}\\geq\\Delta\_\{k\}, and thus

θ^t,u−θ^t,v≥\(θu−θv\)−2​‖𝜽^t−𝜽‖∞≥Δk−Δk2\>0\.\\hat\{\\theta\}\_\{t,u\}\-\\hat\{\\theta\}\_\{t,v\}\\geq\(\\theta\_\{u\}\-\\theta\_\{v\}\)\-2\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\-\\bm\{\\theta\}\\\|\_\{\\infty\}\\geq\\Delta\_\{k\}\-\\frac\{\\Delta\_\{k\}\}\{2\}\>0\.ThereforeS^t=S∗​\(𝜽\)\\hat\{S\}\_\{t\}=S^\{\*\}\(\\bm\{\\theta\}\)andB^t=B​\(𝜽\)\\hat\{B\}\_\{t\}=B\(\\bm\{\\theta\}\)for allt≥tstabt\\geq t\_\{\\mathrm\{stab\}\}∎

## Appendix CProof of Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)

##### Outline\.

The proof decomposes into \(i\) online learning error, \(ii\) statistical error from estimating𝜽\\bm\{\\theta\}, and \(iii\) C\-tracking/mixing bias\. First, Lemma[C\.1](https://arxiv.org/html/2607.08979#A3.Thmtheorem1)establishes that the stochastic gradients are unbiased estimators of the gradients and are uniformly bounded, which is the bounded\-noise condition needed for exponential\-weights/FTRL analysis\. Next, Lemma[C\.2](https://arxiv.org/html/2607.08979#A3.Thmtheorem2)gives high probability regret bounds for both the primal and dual players\. Lemma[C\.3](https://arxiv.org/html/2607.08979#A3.Thmtheorem3)bounds the duality gap by averaged regret\. Finally, we combine these bounds\. We introduce a burn\-in indexbt:=⌈t1/4⌉b\_\{t\}:=\\lceil t^\{1/4\}\\rceilso that, with high probability, the set of boundary pairs is correct for every times∈\{bt,…,t\}s\\in\\\{b\_\{t\},\\ldots,t\\\}\.

Some supporting lemmas are applications of standard online learning regret bounds and online saddle point games; we include full proofs for completeness\.

Fix𝜽∈Θgap\\bm\{\\theta\}\\in\\Theta^\{\\mathrm\{gap\}\}and defineB:=B​\(𝜽\)B:=B\(\\bm\{\\theta\}\)andm:=\|B\|=k​\(n−k\)m:=\|B\|=k\(n\-k\)\. LetDmaxD\_\{\\max\}andLLbe as in Lemma[F\.3](https://arxiv.org/html/2607.08979#A6.Thmtheorem3), and defineG:=m​DmaxG:=mD\_\{\\max\}\. For each horizonT≥1T\\geq 1setbT:=⌈T1/4⌉b\_\{T\}:=\\lceil T^\{1/4\}\\rceil\.

###### Lemma C\.1\.

Taket≥1t\\geq 1andIt∼Unif​\(B\)I\_\{t\}\\sim\\mathrm\{Unif\}\(B\)is sampled after\(𝛉^t,𝐰t,𝐪t\)\(\\hat\{\\bm\{\\theta\}\}\_\{t\},\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\)are formed\. Recall the stochastic gradients from \([3](https://arxiv.org/html/2607.08979#S5.E3)\):

g^t,a​b\(w\):=m​qt,It​da​b​\(𝜽^t,𝜽t∗\),\(a,b\)∈𝒫,\\displaystyle\\hat\{g\}^\{\(w\)\}\_\{t,ab\}:=m\\,q\_\{t,I\_\{t\}\}\\,d\_\{ab\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\},\\bm\{\\theta\}\_\{t\}^\{\*\}\),\\ \(a,b\)\\in\\mathcal\{P\},g^t,i​j\(q\):=m​γt​1​\{\(i,j\)=It\},\(i,j\)∈B\.\\displaystyle\\hat\{g\}^\{\(q\)\}\_\{t,ij\}:=m\\,\\gamma\_\{t\}\\,\\mathbf\{1\}\\\{\(i,j\)=I\_\{t\}\\\},\\ \(i,j\)\\in B\.Then,

𝔼​\[g^t,a​b\(w\)∣𝜽^t,𝒘t,𝒒t\]=∂F∂wa​b​\(𝒘t,𝒒t;𝜽^t\),\\displaystyle\\mathbb\{E\}\\\!\\left\[\\hat\{g\}^\{\(w\)\}\_\{t,ab\}\\mid\\hat\{\\bm\{\\theta\}\}\_\{t\},\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\\right\]=\\frac\{\\partial F\}\{\\partial w\_\{ab\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\),𝔼​\[g^t,i​j\(q\)∣𝜽^t,𝒘t,𝒒t\]=∂F∂qi​j​\(𝒘t,𝒒t;𝜽^t\),\\displaystyle\\mathbb\{E\}\\\!\\left\[\\hat\{g\}^\{\(q\)\}\_\{t,ij\}\\mid\\hat\{\\bm\{\\theta\}\}\_\{t\},\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\\right\]=\\frac\{\\partial F\}\{\\partial q\_\{ij\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\),additionally,‖𝐠^t\(w\)‖∞≤m​Dmax\\\|\\hat\{\\bm\{g\}\}^\{\(w\)\}\_\{t\}\\\|\_\{\\infty\}\\leq mD\_\{\\max\}and‖𝐠^t\(q\)‖∞≤m​Dmax\\\|\\hat\{\\bm\{g\}\}^\{\(q\)\}\_\{t\}\\\|\_\{\\infty\}\\leq mD\_\{\\max\}almost surely\. IfG​\(𝐰t\)G\(\\bm\{w\}\_\{t\}\)is connected \(so the KL projections are unique\), the right\-hand sides above are gradients; otherwise, they are supergradients\.

###### Proof\.

Choose\(a,b\)∈𝒫\(a,b\)\\in\\mathcal\{P\}\. Each boundary pair has probability1/m1/mof being selected\. It follows that

𝔼​\[g^t,a​b\(w\)∣𝜽^t,𝒘t,𝒒t\]=∑\(i,j\)∈Bqt,i​j​da​b​\(𝜽^t,𝜽i​j∗​\(𝒘t;𝜽^t\)\),\\mathbb\{E\}\\\!\\left\[\\hat\{g\}^\{\(w\)\}\_\{t,ab\}\\mid\\hat\{\\bm\{\\theta\}\}\_\{t\},\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\\right\]=\\sum\_\{\(i,j\)\\in B\}q\_\{t,ij\}\\,d\_\{ab\}\\\!\\big\(\\hat\{\\bm\{\\theta\}\}\_\{t\},\\bm\{\\theta\}^\{\*\}\_\{ij\}\(\\bm\{w\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\\big\),
Similarly, for each\(i,j\)∈B\(i,j\)\\in B,

𝔼​\[g^t,i​j\(q\)∣𝜽^t,𝒘t,𝒒t\]\\displaystyle\\mathbb\{E\}\\\!\\left\[\\hat\{g\}^\{\(q\)\}\_\{t,ij\}\\mid\\hat\{\\bm\{\\theta\}\}\_\{t\},\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\\right\]=∑\(p,ℓ\)∈B1m​m​γp​ℓ​\(𝒘t;𝜽^t\)​1​\{\(p,ℓ\)=\(i,j\)\}\\displaystyle\\quad=\\sum\_\{\(p,\\ell\)\\in B\}\\frac\{1\}\{m\}\\,m\\,\\gamma\_\{p\\ell\}\(\\bm\{w\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\\,\\mathbf\{1\}\\\{\(p,\\ell\)=\(i,j\)\\\}=γi​j​\(𝒘t;𝜽^t\)=∂F∂qi​j​\(𝒘t,𝒒t;𝜽^t\)\.\\displaystyle\\quad=\\gamma\_\{ij\}\(\\bm\{w\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\)=\\frac\{\\partial F\}\{\\partial q\_\{ij\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\};\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\.Finally,0≤qt,It≤10\\leq q\_\{t,I\_\{t\}\}\\leq 1,0≤da​b​\(⋅,⋅\)≤Dmax0\\leq d\_\{ab\}\(\\cdot,\\cdot\)\\leq D\_\{\\max\}, and0≤γt≤Dmax0\\leq\\gamma\_\{t\}\\leq D\_\{\\max\}, so\|g^t,a​b\(w\)\|≤m​Dmax\|\\hat\{g\}^\{\(w\)\}\_\{t,ab\}\|\\leq mD\_\{\\max\}and\|g^t,i​j\(q\)\|≤m​Dmax\|\\hat\{g\}^\{\(q\)\}\_\{t,ij\}\|\\leq mD\_\{\\max\}\. ∎

###### Lemma C\.2\.

FixT≥1T\\geq 1, Define

Regw⁡\(T\):=sup𝒘∈Δ𝒫∑t=bTT\(F𝜽^t​\(𝒘,𝒒t\)−F𝜽^t​\(𝒘t,𝒒t\)\),\\displaystyle\\operatorname\{Reg\}\_\{w\}\(T\):=\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\Big\(F\_\{\\hat\{\\bm\{\\theta\}\}\_\{t\}\}\(\\bm\{w\},\\bm\{q\}\_\{t\}\)\-F\_\{\\hat\{\\bm\{\\theta\}\}\_\{t\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\)\\Big\),Regq⁡\(T\):=sup𝒒∈ΔB∑t=bTT\(F𝜽^t​\(𝒘t,𝒒t\)−F𝜽^t​\(𝒘t,𝒒\)\)\.\\displaystyle\\operatorname\{Reg\}\_\{q\}\(T\):=\\sup\_\{\\bm\{q\}\\in\\Delta\_\{B\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\Big\(F\_\{\\hat\{\\bm\{\\theta\}\}\_\{t\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\)\-F\_\{\\hat\{\\bm\{\\theta\}\}\_\{t\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\)\\Big\)\.Then for everyδ∈\(0,1\)\\delta\\in\(0,1\), with probability at least1−δ1\-\\delta,

Regw⁡\(T\)\\displaystyle\\operatorname\{Reg\}\_\{w\}\(T\)≤D𝒘​\(1\)μT\+G22​∑t=1Tμt\+2​G​\(bT−1\)\\displaystyle\\leq\\frac\{D\_\{\\bm\{w\}\}\(1\)\}\{\\mu\_\{T\}\}\+\\frac\{G^\{2\}\}\{2\}\\sum\_\{t=1\}^\{T\}\\mu\_\{t\}\+2G\(b\_\{T\}\-1\)\+4​G​2​\(T−bT\+1\)​log⁡\(8​\|𝒫\|δ\),\\displaystyle\\quad\+4G\\sqrt\{2\(T\-b\_\{T\}\+1\)\\log\\\!\\Big\(\\frac\{8\|\\mathcal\{P\}\|\}\{\\delta\}\\Big\)\},\(10\)Regq⁡\(T\)\\displaystyle\\operatorname\{Reg\}\_\{q\}\(T\)≤D𝒒​\(1\)μT\+G22​∑t=1Tμt\+2​G​\(bT−1\)\\displaystyle\\leq\\frac\{D\_\{\\bm\{q\}\}\(1\)\}\{\\mu\_\{T\}\}\+\\frac\{G^\{2\}\}\{2\}\\sum\_\{t=1\}^\{T\}\\mu\_\{t\}\+2G\(b\_\{T\}\-1\)\+4​G​2​\(T−bT\+1\)​log⁡\(8​mδ\)\.\\displaystyle\\quad\+4G\\sqrt\{2\(T\-b\_\{T\}\+1\)\\log\\\!\\Big\(\\frac\{8m\}\{\\delta\}\\Big\)\}\.\(11\)where

D𝒘​\(1\):=sup𝒘∈Δ𝒫KL⁡\(𝒘∥𝒘1\)=max\(a,b\)∈𝒫⁡log⁡1w1,a​b,\\displaystyle D\_\{\\bm\{w\}\}\(1\):=\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\operatorname\{KL\}\(\\bm\{w\}\\\|\\bm\{w\}\_\{1\}\)=\\max\_\{\(a,b\)\\in\\mathcal\{P\}\}\\log\\frac\{1\}\{w\_\{1,ab\}\},D𝒒​\(1\):=sup𝒒∈ΔBKL⁡\(𝒒∥𝒒1\)=max\(i,j\)∈B⁡log⁡1q1,i​j\.\\displaystyle D\_\{\\bm\{q\}\}\(1\):=\\sup\_\{\\bm\{q\}\\in\\Delta\_\{B\}\}\\operatorname\{KL\}\(\\bm\{q\}\\\|\\bm\{q\}\_\{1\}\)=\\max\_\{\(i,j\)\\in B\}\\log\\frac\{1\}\{q\_\{1,ij\}\}\.

###### Proof\.

We prove \([10](https://arxiv.org/html/2607.08979#A3.E10)\)\. For eacht∈\{bT,…,T\}t\\in\\\{b\_\{T\},\\ldots,T\\\}define𝒈t\(w\):=∇𝒘F𝜽^t​\(𝒘t,𝒒t\)\\bm\{g\}^\{\(w\)\}\_\{t\}:=\\nabla\_\{\\bm\{w\}\}F\_\{\\hat\{\\bm\{\\theta\}\}\_\{t\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\)\. So for all𝒘∈Δ𝒫\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\},

F𝜽^t​\(𝒘,𝒒t\)−F𝜽^t​\(𝒘t,𝒒t\)≤⟨𝒈t\(w\),𝒘−𝒘t⟩\.F\_\{\\hat\{\\bm\{\\theta\}\}\_\{t\}\}\(\\bm\{w\},\\bm\{q\}\_\{t\}\)\-F\_\{\\hat\{\\bm\{\\theta\}\}\_\{t\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\)\\leq\\langle\\bm\{g\}^\{\(w\)\}\_\{t\},\\,\\bm\{w\}\-\\bm\{w\}\_\{t\}\\rangle\.Summing and taking the supremum over𝒘\\bm\{w\}gives

Regw⁡\(T\)≤sup𝒘∈Δ𝒫∑t=bTT⟨𝒈t\(w\),𝒘−𝒘t⟩\.\\operatorname\{Reg\}\_\{w\}\(T\)\\leq\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\langle\\bm\{g\}^\{\(w\)\}\_\{t\},\\,\\bm\{w\}\-\\bm\{w\}\_\{t\}\\rangle\.Now decompose𝒈t\(w\)=𝒈^t\(w\)\+𝚫t\(g\)\\bm\{g\}^\{\(w\)\}\_\{t\}=\\hat\{\\bm\{g\}\}^\{\(w\)\}\_\{t\}\+\\bm\{\\Delta\}^\{\(g\)\}\_\{t\}, where𝚫t\(g\):=𝒈t\(w\)−𝒈^t\(w\)\\bm\{\\Delta\}^\{\(g\)\}\_\{t\}:=\\bm\{g\}^\{\(w\)\}\_\{t\}\-\\hat\{\\bm\{g\}\}^\{\(w\)\}\_\{t\}\. Then

sup𝒘∑t=bTT⟨𝒈t\(w\),𝒘−𝒘t⟩\\displaystyle\\sup\_\{\\bm\{w\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\langle\\bm\{g\}^\{\(w\)\}\_\{t\},\\,\\bm\{w\}\-\\bm\{w\}\_\{t\}\\rangle≤sup𝒘∑t=bTT⟨𝒈^t\(w\),𝒘−𝒘t⟩⏟\(I\)\+sup𝒘∑t=bTT⟨𝚫t\(g\),𝒘−𝒘t⟩⏟\(II\)\.\\displaystyle\\quad\\leq\\underbrace\{\\sup\_\{\\bm\{w\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\langle\\hat\{\\bm\{g\}\}^\{\(w\)\}\_\{t\},\\,\\bm\{w\}\-\\bm\{w\}\_\{t\}\\rangle\}\_\{\(\\mathrm\{I\}\)\}\+\\underbrace\{\\sup\_\{\\bm\{w\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\langle\\bm\{\\Delta\}^\{\(g\)\}\_\{t\},\\,\\bm\{w\}\-\\bm\{w\}\_\{t\}\\rangle\}\_\{\(\\mathrm\{II\}\)\}\.We can bound term \(I\) by applying Lemma[F\.4](https://arxiv.org/html/2607.08979#A6.Thmtheorem4)to𝒈^t=𝒈^t\(w\)\\hat\{\\bm\{g\}\}\_\{t\}=\\hat\{\\bm\{g\}\}^\{\(w\)\}\_\{t\}to get

\(I\)≤D𝒘​\(1\)μT\+G22​∑t=1Tμt\+2​G​\(bT−1\)\.\(\\mathrm\{I\}\)\\leq\\frac\{D\_\{\\bm\{w\}\}\(1\)\}\{\\mu\_\{T\}\}\+\\frac\{G^\{2\}\}\{2\}\\sum\_\{t=1\}^\{T\}\\mu\_\{t\}\+2G\(b\_\{T\}\-1\)\.
To bound term \(II\) we split it into two parts\. Letℋt\\mathcal\{H\}\_\{t\}denote the algorithm history at roundttafter observingYtY\_\{t\}and forming\(𝜽^t,𝒘t,𝒒t\)\(\\hat\{\\bm\{\\theta\}\}\_\{t\},\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\), but before samplingItI\_\{t\}; and let𝒢t\\mathcal\{G\}\_\{t\}denote the history up to timettincluding the drawItI\_\{t\}\. By Lemma[C\.1](https://arxiv.org/html/2607.08979#A3.Thmtheorem1), for each coordinate\(a,b\)\(a,b\),

𝔼​\[g^t,a​b\(w\)∣ℋt\]=gt,a​b\(w\),\\mathbb\{E\}\[\\hat\{g\}^\{\(w\)\}\_\{t,ab\}\\mid\\mathcal\{H\}\_\{t\}\]=g^\{\(w\)\}\_\{t,ab\},so𝔼​\[Δt,a​b\(g\)∣ℋt\]=0\\mathbb\{E\}\[\\Delta^\{\(g\)\}\_\{t,ab\}\\mid\\mathcal\{H\}\_\{t\}\]=0\. Since𝒢t−1⊆ℋt\\mathcal\{G\}\_\{t\-1\}\\subseteq\\mathcal\{H\}\_\{t\}, we also have

𝔼​\[Δt,a​b\(g\)∣𝒢t−1\]=0,\\mathbb\{E\}\[\\Delta^\{\(g\)\}\_\{t,ab\}\\mid\\mathcal\{G\}\_\{t\-1\}\]=0,and therefore\(Δt,a​b\(g\)\)t≥bT\(\\Delta^\{\(g\)\}\_\{t,ab\}\)\_\{t\\geq b\_\{T\}\}is a martingale difference sequence with respect to\(𝒢t\)\(\\mathcal\{G\}\_\{t\}\)\. Also,

\|Δt,a​b\(g\)\|≤\|gt,a​b\(w\)\|\+\|g^t,a​b\(w\)\|≤Dmax\+G≤2​G\.\|\\Delta^\{\(g\)\}\_\{t,ab\}\|\\leq\|g^\{\(w\)\}\_\{t,ab\}\|\+\|\\hat\{g\}^\{\(w\)\}\_\{t,ab\}\|\\leq D\_\{\\max\}\+G\\leq 2G\.Thus each increment is bounded in\[−2​G,2​G\]\[\-2G,2G\]\. For any𝒘∈Δ𝒫\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\},

∑t=bTT⟨𝚫t\(g\),𝒘−𝒘t⟩\\displaystyle\\sum\_\{t=b\_\{T\}\}^\{T\}\\langle\\bm\{\\Delta\}^\{\(g\)\}\_\{t\},\\bm\{w\}\-\\bm\{w\}\_\{t\}\\rangle=⟨∑t=bTT𝚫t\(g\),𝒘⟩−∑t=bTT⟨𝚫t\(g\),𝒘t⟩\\displaystyle\\quad=\\left\\langle\\sum\_\{t=b\_\{T\}\}^\{T\}\\bm\{\\Delta\}^\{\(g\)\}\_\{t\},\\,\\bm\{w\}\\right\\rangle\-\\sum\_\{t=b\_\{T\}\}^\{T\}\\langle\\bm\{\\Delta\}^\{\(g\)\}\_\{t\},\\bm\{w\}\_\{t\}\\rangle≤max\(a,b\)∈𝒫​∑t=bTTΔt,a​b\(g\)⏟\(IIa\)\+\|∑t=bTT⟨𝚫t\(g\),𝒘t⟩\|⏟\(IIb\)\.\\displaystyle\\quad\\leq\\underbrace\{\\max\_\{\(a,b\)\\in\\mathcal\{P\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\Delta^\{\(g\)\}\_\{t,ab\}\}\_\{\(\\mathrm\{IIa\}\)\}\+\\underbrace\{\\left\|\\sum\_\{t=b\_\{T\}\}^\{T\}\\langle\\bm\{\\Delta\}^\{\(g\)\}\_\{t\},\\bm\{w\}\_\{t\}\\rangle\\right\|\}\_\{\(\\mathrm\{IIb\}\)\}\.*\(IIa\) Boundingmax\(a,b\)​∑Δt,a​b\(g\)\\max\_\{\(a,b\)\}\\sum\\Delta^\{\(g\)\}\_\{t,ab\}\.*Fix a coordinate\(a,b\)\(a,b\)and define the martingale

Msa​b:=∑t=bTsΔt,a​b\(g\),s=bT,…,T\.M\_\{s\}^\{ab\}:=\\sum\_\{t=b\_\{T\}\}^\{s\}\\Delta^\{\(g\)\}\_\{t,ab\},\\qquad s=b\_\{T\},\\ldots,T\.It has bounded increments\|Δt,a​b\(g\)\|≤2​G\|\\Delta^\{\(g\)\}\_\{t,ab\}\|\\leq 2Gand mean\-zero conditional increments\. By Azuma–Hoeffding, for anyx\>0x\>0,

ℙ​\(MTa​b≥x\)≤exp⁡\(−x28​G2​\(T−bT\+1\)\)\.\\mathbb\{P\}\\\!\\left\(M\_\{T\}^\{ab\}\\geq x\\right\)\\leq\\exp\\\!\\left\(\-\\frac\{x^\{2\}\}\{8G^\{2\}\(T\-b\_\{T\}\+1\)\}\\right\)\.Choose

x:=2​G​2​\(T−bT\+1\)​log⁡\(4​\|𝒫\|δ\)\.x:=2G\\sqrt\{2\(T\-b\_\{T\}\+1\)\\log\\\!\\Big\(\\frac\{4\|\\mathcal\{P\}\|\}\{\\delta\}\\Big\)\}\.Thenℙ​\(MTa​b≥x\)≤δ/\(4​\|𝒫\|\)\\mathbb\{P\}\(M\_\{T\}^\{ab\}\\geq x\)\\leq\\delta/\(4\|\\mathcal\{P\}\|\)\. A union bound over\(a,b\)∈𝒫\(a,b\)\\in\\mathcal\{P\}gives

ℙ\(max\(a,b\)∈𝒫∑t=bTTΔt,a​b\(g\)\\displaystyle\\mathbb\{P\}\\\!\\bigg\(\\max\_\{\(a,b\)\\in\\mathcal\{P\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\Delta^\{\(g\)\}\_\{t,ab\}≥2G2​\(T−bT\+1\)​log⁡\(4​\|𝒫\|δ\)\)≤δ4\.\\displaystyle\\qquad\\geq 2G\\sqrt\{2\(T\-b\_\{T\}\+1\)\\log\\\!\\Big\(\\frac\{4\|\\mathcal\{P\}\|\}\{\\delta\}\\Big\)\}\\bigg\)\\leq\\frac\{\\delta\}\{4\}\.*\(IIb\) Bounding\|∑⟨𝚫t\(g\),𝐰t⟩\|\\left\|\\sum\\langle\\bm\{\\Delta\}^\{\(g\)\}\_\{t\},\\bm\{w\}\_\{t\}\\rangle\\right\|\.*Define the martingale

Ms:=∑t=bTs⟨𝚫t\(g\),𝒘t⟩,s=bT,…,T,M\_\{s\}:=\\sum\_\{t=b\_\{T\}\}^\{s\}\\langle\\bm\{\\Delta\}^\{\(g\)\}\_\{t\},\\bm\{w\}\_\{t\}\\rangle,\\qquad s=b\_\{T\},\\ldots,T,We have,

\|⟨𝚫t\(g\),𝒘t⟩\|≤‖𝚫t\(g\)‖∞​‖𝒘t‖1≤2​G\.\|\\langle\\bm\{\\Delta\}^\{\(g\)\}\_\{t\},\\bm\{w\}\_\{t\}\\rangle\|\\leq\\\|\\bm\{\\Delta\}^\{\(g\)\}\_\{t\}\\\|\_\{\\infty\}\\\|\\bm\{w\}\_\{t\}\\\|\_\{1\}\\leq 2G\.By Azuma–Hoeffding, for anyy\>0y\>0,

ℙ​\(\|MT\|≥y\)≤2​exp⁡\(−y28​G2​\(T−bT\+1\)\)\.\\mathbb\{P\}\(\|M\_\{T\}\|\\geq y\)\\leq 2\\exp\\\!\\left\(\-\\frac\{y^\{2\}\}\{8G^\{2\}\(T\-b\_\{T\}\+1\)\}\\right\)\.Choose

y:=2​G​2​\(T−bT\+1\)​log⁡\(8δ\)\.y:=2G\\sqrt\{2\(T\-b\_\{T\}\+1\)\\log\\\!\\Big\(\\frac\{8\}\{\\delta\}\\Big\)\}\.Thenℙ​\(\|MT\|≥y\)≤δ/4\\mathbb\{P\}\(\|M\_\{T\}\|\\geq y\)\\leq\\delta/4\. Combining \(IIa\) and \(IIb\) and using thatlog⁡\(8/δ\)≤log⁡\(8​\|𝒫\|/δ\)\\log\(8/\\delta\)\\leq\\log\(8\|\\mathcal\{P\}\|/\\delta\)we get that with probability at least1−δ/21\-\\delta/2,

sup𝒘∈Δ𝒫∑t=bTT⟨𝚫t\(g\),𝒘−𝒘t⟩≤4​G​2​\(T−bT\+1\)​log⁡\(8​\|𝒫\|δ\)\.\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\langle\\bm\{\\Delta\}^\{\(g\)\}\_\{t\},\\bm\{w\}\-\\bm\{w\}\_\{t\}\\rangle\\leq 4G\\sqrt\{2\(T\-b\_\{T\}\+1\)\\log\\\!\\Big\(\\frac\{8\|\\mathcal\{P\}\|\}\{\\delta\}\\Big\)\}\.We combine the bounds of terms I, IIa, and IIb to get that with probability at least1−δ/21\-\\delta/2

Regw⁡\(T\)≤D𝒘​\(1\)μT\+G22​∑t=1Tμt\+2​G​\(bT−1\)\\displaystyle\\operatorname\{Reg\}\_\{w\}\(T\)\\leq\\frac\{D\_\{\\bm\{w\}\}\(1\)\}\{\\mu\_\{T\}\}\+\\frac\{G^\{2\}\}\{2\}\\sum\_\{t=1\}^\{T\}\\mu\_\{t\}\+2G\(b\_\{T\}\-1\)\+4​G​2​\(T−bT\+1\)​log⁡\(8​\|𝒫\|δ\)\.\\displaystyle\\quad\+4G\\sqrt\{2\(T\-b\_\{T\}\+1\)\\log\\\!\\Big\(\\frac\{8\|\\mathcal\{P\}\|\}\{\\delta\}\\Big\)\}\.This is \([10](https://arxiv.org/html/2607.08979#A3.E10)\)\. \([11](https://arxiv.org/html/2607.08979#A3.E11)\) follows by the similar argument applied to𝒒\\bm\{q\}\. The𝒒\\bm\{q\}\-player is minimizing, and the update𝒓t\+1=exp⁡\(−μt\+1​𝚿t\(q\)\)\\bm\{r\}\_\{t\+1\}=\\exp\(\-\\mu\_\{t\+1\}\\bm\{\\Psi\}^\{\(q\)\}\_\{t\}\)corresponds to exponential weights on losses𝒈^t\(q\)\\hat\{\\bm\{g\}\}^\{\(q\)\}\_\{t\}\(equivalently, exponential weights on gains−𝒈^t\(q\)\-\\hat\{\\bm\{g\}\}^\{\(q\)\}\_\{t\}\)\. Applying Lemma[F\.4](https://arxiv.org/html/2607.08979#A6.Thmtheorem4)onΔB\\Delta\_\{B\}\(withD𝒒​\(1\)D\_\{\\bm\{q\}\}\(1\)in place ofD𝒘​\(1\)D\_\{\\bm\{w\}\}\(1\)\) and the same martingale argument gives \([11](https://arxiv.org/html/2607.08979#A3.E11)\) with probability at least1−δ/21\-\\delta/2\. By a union bound, both inequalities \([10](https://arxiv.org/html/2607.08979#A3.E10)\)–\([11](https://arxiv.org/html/2607.08979#A3.E11)\) hold simultaneously with probability at least1−δ1\-\\delta\. ∎

###### Lemma C\.3\.

LetT≥1T\\geq 1and defineT′:=T−bT\+1T^\{\\prime\}:=T\-b\_\{T\}\+1\. For any sequence\(𝐰t,𝐪t\)t=bTT\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\)\_\{t=b\_\{T\}\}^\{T\}, define the averages

𝒘¯bT:T:=1T′​∑t=bTT𝒘t,𝒒¯bT:T:=1T′​∑t=bTT𝒒t\.\\bar\{\\bm\{w\}\}\_\{b\_\{T\}:T\}:=\\frac\{1\}\{T^\{\\prime\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\bm\{w\}\_\{t\},\\qquad\\bar\{\\bm\{q\}\}\_\{b\_\{T\}:T\}:=\\frac\{1\}\{T^\{\\prime\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\bm\{q\}\_\{t\}\.Define the duality gap forF𝛉F\_\{\\bm\{\\theta\}\}:

gapT𝜽:=sup𝒘∈Δ𝒫F𝜽​\(𝒘,𝒒¯bT:T\)−inf𝒒∈ΔBF𝜽​\(𝒘¯bT:T,𝒒\)\.\\operatorname\{gap\}\_\{T\}^\{\\bm\{\\theta\}\}:=\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bar\{\\bm\{q\}\}\_\{b\_\{T\}:T\}\)\-\\inf\_\{\\bm\{q\}\\in\\Delta\_\{B\}\}F\_\{\\bm\{\\theta\}\}\(\\bar\{\\bm\{w\}\}\_\{b\_\{T\}:T\},\\bm\{q\}\)\.Then

gapT𝜽≤Regw𝜽⁡\(T\)\+Regq𝜽⁡\(T\)T′,\\operatorname\{gap\}\_\{T\}^\{\\bm\{\\theta\}\}\\ \\leq\\ \\frac\{\\operatorname\{Reg\}\_\{w\}^\{\\bm\{\\theta\}\}\(T\)\+\\operatorname\{Reg\}\_\{q\}^\{\\bm\{\\theta\}\}\(T\)\}\{T^\{\\prime\}\},where

Regw𝜽⁡\(T\):=sup𝒘∈Δ𝒫∑t=bTT\(F𝜽​\(𝒘,𝒒t\)−F𝜽​\(𝒘t,𝒒t\)\),\\displaystyle\\operatorname\{Reg\}\_\{w\}^\{\\bm\{\\theta\}\}\(T\):=\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\big\(F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bm\{q\}\_\{t\}\)\-F\_\{\\bm\{\\theta\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\)\\big\),Regq𝜽⁡\(T\):=sup𝒒∈ΔB∑t=bTT\(F𝜽​\(𝒘t,𝒒t\)−F𝜽​\(𝒘t,𝒒\)\)\.\\displaystyle\\operatorname\{Reg\}\_\{q\}^\{\\bm\{\\theta\}\}\(T\):=\\sup\_\{\\bm\{q\}\\in\\Delta\_\{B\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\big\(F\_\{\\bm\{\\theta\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\)\-F\_\{\\bm\{\\theta\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\)\\big\)\.

###### Proof\.

By linearity in𝒒\\bm\{q\},

F𝜽​\(𝒘,𝒒¯bT:T\)=1T′​∑t=bTTF𝜽​\(𝒘,𝒒t\)F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bar\{\\bm\{q\}\}\_\{b\_\{T\}:T\}\)=\\frac\{1\}\{T^\{\\prime\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bm\{q\}\_\{t\}\)so

sup𝒘F𝜽​\(𝒘,𝒒¯bT:T\)=sup𝒘1T′​∑t=bTTF𝜽​\(𝒘,𝒒t\)\.\\sup\_\{\\bm\{w\}\}F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bar\{\\bm\{q\}\}\_\{b\_\{T\}:T\}\)=\\sup\_\{\\bm\{w\}\}\\frac\{1\}\{T^\{\\prime\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bm\{q\}\_\{t\}\)\.Additionally, for any fixed𝒒\\bm\{q\}, concavity of𝒘↦F𝜽​\(𝒘,𝒒\)\\bm\{w\}\\mapsto F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bm\{q\}\)and Jensen’s inequality give

F𝜽​\(𝒘¯bT:T,𝒒\)≥1T′​∑t=bTTF𝜽​\(𝒘t,𝒒\),F\_\{\\bm\{\\theta\}\}\(\\bar\{\\bm\{w\}\}\_\{b\_\{T\}:T\},\\bm\{q\}\)\\ \\geq\\ \\frac\{1\}\{T^\{\\prime\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}F\_\{\\bm\{\\theta\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\),so

inf𝒒F𝜽​\(𝒘¯bT:T,𝒒\)≥inf𝒒1T′​∑t=bTTF𝜽​\(𝒘t,𝒒\)\.\\inf\_\{\\bm\{q\}\}F\_\{\\bm\{\\theta\}\}\(\\bar\{\\bm\{w\}\}\_\{b\_\{T\}:T\},\\bm\{q\}\)\\ \\geq\\ \\inf\_\{\\bm\{q\}\}\\frac\{1\}\{T^\{\\prime\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}F\_\{\\bm\{\\theta\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\)\.Thus,

gapT𝜽\\displaystyle\\operatorname\{gap\}\_\{T\}^\{\\bm\{\\theta\}\}≤sup𝒘1T′​∑t=bTTF𝜽​\(𝒘,𝒒t\)−inf𝒒1T′​∑t=bTTF𝜽​\(𝒘t,𝒒\)\\displaystyle\\leq\\sup\_\{\\bm\{w\}\}\\frac\{1\}\{T^\{\\prime\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bm\{q\}\_\{t\}\)\-\\inf\_\{\\bm\{q\}\}\\frac\{1\}\{T^\{\\prime\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}F\_\{\\bm\{\\theta\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\)≤1T′​\(sup𝒘∑t=bTT\(F𝜽​\(𝒘,𝒒t\)−F𝜽​\(𝒘t,𝒒t\)\)\)\\displaystyle\\leq\\frac\{1\}\{T^\{\\prime\}\}\\left\(\\sup\_\{\\bm\{w\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\big\(F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bm\{q\}\_\{t\}\)\-F\_\{\\bm\{\\theta\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\)\\big\)\\right\)\+1T′​\(sup𝒒∑t=bTT\(F𝜽​\(𝒘t,𝒒t\)−F𝜽​\(𝒘t,𝒒\)\)\)\\displaystyle\\quad\+\\frac\{1\}\{T^\{\\prime\}\}\\left\(\\sup\_\{\\bm\{q\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\big\(F\_\{\\bm\{\\theta\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\_\{t\}\)\-F\_\{\\bm\{\\theta\}\}\(\\bm\{w\}\_\{t\},\\bm\{q\}\)\\big\)\\right\)=Regw𝜽⁡\(T\)\+Regq𝜽⁡\(T\)T′\.\\displaystyle=\\frac\{\\operatorname\{Reg\}\_\{w\}^\{\\bm\{\\theta\}\}\(T\)\+\\operatorname\{Reg\}\_\{q\}^\{\\bm\{\\theta\}\}\(T\)\}\{T^\{\\prime\}\}\.∎

We now prove Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)\.

### C\.1Proof of Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)

###### Proof\.

Fixp\>1p\>1and define

bt:=⌈t1/4⌉,T′:=t−bt\+1,Δk:=θ\(k\)−θ\(k\+1\)\>0\.b\_\{t\}:=\\left\\lceil t^\{1/4\}\\right\\rceil,\\quad T^\{\\prime\}:=t\-b\_\{t\}\+1,\\quad\\Delta\_\{k\}:=\\theta\_\{\(k\)\}\-\\theta\_\{\(k\+1\)\}\>0\.For eachs∈\{bt,…,t\}s\\in\\\{b\_\{t\},\\ldots,t\\\}define the event

ℳt,s:=\{∥𝜽^s−𝜽∥2≤2​σa¯n​log⁡\(2​s​\|𝒫\|δt,s\)Nmin​\(s\)\}\.\\mathcal\{M\}\_\{t,s\}:=\\left\\\{\\\|\\hat\{\\bm\{\\theta\}\}\_\{s\}\-\\bm\{\\theta\}\\\|\_\{2\}\\leq\\frac\{2\\sigma\}\{\\underline\{a\}\}\\sqrt\{\\frac\{n\\log\\\!\\big\(\\frac\{2s\|\\mathcal\{P\}\|\}\{\\delta\_\{t,s\}\}\\big\)\}\{N\_\{\\min\}\(s\)\}\}\\right\\\}\.whereδt,s=t−\(p\+2\)\\delta\_\{t,s\}=t^\{\-\(p\+2\)\}\. Define the joint event

ℳt:=⋂s=bttℳt,s\.\\mathcal\{M\}\_\{t\}:=\\bigcap\_\{s=b\_\{t\}\}^\{t\}\\mathcal\{M\}\_\{t,s\}\.By Lemma[6\.1](https://arxiv.org/html/2607.08979#S6.Thmtheorem1), for each fixedss,ℙ𝜽​\(ℳt,sc\)≤δt,s=t−\(p\+2\)\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\mathcal\{M\}\_\{t,s\}^\{c\}\)\\leq\\delta\_\{t,s\}=t^\{\-\(p\+2\)\}\.Therefore, by a union bound over theT′=t−bt\+1T^\{\\prime\}=t\-b\_\{t\}\+1indicess=bt,…,ts=b\_\{t\},\\ldots,t,

ℙ𝜽​\(ℳtc\)≤∑s=bttℙ𝜽​\(ℳt,sc\)≤\(t−bt\+1\)​t−\(p\+2\)≤t−\(p\+1\)\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\mathcal\{M\}\_\{t\}^\{c\}\)\\leq\\sum\_\{s=b\_\{t\}\}^\{t\}\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\mathcal\{M\}\_\{t,s\}^\{c\}\)\\leq\(t\-b\_\{t\}\+1\)t^\{\-\(p\+2\)\}\\leq t^\{\-\(p\+1\)\}\.\(12\)
Note that for alls≤ts\\leq t,

log⁡\(2​s​\|𝒫\|δt,s\)=log⁡\(2​s​\|𝒫\|​tp\+2\)≤log⁡\(2​\|𝒫\|​tp\+3\)\.\\log\\\!\\Big\(\\frac\{2s\|\\mathcal\{P\}\|\}\{\\delta\_\{t,s\}\}\\Big\)=\\log\\\!\\big\(2s\|\\mathcal\{P\}\|\\,t^\{p\+2\}\\big\)\\leq\\log\\\!\\big\(2\|\\mathcal\{P\}\|\\,t^\{p\+3\}\\big\)\.Also, by Lemma[F\.1](https://arxiv.org/html/2607.08979#A6.Thmtheorem1)*\(e\)*, for eachss,

Nmin​\(s\)≥1\|𝒫\|⋅\(s\+1\)1−γ−11−γ−\(\|𝒫\|−1\)\.N\_\{\\min\}\(s\)\\ \\geq\\ \\frac\{1\}\{\|\\mathcal\{P\}\|\}\\cdot\\frac\{\(s\+1\)^\{1\-\\gamma\}\-1\}\{1\-\\gamma\}\\ \-\\ \(\|\\mathcal\{P\}\|\-1\)\.Define

sγ:=⌈max⁡\{21/\(1−γ\),\(4​\|𝒫\|​\(1−γ\)​\(\|𝒫\|−1\)\)1/\(1−γ\)\}⌉s\_\{\\gamma\}:=\\left\\lceil\\max\\\!\\left\\\{2^\{1/\(1\-\\gamma\)\},\\ \\big\(4\|\\mathcal\{P\}\|\(1\-\\gamma\)\(\|\\mathcal\{P\}\|\-1\)\\big\)^\{1/\(1\-\\gamma\)\}\\right\\\}\\right\\rceilandcγ:=14​\|𝒫\|​\(1−γ\)c\_\{\\gamma\}:=\\frac\{1\}\{4\|\\mathcal\{P\}\|\(1\-\\gamma\)\}\. For alls≥sγs\\geq s\_\{\\gamma\}we haveNmin​\(s\)≥cγ​s1−γN\_\{\\min\}\(s\)\\geq c\_\{\\gamma\}s^\{1\-\\gamma\}\(by Fact[F\.2](https://arxiv.org/html/2607.08979#A6.Thmtheorem2)\)\. Now define the time

tp:=\\displaystyle t\_\{p\}:=\{\}min\{t≥max\{2,sγ4\}:\\displaystyle\\min\\Bigg\\\{t\\geq\\max\\\{2,\\ s\_\{\\gamma\}^\{4\}\\\}:2​σa¯n​log⁡\(2​\|𝒫\|​tp\+3\)cγ​bt1−γ≤Δk4\}\.\\displaystyle\\qquad\\frac\{2\\sigma\}\{\\underline\{a\}\}\\sqrt\{\\frac\{n\\log\\\!\\big\(2\|\\mathcal\{P\}\|\\,t^\{p\+3\}\\big\)\}\{c\_\{\\gamma\}\\,b\_\{t\}^\{1\-\\gamma\}\}\}\\ \\leq\\ \\frac\{\\Delta\_\{k\}\}\{4\}\\Bigg\\\}\.Fix anyt≥tpt\\geq t\_\{p\}\. Thenbt≥sγb\_\{t\}\\geq s\_\{\\gamma\}, so for everys∈\{bt,…,t\}s\\in\\\{b\_\{t\},\\ldots,t\\\}we haveNmin​\(s\)≥cγ​s1−γ≥cγ​bt1−γN\_\{\\min\}\(s\)\\geq c\_\{\\gamma\}s^\{1\-\\gamma\}\\geq c\_\{\\gamma\}b\_\{t\}^\{1\-\\gamma\}\. Onℳt\\mathcal\{M\}\_\{t\}, for each suchss,

‖𝜽^s−𝜽‖2≤2​σa¯​n​log⁡\(2​\|𝒫\|​tp\+3\)cγ​bt1−γ≤Δk4,\\\|\\hat\{\\bm\{\\theta\}\}\_\{s\}\-\\bm\{\\theta\}\\\|\_\{2\}\\leq\\frac\{2\\sigma\}\{\\underline\{a\}\}\\sqrt\{\\frac\{n\\log\\\!\\big\(2\|\\mathcal\{P\}\|\\,t^\{p\+3\}\\big\)\}\{c\_\{\\gamma\}\\,b\_\{t\}^\{1\-\\gamma\}\}\}\\leq\\frac\{\\Delta\_\{k\}\}\{4\},Thus‖𝜽^s−𝜽‖∞≤Δk/4\\\|\\hat\{\\bm\{\\theta\}\}\_\{s\}\-\\bm\{\\theta\}\\\|\_\{\\infty\}\\leq\\Delta\_\{k\}/4for alls∈\{bt,…,t\}s\\in\\\{b\_\{t\},\\ldots,t\\\}, and by the same argument in Corollary[6\.2](https://arxiv.org/html/2607.08979#S6.Thmtheorem2)

S^s=S∗​\(𝜽\)andB^s=B​\(𝜽\)∀s=bt,…,t\.\\hat\{S\}\_\{s\}=S^\{\*\}\(\\bm\{\\theta\}\)\\quad\\text\{and\}\\quad\\hat\{B\}\_\{s\}=B\(\\bm\{\\theta\}\)\\qquad\\forall s=b\_\{t\},\\ldots,t\.Now we bound the failure probability of the regret bounds derived in Lemma[C\.2](https://arxiv.org/html/2607.08979#A3.Thmtheorem2)\. Letδt:=t−\(p\+2\)\\delta\_\{t\}:=t^\{\-\(p\+2\)\}andℛt:=\{both bounds \([10](https://arxiv.org/html/2607.08979#A3.E10)\)–\([11](https://arxiv.org/html/2607.08979#A3.E11)\) hold at horizon​T=t\}\\mathcal\{R\}\_\{t\}:=\\\{\\text\{both bounds \\eqref\{eq:hp\-regret\-w\}\-\-\\eqref\{eq:hp\-regret\-q\} hold at horizon \}T=t\\\}\. By Lemma[C\.2](https://arxiv.org/html/2607.08979#A3.Thmtheorem2)with confidenceδt\\delta\_\{t\},

ℙ𝜽​\(ℛtc\)≤δt=t−\(p\+2\)\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\mathcal\{R\}\_\{t\}^\{c\}\)\\leq\\delta\_\{t\}=t^\{\-\(p\+2\)\}\.\(13\)
Next, we boundRegw⁡\(t\)\\operatorname\{Reg\}\_\{w\}\(t\)andRegq⁡\(t\)\\operatorname\{Reg\}\_\{q\}\(t\)from Lemma[C\.2](https://arxiv.org/html/2607.08979#A3.Thmtheorem2)by powers oftt\. We havebt/t≤t−3/4b\_\{t\}/t\\leq t^\{\-3/4\}\. Alsoμs=s−α\\mu\_\{s\}=s^\{\-\\alpha\}so Fact[F\.2](https://arxiv.org/html/2607.08979#A6.Thmtheorem2)gives∑s=1tμs≤1\+t1−α−11−α\\sum\_\{s=1\}^\{t\}\\mu\_\{s\}\\leq 1\+\\frac\{t^\{1\-\\alpha\}\-1\}\{1\-\\alpha\}\. AlsoT′=t−bt\+1≥t/2T^\{\\prime\}=t\-b\_\{t\}\+1\\geq t/2for allt≥2t\\geq 2\. Therefore, onℛt\\mathcal\{R\}\_\{t\},

Regw⁡\(t\)\+Regq⁡\(t\)T′\\displaystyle\\frac\{\\operatorname\{Reg\}\_\{w\}\(t\)\+\\operatorname\{Reg\}\_\{q\}\(t\)\}\{T^\{\\prime\}\}≤Cp,1\(t−\(1−α\)\+t−α\\displaystyle\\leq C\_\{p,1\}\\Big\(t^\{\-\(1\-\\alpha\)\}\+t^\{\-\\alpha\}\+log⁡tt\+t−3/4\),\\displaystyle\\qquad\+\\sqrt\{\\tfrac\{\\log t\}\{t\}\}\+t^\{\-3/4\}\\Big\),\(14\)for a finite constantCp,1C\_\{p,1\}depending only on fixed parameters\.

These regret bounds are with𝜽^\\hat\{\\bm\{\\theta\}\}we now use them to bound the ‘true’ regret\. Define the true\-game regrets:

Regw𝜽⁡\(t\)\\displaystyle\\operatorname\{Reg\}\_\{w\}^\{\\bm\{\\theta\}\}\(t\):=sup𝒘∈Δ𝒫∑s=btt\(F𝜽​\(𝒘,𝒒s\)−F𝜽​\(𝒘s,𝒒s\)\),\\displaystyle:=\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\sum\_\{s=b\_\{t\}\}^\{t\}\\big\(F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bm\{q\}\_\{s\}\)\-F\_\{\\bm\{\\theta\}\}\(\\bm\{w\}\_\{s\},\\bm\{q\}\_\{s\}\)\\big\),Regq𝜽⁡\(t\)\\displaystyle\\operatorname\{Reg\}\_\{q\}^\{\\bm\{\\theta\}\}\(t\):=sup𝒒∈ΔB∑s=btt\(F𝜽​\(𝒘s,𝒒s\)−F𝜽​\(𝒘s,𝒒\)\)\.\\displaystyle:=\\sup\_\{\\bm\{q\}\\in\\Delta\_\{B\}\}\\sum\_\{s=b\_\{t\}\}^\{t\}\\big\(F\_\{\\bm\{\\theta\}\}\(\\bm\{w\}\_\{s\},\\bm\{q\}\_\{s\}\)\-F\_\{\\bm\{\\theta\}\}\(\\bm\{w\}\_\{s\},\\bm\{q\}\)\\big\)\.For eachssdefineΔ​Fs:=sup𝒘∈Δ𝒫,𝒒∈ΔB\|F𝜽^s​\(𝒘,𝒒\)−F𝜽​\(𝒘,𝒒\)\|\.\\Delta F\_\{s\}:=\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\},\\,\\bm\{q\}\\in\\Delta\_\{B\}\}\|F\_\{\\hat\{\\bm\{\\theta\}\}\_\{s\}\}\(\\bm\{w\},\\bm\{q\}\)\-F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bm\{q\}\)\|\.Then for any𝒘\\bm\{w\}and eachss,

F𝜽​\(𝒘,𝒒s\)−F𝜽​\(𝒘s,𝒒s\)≤F𝜽^s​\(𝒘,𝒒s\)−F𝜽^s​\(𝒘s,𝒒s\)\+2​Δ​Fs\.F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bm\{q\}\_\{s\}\)\-F\_\{\\bm\{\\theta\}\}\(\\bm\{w\}\_\{s\},\\bm\{q\}\_\{s\}\)\\leq F\_\{\\hat\{\\bm\{\\theta\}\}\_\{s\}\}\(\\bm\{w\},\\bm\{q\}\_\{s\}\)\-F\_\{\\hat\{\\bm\{\\theta\}\}\_\{s\}\}\(\\bm\{w\}\_\{s\},\\bm\{q\}\_\{s\}\)\+2\\Delta F\_\{s\}\.Summing froms=bts=b\_\{t\}tottand takingsup𝒘\\sup\_\{\\bm\{w\}\}gives

Regw𝜽⁡\(t\)\\displaystyle\\operatorname\{Reg\}\_\{w\}^\{\\bm\{\\theta\}\}\(t\)≤Regw⁡\(t\)\+2​∑s=bttΔ​Fs,\\displaystyle\\leq\\operatorname\{Reg\}\_\{w\}\(t\)\+2\\sum\_\{s=b\_\{t\}\}^\{t\}\\Delta F\_\{s\},Regq𝜽⁡\(t\)\\displaystyle\\operatorname\{Reg\}\_\{q\}^\{\\bm\{\\theta\}\}\(t\)≤Regq⁡\(t\)\+2​∑s=bttΔ​Fs,\\displaystyle\\leq\\operatorname\{Reg\}\_\{q\}\(t\)\+2\\sum\_\{s=b\_\{t\}\}^\{t\}\\Delta F\_\{s\},hence

Regw𝜽⁡\(t\)\+Regq𝜽⁡\(t\)≤Regw⁡\(t\)\+Regq⁡\(t\)\+4​∑s=bttΔ​Fs\.\\operatorname\{Reg\}\_\{w\}^\{\\bm\{\\theta\}\}\(t\)\+\\operatorname\{Reg\}\_\{q\}^\{\\bm\{\\theta\}\}\(t\)\\leq\\operatorname\{Reg\}\_\{w\}\(t\)\+\\operatorname\{Reg\}\_\{q\}\(t\)\+4\\sum\_\{s=b\_\{t\}\}^\{t\}\\Delta F\_\{s\}\.\(15\)By Lemma[F\.3](https://arxiv.org/html/2607.08979#A6.Thmtheorem3),Δ​Fs≤L​‖𝜽^s−𝜽‖2\\Delta F\_\{s\}\\leq L\\\|\\hat\{\\bm\{\\theta\}\}\_\{s\}\-\\bm\{\\theta\}\\\|\_\{2\}\. Therefore,

1T′​∑s=bttΔ​Fs≤L⋅1T′​∑s=btt‖𝜽^s−𝜽‖2\.\\frac\{1\}\{T^\{\\prime\}\}\\sum\_\{s=b\_\{t\}\}^\{t\}\\Delta F\_\{s\}\\leq L\\cdot\\frac\{1\}\{T^\{\\prime\}\}\\sum\_\{s=b\_\{t\}\}^\{t\}\\\|\\hat\{\\bm\{\\theta\}\}\_\{s\}\-\\bm\{\\theta\}\\\|\_\{2\}\.Onℳt\\mathcal\{M\}\_\{t\}, for eachs∈\{bt,…,t\}s\\in\\\{b\_\{t\},\\ldots,t\\\}we already have

‖𝜽^s−𝜽‖2≤2​σa¯​n​log⁡\(2​\|𝒫\|​tp\+3\)cγ​s1−γ=C​log⁡t​s−\(1−γ\)/2\\\|\\hat\{\\bm\{\\theta\}\}\_\{s\}\-\\bm\{\\theta\}\\\|\_\{2\}\\leq\\frac\{2\\sigma\}\{\\underline\{a\}\}\\sqrt\{\\frac\{n\\log\\\!\\big\(2\|\\mathcal\{P\}\|\\,t^\{p\+3\}\\big\)\}\{c\_\{\\gamma\}\\,s^\{1\-\\gamma\}\}\}=C\\sqrt\{\\log t\}\\,s^\{\-\(1\-\\gamma\)/2\}for a finite constantCC\. SinceT′≥t/2T^\{\\prime\}\\geq t/2fort≥2t\\geq 2, and by Fact[F\.2](https://arxiv.org/html/2607.08979#A6.Thmtheorem2)

1T′​∑s=btts−\(1−γ\)/2\\displaystyle\\frac\{1\}\{T^\{\\prime\}\}\\sum\_\{s=b\_\{t\}\}^\{t\}s^\{\-\(1\-\\gamma\)/2\}≤2t​\(1\+t\(1\+γ\)/2\(1\+γ\)/2\)\\displaystyle\\leq\\frac\{2\}\{t\}\\left\(1\+\\frac\{t^\{\(1\+\\gamma\)/2\}\}\{\(1\+\\gamma\)/2\}\\right\)≤81\+γ​t−\(1−γ\)/2\(t≥2\)\.\\displaystyle\\leq\\frac\{8\}\{1\+\\gamma\}\\,t^\{\-\(1\-\\gamma\)/2\}\\qquad\(t\\geq 2\)\.Thus onℳt\\mathcal\{M\}\_\{t\},

1T′​∑s=btt‖𝜽^s−𝜽‖2≤Cp,2​t−\(1−γ\)/2​log⁡t\.\\frac\{1\}\{T^\{\\prime\}\}\\sum\_\{s=b\_\{t\}\}^\{t\}\\\|\\hat\{\\bm\{\\theta\}\}\_\{s\}\-\\bm\{\\theta\}\\\|\_\{2\}\\leq C\_\{p,2\}\\,t^\{\-\(1\-\\gamma\)/2\}\\sqrt\{\\log t\}\.\(16\)Combining \([15](https://arxiv.org/html/2607.08979#A3.E15)\), \([C\.1](https://arxiv.org/html/2607.08979#A3.Ex119)\), and \([16](https://arxiv.org/html/2607.08979#A3.E16)\) and choosing finite constantCp,3C\_\{p,3\}gives that onℳt∩ℛt\\mathcal\{M\}\_\{t\}\\cap\\mathcal\{R\}\_\{t\},

Regw𝜽⁡\(t\)\+Regq𝜽⁡\(t\)T′\\displaystyle\\frac\{\\operatorname\{Reg\}\_\{w\}^\{\\bm\{\\theta\}\}\(t\)\+\\operatorname\{Reg\}\_\{q\}^\{\\bm\{\\theta\}\}\(t\)\}\{T^\{\\prime\}\}≤Cp,3\(t−\(1−α\)\+t−α\+log⁡tt\\displaystyle\\leq C\_\{p,3\}\\Big\(t^\{\-\(1\-\\alpha\)\}\+t^\{\-\\alpha\}\+\\sqrt\{\\tfrac\{\\log t\}\{t\}\}\+t−3/4\+t−\(1−γ\)/2log⁡t\)\\displaystyle\\qquad\+t^\{\-3/4\}\+t^\{\-\(1\-\\gamma\)/2\}\\sqrt\{\\log t\}\\Big\)\(17\)By Lemma[C\.3](https://arxiv.org/html/2607.08979#A3.Thmtheorem3),

gapt𝜽≤Regw𝜽⁡\(t\)\+Regq𝜽⁡\(t\)T′\.\\operatorname\{gap\}\_\{t\}^\{\\bm\{\\theta\}\}\\leq\\frac\{\\operatorname\{Reg\}\_\{w\}^\{\\bm\{\\theta\}\}\(t\)\+\\operatorname\{Reg\}\_\{q\}^\{\\bm\{\\theta\}\}\(t\)\}\{T^\{\\prime\}\}\.Also,sup𝒘F𝜽​\(𝒘,𝒒¯bt:t\)≥Γ∗​\(𝜽\)\\sup\_\{\\bm\{w\}\}F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bar\{\\bm\{q\}\}\_\{b\_\{t\}:t\}\)\\geq\\Gamma^\{\*\}\(\\bm\{\\theta\}\), hence

Γ​\(𝒘¯bt:t;𝜽\)=inf𝒒F𝜽​\(𝒘¯bt:t,𝒒\)≥Γ∗​\(𝜽\)−gapt𝜽\.\\Gamma\(\\bar\{\\bm\{w\}\}\_\{b\_\{t\}:t\};\\bm\{\\theta\}\)=\\inf\_\{\\bm\{q\}\}F\_\{\\bm\{\\theta\}\}\(\\bar\{\\bm\{w\}\}\_\{b\_\{t\}:t\},\\bm\{q\}\)\\geq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\operatorname\{gap\}\_\{t\}^\{\\bm\{\\theta\}\}\.Therefore, onℳt∩ℛt\\mathcal\{M\}\_\{t\}\\cap\\mathcal\{R\}\_\{t\}, combining with \([C\.1](https://arxiv.org/html/2607.08979#A3.Ex129)\) gives

Γ​\(𝒘¯bt:t;𝜽\)\\displaystyle\\Gamma\(\\bar\{\\bm\{w\}\}\_\{b\_\{t\}:t\};\\bm\{\\theta\}\)≥Γ∗​\(𝜽\)\\displaystyle\\geq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)−Cp,3\(t−\(1−α\)\+t−α\+log⁡tt\\displaystyle\\quad\-C\_\{p,3\}\\Big\(t^\{\-\(1\-\\alpha\)\}\+t^\{\-\\alpha\}\+\\sqrt\{\\tfrac\{\\log t\}\{t\}\}\+t−3/4\+t−\(1−γ\)/2log⁡t\)\\displaystyle\\qquad\\qquad\+t^\{\-3/4\}\+t^\{\-\(1\-\\gamma\)/2\}\\sqrt\{\\log t\}\\Big\)Next, we transfer this guarantee to𝒘temp\\bm\{w\}\_\{t\}^\{\\mathrm\{emp\}\}\. First,

‖𝒘¯t−𝒘¯bt:t‖1≤2​\(bt−1\)t≤2​t−3/4\.\\\|\\bar\{\\bm\{w\}\}\_\{t\}\-\\bar\{\\bm\{w\}\}\_\{b\_\{t\}:t\}\\\|\_\{1\}\\leq\\frac\{2\(b\_\{t\}\-1\)\}\{t\}\\leq 2t^\{\-3/4\}\.Second,\|Γ​\(𝒘;𝜽\)−Γ​\(𝒘′;𝜽\)\|≤Dmax​‖𝒘−𝒘′‖1\|\\Gamma\(\\bm\{w\};\\bm\{\\theta\}\)\-\\Gamma\(\\bm\{w\}^\{\\prime\};\\bm\{\\theta\}\)\|\\leq D\_\{\\max\}\\\|\\bm\{w\}\-\\bm\{w\}^\{\\prime\}\\\|\_\{1\}\. Combining these gives

Γ​\(𝒘¯t;𝜽\)≥Γ​\(𝒘¯bt:t;𝜽\)−2​Dmax​t−3/4\.\\Gamma\(\\bar\{\\bm\{w\}\}\_\{t\};\\bm\{\\theta\}\)\\geq\\Gamma\(\\bar\{\\bm\{w\}\}\_\{b\_\{t\}:t\};\\bm\{\\theta\}\)\-2D\_\{\\max\}t^\{\-3/4\}\.Finally, by Lemma[F\.1](https://arxiv.org/html/2607.08979#A6.Thmtheorem1)*\(c\)*,

‖𝒘^temp−𝒘¯t‖∞≤\|𝒫\|−1t\+1t​∑s=1tρs≤\|𝒫\|−1t\+t−γ1−γ\\\|\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\}\-\\bar\{\\bm\{w\}\}\_\{t\}\\\|\_\{\\infty\}\\leq\\frac\{\|\\mathcal\{P\}\|\-1\}\{t\}\+\\frac\{1\}\{t\}\\sum\_\{s=1\}^\{t\}\\rho\_\{s\}\\leq\\frac\{\|\\mathcal\{P\}\|\-1\}\{t\}\+\\frac\{t^\{\-\\gamma\}\}\{1\-\\gamma\}Therefore,

‖𝒘^temp−𝒘¯t‖1≤\|𝒫\|​\(\|𝒫\|−1t\+t−γ1−γ\)\\\|\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\}\-\\bar\{\\bm\{w\}\}\_\{t\}\\\|\_\{1\}\\leq\|\\mathcal\{P\}\|\\left\(\\frac\{\|\\mathcal\{P\}\|\-1\}\{t\}\+\\frac\{t^\{\-\\gamma\}\}\{1\-\\gamma\}\\right\)By Lipschitzness ofΓ\\Gammaand the previous bounds, onℳt∩ℛt\\mathcal\{M\}\_\{t\}\\cap\\mathcal\{R\}\_\{t\}we obtain

Γ​\(𝒘^temp;𝜽\)≥\\displaystyle\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\ \\geqΓ∗​\(𝜽\)\\displaystyle\\Gamma^\{\*\}\(\\bm\{\\theta\}\)−Cp,3​\(t−\(1−α\)\+t−α\+log⁡tt\+t−3/4\)\\displaystyle\-C\_\{p,3\}\\Big\(t^\{\-\(1\-\\alpha\)\}\+t^\{\-\\alpha\}\+\\sqrt\{\\tfrac\{\\log t\}\{t\}\}\+t^\{\-3/4\}\\Big\)−Cp,3​t−\(1−γ\)/2​log⁡t\\displaystyle\-C\_\{p,3\}\\,t^\{\-\(1\-\\gamma\)/2\}\\sqrt\{\\log t\}−Dmax​\|𝒫\|​\(\|𝒫\|−1t\+11−γ​t−γ\)\\displaystyle\-D\_\{\\max\}\|\\mathcal\{P\}\|\\left\(\\frac\{\|\\mathcal\{P\}\|\-1\}\{t\}\+\\frac\{1\}\{1\-\\gamma\}\\,t^\{\-\\gamma\}\\right\)−2​Dmax​t−3/4\.\\displaystyle\-2D\_\{\\max\}t^\{\-3/4\}\.This is the inequality in Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)\.

Fort≥tpt\\geq t\_\{p\}, we have shownℳt∩ℛt\\mathcal\{M\}\_\{t\}\\cap\\mathcal\{R\}\_\{t\}implies this inequality\. Therefore, the failure event at timettis contained inℳtc∪ℛtc\\mathcal\{M\}\_\{t\}^\{c\}\\cup\\mathcal\{R\}\_\{t\}^\{c\}\. Hence, by \([12](https://arxiv.org/html/2607.08979#A3.E12)\) and \([13](https://arxiv.org/html/2607.08979#A3.E13)\),

ℙ𝜽​\(failure at time​t\)\\displaystyle\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\text\{failure at time \}t\)≤ℙ𝜽​\(ℳtc\)\+ℙ𝜽​\(ℛtc\)\\displaystyle\\leq\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\mathcal\{M\}\_\{t\}^\{c\}\)\+\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\mathcal\{R\}\_\{t\}^\{c\}\)≤t−\(p\+1\)\+t−\(p\+2\)\\displaystyle\\leq t^\{\-\(p\+1\)\}\+t^\{\-\(p\+2\)\}≤t−p\(∀t≥2\)\.\\displaystyle\\leq t^\{\-p\}\\qquad\(\\forall t\\geq 2\)\.This proves Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)for allt≥tpt\\geq t\_\{p\}\. ∎

## Appendix DProof of Proposition[6\.4](https://arxiv.org/html/2607.08979#S6.Thmtheorem4)

##### Outline\.

This section proves the two parts of Proposition[6\.4](https://arxiv.org/html/2607.08979#S6.Thmtheorem4): \(i\) theδ\\delta\-correct risk bound and \(ii\) almost sure finite stopping\. For \(i\), we first introduce a mixture likelihood\-ratio martingale valid under adaptive sampling \(Lemma[D\.1](https://arxiv.org/html/2607.08979#A4.Thmtheorem1)\), then lower bound it to obtain the stopping threshold \(Lemma[D\.2](https://arxiv.org/html/2607.08979#A4.Thmtheorem2)\), and finally apply Ville’s inequality to get the risk bound \(Proposition[D\.3](https://arxiv.org/html/2607.08979#A4.Thmtheorem3)\)\. For \(ii\), we need to show that the GLR statistic eventually will be larger than the stopping threshold\. To do this, bound the difference of the normalized GLR statistic to the information rate once the boundary constraints are correct \(Lemma[D\.5](https://arxiv.org/html/2607.08979#A4.Thmtheorem5)\)\. Then \(Lemma[D\.6](https://arxiv.org/html/2607.08979#A4.Thmtheorem6)and Corollary[D\.7](https://arxiv.org/html/2607.08979#A4.Thmtheorem7)\) show this difference goes to 0 almost surely\. Finally, \(Lemma[D\.8](https://arxiv.org/html/2607.08979#A4.Thmtheorem8)and Proposition[D\.9](https://arxiv.org/html/2607.08979#A4.Thmtheorem9)\) show that the GLR statistic grows linearly and the stopping threshold grows sublinearly, to guarantee a finite stopping time\. The combination of Propositions[D\.3](https://arxiv.org/html/2607.08979#A4.Thmtheorem3)and[D\.9](https://arxiv.org/html/2607.08979#A4.Thmtheorem9)gives Proposition[6\.4](https://arxiv.org/html/2607.08979#S6.Thmtheorem4)\.

### D\.1Part I: Risk Bound

###### Lemma D\.1\.

Fix𝛉∈Θ\\bm\{\\theta\}\\in\\Thetaand letπ\\pibe any probability distribution onΘ0\\Theta\_\{0\}\. Define

Mt:=∫Θ0exp⁡\(ℓt​\(𝒖\)−ℓt​\(𝜽\)\)​π​\(d​𝒖\),M0:=1\.M\_\{t\}\\;:=\\;\\int\_\{\\Theta\_\{0\}\}\\exp\\\!\\big\(\\ell\_\{t\}\(\\bm\{u\}\)\-\\ell\_\{t\}\(\\bm\{\\theta\}\)\\big\)\\,\\pi\(d\\bm\{u\}\),\\qquad M\_\{0\}:=1\.Then\(Mt\)t≥0\(M\_\{t\}\)\_\{t\\geq 0\}is a nonnegative martingale with respect toℱt=σ\(\(As,Ys\):s≤t\)\\mathcal\{F\}\_\{t\}=\\sigma\(\(A\_\{s\},Y\_\{s\}\):s\\leq t\)for*any*adaptive sampling rule\.

###### Proof\.

SeeKaufmann & Koolen \([2021](https://arxiv.org/html/2607.08979#bib.bib21)\), Sec\. 2\.3\. ∎

The argument in Lemma[D\.2](https://arxiv.org/html/2607.08979#A4.Thmtheorem2)closely follows existing work on stopping thresholds for linear bandits, for example, seeAbbasi\-Yadkori et al\. \([2011](https://arxiv.org/html/2607.08979#bib.bib1)\)\.

###### Lemma D\.2\.

Fix𝛉∈Θ\\bm\{\\theta\}\\in\\Thetaand letd=dim\(Θ0\)=n−1d=\\dim\(\\Theta\_\{0\}\)=n\-1\. Letλ\>0\\lambda\>0\. Let𝛇∼𝒩​\(0,λ−1​In\)\\bm\{\\zeta\}\\sim\\mathcal\{N\}\(0,\\lambda^\{\-1\}I\_\{n\}\)inℝn\\mathbb\{R\}^\{n\}and let

Π:=In−1n​𝟏𝟏⊤\\Pi\\;:=\\;I\_\{n\}\-\\frac\{1\}\{n\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}denote the orthogonal projector ontoΘ0\\Theta\_\{0\}\. Defineϑ:=Π​𝛇∈Θ0\\bm\{\\vartheta\}:=\\Pi\\bm\{\\zeta\}\\in\\Theta\_\{0\}, and letπλ\\pi\_\{\\lambda\}denote its distribution\. Define the mixture process

Mt:=𝔼​\[exp⁡\(ℓt​\(ϑ\)−ℓt​\(𝜽\)\)\],M0:=1,M\_\{t\}\\;:=\\;\\mathbb\{E\}\\\!\\left\[\\exp\\\!\\big\(\\ell\_\{t\}\(\\bm\{\\vartheta\}\)\-\\ell\_\{t\}\(\\bm\{\\theta\}\)\\big\)\\right\],\\qquad M\_\{0\}:=1,and let𝛉^t∈argmaxϑ∈Θℓt​\(ϑ\)\\hat\{\\bm\{\\theta\}\}\_\{t\}\\in\\mathop\{\\mathrm\{argmax\}\}\_\{\\bm\{\\vartheta\}\\in\\Theta\}\\ell\_\{t\}\(\\bm\{\\vartheta\}\)be a constrained MLE\. Define the Laplacian at timett,ℒt:=∑s=1t𝐱As​𝐱As⊤\\mathcal\{L\}\_\{t\}:=\\sum\_\{s=1\}^\{t\}\\bm\{x\}\_\{A\_\{s\}\}\\bm\{x\}\_\{A\_\{s\}\}^\{\\top\}

Then for allt≥1t\\geq 1,

Mt\\displaystyle M\_\{t\}≥exp⁡\(ℓt​\(𝜽^t\)−ℓt​\(𝜽\)\)\\displaystyle\\;\\geq\\;\\exp\\\!\\big\(\\ell\_\{t\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\-\\ell\_\{t\}\(\\bm\{\\theta\}\)\\big\)⋅exp⁡\(−λ2​‖𝜽^t‖22\)⋅det\(In\+σ¯2λ​ℒt\)−1/2\.\\displaystyle\\quad\\cdot\\exp\\\!\\Big\(\-\\frac\{\\lambda\}\{2\}\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\_\{2\}^\{2\}\\Big\)\\cdot\\det\\\!\\Big\(I\_\{n\}\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\mathcal\{L\}\_\{t\}\\Big\)^\{\-1/2\}\.\(18\)Equivalently,

ℓt​\(𝜽^t\)−ℓt​\(𝜽\)≤log⁡Mt\+λ2​‖𝜽^t‖22\+12​log​det\(In\+σ¯2λ​ℒt\)\.\\ell\_\{t\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\-\\ell\_\{t\}\(\\bm\{\\theta\}\)\\;\\leq\\;\\log M\_\{t\}\+\\frac\{\\lambda\}\{2\}\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\_\{2\}^\{2\}\+\\frac\{1\}\{2\}\\log\\det\\\!\\Big\(I\_\{n\}\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\mathcal\{L\}\_\{t\}\\Big\)\.

###### Proof\.

TakeU∈ℝn×dU\\in\\mathbb\{R\}^\{n\\times d\}that has orthonormal columns spanningΘ0\\Theta\_\{0\}, soU⊤​U=IdU^\{\\top\}U=I\_\{d\}andU⊤​𝟏=0U^\{\\top\}\\mathbf\{1\}=0\. ThenΠ=U​U⊤\\Pi=UU^\{\\top\}, and everyϑ∈Θ0\\bm\{\\vartheta\}\\in\\Theta\_\{0\}can be written uniquely asϑ=U​𝒛\\bm\{\\vartheta\}=U\\bm\{z\}with𝒛∈ℝd\\bm\{z\}\\in\\mathbb\{R\}^\{d\}and‖ϑ‖2=‖𝒛‖2\\\|\\bm\{\\vartheta\}\\\|\_\{2\}=\\\|\\bm\{z\}\\\|\_\{2\}\. Define

ℓ~t​\(𝒛\):=ℓt​\(U​𝒛\),𝒛𝜽:=U⊤​𝜽,𝒛^t:=U⊤​𝜽^t\.\\tilde\{\\ell\}\_\{t\}\(\\bm\{z\}\):=\\ell\_\{t\}\(U\\bm\{z\}\),\\qquad\\bm\{z\}\_\{\\bm\{\\theta\}\}:=U^\{\\top\}\\bm\{\\theta\},\\qquad\\hat\{\\bm\{z\}\}\_\{t\}:=U^\{\\top\}\\hat\{\\bm\{\\theta\}\}\_\{t\}\.With𝜻\\bm\{\\zeta\}as in the statement, set𝒁:=U⊤​𝜻\\bm\{Z\}:=U^\{\\top\}\\bm\{\\zeta\}\. Then𝒁∼𝒩​\(0,λ−1​Id\)\\bm\{Z\}\\sim\\mathcal\{N\}\(0,\\lambda^\{\-1\}I\_\{d\}\)andϑ=Π​𝜻=U​U⊤​𝜻=U​𝒁\.\\bm\{\\vartheta\}=\\Pi\\bm\{\\zeta\}=UU^\{\\top\}\\bm\{\\zeta\}=U\\bm\{Z\}\.Therefore

Mt\\displaystyle M\_\{t\}=𝔼​\[exp⁡\(ℓ~t​\(𝒁\)−ℓ~t​\(𝒛𝜽\)\)\]\\displaystyle=\\mathbb\{E\}\\\!\\left\[\\exp\\\!\\big\(\\tilde\{\\ell\}\_\{t\}\(\\bm\{Z\}\)\-\\tilde\{\\ell\}\_\{t\}\(\\bm\{z\}\_\{\\bm\{\\theta\}\}\)\\big\)\\right\]\(19\)=∫ℝdexp⁡\(ℓ~t​\(𝒛\)−ℓ~t​\(𝒛𝜽\)\)​φλ​\(𝒛\)​𝑑𝒛\\displaystyle=\\int\_\{\\mathbb\{R\}^\{d\}\}\\exp\\\!\\big\(\\tilde\{\\ell\}\_\{t\}\(\\bm\{z\}\)\-\\tilde\{\\ell\}\_\{t\}\(\\bm\{z\}\_\{\\bm\{\\theta\}\}\)\\big\)\\,\\varphi\_\{\\lambda\}\(\\bm\{z\}\)\\,d\\bm\{z\}=\(λ2​π\)d/2​e−ℓ~t​\(𝒛𝜽\)​∫ℝdexp⁡\(ℓ~t​\(𝒛\)−λ2​‖𝒛‖22\)​𝑑𝒛\.\\displaystyle=\\Big\(\\frac\{\\lambda\}\{2\\pi\}\\Big\)^\{d/2\}e^\{\-\\tilde\{\\ell\}\_\{t\}\(\\bm\{z\}\_\{\\bm\{\\theta\}\}\)\}\\int\_\{\\mathbb\{R\}^\{d\}\}\\exp\\\!\\Big\(\\tilde\{\\ell\}\_\{t\}\(\\bm\{z\}\)\-\\frac\{\\lambda\}\{2\}\\\|\\bm\{z\}\\\|\_\{2\}^\{2\}\\Big\)\\,d\\bm\{z\}\.Next, we bound the Hessian\. For anyϑ∈Θ0\\bm\{\\vartheta\}\\in\\Theta\_\{0\}, by Assumption[1](https://arxiv.org/html/2607.08979#Thmassumption1)

−∇2ℓt​\(ϑ\)\\displaystyle\-\\nabla^\{2\}\\ell\_\{t\}\(\\bm\{\\vartheta\}\)=∑s=1tA′′​\(ηAs​\(ϑ\)\)​𝒙As​𝒙As⊤\\displaystyle=\\sum\_\{s=1\}^\{t\}A^\{\\prime\\prime\}\\\!\\big\(\\eta\_\{A\_\{s\}\}\(\\bm\{\\vartheta\}\)\\big\)\\,\\bm\{x\}\_\{A\_\{s\}\}\\bm\{x\}\_\{A\_\{s\}\}^\{\\top\}⪯σ¯2​∑s=1t𝒙As​𝒙As⊤=σ¯2​ℒt\\displaystyle\\preceq\\overline\{\\sigma\}^\{2\}\\sum\_\{s=1\}^\{t\}\\bm\{x\}\_\{A\_\{s\}\}\\bm\{x\}\_\{A\_\{s\}\}^\{\\top\}=\\overline\{\\sigma\}^\{2\}\\mathcal\{L\}\_\{t\}Projecting ontoΘ0\\Theta\_\{0\}gives, for all𝒛∈ℝd\\bm\{z\}\\in\\mathbb\{R\}^\{d\},

−∇2ℓ~t​\(𝒛\)=−U⊤​∇2ℓt​\(U​𝒛\)​U⪯σ¯2​U⊤​ℒt​U\.\-\\nabla^\{2\}\\tilde\{\\ell\}\_\{t\}\(\\bm\{z\}\)=\-U^\{\\top\}\\nabla^\{2\}\\ell\_\{t\}\(U\\bm\{z\}\)\\,U\\ \\preceq\\ \\overline\{\\sigma\}^\{2\}\\,U^\{\\top\}\\mathcal\{L\}\_\{t\}U\.Next, we lower boundft​\(𝒛\):=ℓ~t​\(𝒛\)−λ2​‖𝒛‖22\.f\_\{t\}\(\\bm\{z\}\)\\ :=\\ \\tilde\{\\ell\}\_\{t\}\(\\bm\{z\}\)\-\\frac\{\\lambda\}\{2\}\\\|\\bm\{z\}\\\|\_\{2\}^\{2\}\.For all𝒛∈ℝd\\bm\{z\}\\in\\mathbb\{R\}^\{d\},

−∇2ft\(𝒛\)=−∇2ℓ~t\(𝒛\)\+λId⪯σ¯2U⊤ℒtU\+λId=:Ht,\-\\nabla^\{2\}f\_\{t\}\(\\bm\{z\}\)=\-\\nabla^\{2\}\\tilde\{\\ell\}\_\{t\}\(\\bm\{z\}\)\+\\lambda I\_\{d\}\\ \\preceq\\ \\overline\{\\sigma\}^\{2\}\\,U^\{\\top\}\\mathcal\{L\}\_\{t\}U\+\\lambda I\_\{d\}=:H\_\{t\},whereHt≻0H\_\{t\}\\succ 0sinceλ\>0\\lambda\>0\. Let𝒛tλ∈argmax𝒛∈ℝdft​\(𝒛\)\\bm\{z\}\_\{t\}^\{\\lambda\}\\in\\mathop\{\\mathrm\{argmax\}\}\_\{\\bm\{z\}\\in\\mathbb\{R\}^\{d\}\}f\_\{t\}\(\\bm\{z\}\), which exists and is unique byλ\\lambda\-strong concavity\. Since∇ft​\(𝒛tλ\)=0\\nabla f\_\{t\}\(\\bm\{z\}\_\{t\}^\{\\lambda\}\)=0and−∇2ft⪯Ht\-\\nabla^\{2\}f\_\{t\}\\preceq H\_\{t\}everywhere, Taylor’s theorem gives, for all𝒛∈ℝd\\bm\{z\}\\in\\mathbb\{R\}^\{d\},

ft​\(𝒛\)≥ft​\(𝒛tλ\)−12​\(𝒛−𝒛tλ\)⊤​Ht​\(𝒛−𝒛tλ\)\.f\_\{t\}\(\\bm\{z\}\)\\ \\geq\\ f\_\{t\}\(\\bm\{z\}\_\{t\}^\{\\lambda\}\)\\ \-\\ \\frac\{1\}\{2\}\(\\bm\{z\}\-\\bm\{z\}\_\{t\}^\{\\lambda\}\)^\{\\top\}H\_\{t\}\(\\bm\{z\}\-\\bm\{z\}\_\{t\}^\{\\lambda\}\)\.Thus,

∫ℝdeft​\(𝒛\)​𝑑𝒛\\displaystyle\\int\_\{\\mathbb\{R\}^\{d\}\}e^\{f\_\{t\}\(\\bm\{z\}\)\}d\\bm\{z\}≥eft​\(𝒛tλ\)​∫ℝdexp⁡\(−12​\(𝒛−𝒛tλ\)⊤​Ht​\(𝒛−𝒛tλ\)\)​𝑑𝒛\\displaystyle\\geq e^\{f\_\{t\}\(\\bm\{z\}\_\{t\}^\{\\lambda\}\)\}\\\!\\int\_\{\\mathbb\{R\}^\{d\}\}\\\!\\exp\\\!\\big\(\-\\tfrac\{1\}\{2\}\(\\bm\{z\}\-\\bm\{z\}\_\{t\}^\{\\lambda\}\)^\{\\top\}\\\!H\_\{t\}\(\\bm\{z\}\-\\bm\{z\}\_\{t\}^\{\\lambda\}\)\\big\)d\\bm\{z\}=eft​\(𝒛tλ\)​\(2​π\)d/2det\(Ht\)1/2\.\\displaystyle=e^\{f\_\{t\}\(\\bm\{z\}\_\{t\}^\{\\lambda\}\)\}\\frac\{\(2\\pi\)^\{d/2\}\}\{\\det\(H\_\{t\}\)^\{1/2\}\}\.Plugging into \([19](https://arxiv.org/html/2607.08979#A4.E19)\) yields

Mt≥exp⁡\(ft​\(𝒛tλ\)−ℓ~t​\(𝒛𝜽\)\)⋅λd/2det\(Ht\)1/2\.M\_\{t\}\\ \\geq\\ \\exp\\\!\\big\(f\_\{t\}\(\\bm\{z\}\_\{t\}^\{\\lambda\}\)\-\\tilde\{\\ell\}\_\{t\}\(\\bm\{z\}\_\{\\bm\{\\theta\}\}\)\\big\)\\cdot\\frac\{\\lambda^\{d/2\}\}\{\\det\(H\_\{t\}\)^\{1/2\}\}\.\(20\)𝒛tλ\\bm\{z\}\_\{t\}^\{\\lambda\}maximizesftf\_\{t\}overℝd\\mathbb\{R\}^\{d\}, we haveft​\(𝒛tλ\)≥ft​\(𝒛^t\)f\_\{t\}\(\\bm\{z\}\_\{t\}^\{\\lambda\}\)\\geq f\_\{t\}\(\\hat\{\\bm\{z\}\}\_\{t\}\)\. Notingℓ~t​\(𝒛^t\)=ℓt​\(𝜽^t\)\\tilde\{\\ell\}\_\{t\}\(\\hat\{\\bm\{z\}\}\_\{t\}\)=\\ell\_\{t\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\)and‖𝒛^t‖2=‖𝜽^t‖2\\\|\\hat\{\\bm\{z\}\}\_\{t\}\\\|\_\{2\}=\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\_\{2\}, we get

ft​\(𝒛^t\)−ℓ~t​\(𝒛𝜽\)=ℓt​\(𝜽^t\)−ℓt​\(𝜽\)−λ2​‖𝜽^t‖22\.f\_\{t\}\(\\hat\{\\bm\{z\}\}\_\{t\}\)\-\\tilde\{\\ell\}\_\{t\}\(\\bm\{z\}\_\{\\bm\{\\theta\}\}\)=\\ell\_\{t\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\-\\ell\_\{t\}\(\\bm\{\\theta\}\)\-\\frac\{\\lambda\}\{2\}\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\_\{2\}^\{2\}\.Also,

Ht=λ​Id\+σ¯2​U⊤​ℒt​U=λ​\(Id\+σ¯2λ​U⊤​ℒt​U\),H\_\{t\}=\\lambda I\_\{d\}\+\\overline\{\\sigma\}^\{2\}\\,U^\{\\top\}\\mathcal\{L\}\_\{t\}U=\\lambda\\Big\(I\_\{d\}\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\,U^\{\\top\}\\mathcal\{L\}\_\{t\}U\\Big\),sodet\(Ht\)=λd​det\(Id\+σ¯2λ​U⊤​ℒt​U\)\\det\(H\_\{t\}\)=\\lambda^\{d\}\\det\(I\_\{d\}\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\,U^\{\\top\}\\mathcal\{L\}\_\{t\}U\)and therefore

λd/2det\(Ht\)1/2\\displaystyle\\frac\{\\lambda^\{d/2\}\}\{\\det\(H\_\{t\}\)^\{1/2\}\}=det\(Id\+σ¯2λ​U⊤​ℒt​U\)−1/2\\displaystyle=\\det\\\!\\Big\(I\_\{d\}\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\,U^\{\\top\}\\mathcal\{L\}\_\{t\}U\\Big\)^\{\-1/2\}=det\(In\+σ¯2λ​ℒt\)−1/2\\displaystyle=\\det\\\!\\Big\(I\_\{n\}\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\mathcal\{L\}\_\{t\}\\Big\)^\{\-1/2\}where the last equality follows fromℒt​𝟏=0\\mathcal\{L\}\_\{t\}\\mathbf\{1\}=0\. Combining these with \([20](https://arxiv.org/html/2607.08979#A4.E20)\) gives \([D\.2](https://arxiv.org/html/2607.08979#A4.Ex146)\)\. ∎

###### Proposition D\.3\.

Fix any𝛉∈Θgap\\bm\{\\theta\}\\in\\Theta^\{\\mathrm\{gap\}\}andλ\>0\\lambda\>0\. Define the threshold

β​\(t,δ\):=log⁡1δ\+λ2​‖𝜽^t‖22\+12​log​det\(In\+σ¯2λ​ℒt\),\\beta\(t,\\delta\)\\;:=\\;\\log\\frac\{1\}\{\\delta\}\+\\frac\{\\lambda\}\{2\}\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\_\{2\}^\{2\}\+\\frac\{1\}\{2\}\\log\\det\\\!\\Big\(I\_\{n\}\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\mathcal\{L\}\_\{t\}\\Big\),\(21\)and the stopping time

τδ:=inf\{t≥1:ZS^t​\(t\)≥β​\(t,δ\)\},output​S^τδ\.\\tau\_\{\\delta\}:=\\inf\\\{t\\geq 1:\\ Z\_\{\\hat\{S\}\_\{t\}\}\(t\)\\geq\\beta\(t,\\delta\)\\\},\\qquad\\text\{output \}\\hat\{S\}\_\{\\tau\_\{\\delta\}\}\.Then

ℙ𝜽​\(S^τδ≠S∗​\(𝜽\)\)≤δ\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\hat\{S\}\_\{\\tau\_\{\\delta\}\}\\neq S^\{\*\}\(\\bm\{\\theta\}\)\)\\ \\leq\\ \\delta\.

###### Proof\.

On the event\{τδ=t,S^t≠S∗​\(𝜽\)\}\\\{\\tau\_\{\\delta\}=t,\\ \\hat\{S\}\_\{t\}\\neq S^\{\*\}\(\\bm\{\\theta\}\)\\\}there existu∈S^t∖S∗​\(𝜽\)u\\in\\hat\{S\}\_\{t\}\\setminus S^\{\*\}\(\\bm\{\\theta\}\)andv∈S∗​\(𝜽\)∖S^tv\\in S^\{\*\}\(\\bm\{\\theta\}\)\\setminus\\hat\{S\}\_\{t\}\. Thenθv≥θu\\theta\_\{v\}\\geq\\theta\_\{u\}, so𝜽∈Θu​v​\(t\)\\bm\{\\theta\}\\in\\Theta\_\{uv\}\(t\)\. And\(u,v\)∈B^t\(u,v\)\\in\\hat\{B\}\_\{t\}, so

Zu​v​\(t\)=ℓt​\(𝜽^t\)−sup𝜽′∈Θu​v​\(t\)ℓt​\(𝜽′\)≤ℓt​\(𝜽^t\)−ℓt​\(𝜽\)\.Z\_\{uv\}\(t\)=\\ell\_\{t\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\-\\sup\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{uv\}\(t\)\}\\ell\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\\leq\\ell\_\{t\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\-\\ell\_\{t\}\(\\bm\{\\theta\}\)\.Sinceτδ=t\\tau\_\{\\delta\}=timpliesZS^t​\(t\)=min\(i,j\)∈B^t⁡Zi​j​\(t\)≥β​\(t,δ\)Z\_\{\\hat\{S\}\_\{t\}\}\(t\)=\\min\_\{\(i,j\)\\in\\hat\{B\}\_\{t\}\}Z\_\{ij\}\(t\)\\geq\\beta\(t,\\delta\), we have

ℓt​\(𝜽^t\)−ℓt​\(𝜽\)≥Zu​v​\(t\)≥β​\(t,δ\)\.\\ell\_\{t\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\)\-\\ell\_\{t\}\(\\bm\{\\theta\}\)\\ \\geq\\ Z\_\{uv\}\(t\)\\ \\geq\\ \\beta\(t,\\delta\)\.Now letπλ\\pi\_\{\\lambda\}be the Gaussian prior onΘ0\\Theta\_\{0\}in Lemma[D\.2](https://arxiv.org/html/2607.08979#A4.Thmtheorem2)and define

Mt:=∫Θ0exp⁡\(ℓt​\(𝒖\)−ℓt​\(𝜽\)\)​πλ​\(d​𝒖\),M0:=1\.M\_\{t\}\\;:=\\;\\int\_\{\\Theta\_\{0\}\}\\exp\\\!\\big\(\\ell\_\{t\}\(\\bm\{u\}\)\-\\ell\_\{t\}\(\\bm\{\\theta\}\)\\big\)\\,\\pi\_\{\\lambda\}\(d\\bm\{u\}\),\\qquad M\_\{0\}:=1\.By Lemma[D\.1](https://arxiv.org/html/2607.08979#A4.Thmtheorem1)\(Mt\)\(M\_\{t\}\)is a nonnegative martingale\. Apply Lemma[D\.2](https://arxiv.org/html/2607.08979#A4.Thmtheorem2)att=τδt=\\tau\_\{\\delta\},

Mτδ\\displaystyle M\_\{\\tau\_\{\\delta\}\}≥exp⁡\(ℓτδ​\(𝜽^τδ\)−ℓτδ​\(𝜽\)\)\\displaystyle\\ \\geq\\ \\exp\\\!\\big\(\\ell\_\{\\tau\_\{\\delta\}\}\(\\hat\{\\bm\{\\theta\}\}\_\{\\tau\_\{\\delta\}\}\)\-\\ell\_\{\\tau\_\{\\delta\}\}\(\\bm\{\\theta\}\)\\big\)⋅exp⁡\(−λ2​‖𝜽^τδ‖22\)⋅det\(In\+σ¯2λ​ℒτδ\)−1/2\.\\displaystyle\\quad\\cdot\\exp\\\!\\Big\(\-\\frac\{\\lambda\}\{2\}\\\|\\hat\{\\bm\{\\theta\}\}\_\{\\tau\_\{\\delta\}\}\\\|\_\{2\}^\{2\}\\Big\)\\cdot\\det\\\!\\Big\(I\_\{n\}\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\mathcal\{L\}\_\{\\tau\_\{\\delta\}\}\\Big\)^\{\-1/2\}\.On the event\{ℓτδ​\(𝜽^τδ\)−ℓτδ​\(𝜽\)≥β​\(τδ,δ\)\}\\\{\\ell\_\{\\tau\_\{\\delta\}\}\(\\hat\{\\bm\{\\theta\}\}\_\{\\tau\_\{\\delta\}\}\)\-\\ell\_\{\\tau\_\{\\delta\}\}\(\\bm\{\\theta\}\)\\geq\\beta\(\\tau\_\{\\delta\},\\delta\)\\\},

Mτδ\\displaystyle M\_\{\\tau\_\{\\delta\}\}≥exp⁡\(β​\(τδ,δ\)\)⋅exp⁡\(−λ2​‖𝜽^τδ‖22\)\\displaystyle\\geq\\exp\\\!\\big\(\\beta\(\\tau\_\{\\delta\},\\delta\)\\big\)\\cdot\\exp\\\!\\Big\(\-\\frac\{\\lambda\}\{2\}\\\|\\hat\{\\bm\{\\theta\}\}\_\{\\tau\_\{\\delta\}\}\\\|\_\{2\}^\{2\}\\Big\)⋅det\(In\+σ¯2λℒτδ\)−1/2\\displaystyle\\qquad\\cdot\\det\\\!\\Big\(I\_\{n\}\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\mathcal\{L\}\_\{\\tau\_\{\\delta\}\}\\Big\)^\{\-1/2\}=exp⁡\(log⁡1δ\+λ2​‖𝜽^τδ‖22\+12​log​det\(In\+σ¯2λ​ℒτδ\)\)\\displaystyle=\\exp\\\!\\Big\(\\log\\tfrac\{1\}\{\\delta\}\+\\tfrac\{\\lambda\}\{2\}\\\|\\hat\{\\bm\{\\theta\}\}\_\{\\tau\_\{\\delta\}\}\\\|\_\{2\}^\{2\}\+\\tfrac\{1\}\{2\}\\log\\det\\\!\\big\(I\_\{n\}\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\mathcal\{L\}\_\{\\tau\_\{\\delta\}\}\\big\)\\Big\)⋅exp⁡\(−λ2​‖𝜽^τδ‖22\)⋅det\(In\+σ¯2λ​ℒτδ\)−1/2\\displaystyle\\qquad\\cdot\\exp\\\!\\Big\(\-\\tfrac\{\\lambda\}\{2\}\\\|\\hat\{\\bm\{\\theta\}\}\_\{\\tau\_\{\\delta\}\}\\\|\_\{2\}^\{2\}\\Big\)\\cdot\\det\\\!\\Big\(I\_\{n\}\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\mathcal\{L\}\_\{\\tau\_\{\\delta\}\}\\Big\)^\{\-1/2\}=1δ\.\\displaystyle=\\frac\{1\}\{\\delta\}\.Hence

\{S^τδ≠S∗​\(𝜽\)\}⊆\{supt≥0Mt≥1/δ\}\.\\\{\\hat\{S\}\_\{\\tau\_\{\\delta\}\}\\neq S^\{\*\}\(\\bm\{\\theta\}\)\\\}\\ \\subseteq\\ \\Big\\\{\\sup\_\{t\\geq 0\}M\_\{t\}\\geq 1/\\delta\\Big\\\}\.Since\(Mt\)\(M\_\{t\}\)is a nonnegative martingale withM0=1M\_\{0\}=1, Ville’s inequality gives

ℙ𝜽​\(supt≥0Mt≥1/δ\)≤δ\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\Big\(\\sup\_\{t\\geq 0\}M\_\{t\}\\geq 1/\\delta\\Big\)\\leq\\delta\.∎

### D\.2Part II: Finite Stopping

First, we show that the data\-dependent threshold from Lemma[D\.2](https://arxiv.org/html/2607.08979#A4.Thmtheorem2)is bounded by some constant\.

###### Lemma D\.4\.

Fixλ\>0\\lambda\>0andδ∈\(0,1\)\\delta\\in\(0,1\), and letβ​\(t,δ\)\\beta\(t,\\delta\)be as in Proposition[D\.3](https://arxiv.org/html/2607.08979#A4.Thmtheorem3)\. Letd=dim\(Θ0\)=n−1d=\\dim\(\\Theta\_\{0\}\)=n\-1and define the constant

Cβ:=2​σ¯2λ\.C\_\{\\beta\}\\ :=\\ \\frac\{2\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\.Then for allt≥1t\\geq 1,

β​\(t,δ\)≤β¯​\(t,δ\):=log⁡1δ\+λ2​n​R2\+d2​log⁡\(1\+Cβ​t\)\.\\beta\(t,\\delta\)\\ \\leq\\ \\bar\{\\beta\}\(t,\\delta\):=\\log\\frac\{1\}\{\\delta\}\+\\frac\{\\lambda\}\{2\}\\,nR^\{2\}\+\\frac\{d\}\{2\}\\log\\\!\\big\(1\+C\_\{\\beta\}t\\big\)\.

###### Proof\.

First,‖𝜽^t‖∞≤R\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\_\{\\infty\}\\leq Rimplies‖𝜽^t‖22≤n​R2\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\_\{2\}^\{2\}\\leq nR^\{2\}\. Next,ℒt=∑s=1t𝒙As​𝒙As⊤⪰0\\mathcal\{L\}\_\{t\}=\\sum\_\{s=1\}^\{t\}\\bm\{x\}\_\{A\_\{s\}\}\\bm\{x\}\_\{A\_\{s\}\}^\{\\top\}\\succeq 0andℒt​𝟏=0\\mathcal\{L\}\_\{t\}\\mathbf\{1\}=0, soℒt\\mathcal\{L\}\_\{t\}has at mostddnonzero eigenvalues\. Also‖𝒙i​j‖22=2\\\|\\bm\{x\}\_\{ij\}\\\|\_\{2\}^\{2\}=2implies‖𝒙As​𝒙As⊤‖op=2\\\|\\bm\{x\}\_\{A\_\{s\}\}\\bm\{x\}\_\{A\_\{s\}\}^\{\\top\}\\\|\_\{\\mathrm\{op\}\}=2, hence‖ℒt‖op≤2​t\\\|\\mathcal\{L\}\_\{t\}\\\|\_\{\\mathrm\{op\}\}\\leq 2t\. Therefore,

det\(In\+σ¯2λ​ℒt\)≤\(1\+σ¯2λ​‖ℒt‖op\)d≤\(1\+Cβ​t\)d,\\det\\\!\\Big\(I\_\{n\}\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\mathcal\{L\}\_\{t\}\\Big\)\\leq\\Big\(1\+\\frac\{\\overline\{\\sigma\}^\{2\}\}\{\\lambda\}\\\|\\mathcal\{L\}\_\{t\}\\\|\_\{\\mathrm\{op\}\}\\Big\)^\{d\}\\leq\(1\+C\_\{\\beta\}t\)^\{d\},∎

The thresholdβ​\(t,δ\)\\beta\(t,\\delta\)grows at most logarithmically withtt\. Thus, to show finite stopping, it is sufficient to show that1t​ZS^t​\(t\)\\frac\{1\}\{t\}Z\_\{\\hat\{S\}\_\{t\}\}\(t\)converges to some positive number\.

For𝜽′∈Θ\\bm\{\\theta\}^\{\\prime\}\\in\\Theta, define

L¯t​\(𝜽′\)\\displaystyle\\bar\{L\}\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\):=−1t​ℓt​\(𝜽′\)\\displaystyle=\-\\frac\{1\}\{t\}\\,\\ell\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)=1t​∑s=1t\(A​\(ηAs​\(𝜽′\)\)−ηAs​\(𝜽′\)​T​\(Ys\)\),\\displaystyle=\\frac\{1\}\{t\}\\sum\_\{s=1\}^\{t\}\\Big\(A\(\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}^\{\\prime\}\)\)\-\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}^\{\\prime\}\)\\,T\(Y\_\{s\}\)\\Big\),Lt​\(𝜽′\)\\displaystyle L\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\):=1t​∑s=1t𝔼𝜽​\[A​\(ηAs​\(𝜽′\)\)−ηAs​\(𝜽′\)​T​\(Ys\)\|ℱs−1\],\\displaystyle=\\frac\{1\}\{t\}\\sum\_\{s=1\}^\{t\}\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\\\!\\Big\[A\(\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}^\{\\prime\}\)\)\-\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}^\{\\prime\}\)\\,T\(Y\_\{s\}\)\\,\\Big\|\\,\\mathcal\{F\}\_\{s\-1\}\\Big\],and the uniform deviation

Δt:=sup𝜽′∈Θ\|L¯t​\(𝜽′\)−Lt​\(𝜽′\)\|\.\\Delta\_\{t\}:=\\sup\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\}\\big\|\\bar\{L\}\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\-L\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\\big\|\.
###### Lemma D\.5\.

Fixt≥1t\\geq 1and𝛉∈Θgap\\bm\{\\theta\}\\in\\Theta^\{\\mathrm\{gap\}\}\.

1. \(a\)For every𝜽′∈Θ\\bm\{\\theta\}^\{\\prime\}\\in\\Theta, Lt​\(𝜽′\)=Lt​\(𝜽\)\+D𝒘^temp​\(𝜽∥𝜽′\)\.L\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\\ =\\ L\_\{t\}\(\\bm\{\\theta\}\)\\ \+\\ D\_\{\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)\.\(22\)
2. \(b\)Suppose that at timettthe estimated boundary constraints are correct, i\.e\.B^t=B​\(𝜽\)\\hat\{B\}\_\{t\}=B\(\\bm\{\\theta\}\)andΘi​j​\(t\)=Θi​j\\Theta\_\{ij\}\(t\)=\\Theta\_\{ij\}for all\(i,j\)∈B​\(𝜽\)\(i,j\)\\in B\(\\bm\{\\theta\}\)\. Then for every\(i,j\)∈B​\(𝜽\)\(i,j\)\\in B\(\\bm\{\\theta\}\), \|1t​Zi​j​\(t\)−γi​j​\(𝒘^temp;𝜽\)\|≤2​Δt,\\left\|\\frac\{1\}\{t\}Z\_\{ij\}\(t\)\\ \-\\ \\gamma\_\{ij\}\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\right\|\\ \\leq\\ 2\\Delta\_\{t\},\(23\)

###### Proof\.

\(a\)

𝔼𝜽​\[T​\(Ys\)∣ℱs−1\]=A′​\(ηAs​\(𝜽\)\)\.\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\\\!\\big\[T\(Y\_\{s\}\)\\mid\\mathcal\{F\}\_\{s\-1\}\\big\]\\ =\\ A^\{\\prime\}\(\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}\)\)\.Therefore,

Lt​\(𝜽′\)−Lt​\(𝜽\)\\displaystyle L\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\-L\_\{t\}\(\\bm\{\\theta\}\)=1t∑s=1t\(A\(ηAs\(𝜽′\)\)−A\(ηAs\(𝜽\)\)\\displaystyle=\\frac\{1\}\{t\}\\sum\_\{s=1\}^\{t\}\\Big\(A\(\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}^\{\\prime\}\)\)\-A\(\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}\)\)−\(ηAs\(𝜽′\)−ηAs\(𝜽\)\)A′\(ηAs\(𝜽\)\)\)\\displaystyle\\qquad\-\\big\(\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}^\{\\prime\}\)\-\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}\)\\big\)\\,A^\{\\prime\}\(\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}\)\)\\Big\)=1t​∑s=1tdAs​\(𝜽,𝜽′\)\.\\displaystyle=\\frac\{1\}\{t\}\\sum\_\{s=1\}^\{t\}d\_\{A\_\{s\}\}\(\\bm\{\\theta\},\\bm\{\\theta\}^\{\\prime\}\)\.Sincew^t,i​jemp=Ni​j​\(t\)/t\\hat\{w\}^\{\\mathrm\{emp\}\}\_\{t,ij\}=N\_\{ij\}\(t\)/t,

D𝒘^temp​\(𝜽∥𝜽′\)=∑\(i,j\)w^t,i​jemp​di​j​\(𝜽,𝜽′\)=1t​∑s=1tdAs​\(𝜽,𝜽′\),D\_\{\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)=\\sum\_\{\(i,j\)\}\\hat\{w\}^\{\\mathrm\{emp\}\}\_\{t,ij\}\\,d\_\{ij\}\(\\bm\{\\theta\},\\bm\{\\theta\}^\{\\prime\}\)=\\frac\{1\}\{t\}\\sum\_\{s=1\}^\{t\}d\_\{A\_\{s\}\}\(\\bm\{\\theta\},\\bm\{\\theta\}^\{\\prime\}\),which proves \([22](https://arxiv.org/html/2607.08979#A4.E22)\)\.

##### Part \(b\)\.

By assumption,B^t=B​\(𝜽\)\\hat\{B\}\_\{t\}=B\(\\bm\{\\theta\}\)and for each\(i,j\)∈B​\(𝜽\)\(i,j\)\\in B\(\\bm\{\\theta\}\), the constrained maximization inZi​j​\(t\)Z\_\{ij\}\(t\)is overΘi​j\\Theta\_\{ij\}\. Let

𝜽^i​j,t∈argmax𝜽′∈Θi​jℓt​\(𝜽′\)⇔𝜽^i​j,t∈argmin𝜽′∈Θi​jL¯t​\(𝜽′\),\\hat\{\\bm\{\\theta\}\}\_\{ij,t\}\\in\\mathop\{\\mathrm\{argmax\}\}\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}\}\\ell\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\\Leftrightarrow\\hat\{\\bm\{\\theta\}\}\_\{ij,t\}\\in\\mathop\{\\mathrm\{argmin\}\}\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}\}\\bar\{L\}\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\),and recall𝜽^t∈argmaxΘℓt​\(𝜽′\)⇔𝜽^t∈argminΘL¯t​\(𝜽′\)\\hat\{\\bm\{\\theta\}\}\_\{t\}\\in\\mathop\{\\mathrm\{argmax\}\}\_\{\\Theta\}\\ell\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\\Leftrightarrow\\hat\{\\bm\{\\theta\}\}\_\{t\}\\in\\mathop\{\\mathrm\{argmin\}\}\_\{\\Theta\}\\bar\{L\}\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\. Then

1t​Zi​j​\(t\)=L¯t​\(𝜽^i​j,t\)−L¯t​\(𝜽^t\)=inf𝜽′∈Θi​jL¯t​\(𝜽′\)−inf𝜽′∈ΘL¯t​\(𝜽′\)\.\\frac\{1\}\{t\}Z\_\{ij\}\(t\)=\\bar\{L\}\_\{t\}\(\\hat\{\\bm\{\\theta\}\}\_\{ij,t\}\)\-\\bar\{L\}\_\{t\}\(\\hat\{\\bm\{\\theta\}\}\_\{t\}\)=\\inf\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}\}\\bar\{L\}\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\-\\inf\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\}\\bar\{L\}\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\.On the other hand, by \([22](https://arxiv.org/html/2607.08979#A4.E22)\) and nonnegativity of KL divergence,inf𝜽′∈ΘLt​\(𝜽′\)=Lt​\(𝜽\)\\inf\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\}L\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)=L\_\{t\}\(\\bm\{\\theta\}\), so

inf𝜽′∈Θi​jLt​\(𝜽′\)\\displaystyle\\inf\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}\}L\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)=Lt​\(𝜽\)\+inf𝜽′∈Θi​jD𝒘^temp​\(𝜽∥𝜽′\)\\displaystyle=L\_\{t\}\(\\bm\{\\theta\}\)\+\\inf\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{ij\}\}D\_\{\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\}\}\(\\bm\{\\theta\}\\\|\\bm\{\\theta\}^\{\\prime\}\)=Lt​\(𝜽\)\+γi​j​\(𝒘^temp;𝜽\)\.\\displaystyle=L\_\{t\}\(\\bm\{\\theta\}\)\+\\gamma\_\{ij\}\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\.Thus

γi​j​\(𝒘^temp;𝜽\)=infΘi​jLt−infΘLt\.\\gamma\_\{ij\}\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)=\\inf\_\{\\Theta\_\{ij\}\}L\_\{t\}\-\\inf\_\{\\Theta\}L\_\{t\}\.Using\|inff−infg\|≤sup\|f−g\|\|\\inf f\-\\inf g\|\\leq\\sup\|f\-g\|,

\|1t​Zi​j​\(t\)−γi​j​\(𝒘^temp;𝜽\)\|\\displaystyle\\left\|\\frac\{1\}\{t\}Z\_\{ij\}\(t\)\-\\gamma\_\{ij\}\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\right\|=\|\(infΘi​jL¯t−infΘL¯t\)\\displaystyle=\\Big\|\\big\(\\inf\_\{\\Theta\_\{ij\}\}\\bar\{L\}\_\{t\}\-\\inf\_\{\\Theta\}\\bar\{L\}\_\{t\}\\big\)−\(infΘi​jLt−infΘLt\)\|\\displaystyle\\qquad\-\\big\(\\inf\_\{\\Theta\_\{ij\}\}L\_\{t\}\-\\inf\_\{\\Theta\}L\_\{t\}\\big\)\\Big\|≤\|infΘi​jL¯t−infΘi​jLt\|\\displaystyle\\leq\\left\|\\inf\_\{\\Theta\_\{ij\}\}\\bar\{L\}\_\{t\}\-\\inf\_\{\\Theta\_\{ij\}\}L\_\{t\}\\right\|\+\|infΘL¯t−infΘLt\|≤2​Δt,\\displaystyle\\qquad\+\\left\|\\inf\_\{\\Theta\}\\bar\{L\}\_\{t\}\-\\inf\_\{\\Theta\}L\_\{t\}\\right\|\\leq 2\\Delta\_\{t\},which proves \([23](https://arxiv.org/html/2607.08979#A4.E23)\)\. ∎ Next, we will show thatΔt→0\\Delta\_\{t\}\\to 0almost surely\. Define

𝑽t:=∑s=1t𝒙As​\(T​\(Ys\)−A′​\(ηAs​\(𝜽\)\)\)∈ℝn\.\\bm\{V\}\_\{t\}\\ :=\\ \\sum\_\{s=1\}^\{t\}\\bm\{x\}\_\{A\_\{s\}\}\\Big\(T\(Y\_\{s\}\)\-A^\{\\prime\}\(\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}\)\)\\Big\)\\ \\in\\ \\mathbb\{R\}^\{n\}\.So for every𝜽′∈Θ\\bm\{\\theta\}^\{\\prime\}\\in\\Theta,

L¯t​\(𝜽′\)−Lt​\(𝜽′\)=−1t​𝜽′⁣⊤​𝑽t\.\\bar\{L\}\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\-L\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\\ =\\ \-\\frac\{1\}\{t\}\\,\\bm\{\\theta\}^\{\\prime\\top\}\\bm\{V\}\_\{t\}\.\(24\)
###### Lemma D\.6\.

For everyt≥1t\\geq 1and everyδ∈\(0,1\)\\delta\\in\(0,1\),

ℙ𝜽​\(Δt≥R​σ​n​2​log⁡\(2​n/δ\)t\)≤δ\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\\Delta\_\{t\}\\ \\geq\\ R\\,\\sigma\\,n\\,\\sqrt\{\\frac\{2\\log\(2n/\\delta\)\}\{t\}\}\\right\)\\ \\leq\\ \\delta\.\(25\)

###### Proof\.

By \([24](https://arxiv.org/html/2607.08979#A4.E24)\),

Δt\\displaystyle\\Delta\_\{t\}=sup𝜽′∈Θ\|𝜽′⁣⊤​𝑽t\|t\\displaystyle=\\sup\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\}\\frac\{\|\\bm\{\\theta\}^\{\\prime\\top\}\\bm\{V\}\_\{t\}\|\}\{t\}≤sup‖𝜽′‖∞≤R\|𝜽′⁣⊤​𝑽t\|t=Rt​‖𝑽t‖1≤R​nt​max1≤i≤n⁡\|\(𝑽t\)i\|\.\\displaystyle\\leq\\sup\_\{\\\|\\bm\{\\theta\}^\{\\prime\}\\\|\_\{\\infty\}\\leq R\}\\frac\{\|\\bm\{\\theta\}^\{\\prime\\top\}\\bm\{V\}\_\{t\}\|\}\{t\}=\\frac\{R\}\{t\}\\\|\\bm\{V\}\_\{t\}\\\|\_\{1\}\\leq\\frac\{Rn\}\{t\}\\max\_\{1\\leq i\\leq n\}\|\(\\bm\{V\}\_\{t\}\)\_\{i\}\|\.Fixi∈\[n\]i\\in\[n\]and write\(𝑽t\)i=∑s=1tξs,i\(\\bm\{V\}\_\{t\}\)\_\{i\}=\\sum\_\{s=1\}^\{t\}\\xi\_\{s,i\}where

ξs,i:=\(𝒙As\)i​\(T​\(Ys\)−A′​\(ηAs​\(𝜽\)\)\)\.\\xi\_\{s,i\}:=\(\\bm\{x\}\_\{A\_\{s\}\}\)\_\{i\}\\Big\(T\(Y\_\{s\}\)\-A^\{\\prime\}\(\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}\)\)\\Big\)\.Conditionally onℱs−1\\mathcal\{F\}\_\{s\-1\},\(𝒙As\)i∈\{−1,0,1\}\(\\bm\{x\}\_\{A\_\{s\}\}\)\_\{i\}\\in\\\{\-1,0,1\\\}is fixed, andT​\(Ys\)−A′​\(ηAs​\(𝜽\)\)T\(Y\_\{s\}\)\-A^\{\\prime\}\(\\eta\_\{A\_\{s\}\}\(\\bm\{\\theta\}\)\)is centered andσ2\\sigma^\{2\}\-sub\-Gaussian by Assumption[2](https://arxiv.org/html/2607.08979#Thmassumption2)\. So for allλ∈ℝ\\lambda\\in\\mathbb\{R\},

𝔼𝜽​\[exp⁡\(λ​ξs,i\)∣ℱs−1\]≤exp⁡\(σ2​λ22\)\.\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\\\!\\left\[\\exp\(\\lambda\\xi\_\{s,i\}\)\\mid\\mathcal\{F\}\_\{s\-1\}\\right\]\\leq\\exp\\\!\\left\(\\frac\{\\sigma^\{2\}\\lambda^\{2\}\}\{2\}\\right\)\.Define

Ms,i​\(λ\):=exp⁡\(λ​\(𝑽s\)i−σ2​λ22​s\)\.M\_\{s,i\}\(\\lambda\):=\\exp\\\!\\left\(\\lambda\(\\bm\{V\}\_\{s\}\)\_\{i\}\-\\frac\{\\sigma^\{2\}\\lambda^\{2\}\}\{2\}s\\right\)\.Then\(Ms,i​\(λ\)\)s≥0\(M\_\{s,i\}\(\\lambda\)\)\_\{s\\geq 0\}is a nonnegative supermartingale, so𝔼𝜽​\[Mt,i​\(λ\)\]≤1\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\[M\_\{t,i\}\(\\lambda\)\]\\leq 1\. By Markov’s inequality, for anya\>0a\>0,

ℙ𝜽​\(\(𝑽t\)i≥a\)≤exp⁡\(−λ​a\+σ2​λ22​t\)\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\(\\bm\{V\}\_\{t\}\)\_\{i\}\\geq a\\right\)\\leq\\exp\\\!\\left\(\-\\lambda a\+\\frac\{\\sigma^\{2\}\\lambda^\{2\}\}\{2\}t\\right\)\.Takeλ=a/\(σ2​t\)\\lambda=a/\(\\sigma^\{2\}t\)to get

ℙ𝜽​\(\(𝑽t\)i≥a\)\\displaystyle\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\(\\bm\{V\}\_\{t\}\)\_\{i\}\\geq a\\right\)≤exp⁡\(−a22​σ2​t\),\\displaystyle\\leq\\exp\\\!\\left\(\-\\frac\{a^\{2\}\}\{2\\sigma^\{2\}t\}\\right\),ℙ𝜽​\(\|\(𝑽t\)i\|≥a\)\\displaystyle\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\|\(\\bm\{V\}\_\{t\}\)\_\{i\}\|\\geq a\\right\)≤2​exp⁡\(−a22​σ2​t\)\.\\displaystyle\\leq 2\\exp\\\!\\left\(\-\\frac\{a^\{2\}\}\{2\\sigma^\{2\}t\}\\right\)\.Witha=σ​2​t​log⁡\(2​n/δ\)a=\\sigma\\sqrt\{2t\\log\(2n/\\delta\)\},

ℙ𝜽​\(\|\(𝑽t\)i\|≥σ​2​t​log⁡\(2​n/δ\)\)≤δn\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\|\(\\bm\{V\}\_\{t\}\)\_\{i\}\|\\geq\\sigma\\sqrt\{2t\\log\(2n/\\delta\)\}\\right\)\\leq\\frac\{\\delta\}\{n\}\.A union bound overi=1,…,ni=1,\\ldots,ngives, with probability at least1−δ1\-\\delta,

max1≤i≤n⁡\|\(𝑽t\)i\|≤σ​2​t​log⁡\(2​n/δ\)\.\\max\_\{1\\leq i\\leq n\}\|\(\\bm\{V\}\_\{t\}\)\_\{i\}\|\\leq\\sigma\\sqrt\{2t\\log\(2n/\\delta\)\}\.Substituting intoΔt≤R​nt​maxi⁡\|\(𝑽t\)i\|\\Delta\_\{t\}\\leq\\frac\{Rn\}\{t\}\\max\_\{i\}\|\(\\bm\{V\}\_\{t\}\)\_\{i\}\|yields \([25](https://arxiv.org/html/2607.08979#A4.E25)\)\. ∎

###### Corollary D\.7\.

Under Assumption[2](https://arxiv.org/html/2607.08979#Thmassumption2),we haveΔt→0\\Delta\_\{t\}\\to 0almost surely\.

###### Proof\.

Apply Lemma[D\.6](https://arxiv.org/html/2607.08979#A4.Thmtheorem6)withδt=t−2\\delta\_\{t\}=t^\{\-2\}and use Borel–Cantelli\. ∎

#### D\.2\.1Finite stopping

###### Lemma D\.8\.

Fix𝛉∈Θgap\\bm\{\\theta\}\\in\\Theta^\{\\mathrm\{gap\}\}\. Under the conditions of Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3),

Γ​\(𝒘^temp;𝜽\)⟶Γ∗​\(𝜽\)almost surely\.\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\ \\longrightarrow\\ \\Gamma^\{\*\}\(\\bm\{\\theta\}\)\\qquad\\text\{almost surely\.\}

###### Proof\.

For eacht≥1t\\geq 1, we haveΓ​\(𝒘^temp;𝜽\)≤Γ∗​\(𝜽\)\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\leq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)by definition\. Letεt\\varepsilon\_\{t\}denote the error term subtracted fromΓ∗​\(𝜽\)\\Gamma^\{\*\}\(\\bm\{\\theta\}\)in Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3), soεt→0\\varepsilon\_\{t\}\\to 0ast→∞t\\to\\infty\. Choose anyp\>1p\>1and takettlarge enough that Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)applies\. Then

ℙ𝜽​\(Γ​\(𝒘^temp;𝜽\)≤Γ∗​\(𝜽\)−εt\)≤t−p\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\Big\(\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\ \\leq\\ \\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\varepsilon\_\{t\}\\Big\)\\ \\leq\\ t^\{\-p\}\.Since∑t≥1t−p<∞\\sum\_\{t\\geq 1\}t^\{\-p\}<\\infty, Borel–Cantelli implies that almost surely,

Γ​\(𝒘^temp;𝜽\)≥Γ∗​\(𝜽\)−εtfor all sufficiently large​t\.\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\ \\geq\\ \\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\varepsilon\_\{t\}\\quad\\text\{for all sufficiently large \}t\.
Takinglim inf\\liminfand usingεt→0\\varepsilon\_\{t\}\\to 0yieldslim inft→∞Γ​\(𝒘^temp;𝜽\)≥Γ∗​\(𝜽\)\\liminf\_\{t\\to\\infty\}\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\geq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\. Combining withΓ​\(𝒘^temp;𝜽\)≤Γ∗​\(𝜽\)\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\leq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)gives almost sure convergence\. ∎

###### Proposition D\.9\.

Fix𝛉∈Θgap\\bm\{\\theta\}\\in\\Theta^\{\\mathrm\{gap\}\}andδ∈\(0,1\)\\delta\\in\(0,1\)\. Letτδ\\tau\_\{\\delta\}be the stopping time defined in \([5](https://arxiv.org/html/2607.08979#S5.E5)\)\. Then

ℙ𝜽​\(τδ<∞\)=1\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\tau\_\{\\delta\}<\\infty\)=1\.

###### Proof\.

By Corollary[6\.2](https://arxiv.org/html/2607.08979#S6.Thmtheorem2), almost surely there exists a finite timetstabt\_\{\\mathrm\{stab\}\}such thatS^t=S∗​\(𝜽\)\\hat\{S\}\_\{t\}=S^\{\*\}\(\\bm\{\\theta\}\)andB^t=B​\(𝜽\)\\hat\{B\}\_\{t\}=B\(\\bm\{\\theta\}\)for allt≥tstabt\\geq t\_\{\\mathrm\{stab\}\}; in particular, for allt≥tstabt\\geq t\_\{\\mathrm\{stab\}\}the correctness condition of Lemma[D\.5](https://arxiv.org/html/2607.08979#A4.Thmtheorem5)*\(b\)*holds\.

On this almost sure event, for allt≥tstabt\\geq t\_\{\\mathrm\{stab\}\}, Lemma[D\.5](https://arxiv.org/html/2607.08979#A4.Thmtheorem5)*\(b\)*gives

\|1t​ZS^t​\(t\)−Γ​\(𝒘^temp;𝜽\)\|≤2​Δt\.\\left\|\\frac\{1\}\{t\}Z\_\{\\hat\{S\}\_\{t\}\}\(t\)\-\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\right\|\\ \\leq\\ 2\\Delta\_\{t\}\.By Corollary[D\.7](https://arxiv.org/html/2607.08979#A4.Thmtheorem7),Δt→0\\Delta\_\{t\}\\to 0almost surely, and by Lemma[D\.8](https://arxiv.org/html/2607.08979#A4.Thmtheorem8),Γ​\(𝒘^temp;𝜽\)→Γ∗​\(𝜽\)\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\to\\Gamma^\{\*\}\(\\bm\{\\theta\}\)almost surely\. Thus,

1t​ZS^t​\(t\)⟶Γ∗​\(𝜽\)almost surely\.\\frac\{1\}\{t\}Z\_\{\\hat\{S\}\_\{t\}\}\(t\)\\ \\longrightarrow\\ \\Gamma^\{\*\}\(\\bm\{\\theta\}\)\\qquad\\text\{almost surely\.\}\(26\)
We now compareZS^t​\(t\)Z\_\{\\hat\{S\}\_\{t\}\}\(t\)to the threshold\. By Lemma[D\.4](https://arxiv.org/html/2607.08979#A4.Thmtheorem4), for each fixedδ∈\(0,1\)\\delta\\in\(0,1\)we haveβ​\(t,δ\)≤β¯​\(t,δ\)\\beta\(t,\\delta\)\\leq\\bar\{\\beta\}\(t,\\delta\)for allttandβ¯​\(t,δ\)/t→0\\bar\{\\beta\}\(t,\\delta\)/t\\to 0ast→∞t\\to\\infty\.

Finally, we show thatΓ∗​\(𝜽\)\>0\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\>0\. For each boundary pair\(u,v\)∈B​\(𝜽\)\(u,v\)\\in B\(\\bm\{\\theta\}\)we haveηu​v​\(𝜽\)=θu−θv\>0\\eta\_\{uv\}\(\\bm\{\\theta\}\)=\\theta\_\{u\}\-\\theta\_\{v\}\>0, and any𝜽′∈Θu​v\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{uv\}hasηu​v​\(𝜽′\)≤0\\eta\_\{uv\}\(\\bm\{\\theta\}^\{\\prime\}\)\\leq 0\. By continuity ofdu​v​\(𝜽,𝜽′\)d\_\{uv\}\(\\bm\{\\theta\},\\bm\{\\theta\}^\{\\prime\}\)and compactness ofΘu​v\\Theta\_\{uv\},inf𝜽′∈Θu​vdu​v​\(𝜽,𝜽′\)\>0\\inf\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\_\{uv\}\}d\_\{uv\}\(\\bm\{\\theta\},\\bm\{\\theta\}^\{\\prime\}\)\>0\. Thus,Γ∗​\(𝜽\)\>0\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\>0\.

So almost surely, for all sufficiently largett,ZS^t​\(t\)≥β​\(t,δ\)Z\_\{\\hat\{S\}\_\{t\}\}\(t\)\\geq\\beta\(t,\\delta\)\. Thus,τδ<∞\\tau\_\{\\delta\}<\\inftyalmost surely\. ∎

## Appendix EProof of Theorem[6\.5](https://arxiv.org/html/2607.08979#S6.Thmtheorem5)

###### Proof\.

Fixε∈\(0,1\)\\varepsilon\\in\(0,1\)and define

tδ:=⌈1\+εΓ∗​\(𝜽\)​log⁡1δ⌉,ε′:=ε2​\(1\+ε\)​Γ∗​\(𝜽\)\>0\.t\_\{\\delta\}:=\\left\\lceil\\frac\{1\+\\varepsilon\}\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}\\log\\frac\{1\}\{\\delta\}\\right\\rceil,\\qquad\\varepsilon^\{\\prime\}:=\\frac\{\\varepsilon\}\{2\(1\+\\varepsilon\)\}\\,\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\>0\.By Lemma[D\.4](https://arxiv.org/html/2607.08979#A4.Thmtheorem4),β​\(t,δ\)≤β¯​\(t,δ\)\\beta\(t,\\delta\)\\leq\\bar\{\\beta\}\(t,\\delta\)for allttandt↦β¯​\(t,δ\)/tt\\mapsto\\bar\{\\beta\}\(t,\\delta\)/tis decreasing\. So for allt≥tδt\\geq t\_\{\\delta\},

β​\(t,δ\)t≤β¯​\(t,δ\)t≤β¯​\(tδ,δ\)tδ,\\frac\{\\beta\(t,\\delta\)\}\{t\}\\ \\leq\\ \\frac\{\\bar\{\\beta\}\(t,\\delta\)\}\{t\}\\ \\leq\\ \\frac\{\\bar\{\\beta\}\(t\_\{\\delta\},\\delta\)\}\{t\_\{\\delta\}\},where

β¯​\(tδ,δ\)tδ\\displaystyle\\frac\{\\bar\{\\beta\}\(t\_\{\\delta\},\\delta\)\}\{t\_\{\\delta\}\}=log⁡\(1/δ\)tδ\+1tδ​\(λ2​n​R2\)\\displaystyle=\\frac\{\\log\(1/\\delta\)\}\{t\_\{\\delta\}\}\+\\frac\{1\}\{t\_\{\\delta\}\}\\left\(\\frac\{\\lambda\}\{2\}nR^\{2\}\\right\)\+\(n−1\)​log⁡\(1\+Cβ​tδ\)2​tδ\.\\displaystyle\\qquad\+\\frac\{\(n\-1\)\\log\(1\+C\_\{\\beta\}t\_\{\\delta\}\)\}\{2t\_\{\\delta\}\}\.By definition oftδt\_\{\\delta\},

log⁡\(1/δ\)tδ≤Γ∗​\(𝜽\)1\+ε\.\\frac\{\\log\(1/\\delta\)\}\{t\_\{\\delta\}\}\\ \\leq\\ \\frac\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}\{1\+\\varepsilon\}\.Sincetδ→∞t\_\{\\delta\}\\to\\inftyasδ→0\\delta\\to 0, there existsδ0∈\(0,1\)\\delta\_\{0\}\\in\(0,1\)such that for allδ∈\(0,δ0\)\\delta\\in\(0,\\delta\_\{0\}\),

1tδ​\(λ2​n​R2\)\+n−12⋅log⁡\(1\+Cβ​tδ\)tδ≤ε′\.\\frac\{1\}\{t\_\{\\delta\}\}\\left\(\\frac\{\\lambda\}\{2\}nR^\{2\}\\right\)\+\\frac\{n\-1\}\{2\}\\cdot\\frac\{\\log\(1\+C\_\{\\beta\}t\_\{\\delta\}\)\}\{t\_\{\\delta\}\}\\ \\leq\\ \\varepsilon^\{\\prime\}\.Therefore, for allδ∈\(0,δ0\)\\delta\\in\(0,\\delta\_\{0\}\)and allt≥tδt\\geq t\_\{\\delta\},

β​\(t,δ\)t≤Γ∗​\(𝜽\)1\+ε\+ε′=Γ∗​\(𝜽\)−ε′\.\\frac\{\\beta\(t,\\delta\)\}\{t\}\\ \\leq\\ \\frac\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}\{1\+\\varepsilon\}\+\\varepsilon^\{\\prime\}\\ =\\ \\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\varepsilon^\{\\prime\}\.\(27\)
Next, fixp\>1p\>1and setδt:=t−\(p\+2\)\\delta\_\{t\}:=t^\{\-\(p\+2\)\}\. Takeδ∈\(0,δ0\)\\delta\\in\(0,\\delta\_\{0\}\)andt≥tδt\\geq t\_\{\\delta\}\. Sinceτδ\>t\\tau\_\{\\delta\}\>timplies the stopping condition fails at timett,

ℙ𝜽​\(τδ\>t\)\\displaystyle\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\tau\_\{\\delta\}\>t\)≤ℙ𝜽​\(ZS^t​\(t\)<β​\(t,δ\)\)\\displaystyle\\leq\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(Z\_\{\\hat\{S\}\_\{t\}\}\(t\)<\\beta\(t,\\delta\)\\right\)=ℙ𝜽​\(1t​ZS^t​\(t\)≤β​\(t,δ\)t\)\.\\displaystyle=\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\\frac\{1\}\{t\}Z\_\{\\hat\{S\}\_\{t\}\}\(t\)\\leq\\frac\{\\beta\(t,\\delta\)\}\{t\}\\right\)\.By \([27](https://arxiv.org/html/2607.08979#A5.E27)\), for allt≥tδt\\geq t\_\{\\delta\}we haveβ​\(t,δ\)/t≤Γ∗​\(𝜽\)−ε′\\beta\(t,\\delta\)/t\\leq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\varepsilon^\{\\prime\}, hence

ℙ𝜽​\(τδ\>t\)≤ℙ𝜽​\(1t​ZS^t​\(t\)≤Γ∗​\(𝜽\)−ε′\)\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\tau\_\{\\delta\}\>t\)\\leq\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\\frac\{1\}\{t\}Z\_\{\\hat\{S\}\_\{t\}\}\(t\)\\leq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\varepsilon^\{\\prime\}\\right\)\.\(28\)
LetΔk:=θ\(k\)−θ\(k\+1\)\>0\\Delta\_\{k\}:=\\theta\_\{\(k\)\}\-\\theta\_\{\(k\+1\)\}\>0\. Choose timestbdry,tΔ,tΓ<∞t\_\{\\mathrm\{bdry\}\},t\_\{\\Delta\},t\_\{\\Gamma\}<\\inftysuch that for allttlarge enough the following hold:

*\(i\) Boundary correctness\.*Apply Lemma[6\.1](https://arxiv.org/html/2607.08979#S6.Thmtheorem1)at timettwith confidenceδt\\delta\_\{t\}\. Using Lemma[F\.1](https://arxiv.org/html/2607.08979#A6.Thmtheorem1)*\(e\)*, the deviation bound tends to0ast→∞t\\to\\infty, so there existstbdryt\_\{\\mathrm\{bdry\}\}such that for allt≥tbdryt\\geq t\_\{\\mathrm\{bdry\}\},

ℙ𝜽​\(S^t≠S∗​\(𝜽\)\)≤δt\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\big\(\\hat\{S\}\_\{t\}\\neq S^\{\*\}\(\\bm\{\\theta\}\)\\big\)\\ \\leq\\ \\delta\_\{t\}\.
*\(ii\) One\-time control ofΔt\\Delta\_\{t\}\.*By Lemma[D\.6](https://arxiv.org/html/2607.08979#A4.Thmtheorem6)with confidenceδt\\delta\_\{t\}, and sinceR​σ​n​2​log⁡\(2​n/δt\)/t=o​\(1\)R\\sigma n\\sqrt\{2\\log\(2n/\\delta\_\{t\}\)/t\}=o\(1\), there existstΔt\_\{\\Delta\}such that for allt≥tΔt\\geq t\_\{\\Delta\},

ℙ𝜽​\(Δt≥ε′4\)≤δt\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\\Delta\_\{t\}\\geq\\frac\{\\varepsilon^\{\\prime\}\}\{4\}\\right\)\\ \\leq\\ \\delta\_\{t\}\.
*\(iii\) One\-time lower bound forΓ​\(𝐰^temp;𝛉\)\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\.*Apply Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)with exponentp\+2p\+2\. ChoosetΓt\_\{\\Gamma\}such that the errorerrt≤ε′/2\\mathrm\{err\}\_\{t\}\\leq\\varepsilon^\{\\prime\}/2for allt≥tΓt\\geq t\_\{\\Gamma\}\. Then for allt≥tΓt\\geq t\_\{\\Gamma\},

ℙ𝜽​\(Γ​\(𝒘^temp;𝜽\)≤Γ∗​\(𝜽\)−ε′2\)≤δt\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\leq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\frac\{\\varepsilon^\{\\prime\}\}\{2\}\\right\)\\ \\leq\\ \\delta\_\{t\}\.Sett∗:=max⁡\{tbdry,tΔ,tΓ\}t\_\{\*\}:=\\max\\\{t\_\{\\mathrm\{bdry\}\},t\_\{\\Delta\},t\_\{\\Gamma\}\\\}and fixt≥t∗t\\geq t\_\{\*\}\. On the event

Et:=\\displaystyle E\_\{t\}:=\{\}\{S^t=S∗​\(𝜽\)\}\\displaystyle\\Big\\\{\\hat\{S\}\_\{t\}=S^\{\*\}\(\\bm\{\\theta\}\)\\Big\\\}∩\{Δt≤ε′/4\}∩\{Γ​\(𝒘^temp;𝜽\)≥Γ∗​\(𝜽\)−ε′/2\},\\displaystyle\\cap\\Big\\\{\\Delta\_\{t\}\\leq\\varepsilon^\{\\prime\}/4\\Big\\\}\\cap\\Big\\\{\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\geq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\varepsilon^\{\\prime\}/2\\Big\\\},Lemma[D\.5](https://arxiv.org/html/2607.08979#A4.Thmtheorem5)*\(b\)*applies at timett, so

1t​ZS^t​\(t\)\\displaystyle\\frac\{1\}\{t\}Z\_\{\\hat\{S\}\_\{t\}\}\(t\)≥Γ​\(𝒘^temp;𝜽\)−2​Δt\\displaystyle\\geq\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\-2\\Delta\_\{t\}≥\(Γ∗​\(𝜽\)−ε′2\)−2⋅ε′4\\displaystyle\\geq\\Big\(\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\frac\{\\varepsilon^\{\\prime\}\}\{2\}\\Big\)\-2\\cdot\\frac\{\\varepsilon^\{\\prime\}\}\{4\}=Γ∗​\(𝜽\)−ε′\.\\displaystyle=\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\varepsilon^\{\\prime\}\.Therefore, for allt≥t∗t\\geq t\_\{\*\},

ℙ𝜽​\(1t​ZS^t​\(t\)≤Γ∗​\(𝜽\)−ε′\)\\displaystyle\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\\frac\{1\}\{t\}Z\_\{\\hat\{S\}\_\{t\}\}\(t\)\\leq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\varepsilon^\{\\prime\}\\right\)≤ℙ𝜽​\(Etc\)\\displaystyle\\leq\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(E\_\{t\}^\{c\}\)≤ℙ𝜽​\(S^t≠S∗​\(𝜽\)\)\+ℙ𝜽​\(Δt≥ε′4\)\\displaystyle\\leq\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\big\(\\hat\{S\}\_\{t\}\\neq S^\{\*\}\(\\bm\{\\theta\}\)\\big\)\+\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\\Delta\_\{t\}\\geq\\frac\{\\varepsilon^\{\\prime\}\}\{4\}\\right\)\+ℙ𝜽​\(Γ​\(𝒘^temp;𝜽\)≤Γ∗​\(𝜽\)−ε′2\)\\displaystyle\\quad\+\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\\!\\left\(\\Gamma\(\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\};\\bm\{\\theta\}\)\\leq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\frac\{\\varepsilon^\{\\prime\}\}\{2\}\\right\)≤3​δt=3​t−\(p\+2\)\.\\displaystyle\\leq 3\\delta\_\{t\}=3t^\{\-\(p\+2\)\}\.Combining with \([28](https://arxiv.org/html/2607.08979#A5.E28)\), we obtain that for allδ∈\(0,δ0\)\\delta\\in\(0,\\delta\_\{0\}\)and allt≥max⁡\{tδ,t∗\}t\\geq\\max\\\{t\_\{\\delta\},t\_\{\*\}\\\},

ℙ𝜽​\(τδ\>t\)≤3​t−\(p\+2\)\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\tau\_\{\\delta\}\>t\)\\ \\leq\\ 3t^\{\-\(p\+2\)\}\.DefineTδ:=max⁡\{tδ,t∗\}\.T\_\{\\delta\}:=\\max\\\{t\_\{\\delta\},t\_\{\*\}\\\}\.

𝔼𝜽​\[τδ\]=∑t=0∞ℙ𝜽​\(τδ\>t\)\\displaystyle\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\[\\tau\_\{\\delta\}\]=\\sum\_\{t=0\}^\{\\infty\}\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\tau\_\{\\delta\}\>t\)≤Tδ\+∑t=Tδ∞ℙ𝜽​\(τδ\>t\)\\displaystyle\\leq T\_\{\\delta\}\+\\sum\_\{t=T\_\{\\delta\}\}^\{\\infty\}\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\tau\_\{\\delta\}\>t\)≤Tδ\+3​∑t=Tδ∞t−\(p\+2\)\\displaystyle\\leq T\_\{\\delta\}\+3\\sum\_\{t=T\_\{\\delta\}\}^\{\\infty\}t^\{\-\(p\+2\)\}≤Tδ\+3p\+1​\(Tδ−1\)−\(p\+1\)\.\\displaystyle\\leq T\_\{\\delta\}\+\\frac\{3\}\{p\+1\}\(T\_\{\\delta\}\-1\)^\{\-\(p\+1\)\}\.
Divide bylog⁡\(1/δ\)\\log\(1/\\delta\)and takelim supδ→0\\limsup\_\{\\delta\\to 0\}\. Sincetδ→∞t\_\{\\delta\}\\to\\inftyasδ→0\\delta\\to 0andt∗t\_\{\*\}is fixed,Tδ=tδT\_\{\\delta\}=t\_\{\\delta\}for all sufficiently smallδ\\delta, and the second term goes to 0 after division bylog⁡\(1/δ\)\\log\(1/\\delta\)\. Therefore,

lim supδ→0𝔼𝜽​\[τδ\]log⁡\(1/δ\)≤lim supδ→0tδlog⁡\(1/δ\)=1\+εΓ∗​\(𝜽\)\.\\limsup\_\{\\delta\\to 0\}\\frac\{\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\[\\tau\_\{\\delta\}\]\}\{\\log\(1/\\delta\)\}\\leq\\limsup\_\{\\delta\\to 0\}\\frac\{t\_\{\\delta\}\}\{\\log\(1/\\delta\)\}=\\frac\{1\+\\varepsilon\}\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}\.Finally, letε↓0\\varepsilon\\downarrow 0\.

∎

## Appendix FAuxiliary Lemmas

### F\.1C\-tracking

##### C\-tracking lemma \(Garivier–Kaufmann\)\.

Let𝒑​\(1\),…,𝒑​\(T\)∈Δ𝒫\\bm\{p\}\(1\),\\ldots,\\bm\{p\}\(T\)\\in\\Delta\_\{\\mathcal\{P\}\}\. DefineΛ​\(k\):=∑s=1k𝒑​\(s\)\\Lambda\(k\):=\\sum\_\{s=1\}^\{k\}\\bm\{p\}\(s\)andN​\(0\)=0N\(0\)=0\. If for eachk∈\{0,…,T−1\}k\\in\\\{0,\\ldots,T\-1\\\},

Ik\+1∈argmax\(i,j\)∈𝒫\(Λi​j​\(k\+1\)−Ni​j​\(k\)\),\\displaystyle I\_\{k\+1\}\\in\\mathop\{\\mathrm\{argmax\}\}\_\{\(i,j\)\\in\\mathcal\{P\}\}\\bigl\(\\Lambda\_\{ij\}\(k\+1\)\-N\_\{ij\}\(k\)\\bigr\),N​\(k\+1\):=N​\(k\)\+δIk\+1,\\displaystyle N\(k\+1\):=N\(k\)\+\\delta\_\{I\_\{k\+1\}\},then

max\(i,j\)∈𝒫⁡\|Ni​j​\(T\)−Λi​j​\(T\)\|≤\|𝒫\|−1\.\\max\_\{\(i,j\)\\in\\mathcal\{P\}\}\\bigl\|N\_\{ij\}\(T\)\-\\Lambda\_\{ij\}\(T\)\\bigr\|\\leq\|\\mathcal\{P\}\|\-1\.
###### Lemma F\.1\.

Run C\-tracking with targets𝐩​\(t\)=𝐰~t\\bm\{p\}\(t\)=\\tilde\{\\bm\{w\}\}\_\{t\}\. Fort≥1t\\geq 1, define

𝒘~¯t:=1t​∑s=1t𝒘~s,𝒘¯t:=1t​∑s=1t𝒘s\.\\bar\{\\tilde\{\\bm\{w\}\}\}\_\{t\}:=\\frac\{1\}\{t\}\\sum\_\{s=1\}^\{t\}\\tilde\{\\bm\{w\}\}\_\{s\},\\qquad\\bar\{\\bm\{w\}\}\_\{t\}:=\\frac\{1\}\{t\}\\sum\_\{s=1\}^\{t\}\\bm\{w\}\_\{s\}\.Then:

1. \(a\)max\(i,j\)∈𝒫⁡\|Ni​j​\(t\)−Pi​j​\(t\)\|≤\|𝒫\|−1\.\\displaystyle\\max\_\{\(i,j\)\\in\\mathcal\{P\}\}\\big\|N\_\{ij\}\(t\)\-P\_\{ij\}\(t\)\\big\|\\leq\|\\mathcal\{P\}\|\-1\.
2. \(b\)‖𝒘^temp−𝒘~¯t‖∞≤\|𝒫\|−1t\.\\displaystyle\\bigl\\\|\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\}\-\\bar\{\\tilde\{\\bm\{w\}\}\}\_\{t\}\\bigr\\\|\_\{\\infty\}\\leq\\frac\{\|\\mathcal\{P\}\|\-1\}\{t\}\.
3. \(c\)‖𝒘^temp−𝒘¯t‖∞≤\|𝒫\|−1t\+1t​∑s=1tρs\.\\displaystyle\\bigl\\\|\\hat\{\\bm\{w\}\}\_\{t\}^\{\\mathrm\{emp\}\}\-\\bar\{\\bm\{w\}\}\_\{t\}\\bigr\\\|\_\{\\infty\}\\leq\\frac\{\|\\mathcal\{P\}\|\-1\}\{t\}\+\\frac\{1\}\{t\}\\sum\_\{s=1\}^\{t\}\\rho\_\{s\}\.
4. \(d\)for every\(i,j\)∈𝒫\(i,j\)\\in\\mathcal\{P\},Ni​j​\(t\)≥1\|𝒫\|​∑s=1tρs−\(\|𝒫\|−1\)N\_\{ij\}\(t\)\\ \\geq\\ \\frac\{1\}\{\|\\mathcal\{P\}\|\}\\sum\_\{s=1\}^\{t\}\\rho\_\{s\}\\ \-\\ \(\|\\mathcal\{P\}\|\-1\)
5. \(e\)Ifρt=t−γ\\rho\_\{t\}=t^\{\-\\gamma\}for someγ∈\(0,1\)\\gamma\\in\(0,1\), then for allt≥1t\\geq 1, Nmin​\(t\)≥1\|𝒫\|⋅\(t\+1\)1−γ−11−γ−\(\|𝒫\|−1\)\.N\_\{\\min\}\(t\)\\ \\geq\\ \\frac\{1\}\{\|\\mathcal\{P\}\|\}\\cdot\\frac\{\(t\+1\)^\{1\-\\gamma\}\-1\}\{1\-\\gamma\}\\ \-\\ \(\|\\mathcal\{P\}\|\-1\)\.In particular,Nmin​\(t\)/log⁡t→∞N\_\{\\min\}\(t\)/\\log t\\to\\infty\.

###### Proof\.

\(a\) Apply the Garivier–Kaufmann C\-tracking lemma stated above withT=tT=tand𝒑​\(s\)=𝒘~s\\bm\{p\}\(s\)=\\tilde\{\\bm\{w\}\}\_\{s\}\. ThenPi​j​\(t\)=Λi​j​\(t\)P\_\{ij\}\(t\)=\\Lambda\_\{ij\}\(t\)andNi​j​\(t\)=Ni​j​\(t\)N\_\{ij\}\(t\)=N\_\{ij\}\(t\), somax\(i,j\)⁡\|Ni​j​\(t\)−Pi​j​\(t\)\|≤\|𝒫\|−1\\max\_\{\(i,j\)\}\|N\_\{ij\}\(t\)\-P\_\{ij\}\(t\)\|\\leq\|\\mathcal\{P\}\|\-1\.

\(b\) Divide \(a\) byttand usew~¯t,i​j=Pi​j​\(t\)/t\\bar\{\\tilde\{w\}\}\_\{t,ij\}=P\_\{ij\}\(t\)/t\.

\(c\)

𝒘~¯t−𝒘¯t=1t​∑s=1tρs​\(𝒖−𝒘s\),\\bar\{\\tilde\{\\bm\{w\}\}\}\_\{t\}\-\\bar\{\\bm\{w\}\}\_\{t\}=\\frac\{1\}\{t\}\\sum\_\{s=1\}^\{t\}\\rho\_\{s\}\\,\(\\bm\{u\}\-\\bm\{w\}\_\{s\}\),and since𝒖,𝒘s∈Δ𝒫\\bm\{u\},\\bm\{w\}\_\{s\}\\in\\Delta\_\{\\mathcal\{P\}\}we have‖𝒖−𝒘s‖∞≤1\\\|\\bm\{u\}\-\\bm\{w\}\_\{s\}\\\|\_\{\\infty\}\\leq 1, hence‖𝒘~¯t−𝒘¯t‖∞≤1t​∑s=1tρs\\\|\\bar\{\\tilde\{\\bm\{w\}\}\}\_\{t\}\-\\bar\{\\bm\{w\}\}\_\{t\}\\\|\_\{\\infty\}\\leq\\frac\{1\}\{t\}\\sum\_\{s=1\}^\{t\}\\rho\_\{s\}\. Combine with \(b\) and the triangle inequality\.

\(d\) From \(a\),Ni​j​\(t\)≥Pi​j​\(t\)−\(\|𝒫\|−1\)N\_\{ij\}\(t\)\\geq P\_\{ij\}\(t\)\-\(\|\\mathcal\{P\}\|\-1\)\. Alsow~s,i​j=\(1−ρs\)​ws,i​j\+ρs​ui​j≥ρs​ui​j=ρs/\|𝒫\|\\tilde\{w\}\_\{s,ij\}=\(1\-\\rho\_\{s\}\)w\_\{s,ij\}\+\\rho\_\{s\}u\_\{ij\}\\geq\\rho\_\{s\}u\_\{ij\}=\\rho\_\{s\}/\|\\mathcal\{P\}\|, hencePi​j​\(t\)=∑s=1tw~s,i​j≥1\|𝒫\|​∑s=1tρsP\_\{ij\}\(t\)=\\sum\_\{s=1\}^\{t\}\\tilde\{w\}\_\{s,ij\}\\geq\\frac\{1\}\{\|\\mathcal\{P\}\|\}\\sum\_\{s=1\}^\{t\}\\rho\_\{s\}\.

\(e\) Follows from \(d\) and Fact[F\.2](https://arxiv.org/html/2607.08979#A6.Thmtheorem2)\. ∎

###### Fact F\.2\.

Fixr∈\(0,1\)r\\in\(0,1\)\. For every integert≥1t\\geq 1,

\(t\+1\)1−r−11−r≤∑s=1ts−r≤1\+t1−r−11−r\.\\frac\{\(t\+1\)^\{1\-r\}\-1\}\{1\-r\}\\ \\leq\\ \\sum\_\{s=1\}^\{t\}s^\{\-r\}\\ \\leq\\ 1\+\\frac\{t^\{1\-r\}\-1\}\{1\-r\}\.And, for any integers1≤a≤b1\\leq a\\leq b,

∑s=abs−r≤1\+b1−r−\(a−1\)1−r1−r≤1\+b1−r1−r\.\\sum\_\{s=a\}^\{b\}s^\{\-r\}\\ \\leq\\ 1\+\\frac\{b^\{1\-r\}\-\(a\-1\)^\{1\-r\}\}\{1\-r\}\\ \\leq\\ 1\+\\frac\{b^\{1\-r\}\}\{1\-r\}\.

### F\.2Supporting Lemmas for the Proof of Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)

###### Lemma F\.3\.

Define

Dmax:=max\(i,j\)∈𝒫​supϑ,ϑ′∈Θdi​j​\(ϑ,ϑ′\),L:=4​R​σ¯2​2\.D\_\{\\max\}:=\\max\_\{\(i,j\)\\in\\mathcal\{P\}\}\\ \\sup\_\{\\bm\{\\vartheta\},\\bm\{\\vartheta\}^\{\\prime\}\\in\\Theta\}d\_\{ij\}\(\\bm\{\\vartheta\},\\bm\{\\vartheta\}^\{\\prime\}\),\\quad L:=4R\\,\\overline\{\\sigma\}^\{2\}\\sqrt\{2\}\.Then for any nonemptyB⊆𝒫B\\subseteq\\mathcal\{P\}, any𝐰∈Δ𝒫\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}, any𝐪∈ΔB\\bm\{q\}\\in\\Delta\_\{B\}, and anyϑ∈Θ\\bm\{\\vartheta\}\\in\\Theta,

0≤γi​j​\(𝒘;ϑ\)≤Dmax,0≤Fϑ​\(𝒘,𝒒\)≤Dmax\.0\\leq\\gamma\_\{ij\}\(\\bm\{w\};\\bm\{\\vartheta\}\)\\leq D\_\{\\max\},\\qquad 0\\leq F\_\{\\bm\{\\vartheta\}\}\(\\bm\{w\},\\bm\{q\}\)\\leq D\_\{\\max\}\.Additionally, for allϑ,ϑ′∈Θ\\bm\{\\vartheta\},\\bm\{\\vartheta\}^\{\\prime\}\\in\\Theta,

sup𝒘∈Δ𝒫,𝒒∈ΔB\|Fϑ​\(𝒘,𝒒\)−Fϑ′​\(𝒘,𝒒\)\|≤L​‖ϑ−ϑ′‖2\.\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\},\\ \\bm\{q\}\\in\\Delta\_\{B\}\}\\big\|F\_\{\\bm\{\\vartheta\}\}\(\\bm\{w\},\\bm\{q\}\)\-F\_\{\\bm\{\\vartheta\}^\{\\prime\}\}\(\\bm\{w\},\\bm\{q\}\)\\big\|\\leq L\\,\\\|\\bm\{\\vartheta\}\-\\bm\{\\vartheta\}^\{\\prime\}\\\|\_\{2\}\.

###### Proof\.

Fixϑ,ϑ′∈Θ\\bm\{\\vartheta\},\\bm\{\\vartheta\}^\{\\prime\}\\in\\Theta,\(a,b\)∈𝒫\(a,b\)\\in\\mathcal\{P\}, and𝜽′′∈Θ\\bm\{\\theta\}^\{\\prime\\prime\}\\in\\Theta, and writeη=ηa​b​\(ϑ\)\\eta=\\eta\_\{ab\}\(\\bm\{\\vartheta\}\),η′=ηa​b​\(ϑ′\)\\eta^\{\\prime\}=\\eta\_\{ab\}\(\\bm\{\\vartheta\}^\{\\prime\}\), andν=ηa​b​\(𝜽′′\)\\nu=\\eta\_\{ab\}\(\\bm\{\\theta\}^\{\\prime\\prime\}\)\. Defineg​\(u\):=d​\(u,ν\)g\(u\):=d\(u,\\nu\), whered​\(u,ν\)=A​\(ν\)−A​\(u\)−A′​\(u\)​\(ν−u\)d\(u,\\nu\)=A\(\\nu\)\-A\(u\)\-A^\{\\prime\}\(u\)\(\\nu\-u\)\. Theng′​\(u\)=A′′​\(u\)​\(u−ν\)g^\{\\prime\}\(u\)=A^\{\\prime\\prime\}\(u\)\(u\-\\nu\)\. Since\|u\|,\|ν\|≤2​R\|u\|,\|\\nu\|\\leq 2Rforu=ηa​b​\(⋅\)u=\\eta\_\{ab\}\(\\cdot\)onΘ\\Theta, we have\|u−ν\|≤4​R\|u\-\\nu\|\\leq 4Rand thus\|g′​\(u\)\|≤4​R​σ¯2\|g^\{\\prime\}\(u\)\|\\leq 4R\\,\\overline\{\\sigma\}^\{2\}for all\|u\|≤2​R\|u\|\\leq 2R\. By the mean value theorem,

\|da​b​\(ϑ,𝜽′′\)−da​b​\(ϑ′,𝜽′′\)\|\\displaystyle\|d\_\{ab\}\(\\bm\{\\vartheta\},\\bm\{\\theta\}^\{\\prime\\prime\}\)\-d\_\{ab\}\(\\bm\{\\vartheta\}^\{\\prime\},\\bm\{\\theta\}^\{\\prime\\prime\}\)\|=\|d​\(η,ν\)−d​\(η′,ν\)\|\\displaystyle=\|d\(\\eta,\\nu\)\-d\(\\eta^\{\\prime\},\\nu\)\|≤4​R​σ¯2​\|η−η′\|\\displaystyle\\leq 4R\\,\\overline\{\\sigma\}^\{2\}\\,\|\\eta\-\\eta^\{\\prime\}\|≤4​R​σ¯2​‖𝒙a​b‖2​‖ϑ−ϑ′‖2\\displaystyle\\leq 4R\\,\\overline\{\\sigma\}^\{2\}\\,\\\|\\bm\{x\}\_\{ab\}\\\|\_\{2\}\\,\\\|\\bm\{\\vartheta\}\-\\bm\{\\vartheta\}^\{\\prime\}\\\|\_\{2\}=4​R​σ¯2​2​‖ϑ−ϑ′‖2\.\\displaystyle=4R\\,\\overline\{\\sigma\}^\{2\}\\sqrt\{2\}\\,\\\|\\bm\{\\vartheta\}\-\\bm\{\\vartheta\}^\{\\prime\}\\\|\_\{2\}\.Averaging over𝒘\\bm\{w\}and using\|inff−infg\|≤sup\|f−g\|\|\\inf f\-\\inf g\|\\leq\\sup\|f\-g\|yields, for each\(i,j\)∈B\(i,j\)\\in B,

sup𝒘∈Δ𝒫\|γi​j​\(𝒘;ϑ\)−γi​j​\(𝒘;ϑ′\)\|≤4​R​σ¯2​2​‖ϑ−ϑ′‖2\.\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\|\\gamma\_\{ij\}\(\\bm\{w\};\\bm\{\\vartheta\}\)\-\\gamma\_\{ij\}\(\\bm\{w\};\\bm\{\\vartheta\}^\{\\prime\}\)\|\\leq 4R\\,\\overline\{\\sigma\}^\{2\}\\sqrt\{2\}\\,\\\|\\bm\{\\vartheta\}\-\\bm\{\\vartheta\}^\{\\prime\}\\\|\_\{2\}\.SinceFϑ​\(𝒘,𝒒\)=∑\(i,j\)∈Bqi​j​γi​j​\(𝒘;ϑ\)F\_\{\\bm\{\\vartheta\}\}\(\\bm\{w\},\\bm\{q\}\)=\\sum\_\{\(i,j\)\\in B\}q\_\{ij\}\\gamma\_\{ij\}\(\\bm\{w\};\\bm\{\\vartheta\}\)with𝒒∈ΔB\\bm\{q\}\\in\\Delta\_\{B\}, we obtain

sup𝒘,𝒒\|Fϑ​\(𝒘,𝒒\)−Fϑ′​\(𝒘,𝒒\)\|≤4​R​σ¯2​2​‖ϑ−ϑ′‖2\.\\sup\_\{\\bm\{w\},\\bm\{q\}\}\\big\|F\_\{\\bm\{\\vartheta\}\}\(\\bm\{w\},\\bm\{q\}\)\-F\_\{\\bm\{\\vartheta\}^\{\\prime\}\}\(\\bm\{w\},\\bm\{q\}\)\\big\|\\leq 4R\\,\\overline\{\\sigma\}^\{2\}\\sqrt\{2\}\\,\\\|\\bm\{\\vartheta\}\-\\bm\{\\vartheta\}^\{\\prime\}\\\|\_\{2\}\.Finally,0≤di​j​\(ϑ,ϑ′\)≤Dmax0\\leq d\_\{ij\}\(\\bm\{\\vartheta\},\\bm\{\\vartheta\}^\{\\prime\}\)\\leq D\_\{\\max\}for all\(i,j\),ϑ,ϑ′\(i,j\),\\bm\{\\vartheta\},\\bm\{\\vartheta\}^\{\\prime\}by definition, hence0≤D𝒘​\(ϑ∥ϑ′\)≤Dmax0\\leq D\_\{\\bm\{w\}\}\(\\bm\{\\vartheta\}\\\|\\bm\{\\vartheta\}^\{\\prime\}\)\\leq D\_\{\\max\}and therefore0≤γi​j​\(𝒘;ϑ\)≤Dmax0\\leq\\gamma\_\{ij\}\(\\bm\{w\};\\bm\{\\vartheta\}\)\\leq D\_\{\\max\}and0≤Fϑ​\(𝒘,𝒒\)≤Dmax0\\leq F\_\{\\bm\{\\vartheta\}\}\(\\bm\{w\},\\bm\{q\}\)\\leq D\_\{\\max\}\. ∎

Lemma[F\.4](https://arxiv.org/html/2607.08979#A6.Thmtheorem4)is a known result with slight modification for our setting; see, e\.g\.,Shalev\-Shwartz \([2012](https://arxiv.org/html/2607.08979#bib.bib34)\); Hazan \([2016](https://arxiv.org/html/2607.08979#bib.bib16)\); Arora et al\. \([2012](https://arxiv.org/html/2607.08979#bib.bib2)\)\. We provide the full proof here\.

###### Lemma F\.4\(entropic\-FTRL bound\)\.

FixT≥1T\\geq 1and definebT:=⌈T1/4⌉b\_\{T\}:=\\lceil T^\{1/4\}\\rceil\. Let\(𝐠^t\)t=1T\(\\hat\{\\bm\{g\}\}\_\{t\}\)\_\{t=1\}^\{T\}be any sequence inℝ𝒫\\mathbb\{R\}^\{\\mathcal\{P\}\}with‖𝐠^t‖∞≤G\\\|\\hat\{\\bm\{g\}\}\_\{t\}\\\|\_\{\\infty\}\\leq Gfor alltt\. Let\(μt\)t≥1\(\\mu\_\{t\}\)\_\{t\\geq 1\}be nonincreasing\. Define cumulative scores𝚿0=𝟎\\bm\{\\Psi\}\_\{0\}=\\mathbf\{0\}and𝚿t=𝚿t−1\+𝐠^t\\bm\{\\Psi\}\_\{t\}=\\bm\{\\Psi\}\_\{t\-1\}\+\\hat\{\\bm\{g\}\}\_\{t\}\. Define the exponential\-weights / FTRL updates

𝒘t=𝒘1⊙exp⁡\(μt​𝚿t−1\)⟨𝒘1,exp⁡\(μt​𝚿t−1\)⟩,t≥1,\\bm\{w\}\_\{t\}=\\frac\{\\bm\{w\}\_\{1\}\\odot\\exp\(\\mu\_\{t\}\\bm\{\\Psi\}\_\{t\-1\}\)\}\{\\langle\\bm\{w\}\_\{1\},\\exp\(\\mu\_\{t\}\\bm\{\\Psi\}\_\{t\-1\}\)\\rangle\},\\qquad t\\geq 1,where𝐰1∈Δ𝒫\\bm\{w\}\_\{1\}\\in\\Delta\_\{\\mathcal\{P\}\}has full support\. Then

sup𝒘∈Δ𝒫∑t=bTT⟨𝒈^t,𝒘−𝒘t⟩≤D𝒘​\(1\)μT\+G22​∑t=1Tμt\+2​G​\(bT−1\),\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\langle\\hat\{\\bm\{g\}\}\_\{t\},\\bm\{w\}\-\\bm\{w\}\_\{t\}\\rangle\\leq\\frac\{D\_\{\\bm\{w\}\}\(1\)\}\{\\mu\_\{T\}\}\+\\frac\{G^\{2\}\}\{2\}\\sum\_\{t=1\}^\{T\}\\mu\_\{t\}\+2G\(b\_\{T\}\-1\),

###### Proof\.

Fort≥0t\\geq 0andη\>0\\eta\>0define

Zt​\(η\):=∑\(a,b\)∈𝒫w1,a​b​exp⁡\(η​Ψt,a​b\),Φt:=1μt​log⁡Zt​\(μt\)\.Z\_\{t\}\(\\eta\):=\\sum\_\{\(a,b\)\\in\\mathcal\{P\}\}w\_\{1,ab\}\\exp\\\!\\big\(\\eta\\Psi\_\{t,ab\}\\big\),\\quad\\Phi\_\{t\}:=\\frac\{1\}\{\\mu\_\{t\}\}\\log Z\_\{t\}\(\\mu\_\{t\}\)\.We first show that for eacht∈\{1,…,T\}t\\in\\\{1,\\ldots,T\\\},

Φt−Φt−1≤⟨𝒘t,𝒈^t⟩\+μt​G22\.\\Phi\_\{t\}\-\\Phi\_\{t\-1\}\\leq\\langle\\bm\{w\}\_\{t\},\\hat\{\\bm\{g\}\}\_\{t\}\\rangle\+\\frac\{\\mu\_\{t\}G^\{2\}\}\{2\}\.Since\(μt\)\(\\mu\_\{t\}\)is nonincreasing,μt≤μt−1\\mu\_\{t\}\\leq\\mu\_\{t\-1\}andx↦xμt/μt−1x\\mapsto x^\{\\mu\_\{t\}/\\mu\_\{t\-1\}\}is concave onℝ\+\\mathbb\{R\}\_\{\+\}, so Jensen gives

Zt−1​\(μt\)\\displaystyle Z\_\{t\-1\}\(\\mu\_\{t\}\)=∑\(a,b\)w1,a​b​\(exp⁡\(μt−1​Ψt−1,a​b\)\)μt/μt−1\\displaystyle=\\sum\_\{\(a,b\)\}w\_\{1,ab\}\\Big\(\\exp\(\\mu\_\{t\-1\}\\Psi\_\{t\-1,ab\}\)\\Big\)^\{\\mu\_\{t\}/\\mu\_\{t\-1\}\}≤Zt−1​\(μt−1\)μt/μt−1\.\\displaystyle\\leq Z\_\{t\-1\}\(\\mu\_\{t\-1\}\)^\{\\mu\_\{t\}/\\mu\_\{t\-1\}\}\.Thus

1μt​log⁡Zt−1​\(μt\)≤1μt−1​log⁡Zt−1​\(μt−1\)=Φt−1\.\\frac\{1\}\{\\mu\_\{t\}\}\\log Z\_\{t\-1\}\(\\mu\_\{t\}\)\\leq\\frac\{1\}\{\\mu\_\{t\-1\}\}\\log Z\_\{t\-1\}\(\\mu\_\{t\-1\}\)=\\Phi\_\{t\-1\}\.Also, since𝚿t=𝚿t−1\+𝒈^t\\bm\{\\Psi\}\_\{t\}=\\bm\{\\Psi\}\_\{t\-1\}\+\\hat\{\\bm\{g\}\}\_\{t\},

Zt​\(μt\)Zt−1​\(μt\)=𝔼X∼𝒘t​\[exp⁡\(μt​g^t,X\)\],\\frac\{Z\_\{t\}\(\\mu\_\{t\}\)\}\{Z\_\{t\-1\}\(\\mu\_\{t\}\)\}=\\mathbb\{E\}\_\{X\\sim\\bm\{w\}\_\{t\}\}\\Big\[\\exp\(\\mu\_\{t\}\\hat\{g\}\_\{t,X\}\)\\Big\],so

Φt−1μt​log⁡Zt−1​\(μt\)=1μt​log⁡𝔼X∼𝒘t​\[exp⁡\(μt​g^t,X\)\]\.\\Phi\_\{t\}\-\\frac\{1\}\{\\mu\_\{t\}\}\\log Z\_\{t\-1\}\(\\mu\_\{t\}\)=\\frac\{1\}\{\\mu\_\{t\}\}\\log\\mathbb\{E\}\_\{X\\sim\\bm\{w\}\_\{t\}\}\\Big\[\\exp\(\\mu\_\{t\}\\hat\{g\}\_\{t,X\}\)\\Big\]\.LetY:=g^t,X∈\[−G,G\]Y:=\\hat\{g\}\_\{t,X\}\\in\[\-G,G\]a\.s\. Defineψ​\(λ\):=log⁡𝔼​\[exp⁡\(λ​Y\)\]\\psi\(\\lambda\):=\\log\\mathbb\{E\}\[\\exp\(\\lambda Y\)\]\. Thenψ′​\(0\)=𝔼​\[Y\]\\psi^\{\\prime\}\(0\)=\\mathbb\{E\}\[Y\]andψ′′​\(λ\)=Varλ⁡\(Y\)≤G2\\psi^\{\\prime\\prime\}\(\\lambda\)=\\operatorname\{Var\}\_\{\\lambda\}\(Y\)\\leq G^\{2\}for allλ\\lambda, hence

1μt​log⁡𝔼​\[exp⁡\(μt​Y\)\]≤𝔼​\[Y\]\+μt​G22=⟨𝒘t,𝒈^t⟩\+μt​G22\.\\frac\{1\}\{\\mu\_\{t\}\}\\log\\mathbb\{E\}\[\\exp\(\\mu\_\{t\}Y\)\]\\leq\\mathbb\{E\}\[Y\]\+\\frac\{\\mu\_\{t\}G^\{2\}\}\{2\}=\\langle\\bm\{w\}\_\{t\},\\hat\{\\bm\{g\}\}\_\{t\}\\rangle\+\\frac\{\\mu\_\{t\}G^\{2\}\}\{2\}\.Combining gives the bound\. Summing fromt=1t=1toTTgives

ΦT≤∑t=1T⟨𝒘t,𝒈^t⟩\+G22​∑t=1Tμt\.\\Phi\_\{T\}\\leq\\sum\_\{t=1\}^\{T\}\\langle\\bm\{w\}\_\{t\},\\hat\{\\bm\{g\}\}\_\{t\}\\rangle\+\\frac\{G^\{2\}\}\{2\}\\sum\_\{t=1\}^\{T\}\\mu\_\{t\}\.Now take any𝒘∈Δ𝒫\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\. Nonnegativity of KL divergence gives

⟨𝒘,𝚿T⟩≤ΦT\+1μT​KL⁡\(𝒘∥𝒘1\),\\langle\\bm\{w\},\\bm\{\\Psi\}\_\{T\}\\rangle\\leq\\Phi\_\{T\}\+\\frac\{1\}\{\\mu\_\{T\}\}\\operatorname\{KL\}\(\\bm\{w\}\\\|\\bm\{w\}\_\{1\}\),so

∑t=1T⟨𝒈^t,𝒘−𝒘t⟩\\displaystyle\\sum\_\{t=1\}^\{T\}\\langle\\hat\{\\bm\{g\}\}\_\{t\},\\bm\{w\}\-\\bm\{w\}\_\{t\}\\rangle=⟨𝒘,𝚿T⟩−∑t=1T⟨𝒘t,𝒈^t⟩\\displaystyle=\\langle\\bm\{w\},\\bm\{\\Psi\}\_\{T\}\\rangle\-\\sum\_\{t=1\}^\{T\}\\langle\\bm\{w\}\_\{t\},\\hat\{\\bm\{g\}\}\_\{t\}\\rangle≤1μT​KL⁡\(𝒘∥𝒘1\)\+G22​∑t=1Tμt\.\\displaystyle\\leq\\frac\{1\}\{\\mu\_\{T\}\}\\operatorname\{KL\}\(\\bm\{w\}\\\|\\bm\{w\}\_\{1\}\)\+\\frac\{G^\{2\}\}\{2\}\\sum\_\{t=1\}^\{T\}\\mu\_\{t\}\.Takingsup𝒘\\sup\_\{\\bm\{w\}\}yields

sup𝒘∈Δ𝒫∑t=1T⟨𝒈^t,𝒘−𝒘t⟩≤D𝒘​\(1\)μT\+G22​∑t=1Tμt\.\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\sum\_\{t=1\}^\{T\}\\langle\\hat\{\\bm\{g\}\}\_\{t\},\\bm\{w\}\-\\bm\{w\}\_\{t\}\\rangle\\leq\\frac\{D\_\{\\bm\{w\}\}\(1\)\}\{\\mu\_\{T\}\}\+\\frac\{G^\{2\}\}\{2\}\\sum\_\{t=1\}^\{T\}\\mu\_\{t\}\.Finally, for eachtt,\|⟨𝒈^t,𝒘−𝒘t⟩\|≤‖𝒈^t‖∞​‖𝒘−𝒘t‖1≤2​G\|\\langle\\hat\{\\bm\{g\}\}\_\{t\},\\bm\{w\}\-\\bm\{w\}\_\{t\}\\rangle\|\\leq\\\|\\hat\{\\bm\{g\}\}\_\{t\}\\\|\_\{\\infty\}\\\|\\bm\{w\}\-\\bm\{w\}\_\{t\}\\\|\_\{1\}\\leq 2G, so removing the first\(bT−1\)\(b\_\{T\}\-1\)terms gives

sup𝒘∈Δ𝒫∑t=bTT⟨𝒈^t,𝒘−𝒘t⟩≤D𝒘​\(1\)μT\+G22​∑t=1Tμt\+2​G​\(bT−1\)\.\\sup\_\{\\bm\{w\}\\in\\Delta\_\{\\mathcal\{P\}\}\}\\sum\_\{t=b\_\{T\}\}^\{T\}\\langle\\hat\{\\bm\{g\}\}\_\{t\},\\bm\{w\}\-\\bm\{w\}\_\{t\}\\rangle\\leq\\frac\{D\_\{\\bm\{w\}\}\(1\)\}\{\\mu\_\{T\}\}\+\\frac\{G^\{2\}\}\{2\}\\sum\_\{t=1\}^\{T\}\\mu\_\{t\}\+2G\(b\_\{T\}\-1\)\.∎

## Appendix GAdditional Discussion

This section contains three supplemental discussions\. Subsection[G\.1](https://arxiv.org/html/2607.08979#A7.SS1)derives how quickly the stopping time converges to the lower bound asδ→0\\delta\\to 0\. Subsection[G\.2](https://arxiv.org/html/2607.08979#A7.SS2)explains where the boundedness of the parameter space is used in the analysis\. Subsection[G\.3](https://arxiv.org/html/2607.08979#A7.SS3)discusses the choice of the learning\-rate exponentα\\alpha\.

### G\.1Rate of Convergence

Here, we derive an upper bound on the convergence error of𝔼𝜽​\[τδ\]log⁡\(1/δ\)\\frac\{\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\[\\tau\_\{\\delta\}\]\}\{\\log\(1/\\delta\)\}to1Γ∗​\(𝜽\)\\frac\{1\}\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}asδ→0\\delta\\to 0\.

###### Proposition G\.1\(Refinement of Theorem[6\.5](https://arxiv.org/html/2607.08979#S6.Thmtheorem5)\)\.

Let

ρ:=min⁡\{α,1−α,γ,1−γ2\}\.\\rho:=\\min\\left\\\{\\alpha,\\,1\-\\alpha,\\,\\gamma,\\,\\frac\{1\-\\gamma\}\{2\}\\right\\\}\.Fix𝛉∈Θgap\\bm\{\\theta\}\\in\\Theta^\{\\mathrm\{gap\}\}and letτδ\\tau\_\{\\delta\}be the stopping time of Algorithm[2](https://arxiv.org/html/2607.08979#alg2)\. Then there exist constantsC𝛉<∞C\_\{\\bm\{\\theta\}\}<\\inftyandδ0​\(𝛉\)∈\(0,1\)\\delta\_\{0\}\(\\bm\{\\theta\}\)\\in\(0,1\)such that, for allδ<δ0​\(𝛉\)\\delta<\\delta\_\{0\}\(\\bm\{\\theta\}\),

𝔼𝜽​\[τδ\]≤log⁡\(1/δ\)Γ∗​\(𝜽\)\+C𝜽​\(log⁡\(1/δ\)\)1−ρ​log⁡log⁡\(1/δ\)\.\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\[\\tau\_\{\\delta\}\]\\leq\\frac\{\\log\(1/\\delta\)\}\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}\+C\_\{\\bm\{\\theta\}\}\(\\log\(1/\\delta\)\)^\{1\-\\rho\}\\sqrt\{\\log\\log\(1/\\delta\)\}\.Hence,

𝔼𝜽​\[τδ\]log⁡\(1/δ\)≤1Γ∗​\(𝜽\)\+O~​\(\(log⁡\(1/δ\)\)−ρ\)\.\\frac\{\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\[\\tau\_\{\\delta\}\]\}\{\\log\(1/\\delta\)\}\\leq\\frac\{1\}\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}\+\\widetilde\{O\}\\\!\\left\(\(\\log\(1/\\delta\)\)^\{\-\\rho\}\\right\)\.

###### Proof\.

We follow the proof of Theorem[6\.5](https://arxiv.org/html/2607.08979#S6.Thmtheorem5)in Appendix[E](https://arxiv.org/html/2607.08979#A5)\. The only change is that the fixed slackε′\\varepsilon^\{\\prime\}used there is replaced by the explicit rate obtained from Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)\.

Apply Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)withp=4p=4, Lemma[D\.6](https://arxiv.org/html/2607.08979#A4.Thmtheorem6)with confidence levelt−4t^\{\-4\}, and Lemma[6\.1](https://arxiv.org/html/2607.08979#S6.Thmtheorem1)with confidence levelt−4t^\{\-4\}\. As in the proof of Theorem[6\.5](https://arxiv.org/html/2607.08979#S6.Thmtheorem5), Lemma[6\.1](https://arxiv.org/html/2607.08979#S6.Thmtheorem1)together with Lemma[F\.1](https://arxiv.org/html/2607.08979#A6.Thmtheorem1)*\(e\)*gives correctness of the estimated boundary constraints for all sufficiently largettwith probability at least1−t−41\-t^\{\-4\}\. Additionally, all error terms in Proposition[6\.3](https://arxiv.org/html/2607.08979#S6.Thmtheorem3)and Lemma[D\.6](https://arxiv.org/html/2607.08979#A4.Thmtheorem6)are bounded, for all sufficiently largett, by a constant multiple oft−ρ​log⁡tt^\{\-\\rho\}\\sqrt\{\\log t\}\. Therefore, by the same union\-bound argument over the three events used in Appendix[E](https://arxiv.org/html/2607.08979#A5), and by the same application of Lemma[D\.5](https://arxiv.org/html/2607.08979#A4.Thmtheorem5)*\(b\)*, there exist constantsC1<∞C\_\{1\}<\\inftyandt∗​\(𝜽\)<∞t^\{\*\}\(\\bm\{\\theta\}\)<\\inftysuch that, for allt≥t∗​\(𝜽\)t\\geq t^\{\*\}\(\\bm\{\\theta\}\),

ℙ𝜽​\(1t​ZS^t​\(t\)≤Γ∗​\(𝜽\)−C1​t−ρ​log⁡t\)≤3​t−4\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\\left\(\\frac\{1\}\{t\}Z\_\{\\hat\{S\}\_\{t\}\}\(t\)\\leq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-C\_\{1\}t^\{\-\\rho\}\\sqrt\{\\log t\}\\right\)\\leq 3t^\{\-4\}\.\(29\)
We now define the deterministic cutofftδt\_\{\\delta\}, the candidate upper bound for the stopping time\. We show that after this time, the probability that the algorithm has not stopped is at most3​t−43t^\{\-4\}\. First, chooseM\>0M\>0large enough so that

\(Γ∗​\(𝜽\)\)2​M4\>2​C1​\(Γ∗​\(𝜽\)\)ρ\.\\frac\{\(\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\)^\{2\}M\}\{4\}\>2C\_\{1\}\(\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\)^\{\\rho\}\.Define

tδ=⌈log⁡\(1/δ\)Γ∗​\(𝜽\)\+M​\(log⁡\(1/δ\)\)1−ρ​log⁡log⁡\(1/δ\)⌉\.t\_\{\\delta\}=\\left\\lceil\\frac\{\\log\(1/\\delta\)\}\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}\+M\(\\log\(1/\\delta\)\)^\{1\-\\rho\}\\sqrt\{\\log\\log\(1/\\delta\)\}\\right\\rceil\.
By Lemma[D\.4](https://arxiv.org/html/2607.08979#A4.Thmtheorem4),β​\(t,δ\)≤β¯​\(t,δ\)\\beta\(t,\\delta\)\\leq\\bar\{\\beta\}\(t,\\delta\), where

β¯​\(t,δ\)=log⁡\(1/δ\)\+λ2​n​R2\+n−12​log⁡\(1\+Cβ​t\),\\bar\{\\beta\}\(t,\\delta\)=\\log\(1/\\delta\)\+\\frac\{\\lambda\}\{2\}nR^\{2\}\+\\frac\{n\-1\}\{2\}\\log\(1\+C\_\{\\beta\}t\),and, as used in Appendix[E](https://arxiv.org/html/2607.08979#A5),t↦β¯​\(t,δ\)/tt\\mapsto\\bar\{\\beta\}\(t,\\delta\)/tis decreasing\.

Sincetδ≍log⁡\(1/δ\)t\_\{\\delta\}\\asymp\\log\(1/\\delta\),

β¯​\(tδ,δ\)=log⁡\(1/δ\)\+O​\(log⁡log⁡\(1/δ\)\)\.\\bar\{\\beta\}\(t\_\{\\delta\},\\delta\)=\\log\(1/\\delta\)\+O\(\\log\\log\(1/\\delta\)\)\.Also,

log⁡log⁡\(1/δ\)=o​\(\(log⁡\(1/δ\)\)1−ρ​log⁡log⁡\(1/δ\)\)\.\\log\\log\(1/\\delta\)=o\\\!\\left\(\(\\log\(1/\\delta\)\)^\{1\-\\rho\}\\sqrt\{\\log\\log\(1/\\delta\)\}\\right\)\.Thus, for all sufficiently smallδ\\delta,

Γ∗​\(𝜽\)​tδ−β¯​\(tδ,δ\)≥Γ∗​\(𝜽\)​M2​\(log⁡\(1/δ\)\)1−ρ​log⁡log⁡\(1/δ\)\.\\Gamma^\{\*\}\(\\bm\{\\theta\}\)t\_\{\\delta\}\-\\bar\{\\beta\}\(t\_\{\\delta\},\\delta\)\\geq\\frac\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)M\}\{2\}\(\\log\(1/\\delta\)\)^\{1\-\\rho\}\\sqrt\{\\log\\log\(1/\\delta\)\}\.Moreover, for all sufficiently smallδ\\delta,

tδ≤2​log⁡\(1/δ\)Γ∗​\(𝜽\)\.t\_\{\\delta\}\\leq\\frac\{2\\log\(1/\\delta\)\}\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}\.Thus dividing bytδt\_\{\\delta\}gives

Γ∗​\(𝜽\)−β¯​\(tδ,δ\)tδ≥\(Γ∗​\(𝜽\)\)2​M4​\(log⁡\(1/δ\)\)−ρ​log⁡log⁡\(1/δ\)\.\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-\\frac\{\\bar\{\\beta\}\(t\_\{\\delta\},\\delta\)\}\{t\_\{\\delta\}\}\\geq\\frac\{\(\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\)^\{2\}M\}\{4\}\(\\log\(1/\\delta\)\)^\{\-\\rho\}\\sqrt\{\\log\\log\(1/\\delta\)\}\.Set

c:=\(Γ∗​\(𝜽\)\)2​M4\.c:=\\frac\{\(\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\)^\{2\}M\}\{4\}\.Sinceβ​\(t,δ\)≤β¯​\(t,δ\)\\beta\(t,\\delta\)\\leq\\bar\{\\beta\}\(t,\\delta\)andβ¯​\(t,δ\)/t\\bar\{\\beta\}\(t,\\delta\)/tis decreasing, for everyt≥tδt\\geq t\_\{\\delta\},

β​\(t,δ\)t≤Γ∗​\(𝜽\)−c​\(log⁡\(1/δ\)\)−ρ​log⁡log⁡\(1/δ\)\.\\frac\{\\beta\(t,\\delta\)\}\{t\}\\leq\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\-c\(\\log\(1/\\delta\)\)^\{\-\\rho\}\\sqrt\{\\log\\log\(1/\\delta\)\}\.\(30\)Sincet↦t−ρ​log⁡tt\\mapsto t^\{\-\\rho\}\\sqrt\{\\log t\}is eventually decreasing andtδ→∞t\_\{\\delta\}\\to\\inftyasδ→0\\delta\\to 0, we chooseδ0​\(𝜽\)\\delta\_\{0\}\(\\bm\{\\theta\}\)small enough so that this monotonicity holds on\[tδ,∞\)\[t\_\{\\delta\},\\infty\)andtδ≥t∗​\(𝜽\)t\_\{\\delta\}\\geq t^\{\*\}\(\\bm\{\\theta\}\)\. Hence, for allδ<δ0​\(𝜽\)\\delta<\\delta\_\{0\}\(\\bm\{\\theta\}\)and allt≥tδt\\geq t\_\{\\delta\},

C1​t−ρ​log⁡t\\displaystyle C\_\{1\}t^\{\-\\rho\}\\sqrt\{\\log t\}≤C1​tδ−ρ​log⁡tδ\\displaystyle\\leq C\_\{1\}t\_\{\\delta\}^\{\-\\rho\}\\sqrt\{\\log t\_\{\\delta\}\}≤2​C1​\(Γ∗​\(𝜽\)\)ρ​\(log⁡\(1/δ\)\)−ρ​log⁡log⁡\(1/δ\)\.\\displaystyle\\leq 2C\_\{1\}\(\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\)^\{\\rho\}\\,\(\\log\(1/\\delta\)\)^\{\-\\rho\}\\sqrt\{\\log\\log\(1/\\delta\)\}\.Definec′:=2​C1​\(Γ∗​\(𝜽\)\)ρ\.c^\{\\prime\}:=2C\_\{1\}\(\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\)^\{\\rho\}\.By the choice ofMM,c\>c′c\>c^\{\\prime\}\. Therefore, combining the high\-probability GLR lower bound \([29](https://arxiv.org/html/2607.08979#A7.E29)\) with the threshold bound \([30](https://arxiv.org/html/2607.08979#A7.E30)\), and using the same argument as in Appendix[E](https://arxiv.org/html/2607.08979#A5), we get

ℙ𝜽​\(τδ\>t\)≤3​t−4,t≥tδ\.\\mathbb\{P\}\_\{\\bm\{\\theta\}\}\(\\tau\_\{\\delta\}\>t\)\\leq 3t^\{\-4\},\\qquad t\\geq t\_\{\\delta\}\.Finally, by the same tail\-sum argument as in the proof of Theorem[6\.5](https://arxiv.org/html/2607.08979#S6.Thmtheorem5),

𝔼𝜽​\[τδ\]≤tδ\+3​∑t=tδ∞t−4=tδ\+O​\(tδ−3\)\.\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\[\\tau\_\{\\delta\}\]\\leq t\_\{\\delta\}\+3\\sum\_\{t=t\_\{\\delta\}\}^\{\\infty\}t^\{\-4\}=t\_\{\\delta\}\+O\(t\_\{\\delta\}^\{\-3\}\)\.Substituting the definition oftδt\_\{\\delta\}and absorbing the ceiling, the tail term, andMMintoC𝜽<∞C\_\{\\bm\{\\theta\}\}<\\inftygives

𝔼𝜽​\[τδ\]≤log⁡\(1/δ\)Γ∗​\(𝜽\)\+C𝜽​\(log⁡\(1/δ\)\)1−ρ​log⁡log⁡\(1/δ\)\.\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\[\\tau\_\{\\delta\}\]\\leq\\frac\{\\log\(1/\\delta\)\}\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}\+C\_\{\\bm\{\\theta\}\}\(\\log\(1/\\delta\)\)^\{1\-\\rho\}\\sqrt\{\\log\\log\(1/\\delta\)\}\.Thus,

𝔼𝜽​\[τδ\]log⁡\(1/δ\)≤1Γ∗​\(𝜽\)\+C𝜽​\(log⁡\(1/δ\)\)−ρ​log⁡log⁡\(1/δ\)\.\\frac\{\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\[\\tau\_\{\\delta\}\]\}\{\\log\(1/\\delta\)\}\\leq\\frac\{1\}\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}\+C\_\{\\bm\{\\theta\}\}\(\\log\(1/\\delta\)\)^\{\-\\rho\}\\sqrt\{\\log\\log\(1/\\delta\)\}\.Equivalently,

𝔼𝜽​\[τδ\]log⁡\(1/δ\)≤1Γ∗​\(𝜽\)\+O~​\(\(log⁡\(1/δ\)\)−ρ\)\.\\frac\{\\mathbb\{E\}\_\{\\bm\{\\theta\}\}\[\\tau\_\{\\delta\}\]\}\{\\log\(1/\\delta\)\}\\leq\\frac\{1\}\{\\Gamma^\{\*\}\(\\bm\{\\theta\}\)\}\+\\widetilde\{O\}\\\!\\left\(\(\\log\(1/\\delta\)\)^\{\-\\rho\}\\right\)\.∎

### G\.2Bounded Parameter Space Assumption

Here, we discuss where the boundedness assumption is used\. First, since every pairwise differenceθi−θj\\theta\_\{i\}\-\\theta\_\{j\}lies in\[−2​R,2​R\]\[\-2R,2R\], we get the bound

a=inf\|η\|≤2​RA′′​\(η\)\>0,a=\\inf\_\{\|\\eta\|\\leq 2R\}A^\{\\prime\\prime\}\(\\eta\)\>0,which is used for the MLE concentration argument\. Second, boundedΘ\\ThetamakesDmaxD\_\{\\max\}finite, which is the Lipschitz constant that controls how muchΓ​\(𝒘;𝜽\)\\Gamma\(\\bm\{w\};\\bm\{\\theta\}\)can change with changes to the allocation𝒘\\bm\{w\}\. Third, it makesLLfinite, which appears in

Δ​Fs:=sup𝒘,𝒒\|F𝜽^s​\(𝒘,𝒒\)−F𝜽​\(𝒘,𝒒\)\|≤L​‖𝜽^s−𝜽‖2,\\Delta F\_\{s\}:=\\sup\_\{\\bm\{w\},\\bm\{q\}\}\\bigl\|F\_\{\\hat\{\\bm\{\\theta\}\}\_\{s\}\}\(\\bm\{w\},\\bm\{q\}\)\-F\_\{\\bm\{\\theta\}\}\(\\bm\{w\},\\bm\{q\}\)\\bigr\|\\leq L\\\|\\hat\{\\bm\{\\theta\}\}\_\{s\}\-\\bm\{\\theta\}\\\|\_\{2\},used to map estimation error to the error in the achieved information rate\. Finally, in the stopping proof, boundedness gives us

Δt=sup𝜽′∈Θ\|L¯t​\(𝜽′\)−Lt​\(𝜽′\)\|\\Delta\_\{t\}=\\sup\_\{\\bm\{\\theta\}^\{\\prime\}\\in\\Theta\}\\bigl\|\\bar\{L\}\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\-L\_\{t\}\(\\bm\{\\theta\}^\{\\prime\}\)\\bigr\|finite and tending to zero, and

λ2​‖𝜽^t‖22≤λ2​n​R2,\\frac\{\\lambda\}\{2\}\\\|\\hat\{\\bm\{\\theta\}\}\_\{t\}\\\|\_\{2\}^\{2\}\\leq\\frac\{\\lambda\}\{2\}nR^\{2\},which allows us to conclude that the stopping threshold grows logarithmically\.

Thus, the boundedness assumption is primarily used as a tool in the proofs \(also a common assumption in linear bandits\), but we do not view it as particularly restrictive in practice\. For example, under the Bradley–Terry model, ifR=5R=5, thenP​\(i≻j\)P\(i\\succ j\)can be as high as0\.999950\.99995\. Thus in most settings, it is likely that any practically relevant utilities lie within such a bounded range\.

### G\.3Hyperparameterα\\alphaTuning

Here we discuss what is meant by “the early gradient directions can be noisy” and the tuning ofα\\alpha\.

There are two mechanisms contributing to this “noise”\. The first is the stochastic gradient approximation itself\. The full𝒘\\bm\{w\}\-gradient averages over all boundary pairs inB​\(𝜽\)B\(\\bm\{\\theta\}\), whereas the update in Algorithm[2](https://arxiv.org/html/2607.08979#alg2)uses only the sampled pairItI\_\{t\}and the corresponding alternative𝜽t∗\\bm\{\\theta\}\_\{t\}^\{\*\}\. Thus, early on, depending on which pair is sampled, the update can put too much weight on comparisons that are especially informative for ruling out that one sampled inversion, rather than on those that matter most for the full gradient\. For example, ifn=4n=4andk=2k=2, the most challenging boundary pair is to differentiate ranks22and33, but we may draw ranks\(1,4\)\(1,4\)several times early on\. The resulting updates overweight comparisons useful for ruling out item44overtaking item11, which causes the cumulative scores𝚿t\(w\)\\bm\{\\Psi\}\_\{t\}^\{\(w\)\}and𝚿t\(q\)\\bm\{\\Psi\}\_\{t\}^\{\(q\)\}to be biased early\. Hence, ifμt=t−α\\mu\_\{t\}=t^\{\-\\alpha\}decays too quickly, later updates will take a long time to undo this bias; if it decays too slowly, then the iterates remain overly sensitive to one\-step noise\. This is only an issue in early rounds, since later, non\-bottleneck boundary pairs receive very little mass under𝒒t\\bm\{q\}\_\{t\}, so even if such a pair is sampled, its effect on the𝒘\\bm\{w\}\-update is small\. This is the mechanism that leads to improved performance of the oracle problem, where𝜽\\bm\{\\theta\}is known, whenα\\alphais moderately small \(smaller than\.5\.5\)\.

When𝜽\\bm\{\\theta\}is not known, there is a second mechanism that arises since the gradients are evaluated at𝜽^t\\hat\{\\bm\{\\theta\}\}\_\{t\}rather than at𝜽\\bm\{\\theta\}\. Even if we were to compute the full gradient, it could still be misaligned early, because the estimated boundary set and the associated𝜽i​j∗\\bm\{\\theta\}\_\{ij\}^\{\*\}may be off\. When tuningα\\alpha, we therefore consider both its effect on the oracle problem and on the full problem with estimation error\. Empirically, these choices are quite similar\. The slower decay that helps mitigate the issue of using a stochastic gradient also gives the MLE and the estimated boundary set time to stabilize\.

Figures[3](https://arxiv.org/html/2607.08979#A7.F3)and[4](https://arxiv.org/html/2607.08979#A7.F4)show the mean stopping time over 100 simulations of the online oracle333This is different from the oracle of the plots in Section[7](https://arxiv.org/html/2607.08979#S7)\. Here the oracle knows𝜽\\bm\{\\theta\}, but still learns𝒘∗\\bm\{w\}^\{\*\}online, so its only advantage is the lack of noise in the MLE\.and Algorithm[2](https://arxiv.org/html/2607.08979#alg2), on the Equally Spaced instance withδ=0\.01\\delta=0\.01\. Figure[3](https://arxiv.org/html/2607.08979#A7.F3)fixesn=50n=50,gap=0\.25\\mathrm\{gap\}=0\.25and varieskk; Figure[4](https://arxiv.org/html/2607.08979#A7.F4)fixesk=5k=5and variesnnandgap\\mathrm\{gap\}\. The plots demonstrate that the choice ofα\\alphais important; values that are too large or too small can result in significantly worse performance\. However, as illustrated in both figures, the algorithm generally performs well forα∈\[0\.15,0\.3\]\\alpha\\in\[0\.15,0\.3\], and this holds robustly for other configurations as well\.

![Refer to caption](https://arxiv.org/html/2607.08979v1/figure_alpha_sweep_k.png)Figure 3:Mean stopping time over 100 simulations as a function of the learning\-rate exponentα\\alphafor varyingkk, on the Equally Spaced instance withn=50n=50,gap=0\.25\\mathrm\{gap\}=0\.25, andδ=0\.01\\delta=0\.01\. Each panel compares Algorithm[2](https://arxiv.org/html/2607.08979#alg2)\(blue\) to the Online Oracle \(orange\) atk∈\{1,2,3,5,10\}k\\in\\\{1,2,3,5,10\\\}\.![Refer to caption](https://arxiv.org/html/2607.08979v1/figure_alpha_sweep_n_gap.png)Figure 4:Mean stopping time over 100 simulations as a function of the learning\-rate exponentα\\alpha, on the Equally Spaced instance withk=5k=5andδ=0\.01\\delta=0\.01\. Rows varyn∈\{20,50,100\}n\\in\\\{20,50,100\\\}; columns varygap∈\{0\.1,0\.25\}\\mathrm\{gap\}\\in\\\{0\.1,0\.25\\\}\. Algorithm[2](https://arxiv.org/html/2607.08979#alg2)\(blue\) versus Online Oracle \(orange\)\.