Stochastic Linear Bandits with Partially Observed Actions

# Stochastic Linear Bandits with Partially Observed Actions††thanks: This work was primarily performed while Vineet Gattani was a PhD student at Arizona State University, and was partially supported by the National Science Foundation award CCF-2048223.
Source: [https://arxiv.org/html/2607.08971](https://arxiv.org/html/2607.08971)
###### Abstract

The stochastic linear bandit, where actions are represented as vectors and rewards are linear, is a central paradigm for sequential decision making\. We study a partially observed variant of this problem in which the learning agent only sees a random subset of coordinates for each action\. Such partial observability arises naturally in settings like recommendation and healthcare, where full action descriptions can be expensive or even impossible to obtain\. In general, this makes sublinear regret information\-theoretically impossible\. However, we show that this barrier can be overcome when the action vectors have low intrinsic dimension\. We propose an algorithm,TOFU\-POV, that estimates the latent action subspace using the masked actions, imputes current actions using an epoch\-wise frozen representation, and runs OFUL in the resulting low\-dimensional coordinates\. Our theory shows thatTOFU\-POVenjoys aT\\sqrt\{T\}regret that scales with the intrinsic action subspace dimension as opposed to the ambient dimension and quantifies the interaction between these quantities and the missingness, decision set size, and subspace conditioning\. We also devise a rank\-adaptive algorithm that does not require the knowledge of the intrinsic dimension\. We complement these guarantees with a lower bound based on a novel product construction that separates usual reward\-learning uncertainty from a missingness\-dependent cost intrinsic to partial observation\. Synthetic and real data experiments support our theory and show thatTOFU\-POVcan substantially improve upon natural baselines in this challenging problem\.

## 1Introduction

The stochastic linear bandit \(SLB\) is an important framework for sequential decision\-making under uncertainty, where the expected reward of a vector\-valued action is assumed to be a linear function of its features\[[1](https://arxiv.org/html/2607.08971#bib.bib1),[2](https://arxiv.org/html/2607.08971#bib.bib2),[3](https://arxiv.org/html/2607.08971#bib.bib3)\]\. At each roundtt, the learning agent is presented with a decision setDt=\{Xt,1,Xt,2,…\}D\_\{t\}=\\\{X\_\{t,1\},X\_\{t,2\},\\ldots\\\}and chooses an actionXt∈DtX\_\{t\}\\in D\_\{t\}, which results in a rewardrt=⟨Xt,θ⋆⟩\+ηtr\_\{t\}=\\langle X\_\{t\},\\theta^\{\\star\}\\rangle\+\\eta\_\{t\}, whereθ⋆∈ℝd\\theta^\{\\star\}\\in\\mathbb\{R\}^\{d\}is unknown andηt\\eta\_\{t\}is conditionally zero\-mean random noise\. The goal here is to minimize the cumulative regret relative to an oracle that \(a\) knowsθ⋆\\theta^\{\\star\}and, therefore, \(b\) chooses the action inDtD\_\{t\}that maximizes the expected reward each round\. A widely studied algorithm in this setting is OFUL\[[1](https://arxiv.org/html/2607.08971#bib.bib1)\], whose regret is known to be bounded above byO~​\(d​T\)\\widetilde\{O\}\(d\\sqrt\{T\}\), matching known lower bounds up to logarithmic factors\. SLBs have found far\-ranging applications in recommendation systems, advertising, and treatment allocation\[[4](https://arxiv.org/html/2607.08971#bib.bib4),[5](https://arxiv.org/html/2607.08971#bib.bib5),[6](https://arxiv.org/html/2607.08971#bib.bib6)\]\.

In many modern applications, however, observing the full feature vector of each action is prohibitively expensive, infeasible, or impossible\. In recommendation systems\[[4](https://arxiv.org/html/2607.08971#bib.bib4)\], due to privacy, storage, or computational constraints, only a sparse subset of item features may be accessible\. Similarly, in scientific or healthcare applications, constraints on sensing or data collection may naturally lead to missing observations\. This motivates a more challenging variant of the SLB problem, where at each round, the agent only observes a subset of entries from each action vector—what we refer to as*partial observability*\. Formally, for each action vectorXt∈ℝdX\_\{t\}\\in\\mathbb\{R\}^\{d\}, each coordinate is revealed independently with probabilityp∈\(0,1\]p\\in\(0,1\]\. Without any further structure, reward\-learning is information\-theoretically impossible here: the agent cannot infer the full linear reward model, and pays a suboptimality price \(i\.e\., regret\) that is linear in time\. Fortunately, many real\-world problems exhibit low\-dimensional structure\. In recommendation systems, for example, user\-item interactions are often governed by a small number of latent factors, implying that the true feature vectors lie near a low\-dimensional subspace\. Motivated by this, we study the SLB problem under*limited observability*and*low\-rank structure*: we assume that the ideal \(fully observed\) action vectorsX∈ℝdX\\in\\mathbb\{R\}^\{d\}lie in an unknownmm\-dimensional subspace, withmmpossibly being significantly smaller thandd\.

There are two lines of work that our setting sits between, but is not covered by\. Recent work on bandits with partially observable features\[[7](https://arxiv.org/html/2607.08971#bib.bib7),[8](https://arxiv.org/html/2607.08971#bib.bib8)\]studies different observation models in which the missing or latent components enter the reward problem in a prescribed way\. In contrast, here the actions themselves lie in a low\-dimensional subspace, and the learner sees \(changing\) random coordinate masks of every offered action\.\[[9](https://arxiv.org/html/2607.08971#bib.bib9)\]studies the low\-rank action\-representation bandit problem where the action vectors are*fully observed*, and pursue a projected\-OFUL style analysis to get regret guarantees for their algorithm, PSLB\. This is distinct from our approach which creates estimates of the latent action subspace and imputes every slate of actions using an epoch\-wise frozen subspace representation\. We then show that inside each epoch, we have an approximate linear bandit whose approximation error can be carefully controlled to get our regret guarantees\.

There is also a technical reason to be careful about importing projected\-OFUL analyses directly\. If the estimated projection is updated over time and applied to past data, the projected noise process is not the standard predictable martingale transform used in OFUL, and the projected design no longer evolves by the usual rank\-one updates\. Moreover, lower\-eigenvalue growth of the covariance of played arms cannot be inferred from the population covariance of offered arms when the played arms are selected by an OFU rule\. This is whyTOFU\-POVfreezes the representation within epochs: conditional on the epoch\-start estimate, reward learning is an ordinary fixed\-coordinate OFUL problem, while subspace and imputation errors are controlled separately\. We give the detailed comparison in Appendix[A](https://arxiv.org/html/2607.08971#A1)\.

We begin by outlining themain contributionsof our paper\.

- •Problem formulation\.We formulate the partially observed low\-rank SLB problem, where each offered action lies in an unknownmm\-dimensional subspace ofℝd\\mathbb\{R\}^\{d\}, but the learner observes only a random subset of its features \(determined by independent coordinate masks\)\.
- •Algorithm\.We introduceTOFU\-POV, an epoch\-wise algorithm that estimates the latent subspace from all offered masked actions, freezes the representation within each epoch, imputes the current action set, and runs OFUL in the resulting low\-dimensional coordinates\.
- •Regret guarantee\.Whenmmis known, under a standard incoherence condition and the standard normalizationBX,‖θ⋆‖=O​\(1\)B\_\{X\},\\\|\\theta^\{\\star\}\\\|=O\(1\), we show that, with high probability, the regret ofTOFU\-POVscales as O~​\(κ2​mp4​K\+m​T\+κ​m​Tp2​K\),\\widetilde\{O\}\\\!\\left\(\\frac\{\\kappa^\{2\}m\}\{p^\{4\}K\}\+m\\sqrt\{T\}\+\\frac\{\\kappa m\\sqrt\{T\}\}\{p^\{2\}\\sqrt\{K\}\}\\right\),whereκ\\kappameasures the conditioning of the action covariance matrix \(see Section[3](https://arxiv.org/html/2607.08971#S3)\)\. Equivalently, suppressing the additive burn\-in cost that does not growTT, the effective scaling isO~​\(m​T\+κ​m​T/\(p2​K\)\)\\widetilde\{O\}\(m\\sqrt\{T\}\+\\kappa m\\sqrt\{T\}/\(p^\{2\}\\sqrt\{K\}\)\)\. This bound replaces the ambient dimensionddby the usually much smallermm, while exposing how missingnesspp, decision set sizeKK, and subspace conditioningκ\\kappaaffect the regret\. To achieve this, our epoch\-based argument isolates the statistical difficulty of OFUL from a controlled misspecification term\. As we show in Appendix[A](https://arxiv.org/html/2607.08971#A1), this resolves key obstacles in projected low\-rank bandit arguments\.
- •Rank adaptivity\.We also devise a rank\-adaptive variant ofTOFU\-POVthat requires no knowledge ofmm\(nor an upper bound on it\) and enjoys the same regret scaling; the rank\-identification cost is absorbed, up to constants, into the imputation burn\-in\.
- •Lower bound\.We complement the upper bound with a lower bound via a novel argument that separates ordinary reward uncertainty from a missingness\-discovery cost\.
- •Experiments\.We evaluateTOFU\-POVand its natural variants on synthetic and real data\. The results corroborate our theory, with the largest gains over the baselines appearing under heavier missingness\.

## 2Related Work

Structured Linear Bandits\.Linear stochastic bandits are usually analyzed in the ambient feature dimensiondd\[[1](https://arxiv.org/html/2607.08971#bib.bib1),[3](https://arxiv.org/html/2607.08971#bib.bib3)\]\. A large literature reduces this dependence by imposing structure, including sparsity ofθ⋆\\theta^\{\\star\}\[[10](https://arxiv.org/html/2607.08971#bib.bib10),[11](https://arxiv.org/html/2607.08971#bib.bib11),[12](https://arxiv.org/html/2607.08971#bib.bib12),[13](https://arxiv.org/html/2607.08971#bib.bib13)\]and low\-rank matrix or bilinear reward structure\[[14](https://arxiv.org/html/2607.08971#bib.bib14),[15](https://arxiv.org/html/2607.08971#bib.bib15),[16](https://arxiv.org/html/2607.08971#bib.bib16),[17](https://arxiv.org/html/2607.08971#bib.bib17)\]\. These exploit parameter structure, whereas our setting exploits low\-rank structure in the action vectors themselves\. Other kinds of structure on the action space itself, e\.g\., spectral structure, are likewise known to aid regret and pure\-exploration performance\[[18](https://arxiv.org/html/2607.08971#bib.bib18),[19](https://arxiv.org/html/2607.08971#bib.bib19),[20](https://arxiv.org/html/2607.08971#bib.bib20)\]\.

Low\-Rank Action Representations\.The closest predecessor isLale et al\. \[[9](https://arxiv.org/html/2607.08971#bib.bib9)\], who study linear bandits with fully observed, approximately low\-rank action representations\. Our paradigm adds coordinate\-wise missingness, but the distinction goes beyond just modeling: a direct projected\-OFUL proof with a continually updated projection does not automatically inherit the standard self\-normalized or elliptical\-potential arguments\. Our epoch\-wise construction avoids this by freezing the representation inside each epoch and controlling the resulting representation bias carefully; see Appendix[A](https://arxiv.org/html/2607.08971#A1)for more on this comparison\.

Bandits with Partially Observable Features\.Recent work also studies bandits with partially observable or latent features\[[7](https://arxiv.org/html/2607.08971#bib.bib7),[8](https://arxiv.org/html/2607.08971#bib.bib8)\], but under different observation models\. In these settings, the missing or latent components enter the reward problem through a prescribed structure or a known sensing channel\. In contrast,TOFU\-POVassumes that the action vectors themselves lie in an unknown low\-rank subspace, while each offered arm is revealed through random arm\- and time\-dependent coordinate masks\.

Subspace Estimation with Missing Data\.Our work is also related to subspace estimation from incomplete observations, which has been widely studied, both in offline settings such as matrix completion\[[21](https://arxiv.org/html/2607.08971#bib.bib21)\]and robust PCA\[[22](https://arxiv.org/html/2607.08971#bib.bib22)\], and in online or streaming settings through methods such as Oja’s algorithm\[[23](https://arxiv.org/html/2607.08971#bib.bib23)\], GROUSE\[[24](https://arxiv.org/html/2607.08971#bib.bib24)\], and PETRELS\[[25](https://arxiv.org/html/2607.08971#bib.bib25)\]\. Recent high\-dimensional analyses also give a unified view of several such online updates\[[26](https://arxiv.org/html/2607.08971#bib.bib26)\]\. We use some similar techniques, but our focus is on controlling the bias induced by approximate subspace estimation and its interaction with the regret of an online algorithm\.

## 3Problem Setup

Low\-dimensional action vector model\.For a natural numbern∈ℕn\\in\\mathbb\{N\}, let\[n\]:=\{1,2,…,n\}\[n\]:=\\\{1,2,\\ldots,n\\\}\. We consider add\-dimensional stochastic linear bandit over a horizon ofTTrounds, and assume that the ideal action vectors lie in an unknownmm\-dimensional subspace ofℝd\\mathbb\{R\}^\{d\}\. Let𝐔∈ℝd×m\\mathbf\{U\}\\in\\mathbb\{R\}^\{d\\times m\}be an orthonormal basis for this unknown subspace\. At each roundt∈\[T\]t\\in\[T\], an ideal decision setDt:=\{Xt,1,…,Xt,K\}D\_\{t\}:=\\\{X\_\{t,1\},\\ldots,X\_\{t,K\}\\\}is generated, where the ideal action vectorsXt,iX\_\{t,i\}are drawn i\.i\.d\. \(across arms and rounds\) from a fixed distribution supported onspan​\(𝐔\)\\mathrm\{span\}\(\\mathbf\{U\}\)\. We make the following two assumptions on this distribution\.

###### Assumption 1\(Bounded actions\)

There is a known constantBXB\_\{X\}such that‖Xt,i‖2≤BX\\\|X\_\{t,i\}\\\|\_\{2\}\\leq B\_\{X\}almost surely for allt∈\[T\]t\\in\[T\]andi∈\[K\]i\\in\[K\]\.

###### Assumption 2\(Action covariance rank\)

The covariance matrixΣ:=𝔼​\[Xt,i​Xt,i⊤\]\\Sigma:=\\mathbb\{E\}\[X\_\{t,i\}X\_\{t,i\}^\{\\top\}\]has rankmm, and its nonzero eigenvalues satisfy

λ¯≥λ1≥⋯≥λm\>0,\\bar\{\\lambda\}\\;\\geq\\;\\lambda\_\{1\}\\;\\geq\\;\\cdots\\;\\geq\\;\\lambda\_\{m\}\\;\>\\;0,for some constantλ¯\>0\\bar\{\\lambda\}\>0\.

As is standard in the bandit literature\[[5](https://arxiv.org/html/2607.08971#bib.bib5),[27](https://arxiv.org/html/2607.08971#bib.bib27),[28](https://arxiv.org/html/2607.08971#bib.bib28),[29](https://arxiv.org/html/2607.08971#bib.bib29),[1](https://arxiv.org/html/2607.08971#bib.bib1)\], we will suppose thatBX=O​\(1\)B\_\{X\}=O\(1\)and is known by the algorithm\. Assumption[2](https://arxiv.org/html/2607.08971#Thmassumption2)says that the action distribution excites every direction of the latent subspace, and provides an envelope on its energy\. We note thatλ¯\\bar\{\\lambda\}need not be known: sinceλ1≤𝔼​‖Xt,i‖22≤BX2\\lambda\_\{1\}\\leq\\mathbb\{E\}\\\|X\_\{t,i\}\\\|\_\{2\}^\{2\}\\leq B\_\{X\}^\{2\}, one may always takeλ¯=BX2\\bar\{\\lambda\}=B\_\{X\}^\{2\}, and a sharper envelope only tightens our bounds\.

Example\.A natural setting satisfying these assumptions is the following loading\-matrix model

Xt,i=𝐔​𝚲​Zt,i,\\displaystyle X\_\{t,i\}=\\mathbf\{U\}\\mathbf\{\\Lambda\}Z\_\{t,i\},\(1\)where𝚲∈ℝm×m\\mathbf\{\\Lambda\}\\in\\mathbb\{R\}^\{m\\times m\}is a fixed diagonal loading matrix and the latent vectorsZt,i∈ℝmZ\_\{t,i\}\\in\\mathbb\{R\}^\{m\}are i\.i\.d\. with𝔼​\[Zt,i\]=0\\mathbb\{E\}\[Z\_\{t,i\}\]=0,𝔼​\[Zt,i​Zt,i⊤\]⪰ν​Im\\mathbb\{E\}\[Z\_\{t,i\}Z\_\{t,i\}^\{\\top\}\]\\succeq\\nu I\_\{m\}for someν\>0\\nu\>0, and‖Zt,i‖∞≤BZ\\\|Z\_\{t,i\}\\\|\_\{\\infty\}\\leq B\_\{Z\}almost surely\. This model satisfies Assumption[1](https://arxiv.org/html/2607.08971#Thmassumption1)withBX=maxj∈\[m\]⁡\|𝚲j​j\|​m​BZB\_\{X\}=\\max\_\{j\\in\[m\]\}\|\\mathbf\{\\Lambda\}\_\{jj\}\|\\sqrt\{m\}\\,B\_\{Z\}and Assumption[2](https://arxiv.org/html/2607.08971#Thmassumption2)withλm≥ν​minj∈\[m\]⁡𝚲j​j2\\lambda\_\{m\}\\geq\\nu\\min\_\{j\\in\[m\]\}\\mathbf\{\\Lambda\}\_\{jj\}^\{2\}\.

Conditioning of the action covariance\.It is important to note that the smallest eigenvalue of the action covariance matrix cannot be dimension\-free: sincem​λm≤tr⁡\(Σ\)=𝔼​‖Xt,i‖22≤BX2m\\lambda\_\{m\}\\leq\\operatorname\{tr\}\(\\Sigma\)=\\mathbb\{E\}\\\|X\_\{t,i\}\\\|\_\{2\}^\{2\}\\leq B\_\{X\}^\{2\}, we necessarily haveλm≤BX2/m\\lambda\_\{m\}\\leq B\_\{X\}^\{2\}/m\. As we will see below, our regret bounds depend on the spectrum through the quantity

κ:=BX​λ¯λm​m,\\kappa:=\\frac\{B\_\{X\}\\sqrt\{\\bar\{\\lambda\}\}\}\{\\lambda\_\{m\}\\sqrt\{m\}\},which measures the conditioning of the action covariance\. Sinceκ2=\(BX2m​λm\)​\(λ¯λm\)\\kappa^\{2\}=\\left\(\\tfrac\{B\_\{X\}^\{2\}\}\{m\\lambda\_\{m\}\}\\right\)\\left\(\\tfrac\{\\bar\{\\lambda\}\}\{\\lambda\_\{m\}\}\\right\), we always haveκ≥1\\kappa\\geq 1\. In the well\-conditioned regimeλm≍λ¯≍BX2/m\\lambda\_\{m\}\\asymp\\bar\{\\lambda\}\\asymp B\_\{X\}^\{2\}/m, i\.e\., when the spectrum is flat and the norm bound is tight on average, we haveκ=Θ​\(1\)\\kappa=\\Theta\(1\)

To reason about subspace recovery from partial observations, an important property is the*incoherence*of the subspace \(with respect to the canonical basis\)\. A coherent subspace may be extremely concentrated on a small set of coordinates and missing these coordinates would make learning impossible\. Incoherence assumptions are standard \(see e\.g\.,\[[21](https://arxiv.org/html/2607.08971#bib.bib21)\]\), and make restrictions on the incoherence parameter, which is defined as follows\.

###### Definition 1\(Incoherence\)

Let𝐔∈ℝd×m\\mathbf\{U\}\\in\\mathbb\{R\}^\{d\\times m\}be a matrix with orthonormal columns\. The incoherence parameter of𝐔\\mathbf\{U\}is defined asμ:=dm⋅maxj∈\[d\]⁡‖𝐔j,:‖2,\\mu:=\\sqrt\{\\frac\{d\}\{m\}\}\\cdot\\max\_\{j\\in\[d\]\}\\\|\\mathbf\{U\}\_\{j,:\}\\\|\_\{2\},where𝐔j,:∈ℝm\\mathbf\{U\}\_\{j,:\}\\in\\mathbb\{R\}^\{m\}denotes thejj\-th row of𝐔\\mathbf\{U\}\.

A smallμ\\mumeans that the subspace energy is spread evenly across coordinates, which is precisely the regime where missing observations still carry useful information about the latent subspace\.

Missingness and observation model\.The learner does not observe the ideal action vectors directly\. Instead, for each roundt∈\[T\]t\\in\[T\], armi∈\[K\]i\\in\[K\], and coordinatej∈\[d\]j\\in\[d\], we draw an observation indicatorst,i\(j\)∼i\.i\.d\.Bernoulli​\(p\)s\_\{t,i\}^\{\(j\)\}\\stackrel\{\{\\scriptstyle\\text\{i\.i\.d\.\}\}\}\{\{\\sim\}\}\\mathrm\{Bernoulli\}\(p\), independently across rounds, arms, and coordinates, and independently of the ideal action vectors\. WritingSt,i:=\(st,i\(1\),…,st,i\(d\)\)∈\{0,1\}dS\_\{t,i\}:=\(s\_\{t,i\}^\{\(1\)\},\\ldots,s\_\{t,i\}^\{\(d\)\}\)\\in\\\{0,1\\\}^\{d\}, the partially observed action vector is defined as

X˙t,i=St,i⊙Xt,i,\\dot\{X\}\_\{t,i\}=S\_\{t,i\}\\odot X\_\{t,i\},\(2\)where⊙\\odotdenotes entrywise multiplication\. Equivalently,X˙t,i\(j\)=Xt,i\(j\)​st,i\(j\)\\dot\{X\}\_\{t,i\}^\{\(j\)\}=X\_\{t,i\}^\{\(j\)\}s\_\{t,i\}^\{\(j\)\}for each coordinatejj\. We suppose that the learner observes the partially observed decision setD˙t:=\{X˙t,1,…,X˙t,K\}\\dot\{D\}\_\{t\}:=\\\{\\dot\{X\}\_\{t,1\},\\ldots,\\dot\{X\}\_\{t,K\}\\\}, and based on this set, it selects an actionX˙t∈D˙t\\dot\{X\}\_\{t\}\\in\\dot\{D\}\_\{t\}and the resulting reward isrt=Xt⊤​θ⋆\+ηt\.r\_\{t\}=X\_\{t\}^\{\\top\}\\theta^\{\\star\}\+\\eta\_\{t\}\.HereXtX\_\{t\}is the correspondingideal action,θ⋆∈span​\(𝐔\)\\theta^\{\\star\}\\in\\text\{span\}\(\\mathbf\{U\}\)is an unknown parameter vector with‖θ⋆‖2≤S\\\|\\theta^\{\\star\}\\\|\_\{2\}\\leq Sfor a known constantSS, which, as withBXB\_\{X\}, we treat asO​\(1\)O\(1\), andηt\\eta\_\{t\}is conditionallyRR\-sub\-Gaussian:𝔼​\[exp⁡\(λ​ηt\)∣ℱt−1\]≤exp⁡\(λ2​R22\)\\mathbb\{E\}\[\\exp\(\\lambda\\eta\_\{t\}\)\\mid\\mathcal\{F\}\_\{t\-1\}\]\\leq\\exp\\left\(\\frac\{\\lambda^\{2\}R^\{2\}\}\{2\}\\right\), for allλ∈ℝ\.\\lambda\\in\\mathbb\{R\}\.

Indeed our goal is to design a learning algorithm withsmall cumulative regret,

RT:=∑t=1T\(Xt⋆−Xt\)⊤​θ⋆,R\_\{T\}:=\\sum\_\{t=1\}^\{T\}\(X\_\{t\}^\{\\star\}\-X\_\{t\}\)^\{\\top\}\\theta^\{\\star\},whereXt⋆=arg⁡maxX∈Dt⁡X⊤​θ⋆X\_\{t\}^\{\\star\}=\\arg\\max\_\{X\\in D\_\{t\}\}X^\{\\top\}\\theta^\{\\star\}is the optimal ideal action at roundtt\.

## 4Our Algorithm:TOFU\-POV

In this section, we describeTOFU\-POV\(*Two\-phase OFUL with Partially Observed Vectors*\), our epoch\-wise algorithm for stochastic linear bandits with partially observed action features\. The algorithm takes as input a burn\-in lengthtbt\_\{b\}, regularizationλ\\lambda, subspace dimensionmm, and a burn\-in policyπburn\\pi\_\{\\rm burn\}\. The Algorithm[1](https://arxiv.org/html/2607.08971#alg1)display gives the formal pseudocode\. During burn\-in, the learner observes each masked decision set, plays according to a burn\-in policyπburn\\pi\_\{\\rm burn\}, and records the reward; in our experiments,πburn\\pi\_\{\\rm burn\}is taken to be standard OFUL where missing action vector coordinates are filled with zero\. After burn\-in, time is divided into epochs whose lengths double withτ0=tb\+1\\tau\_\{0\}=t\_\{b\}\+1,τe\+1=2​τe\\tau\_\{e\+1\}=2\\tau\_\{e\}\. In what follows, we let𝔗e:=\{τe,τe\+1,…,min⁡\(τe\+1−1,T\)\}\\mathfrak\{T\}\_\{e\}:=\\\{\\tau\_\{e\},\\tau\_\{e\}\+1,\\ldots,\\min\(\\tau\_\{e\+1\}\-1,T\)\\\}denote the time indices in theee\-th epoch\.

At the start of epochee, the learner estimates a subspace basis𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}as the top\-mmeigenvectors of a corrected covariance estimator \(Equation \([3](https://arxiv.org/html/2607.08971#S4.E3)\) below\) built from only decision sets observed beforeτe\\tau\_\{e\}, and then freezes this representation throughout the epoch\. For each roundt∈𝔗et\\in\\mathfrak\{T\}\_\{e\}, it imputes the currently offered arms using𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}, and then forms reduced features

zt,i=𝐔^e⊤​X^t,i∈ℝm\.z\_\{t,i\}=\\hat\{\\mathbf\{U\}\}\_\{e\}^\{\\top\}\\hat\{X\}\_\{t,i\}\\in\\mathbb\{R\}^\{m\}\.The learner then runs an OFUL policy inside the epoch using rewards collected earlier in the same epoch\. For notational ease, we letzs:=zs,isz\_\{s\}:=z\_\{s,i\_\{s\}\}denote the reduced feature vector of the arm played at roundss\. Notice that this epoch structure makes the representation predictable relative to the rewards used by OFUL\. Section[4\.1](https://arxiv.org/html/2607.08971#S4.SS1)controls the subspace estimation error, Section[4\.2](https://arxiv.org/html/2607.08971#S4.SS2)controls the imputation error, and Section[4\.3](https://arxiv.org/html/2607.08971#S4.SS3)converts these into epoch\-wise confidence sets for the frozen\-coordinate OFUL problem\. Section[5](https://arxiv.org/html/2607.08971#S5)then combines all of these ingredients into ourm​Tm\\sqrt\{T\}regret guarantee, and Section[6](https://arxiv.org/html/2607.08971#S6)gives the rank\-adaptive extension\. It is instructive to compare the repeated epoch updates with a simpler one\-shot two\-phase strategy that estimates the subspace once and then freezes it for the rest of the horizon\. In Appendix[B](https://arxiv.org/html/2607.08971#A2)we show that this would make the regret scale likeT2/3T^\{2/3\}instead ofT\\sqrt\{T\}\.

Algorithm 1TOFU\-POV: Two\-phase OFUL with Partially Observed Vectors1:Inputs: burn\-in length

tbt\_\{b\}, regularization

λ\\lambda, subspace dimension

mm, burn\-in policy

πburn\\pi\_\{\\rm burn\}
2:for

t=1,…,tbt=1,\\dots,t\_\{b\}do

3:Receive partially observed decision set

D˙t=\{X˙t,1,…,X˙t,K\}\\dot\{D\}\_\{t\}=\\\{\\dot\{X\}\_\{t,1\},\\dots,\\dot\{X\}\_\{t,K\}\\\}
4:Play

it=πburn​\(D˙1,r1,…,D˙t−1,rt−1,D˙t\)i\_\{t\}=\\pi\_\{\\rm burn\}\(\\dot\{D\}\_\{1\},r\_\{1\},\\ldots,\\dot\{D\}\_\{t\-1\},r\_\{t\-1\},\\dot\{D\}\_\{t\}\)and observe reward

rtr\_\{t\}
5:endfor

6:Set

τ0←tb\+1\\tau\_\{0\}\\leftarrow t\_\{b\}\+1and

τe\+1←2​τe,e=0,1,2,…\\tau\_\{e\+1\}\\leftarrow 2\\tau\_\{e\},e=0,1,2,\\ldots
7:forepochs

e=0,1,2,…e=0,1,2,\\dotsdo

8:Estimate subspace

𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}as the top\-

mmeigenvectors of

Σ˙τe−1\\dot\{\\Sigma\}\_\{\\tau\_\{e\}\-1\}in Equation \([3](https://arxiv.org/html/2607.08971#S4.E3)\)\[Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)\]

9:for

t∈𝔗et\\in\\mathfrak\{T\}\_\{e\}do

10:Receive partially observed decision set

D˙t=\{X˙t,1,…,X˙t,K\}\\dot\{D\}\_\{t\}=\\\{\\dot\{X\}\_\{t,1\},\\dots,\\dot\{X\}\_\{t,K\}\\\}
11:Impute each arm using the frozen subspace to obtain

X^t,i\\hat\{X\}\_\{t,i\}\[Lemma[2](https://arxiv.org/html/2607.08971#Thmlemma2)\]

12:Form features

zt,i←𝐔^e⊤​X^t,i∈ℝmz\_\{t,i\}\\leftarrow\\hat\{\\mathbf\{U\}\}\_\{e\}^\{\\top\}\\hat\{X\}\_\{t,i\}\\in\\mathbb\{R\}^\{m\}\[Sec\.[4\.3](https://arxiv.org/html/2607.08971#S4.SS3)\]

13:

Ve,t←λ​Im\+∑s∈𝔗es<tzs​zs⊤,ϑ^e,t←Ve,t−1​∑s∈𝔗es<tzs​rsV\_\{e,t\}\\leftarrow\\lambda I\_\{m\}\+\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}z\_\{s\}^\{\\top\},\\qquad\\hat\{\\vartheta\}\_\{e,t\}\\leftarrow V\_\{e,t\}^\{\-1\}\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}r\_\{s\}\[Thm\.[1](https://arxiv.org/html/2607.08971#Thmtheorem1)\]

14:Choose

it∈arg⁡maxi∈\[K\]⁡\{⟨zt,i,ϑ^e,t⟩\+βe,t​‖zt,i‖Ve,t−1\}i\_\{t\}\\in\\arg\\max\_\{i\\in\[K\]\}\\left\\\{\\langle z\_\{t,i\},\\hat\{\\vartheta\}\_\{e,t\}\\rangle\+\\beta\_\{e,t\}\\\|z\_\{t,i\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}\\;\\right\\\}\[Thm\.[1](https://arxiv.org/html/2607.08971#Thmtheorem1)\]

15:Play arm

iti\_\{t\}and observe reward

rtr\_\{t\}
16:endfor

17:endfor

Practical implementation\.Algorithm[1](https://arxiv.org/html/2607.08971#alg1)is the conservative version used in our regret analysis below\. In some of our experiments, we make a natural data\-reuse modification: at the start of each epochee, after computing𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}, we re\-impute every previously played arm using the frozen representation𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}\. We then form the corresponding coordinateszs\(e\)z\_\{s\}^\{\(e\)\}, and initialize the epoch design matrix and response vector with all past reward observations \(with newly imputed actions\)\. Analyzing this “warm\-start” variant requires handling the dependence between the design matrix and𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}\. We expect this can be done by a careful self\-normalized confidence argument over a neighborhood of the true subspace, or by a sample\-splitting construction; we leave a formal regret analysis to future work\.

### 4\.1Estimating the Subspace from Partial Observations

If the action vectors were fully observed, the low\-dimensional subspace could be estimated by applying PCA to their empirical covariance\. Under partial observation, the naive covariance of the masked vectors is biased: unlike diagonal entries, off\-diagonal entries are observed only when two coordinates are simultaneously revealed\. We therefore use an inverse\-probability correction that treats diagonal and off\-diagonal entries differently\. Fort≥1t\\geq 1, our corrected estimator, which uses all partially observed vectors offered in all previous rounds is given as follows:

Σ˙t:=1t​K​∑s=1t∑i=1K\[1p2​X˙s,i​X˙s,i⊤\+\(1p−1p2\)​diag​\(X˙s,i​X˙s,i⊤\)\]\.\\displaystyle\\dot\{\\Sigma\}\_\{t\}:=\\frac\{1\}\{tK\}\\sum\_\{s=1\}^\{t\}\\sum\_\{i=1\}^\{K\}\\left\[\\frac\{1\}\{p^\{2\}\}\\dot\{X\}\_\{s,i\}\\dot\{X\}\_\{s,i\}^\{\\top\}\+\\left\(\\frac\{1\}\{p\}\-\\frac\{1\}\{p^\{2\}\}\\right\)\\mathrm\{diag\}\\bigl\(\\dot\{X\}\_\{s,i\}\\dot\{X\}\_\{s,i\}^\{\\top\}\\bigr\)\\right\]\.\(3\)Note that this estimator includes arms that were not played as well and that the offered arms are i\.i\.d\. and independent of the learner’s policy\. These estimators appear in the matrix completion and missing\-data covariance estimation literature \(see e\.g\.,\[[21](https://arxiv.org/html/2607.08971#bib.bib21),[30](https://arxiv.org/html/2607.08971#bib.bib30)\]\)\. Let𝐔^t∈ℝd×m\\hat\{\\mathbf\{U\}\}\_\{t\}\\in\\mathbb\{R\}^\{d\\times m\}denote the matrix of top\-mmeigenvectors ofΣ˙t\\dot\{\\Sigma\}\_\{t\}, and let𝐏^t:=𝐔^t​𝐔^t⊤,𝐏:=𝐔𝐔⊤\\hat\{\\mathbf\{P\}\}\_\{t\}:=\\hat\{\\mathbf\{U\}\}\_\{t\}\\hat\{\\mathbf\{U\}\}\_\{t\}^\{\\top\},\\quad\\mathbf\{P\}:=\\mathbf\{U\}\\mathbf\{U\}^\{\\top\}be the estimated and true projection matrices\. A standard way to measure subspace error is

dist​\(𝐔^t,𝐔\):=‖\(I−𝐔^t​𝐔^t⊤\)​𝐔‖2=‖\(I−𝐔𝐔⊤\)​𝐔^t‖2,\\mathrm\{dist\}\(\\hat\{\\mathbf\{U\}\}\_\{t\},\\mathbf\{U\}\):=\\\|\(I\-\\hat\{\\mathbf\{U\}\}\_\{t\}\\hat\{\\mathbf\{U\}\}\_\{t\}^\{\\top\}\)\\mathbf\{U\}\\\|\_\{2\}=\\left\\\|\(I\-\\mathbf\{U\}\\mathbf\{U\}^\{\\top\}\)\\hat\{\\mathbf\{U\}\}\_\{t\}\\right\\\|\_\{2\},which equals the sine of the largest principal angle between the estimated and true subspaces\. Since both projectors have rankmm, this quantity also coincides with the projector distance‖𝐏^t−𝐏‖2\\\|\\hat\{\\mathbf\{P\}\}\_\{t\}\-\\mathbf\{P\}\\\|\_\{2\}, and we work with the latter in what follows\.

###### Lemma 1\(Subspace recovery from partial observations\)

Suppose Assumptions[1](https://arxiv.org/html/2607.08971#Thmassumption1)and[2](https://arxiv.org/html/2607.08971#Thmassumption2)hold, and consider the Bernoulli missingness model in Equation \([2](https://arxiv.org/html/2607.08971#S3.E2)\)\. Then, with probability at least1−δ1\-\\delta, simultaneously for allt∈\[T\]t\\in\[T\],

‖𝐏^t−𝐏‖2≤ϵt,ϵt:=Csub​κp​mt​K​log⁡\(2​d​Tδ\)\.\\displaystyle\{\\left\\\|\\hat\{\\mathbf\{P\}\}\_\{t\}\-\\mathbf\{P\}\\right\\\|\_\{2\}\\leq\\epsilon\_\{t\},\}\\qquad\\epsilon\_\{t\}:=C\_\{\\mathrm\{sub\}\}\\frac\{\\kappa\}\{p\}\\sqrt\{\\frac\{m\}\{tK\}\\log\\\!\\left\(\\frac\{2dT\}\{\\delta\}\\right\)\}\.\(4\)HereCsub\>0C\_\{\\mathrm\{sub\}\}\>0is a universal numerical constant, andκ\\kappais the action subspace conditioning constant from Section[3](https://arxiv.org/html/2607.08971#S3)\. Equivalently, suppressing constants and logarithms, this is the rateϵt=O~​\(κp​mt​K\)\.\\epsilon\_\{t\}=\\widetilde\{O\}\\\!\\left\(\\frac\{\\kappa\}\{p\}\\sqrt\{\\frac\{m\}\{tK\}\}\\right\)\.

Proof sketch\.We begin by establishing thatΣ˙t\\dot\{\\Sigma\}\_\{t\}is unbiased, and we then show that we can control‖Σ˙t−Σ‖2\\\|\\dot\{\\Sigma\}\_\{t\}\-\\Sigma\\\|\_\{2\}using a matrix Bernstein bound with a variance proxy of orderBX2​λ¯/p2B\_\{X\}^\{2\}\\bar\{\\lambda\}/p^\{2\}\. Since themm\-th eigenvalue ofΣ\\Sigmaisλm\\lambda\_\{m\}and the\(m\+1\)\(m\{\+\}1\)\-st is zero, we may then invoke the Davis–Kahansin⁡Θ\\sin\\Thetatheorem\[[31](https://arxiv.org/html/2607.08971#bib.bib31)\]to convert this covariance error into the subspace bound in \([4](https://arxiv.org/html/2607.08971#S4.E4)\), provided the covariance error is belowλm/2\\lambda\_\{m\}/2; for the \(early\) rounds where this fails,ϵt\\epsilon\_\{t\}exceeds a universal constant and the bound holds trivially since‖𝐏^t−𝐏‖2≤1\\\|\\hat\{\\mathbf\{P\}\}\_\{t\}\-\\mathbf\{P\}\\\|\_\{2\}\\leq 1\. Full details are in Appendix[C](https://arxiv.org/html/2607.08971#A3)\.

Burn\-in period\.The least\-squares imputation step below requires the observed rows of the estimated subspace to be well\-conditioned\. Since the true observed Gram matrix concentrates aroundp​ImpI\_\{m\}, it is sufficient for the projection error to be a small constant multiple ofpp; we use the convenient conditionϵt≤p/32\\epsilon\_\{t\}\\leq p/32\. Solving Equation \([4](https://arxiv.org/html/2607.08971#S4.E4)\) for this condition gives us the burn\-in length

tb:=⌈Cb​κ2​mp4​K​log⁡\(2​d​Tδ\)⌉,\\displaystyle t\_\{b\}:=\\left\\lceil C\_\{b\}\\,\\frac\{\\kappa^\{2\}m\}\{p^\{4\}K\}\\log\\\!\\left\(\\frac\{2dT\}\{\\delta\}\\right\)\\right\\rceil,\(5\)for a universal constantCb\>0C\_\{b\}\>0\(Cb=\(32​Csub\)2C\_\{b\}=\(32C\_\{\\mathrm\{sub\}\}\)^\{2\}suffices\)\. That is, the algorithm needs to wait for this number of rounds before the imputation step starts helping\. Thep−4p^\{\-4\}dependence here reflects our specific technique; improving this dependence is an interesting direction for future work\. However, this term is anadditive burn\-in cost\. For fixed problem parameters, it does not scale withTTexcept through logarithmic factors\.

### 4\.2Imputing the Partially Observed Actions and Controlling Errors

Once the subspace is estimated, the observed coordinates of the action vectors are used to impute the hidden ones using least squares\. The key technical step here that allows us to control the quality of the imputation is to ensure that the observed rows of the estimated basis contains enough information about every latent direction\. Under incoherence and sufficient observations \(generated via a Bernoulli mask\), we show that the true observed Gram matrix is well\-conditioned at scalepp\. And, after burn\-in, the estimated projector is close enough to the true projector that the estimated observed Gram matrix remains well\-conditioned\.

An important subtlety \(for our analysis\) here is that the algorithm imputes with the*frozen*epoch basis: within epochee, every arm is reconstructed using𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}, which is computed from theτe−1\\tau\_\{e\}\-1rounds preceding the epoch and is not updated as the epoch progresses\. Fix an epochee, a roundt∈𝔗et\\in\\mathfrak\{T\}\_\{e\}, and an armii\. LetΩt,i⊂\[d\]\\Omega\_\{t,i\}\\subset\[d\]be the observed coordinates, and for a matrixA∈ℝd×mA\\in\\mathbb\{R\}^\{d\\times m\}writeAΩt,iA\_\{\\Omega\_\{t,i\}\}for the submatrix formed by selecting rows inΩt,i\\Omega\_\{t,i\}\. Given the frozen basis𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}, the imputed actionX^t,i\\hat\{X\}\_\{t,i\}keeps the observed entries unchanged and fills in the missing entries as follows \(since the epochs partition the horizon, the round index determines the epoch, and we leave the epoch implicit inX^t,i\\hat\{X\}\_\{t,i\}; we use an epoch superscript only where an action is re\-imputed under a different epoch’s basis, as in the practical variant discussed after Algorithm[1](https://arxiv.org/html/2607.08971#alg1)\)

\(X^t,i\)\(Ωt,ic\):=𝐔^e,Ωt,ic​a^t,i,where​a^t,i:=\(𝐔^e,Ωt,i⊤​𝐔^e,Ωt,i\)−1​𝐔^e,Ωt,i⊤​Xt,i\(Ωt,i\)\.\\displaystyle\\big\(\\hat\{X\}\_\{t,i\}\\big\)^\{\(\\Omega\_\{t,i\}^\{c\}\)\}:=\\hat\{\\mathbf\{U\}\}\_\{e,\\Omega\_\{t,i\}^\{c\}\}\\hat\{a\}\_\{t,i\},\\;\\mbox\{where \}\\hat\{a\}\_\{t,i\}:=\(\\hat\{\\mathbf\{U\}\}\_\{e,\\Omega\_\{t,i\}\}^\{\\top\}\\hat\{\\mathbf\{U\}\}\_\{e,\\Omega\_\{t,i\}\}\)^\{\-1\}\\hat\{\\mathbf\{U\}\}\_\{e,\\Omega\_\{t,i\}\}^\{\\top\}X\_\{t,i\}^\{\(\\Omega\_\{t,i\}\)\}\.\(6\)
###### Lemma 2\(Uniform imputation error\)

Assume the conditions of Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)\. Assume also that the true subspace isμ\\mu\-incoherent \(as in Definition[1](https://arxiv.org/html/2607.08971#Thmdefinition1)\), and that

p≥Cμ​μ2​md​log⁡\(m​T​Kδ\)\\displaystyle p\\geq C\_\{\\mu\}\\frac\{\\mu^\{2\}m\}\{d\}\\log\\\!\\left\(\\frac\{mTK\}\{\\delta\}\\right\)\(7\)for a sufficiently large universal constantCμ\>0C\_\{\\mu\}\>0\. Lettbt\_\{b\}be as in Equation \([5](https://arxiv.org/html/2607.08971#S4.E5)\), and writeϵe:=ϵτe−1\\epsilon\_\{e\}:=\\epsilon\_\{\\tau\_\{e\}\-1\}for the subspace error bound of Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)at the start of epochee\. Then, with probability at least1−3​δ1\-3\\delta, for every epochee, everyt∈𝔗et\\in\\mathfrak\{T\}\_\{e\}, and everyi∈\[K\]i\\in\[K\],‖Xt,i−X^t,i‖2≤\(1\+2p\)​BX​ϵe\.\\\|X\_\{t,i\}\-\\hat\{X\}\_\{t,i\}\\\|\_\{2\}\\leq\\left\(1\+\\frac\{2\}\{p\}\\right\)B\_\{X\}\\epsilon\_\{e\}\.

Proof sketch\.We first show, via a matrix Chernoff bound, that the true observed Gram matrix satisfies𝐔Ωt,i⊤​𝐔Ωt,i≳p​Im\\mathbf\{U\}\_\{\\Omega\_\{t,i\}\}^\{\\top\}\\mathbf\{U\}\_\{\\Omega\_\{t,i\}\}\\gtrsim pI\_\{m\}\. Sinceτe−1≥tb\\tau\_\{e\}\-1\\geq t\_\{b\}, we can then transfer this conditioning to the frozen estimated Gram matrices\. A technical detail here is that Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)controls only the subspace distance, while the error analysis compares the matrices𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}and𝐔\\mathbf\{U\}directly, and the latter is not determined by the former \(𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}is only defined up to a right rotation\)\. However, the imputed vector is invariant under such rotations, so we may analyze the best\-aligned basis, whose distance to𝐔\\mathbf\{U\}is at mostϵe\\epsilon\_\{e\}\(see Corollary[1](https://arxiv.org/html/2607.08971#Thmcorollary1); the2\\sqrt\{2\}factor from basis alignment is absorbed into the constantCsubC\_\{\\mathrm\{sub\}\}\)\. We then decompose the least\-squares imputation error into the subspace perturbation plus the induced coefficient error, the latter amplified by the inverse Gram matrix \(whose norm is2/p2/p\); combining the pieces gives the result\. The complete proof is in Appendix[D](https://arxiv.org/html/2607.08971#A4)\.

Epoch\-wise representation event\.For a target representation failure probabilityδrep\\delta\_\{\\rm rep\}, letℰrep\\mathcal\{E\}\_\{\\rm rep\}denote the event on which the subspace recovery guarantee in Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)\(simultaneously for allt∈\[T\]t\\in\[T\]\) and the epoch\-wise imputation guarantee in Lemma[2](https://arxiv.org/html/2607.08971#Thmlemma2)\(simultaneously for all epochsee, roundst∈𝔗et\\in\\mathfrak\{T\}\_\{e\}, and arms\) hold, with both lemmas invoked at confidence parameterδrep/4\\delta\_\{\\rm rep\}/4\. By the two lemmas and a union bound,ℙ​\(ℰrep\)≥1−δrep\\mathbb\{P\}\(\\mathcal\{E\}\_\{\\rm rep\}\)\\geq 1\-\\delta\_\{\\rm rep\}\.

### 4\.3Epoch\-wise Surrogate Model and Estimation Error

We now fix an epocheeand analyze the bandit problem induced by the frozen representation𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}\. For each roundt∈𝔗et\\in\\mathfrak\{T\}\_\{e\}and armi∈\[K\]i\\in\[K\], define

zt,i:=𝐔^e⊤​X^t,i∈ℝm,ϑe⋆:=𝐔^e⊤​θ⋆∈ℝm,z\_\{t,i\}:=\\hat\{\\mathbf\{U\}\}\_\{e\}^\{\\top\}\\hat\{X\}\_\{t,i\}\\in\\mathbb\{R\}^\{m\},\\qquad\\vartheta\_\{e\}^\{\\star\}:=\\hat\{\\mathbf\{U\}\}\_\{e\}^\{\\top\}\\theta^\{\\star\}\\in\\mathbb\{R\}^\{m\},whereX^t,i\\hat\{X\}\_\{t,i\}is the epoch\-eeimputed action from Section[4\.2](https://arxiv.org/html/2607.08971#S4.SS2)\. We also write

μ¯t,i:=⟨zt,i,ϑe⋆⟩,be:=S​BX​\(2\+2p\)​ϵe,\\bar\{\\mu\}\_\{t,i\}:=\\langle z\_\{t,i\},\\vartheta\_\{e\}^\{\\star\}\\rangle,\\qquad b\_\{e\}:=SB\_\{X\}\\left\(2\+\\frac\{2\}\{p\}\\right\)\\epsilon\_\{e\},whereϵe:=ϵτe−1\\epsilon\_\{e\}:=\\epsilon\_\{\\tau\_\{e\}\-1\}is the subspace error bound of Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)at the start of epochee, as in Lemma[2](https://arxiv.org/html/2607.08971#Thmlemma2)\.

The quantityμ¯t,i\\bar\{\\mu\}\_\{t,i\}is the surrogate mean inside the epoch\. Indeed, it is only an approximation of the true mean⟨Xt,i,θ⋆⟩\\langle X\_\{t,i\},\\theta^\{\\star\}\\rangle, since the learner works with an estimated subspace and imputed actions rather than the true action vectors\. Our first lemma shows that, on the representation event, this approximation error is uniformly controlled bybeb\_\{e\}\.

###### Lemma 3\(Surrogate approximation inside epoch\)

On the eventℰrep\\mathcal\{E\}\_\{\\rm rep\}defined in Section[2](https://arxiv.org/html/2607.08971#Thmlemma2), for every epochee, every roundt∈𝔗et\\in\\mathfrak\{T\}\_\{e\}, and every armi∈\[K\]i\\in\[K\],

\|⟨Xt,i,θ⋆⟩−μ¯t,i\|≤be\.\\left\|\\langle X\_\{t,i\},\\theta^\{\\star\}\\rangle\-\\bar\{\\mu\}\_\{t,i\}\\right\|\\leq b\_\{e\}\.

The proof is in Appendix[E](https://arxiv.org/html/2607.08971#A5)\. An immediate consequence is that, onℰrep\\mathcal\{E\}\_\{\\rm rep\}, the learner faces an ordinarymm\-dimensional linear bandit with bounded misspecification inside each epoch\. Recalling thatzt:=zt,itz\_\{t\}:=z\_\{t,i\_\{t\}\}denotes, for notational convenience, the reduced feature of the arm played at roundt∈𝔗et\\in\\mathfrak\{T\}\_\{e\}, the observed reward satisfies

rt=⟨zt,ϑe⋆⟩\+ξt\+ηt,ξt:=⟨Xt,it,θ⋆⟩−μ¯t,it,\|ξt\|≤be,r\_\{t\}=\\langle z\_\{t\},\\vartheta\_\{e\}^\{\\star\}\\rangle\+\\xi\_\{t\}\+\\eta\_\{t\},\\qquad\\xi\_\{t\}:=\\langle X\_\{t,i\_\{t\}\},\\theta^\{\\star\}\\rangle\-\\bar\{\\mu\}\_\{t,i\_\{t\}\},\\qquad\|\\xi\_\{t\}\|\\leq b\_\{e\},whereηt\\eta\_\{t\}is the reward noise from Section[3](https://arxiv.org/html/2607.08971#S3)\.

Our next goal is an OFUL\-style confidence set for the estimation of the surrogate parameterϑe⋆\\vartheta\_\{e\}^\{\\star\}, centered at the epoch’s estimatorϑ^e,t\\hat\{\\vartheta\}\_\{e,t\}from Algorithm[1](https://arxiv.org/html/2607.08971#alg1)\. Indeed, as in standard OFUL, this is the object that drives the optimistic arm selection\. Notice that one may substitute the reward decomposition above into the definition ofϑ^e,t\\hat\{\\vartheta\}\_\{e,t\}to get:

Ve,t​\(ϑ^e,t−ϑe⋆\)=−λ​ϑe⋆\+∑s∈𝔗es<tzs​ξs\+∑s∈𝔗es<tzs​ηs,\\displaystyle V\_\{e,t\}\\left\(\\hat\{\\vartheta\}\_\{e,t\}\-\\vartheta\_\{e\}^\{\\star\}\\right\)=\-\\lambda\\vartheta\_\{e\}^\{\\star\}\+\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}\\xi\_\{s\}\+\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}\\eta\_\{s\},\(8\)where, we recall that the design matrixVe,t=λ​Im\+∑s∈𝔗e,s<tzs​zs⊤V\_\{e,t\}=\\lambda I\_\{m\}\+\\sum\_\{s\\in\\mathfrak\{T\}\_\{e\},\\,s<t\}z\_\{s\}z\_\{s\}^\{\\top\}\. The first term is the usual \(ridge regularization\) bias\. The second term∑s<tzs​ηs\\sum\_\{s<t\}z\_\{s\}\\eta\_\{s\}is handled exactly as in standard OFUL\. It is worth noting that since the feature map is frozen over the epoch, eachzsz\_\{s\}is predictable \(measurable given the history before the rewardrsr\_\{s\}is revealed\), and therefore the standard self\-normalized inequality machinery applies\. The third term∑s<tzs​ξs\\sum\_\{s<t\}z\_\{s\}\\xi\_\{s\}is specific to our setting, and the next lemma shows how we control it\.

###### Lemma 4\(Misspecification control inside an epoch\)

Fix an epocheeand suppose that for all roundss∈𝔗es\\in\\mathfrak\{T\}\_\{e\},rs=⟨zs,ϑe⋆⟩\+ξs\+ηs,\|ξs\|≤be,r\_\{s\}=\\langle z\_\{s\},\\vartheta\_\{e\}^\{\\star\}\\rangle\+\\xi\_\{s\}\+\\eta\_\{s\},\\;\|\\xi\_\{s\}\|\\leq b\_\{e\},whereηs\\eta\_\{s\}is conditionallyRR\-sub\-Gaussian\. Then for everyt∈𝔗et\\in\\mathfrak\{T\}\_\{e\},‖∑s∈𝔗es<tzs​ξs‖Ve,t−1≤be​t−τe,\\left\\\|\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}\\xi\_\{s\}\\right\\\|\_\{V\_\{e,t\}^\{\-1\}\}\\leq b\_\{e\}\\sqrt\{t\-\\tau\_\{e\}\},whereVe,t:=λ​Im\+∑s∈𝔗e,s<tzs​zs⊤V\_\{e,t\}:=\\lambda I\_\{m\}\+\\sum\_\{s\\in\\mathfrak\{T\}\_\{e\},\\,s<t\}z\_\{s\}z\_\{s\}^\{\\top\}\.

A naive triangle\-inequality argument here would bound the vectorszs​ξsz\_\{s\}\\xi\_\{s\}one at a time and we will have to pay a price through their leverage scores\. While this is valid, it ignores that the same featureszsz\_\{s\}also buildVe,tV\_\{e,t\}, and therefore loses an extra factor on the order ofm\\sqrt\{m\}\. We highlight the suboptimality of this approach in Appendix[G](https://arxiv.org/html/2607.08971#A7)\. We instead pursue a sharper argument \(detailed in Appendix[E](https://arxiv.org/html/2607.08971#A5)\) that keeps the misspecification aggregated\. With both error sources under control, we can now state the confidence set guarantee for the surrogate parameterϑe⋆\\vartheta\_\{e\}^\{\\star\}\.

###### Theorem 1\(Surrogate estimation error inside epoch\)

Fix an epocheeand a confidence levelδe∈\(0,1\)\\delta\_\{e\}\\in\(0,1\)\. Fort∈𝔗et\\in\\mathfrak\{T\}\_\{e\}, define

Ve,t:=λ​Im\+∑s∈𝔗es<tzs​zs⊤,ϑ^e,t:=Ve,t−1​∑s∈𝔗es<tzs​rs\.V\_\{e,t\}:=\\lambda I\_\{m\}\+\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}z\_\{s\}^\{\\top\},\\qquad\\hat\{\\vartheta\}\_\{e,t\}:=V\_\{e,t\}^\{\-1\}\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}r\_\{s\}\.Then, on the representation eventℰrep\\mathcal\{E\}\_\{\\rm rep\}, with probability at least1−δe1\-\\delta\_\{e\}, the surrogate parameterϑe⋆\\vartheta\_\{e\}^\{\\star\}satisfies, simultaneously for allt∈𝔗et\\in\\mathfrak\{T\}\_\{e\},

‖ϑ^e,t−ϑe⋆‖Ve,t≤βe,t:=λ​S\+R​2​log⁡\(det\(Ve,t\)1/2det\(λ​Im\)1/2​δe\)\+be​t−τe\.\\\|\\hat\{\\vartheta\}\_\{e,t\}\-\\vartheta\_\{e\}^\{\\star\}\\\|\_\{V\_\{e,t\}\}\\leq\\beta\_\{e,t\}:=\\sqrt\{\\lambda\}S\+R\\sqrt\{2\\log\\\!\\left\(\\frac\{\\det\(V\_\{e,t\}\)^\{1/2\}\}\{\\det\(\\lambda I\_\{m\}\)^\{1/2\}\\delta\_\{e\}\}\\right\)\}\+b\_\{e\}\\sqrt\{t\-\\tau\_\{e\}\}\.

We writeℰconf,e\\mathcal\{E\}\_\{\{\\rm conf\},e\}for the event in Theorem[1](https://arxiv.org/html/2607.08971#Thmtheorem1)on which the above bound holds simultaneously for allt∈𝔗et\\in\\mathfrak\{T\}\_\{e\}\. Conditional onℰrep\\mathcal\{E\}\_\{\\rm rep\}, this event has probability at least1−δe1\-\\delta\_\{e\}\.

Proof sketch\.We takeVe,t−1V\_\{e,t\}^\{\-1\}\-weighted norms in the error decomposition of \([8](https://arxiv.org/html/2607.08971#S4.E8)\), and we bound its three terms by the three terms ofβe,t\\beta\_\{e,t\}, respectively: the regularization term using‖ϑe⋆‖2≤S\\\|\\vartheta\_\{e\}^\{\\star\}\\\|\_\{2\}\\leq S, the stochastic term via the self\-normalized inequality \(as discussed above\), and the misspecification term via Lemma[4](https://arxiv.org/html/2607.08971#Thmlemma4)\. The full proof is in Appendix[E](https://arxiv.org/html/2607.08971#A5)\.

Consequently, for every candidate arm with epoch\-eesurrogate featurezz, Theorem[1](https://arxiv.org/html/2607.08971#Thmtheorem1)implies\|⟨z,ϑ^e,t−ϑe⋆⟩\|≤βe,t​‖z‖Ve,t−1\.\|\\langle z,\\hat\{\\vartheta\}\_\{e,t\}\-\\vartheta\_\{e\}^\{\\star\}\\rangle\|\\leq\\beta\_\{e,t\}\\\|z\\\|\_\{V\_\{e,t\}^\{\-1\}\}\.Thus the optimistic score used in Algorithm[1](https://arxiv.org/html/2607.08971#alg1)is the upper confidence bound induced by the frozen\-epoch estimation guarantee\. The next section converts this surrogate optimism into a true\-regret bound by combining the surrogate approximation error with Theorem[1](https://arxiv.org/html/2607.08971#Thmtheorem1)\.

## 5Regret Analysis

The preceding section reduces the post\-burn\-in analysis to a sequence of fixed\-coordinate OFUL problems, one for each frozen epoch, with an additional approximation error from subspace estimation and imputation\. The regret proof combines these two effects\. Within an epoch, we get the OFUL\-style control \(in dimensionmm\)\. Across epochs, the doubling schedule makes the shrinking representation error \(which avoids an extra factor ofm\\sqrt\{m\}as discussed above\) summable at theT\\sqrt\{T\}scale\. We now state the resulting bound\.

###### Theorem 2\(TOFU\-POVregret\)

Fix a confidence parameterδ∈\(0,1\)\\delta\\in\(0,1\)and setδrep:=δ/2\\delta\_\{\\rm rep\}:=\\delta/2\. Suppose Assumptions[1](https://arxiv.org/html/2607.08971#Thmassumption1)and[2](https://arxiv.org/html/2607.08971#Thmassumption2)hold, the true subspace isμ\\mu\-incoherent, andppsatisfies Equation \([7](https://arxiv.org/html/2607.08971#S4.E7)\) withδrep/4\\delta\_\{\\rm rep\}/4in place ofδ\\delta\. Choose the burn\-in timetbt\_\{b\}as in Equation \([5](https://arxiv.org/html/2607.08971#S4.E5)\) withδrep/4\\delta\_\{\\rm rep\}/4in place ofδ\\delta, and set the regularization toλ:=4​BX2\\lambda:=4B\_\{X\}^\{2\}\. LetE:=max⁡\{e:𝔗e≠∅\}E:=\\max\\\{e:\\mathfrak\{T\}\_\{e\}\\neq\\emptyset\\\}denote the final epoch index \(sinceτe\+1=2​τe\\tau\_\{e\+1\}=2\\tau\_\{e\},E≤⌈log2⁡T⌉\+1E\\leq\\lceil\\log\_\{2\}T\\rceil\+1\), and setδe:=δ/\(2​\(E\+1\)\)\\delta\_\{e\}:=\\delta/\(2\(E\+1\)\)\.

Then, with probability at least1−δ1\-\\delta, the epoch\-wise algorithm satisfies

RT≤O~​\(S​BX​κ2​mp4​K\)\+O~​\(\(R\+S​BX\)​m​T\)\+O~​\(S​BX​κ​m​Tp2​K\)\.R\_\{T\}\\leq\\widetilde\{O\}\\\!\\left\(SB\_\{X\}\\frac\{\\kappa^\{2\}m\}\{p^\{4\}K\}\\right\)\+\\widetilde\{O\}\\\!\\left\(\(R\+SB\_\{X\}\)\\,m\\sqrt\{T\}\\right\)\+\\widetilde\{O\}\\\!\\left\(SB\_\{X\}\\frac\{\\kappa m\\sqrt\{T\}\}\{p^\{2\}\\sqrt\{K\}\}\\right\)\.

Interpretation\.The first term in this bound is theadditiveburn\-in cost needed for the imputation step to be stable\. The second term is the usual stochastic linear bandit regret, which scales inmmgiven how we set the intra\-epoch linear bandit problem up\. The third term is the cost of learning and using the representation from partially observed actions\. Its1/K1/\\sqrt\{K\}dependence reflects that allKKdisplayed arms contribute to subspace estimation, while only one arm is played for reward\. Under the standard normalizationBX,S,R=O​\(1\)B\_\{X\},S,R=O\(1\), the bound reveals the scalingO~​\(κ2​m/\(p4​K\)\+m​T\+κ​m​T/\(p2​K\)\)\\widetilde\{O\}\\big\(\\kappa^\{2\}m/\(p^\{4\}K\)\+m\\sqrt\{T\}\+\\kappa m\\sqrt\{T\}/\(p^\{2\}\\sqrt\{K\}\)\\big\)\. In particular, for well\-conditioned action covariances \(κ=Θ​\(1\)\\kappa=\\Theta\(1\)\), the regret bound forTOFU\-POVscales asO~​\(m​T\)\\widetilde\{O\}\(m\\sqrt\{T\}\)\.

Proof sketch\.The full proof is in Appendix[F](https://arxiv.org/html/2607.08971#A6)\. At a high level, the representation event lets us compare the true rewards to the frozen surrogate model, costing us a per\-epoch biasbeb\_\{e\}\. Conditional on this event, the confidence set and optimistic action choice give us the usual OFUL regret term inside each epoch\. Because the coordinates are frozen, we can control the accumulated uncertainty by the standard elliptical\-potential argument in the fixedmm\-dimensional coordinates \(Lemma[12](https://arxiv.org/html/2607.08971#Thmlemma12)in Appendix[F](https://arxiv.org/html/2607.08971#A6)\); the choiceλ=4​BX2\\lambda=4B\_\{X\}^\{2\}suffices for this argument since the frozen features have norm at most2​BX2B\_\{X\}after burn\-in\. The remaining cost is the surrogate approximation error\. Since the epochs double andbeb\_\{e\}decays like1/τe1/\\sqrt\{\\tau\_\{e\}\}, these approximation errors also sum at theT\\sqrt\{T\}scale\. Combining the OFUL and approximation contributions over all epochs gives us the bound above\. In the next section, we show how to extend the method to unknownmm\.

## 6Adaptivity to Unknown Subspace Dimension

The epoch\-wise algorithm described above assumes that the latent dimensionalitymm, or equivalently, the rank of the action covarianceΣ\\Sigma\(Assumption[2](https://arxiv.org/html/2607.08971#Thmassumption2)\), is known\. This assumption can be removed by implementing a thresholding procedure on the eigenspectrum of the estimated covariances\. That is, the learner estimates the spectrum of the corrected covariance from all decision sets observed beforeτe\\tau\_\{e\}, keeps the empirical eigenvectors whose eigenvalues clear a confidence threshold, and then runs the same frozen\-coordinate OFUL procedure in the selected dimension\. We call this variantRank\-AdaptiveTOFU\-POV\.

Concretely, letλ^t,1≥⋯≥λ^t,d\\hat\{\\lambda\}\_\{t,1\}\\geq\\cdots\\geq\\hat\{\\lambda\}\_\{t,d\}denote the eigenvalues of the corrected covariance estimatorΣ˙t\\dot\{\\Sigma\}\_\{t\}from Equation \([3](https://arxiv.org/html/2607.08971#S4.E3)\)\. For a target failure probabilityδrank\\delta\_\{\\rm rank\}, at the start of epocheethe learner first selects the dimension

m^e:=\#​\{j∈\[d\]:λ^τe−1,j≥2​ρτe−1\},ρt:=2​BX​λ¯p2​t​K​log⁡2​d​Tδrank\+2​BX2p2​t​K​log⁡2​d​Tδrank,\\hat\{m\}\_\{e\}:=\\\#\\left\\\{j\\in\[d\]:\\hat\{\\lambda\}\_\{\\tau\_\{e\}\-1,j\}\\ \\geq\\ 2\\rho\_\{\\tau\_\{e\}\-1\}\\right\\\},\\qquad\\rho\_\{t\}:=2B\_\{X\}\\sqrt\{\\frac\{\\bar\{\\lambda\}\}\{p^\{2\}tK\}\\log\\frac\{2dT\}\{\\delta\_\{\\rm rank\}\}\}\+\\frac\{2B\_\{X\}^\{2\}\}\{p^\{2\}tK\}\\log\\frac\{2dT\}\{\\delta\_\{\\rm rank\}\},and then uses the topm^e\\hat\{m\}\_\{e\}eigenvectors ofΣ˙τe−1\\dot\{\\Sigma\}\_\{\\tau\_\{e\}\-1\}as the frozen basis𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}for epochee\(as in Algorithm[1](https://arxiv.org/html/2607.08971#alg1), only decision sets observed before the epoch are used\)\. The thresholdρt\\rho\_\{t\}is a high\-probability upper bound on‖Σ˙t−Σ‖2\\\|\\dot\{\\Sigma\}\_\{t\}\-\\Sigma\\\|\_\{2\}, and is computable from\(BX,p,K\)\(B\_\{X\},p,K\)alone \(recall that one may always takeλ¯=BX2\\bar\{\\lambda\}=B\_\{X\}^\{2\}\)\. In particular, the learner needs no knowledge ofmmor even an upper bound on it\.

###### Theorem 3\(Rank\-AdaptiveTOFU\-POVregret\)

Suppose the assumptions of Theorem[2](https://arxiv.org/html/2607.08971#Thmtheorem2)hold, butmmis unknown to the algorithm\. Then, with probability at least1−δ1\-\\delta\(under a natural allocation ofδ\\deltaacross various events\),Rank\-AdaptiveTOFU\-POVsatisfies

RT≤O~​\(S​BX​tid\)\+O~​\(\(R\+S​BX\)​m​T\)\+O~​\(S​BX​κ​m​Tp2​K\),tid:=max⁡\{tb,trank\},R\_\{T\}\\leq\\widetilde\{O\}\\\!\\left\(SB\_\{X\}\\,t\_\{\\rm id\}\\right\)\+\\widetilde\{O\}\\\!\\left\(\(R\+SB\_\{X\}\)\\,m\\sqrt\{T\}\\right\)\+\\widetilde\{O\}\\\!\\left\(SB\_\{X\}\\frac\{\\kappa m\\sqrt\{T\}\}\{p^\{2\}\\sqrt\{K\}\}\\right\),\\qquad t\_\{\\rm id\}:=\\max\\\{t\_\{b\},\\,t\_\{\\rm rank\}\\\},wheretrank=O~​\(κ2​m/\(p2​K\)\)t\_\{\\rm rank\}=\\widetilde\{O\}\\big\(\\kappa^\{2\}m/\(p^\{2\}K\)\\big\)is the time at which the threshold separates themmsignal eigenvalues from the null ones\.

The full statement with explicit constants, and the proof, appear in Appendix[I](https://arxiv.org/html/2607.08971#A9)\. Informally, once the threshold is able to separate the signal from the noise floor,m^e=m\\hat\{m\}\_\{e\}=m\. Indeed, after this, the algorithm coincides with the known\-rank procedure\. In our analysis, we charge all regret before this point at the worst\-case rate, giving the first term\. Moreover, somewhat unsurprisingly \(given the complexity of the tasks in question\), rank identification is faster than stable imputation\. Indeed, comparing rates,trank=O~​\(κ2​m/\(p2​K\)\)t\_\{\\rm rank\}=\\widetilde\{O\}\\big\(\\kappa^\{2\}m/\(p^\{2\}K\)\\big\)whiletb=O~​\(κ2​m/\(p4​K\)\)t\_\{b\}=\\widetilde\{O\}\\big\(\\kappa^\{2\}m/\(p^\{4\}K\)\\big\), sotid=tbt\_\{\\rm id\}=t\_\{b\}up to constants\. Therefore, the identification cost is absorbed into the burn\-in term of Theorem[2](https://arxiv.org/html/2607.08971#Thmtheorem2)and we essentially get adaptivity to the unknown subspace dimension for free\.

## 7A Lower Bound: Bandit Learning and Missingness Discovery

The above analysis shows us the effect of the cost of reward learning and of missing coordinates on the final regret ofTOFU\-POV\. In this section, we show that both of these costs are essentially unavoidable\. Towards this end, we provide a novel construction of a hard family of bandit instances which is parametrized by two independent \(hidden\) signs: one parametrizes noisy reward\-learning, while the other selects one of two completions that cannot be distinguished from single\-coordinate observations and is revealed only when a particular coordinate pair is co\-observed\. The regret for any policyπ\\piis denoted byRTπR\_\{T\}^\{\\pi\}, and is measured against the full\-information oracle that sees the complete action set in each round, which only strengthens the lower bound\.

###### Theorem 4\(Lower bound\)

AssumeK≥4K\\geq 4,p∈\(0,1/2\]p\\in\(0,1/2\], and Gaussian reward noise with varianceR2R^\{2\}\. There exist universal constantsc,c0\>0c,c\_\{0\}\>0such that the following holds\. For anyT≥1T\\geq 1,S\>0S\>0, and action norm boundBX\>0B\_\{X\}\>0satisfyingK​p2\+S2​BX2R2≤c0,Kp^\{2\}\+\\frac\{S^\{2\}B\_\{X\}^\{2\}\}\{R^\{2\}\}\\leq c\_\{0\},there is a four\-instance familyℑ=\{ℐν,σ:\(ν,σ\)∈\{±1\}2\}\\mathfrak\{I\}=\\\{\\mathcal\{I\}\_\{\\nu,\\sigma\}:\(\\nu,\\sigma\)\\in\\\{\\pm 1\\\}^\{2\}\\\}of Bernoulli\-ppmissing\-feature linear\-bandit instances whose i\.i\.d\.KK\-arm slate distributions are supported on rank\-three subspaces ofℝ4\\mathbb\{R\}^\{4\}and satisfy‖Xt,i‖2≤BX\\\|X\_\{t,i\}\\\|\_\{2\}\\leq B\_\{X\}and‖θ⋆‖2≤S\\\|\\theta^\{\\star\}\\\|\_\{2\}\\leq S, such that every policyπ\\piobeys

supℐ∈ℑ𝔼ℐ​RTπ\\displaystyle\\sup\_\{\\mathcal\{I\}\\in\\mathfrak\{I\}\}\\mathbb\{E\}\_\{\\mathcal\{I\}\}R\_\{T\}^\{\\pi\}≥c​min⁡\{S​BX​T,R​T\}\+c​S​BX​min⁡\{T,1K​p2\+S2​BX2/R2\}\.\\displaystyle\\geq c\\min\\\{SB\_\{X\}T,\\,R\\sqrt\{T\}\\\}\+cSB\_\{X\}\\min\\left\\\{T,\\frac\{1\}\{Kp^\{2\}\+S^\{2\}B\_\{X\}^\{2\}/R^\{2\}\}\\right\\\}\.\(9\)In particular, ifR≥S​BX/\(p​K\)R\\geq SB\_\{X\}/\(p\\sqrt\{K\}\), thensupℐ∈ℑ𝔼ℐ​RTπ≥c​min⁡\{S​BX​T,R​T\}\+c​S​BX​min⁡\{T,1K​p2\}\.\\sup\_\{\\mathcal\{I\}\\in\\mathfrak\{I\}\}\\mathbb\{E\}\_\{\\mathcal\{I\}\}R\_\{T\}^\{\\pi\}\\geq c\\min\\\{SB\_\{X\}T,\\,R\\sqrt\{T\}\\\}\+cSB\_\{X\}\\min\\left\\\{T,\\frac\{1\}\{Kp^\{2\}\}\\right\\\}\.

The first term in Theorem[4](https://arxiv.org/html/2607.08971#Thmtheorem4)is the standard stochastic\-bandit cost\. The second term is specific to missing features: we show that we can obfuscate the sign from any agent until some side\-information arm reveals both relevant coordinates, an event occurring at rateΘ​\(K​p2\)\\Theta\(Kp^\{2\}\)per round \(whereKKarms are revealed\), or until rewards identify the sign at rateΘ​\(S2​BX2/R2\)\\Theta\(S^\{2\}B\_\{X\}^\{2\}/R^\{2\}\)\. Thus, in the context\-limited regime, missingness contributes an additional unavoidable cost of orderS​BX​min⁡\{T,1/\(K​p2\)\}SB\_\{X\}\\min\\\{T,1/\(Kp^\{2\}\)\\\}\. We note that the hard instances lie within the class covered by our upper bound: the action vectors are bounded byBXB\_\{X\}almost surely and the action distribution is i\.i\.d\. with a rank\-three covariance, so Assumptions[1](https://arxiv.org/html/2607.08971#Thmassumption1)and[2](https://arxiv.org/html/2607.08971#Thmassumption2)hold\. Comparing with Theorem[2](https://arxiv.org/html/2607.08971#Thmtheorem2), both bounds exhibit a missingness cost that decreases withKKandpp, but the scaling with respect to these parameters does not match exactly\. Identifying a lower bound construction that provides a sharp dependence onppandKKas well remains an interesting open problem\. The full gap\-dependent statement and proof appear in Appendix[H](https://arxiv.org/html/2607.08971#A8)\.

## 8Simulations

Experimental setup\.We evaluateTOFU\-POVwith the practical choices described after Algorithm[1](https://arxiv.org/html/2607.08971#alg1); all practical variants use zero\-imputed OFUL during burn\-in\. In synthetic experiments,TOFUandRA\-TOFUdenote the known\-rank and rank\-adaptive versions\. In real\-feature experiments,TOFU\-FHandRA\-TOFU\-FHdenote full\-history replay variants that re\-impute and replay past selected rewards at each epoch start\. Baselines areZF\-OFUL, ambient OFUL on zero\-filled masked arms, andZF\-PSLB, the analogous zero\-filled adaptation of PSLB\[[9](https://arxiv.org/html/2607.08971#bib.bib9)\]\.

The mainsynthetic experimentsused=30d=30, true rankm⋆=3m^\{\\star\}=3,K=8K=8, horizonT=400T=400, burn\-intb=30t\_\{b\}=30, and Gaussian reward noise with standard deviation0\.050\.05\. We vary the observation probability overp∈\{0\.8,0\.6,0\.4,0\.3,0\.2\}p\\in\\\{0\.8,0\.6,0\.4,0\.3,0\.2\\\}\. Figure[1](https://arxiv.org/html/2607.08971#S8.F1)reports cumulative regret over time and final regret as a function of missingness\. Means and standard errors over 20 random seeds are reported\.

![Refer to caption](https://arxiv.org/html/2607.08971v1/x1.png)\(a\)Regret over time\.
![Refer to caption](https://arxiv.org/html/2607.08971v1/x2.png)\(b\)Final regret vs\.pp\.

Figure 1:Synthetic missing\-feature experiments\. TOFU and RA\-TOFU gain most when missingness is substantial: ZF\-OFUL and ZF\-PSLB degrade asppdecreases, while the TOFU variants continue to exploit the corrected low\-rank structure\.These results show that exploiting the low\-rank structure becomes increasingly valuable when missingness is substantial\. At high observation probability, ZF\-OFUL is competitive because the ambient zero\-filled representation retains enough information, and asppdecreases, ambient learning becomes increasingly biased and sample\-inefficient\. TOFU and RA\-TOFU remain close in moderate missingness and substantially outperform both ZF\-OFUL and ZF\-PSLB in sparse regimes\. In the real\-feature experiments below we use the full\-history replay variants; Appendix[J](https://arxiv.org/html/2607.08971#A10)repeats this synthetic study with TOFU\-FH and RA\-TOFU\-FH, which gives significant additional improvement\.

MNIST product\-context experiment\. \(Figure[2](https://arxiv.org/html/2607.08971#S9.F2)\)For MNIST\[[32](https://arxiv.org/html/2607.08971#bib.bib32)\], we train a small CNN withm=4m=4\-dimensional penultimate featureh​\(x\)h\(x\)and class\-head weightswkw\_\{k\}\. The class\-kkarm isXk​\(x\)=h​\(x\)⊙wkX\_\{k\}\(x\)=h\(x\)\\odot w\_\{k\}, preserving the classifier score through⟨Xk​\(x\),𝟏m⟩=wk⊤​h​\(x\)\\langle X\_\{k\}\(x\),\\mathbf\{1\}\_\{m\}\\rangle=w\_\{k\}^\{\\top\}h\(x\)\. We lift the ten product\-context arms intoℝ100\\mathbb\{R\}^\{100\}by a fixed orthonormal map and mask coordinates\. Rewards are classification rewards;T=5000T=5000,tb=500t\_\{b\}=500,p∈\{0\.7,0\.5,0\.3,0\.2\}p\\in\\\{0\.7,0\.5,0\.3,0\.2\\\}, and ranks, thresholds, and confidence parameters are chosen on validation seeds disjoint from reporting seeds\.

The MNIST product\-context experiment gives a real\-feature setting where the low\-rank reward geometry is present by construction\. TOFU\-FH matches ZF\-OFUL at mild missingness and increasingly outperforms it as features become sparse\. ZF\-PSLB is consistently worse, with the gap widening at lower observation probabilities\.

Additional experiments\.Appendix[J](https://arxiv.org/html/2607.08971#A10)contains several supporting experiments: real\-feature synthetic tasks using optical digit covariates\[[33](https://arxiv.org/html/2607.08971#bib.bib33)\], rank\-recovery and fixed\-rank misspecification diagnostics, warm\-start comparisons, MNIST rank validation, and a 20 Newsgroups product\-context experiment\[[34](https://arxiv.org/html/2607.08971#bib.bib34)\]with approximately low\-rank features\. Thecode and scriptsfor reproducing the experimental results are available at[https://github\.com/gautamdasarathy/tofu\-pov\-arxiv](https://github.com/gautamdasarathy/tofu-pov-arxiv)\.

## 9Discussion

![Refer to caption](https://arxiv.org/html/2607.08971v1/x3.png)

Figure 2:MNIST experiment\. TOFU\-FH wins or ties, and the gap over baselines grows asppdecreases\.

This paper studies contextual decision making when the learner must act from incomplete action descriptions\. We focus on the case of i\.i\.d\. action sets with low\-rank structure and missing\-at\-random coordinates\. Natural extensions include approximately low\-rank action models, where the tail of the spectrum would create an additional approximation term, and missing\-not\-at\-random observation patterns, where the missingness process itself may be informative or biased\. Both settings would require separating representation error, imputation bias, and reward\-learning uncertainty more carefully and are promising avenues for future work\. The experiments partly probe beyond the theory by using public benchmark\-derived covariates and approximately low\-rank nuisance directions; these results suggest the method is not brittle to such deviations\. Our epoch\-wise freezing is deliberately conservative, and a truly sequential subspace\-identification procedure would be more natural; this would require new concentration and potential arguments for learned, time\-varying representations\. Finally, when the learner has some control over which coordinates are revealed, the problem acquires an active learning flavor and adaptively targeting informative coordinates \(along the lines of techniques in\[[35](https://arxiv.org/html/2607.08971#bib.bib35)\]\) could sharpen both the subspace estimate and the burn\-in cost\. If the learner can instead choose the*amount*of missingness across rounds and arms, by paying more for a higher observation probabilitypp, this becomes a multi\-fidelity decision\-making problem, and the techniques developed in the multi\-fidelity bandit and optimization literature\[[36](https://arxiv.org/html/2607.08971#bib.bib36),[37](https://arxiv.org/html/2607.08971#bib.bib37),[38](https://arxiv.org/html/2607.08971#bib.bib38),[39](https://arxiv.org/html/2607.08971#bib.bib39)\]offer a natural starting point\. On the lower\-bound side, our construction shows that partial observation creates a genuine missingness\-discovery cost, but it does not settle the sharp dependence onpp\. In particular, it remains unclear whether theT/p2\\sqrt\{T\}/p^\{2\}\-type term in the upper bound is intrinsic or an artifact of worst\-case subspace estimation and imputation control\.

## References

- Abbasi\-Yadkori et al\. \[2011\]Yasin Abbasi\-Yadkori, Dávid Pál, and Csaba Szepesvári\.Improved algorithms for linear stochastic bandits\.*Advances in neural information processing systems*, 24, 2011\.
- Rusmevichientong and Tsitsiklis \[2010\]Paat Rusmevichientong and John N Tsitsiklis\.Linearly parameterized bandits\.*Mathematics of Operations Research*, 35\(2\):395–411, 2010\.
- Lattimore and Szepesvári \[2020\]Tor Lattimore and Csaba Szepesvári\.*Bandit algorithms*\.Cambridge University Press, 2020\.
- Li et al\. \[2010\]Lihong Li, Wei Chu, John Langford, and Robert E Schapire\.A contextual\-bandit approach to personalized news article recommendation\.In*Proceedings of the 19th international conference on World wide web*, pages 661–670, 2010\.
- Chu et al\. \[2011\]Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire\.Contextual bandits with linear payoff functions\.In*Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics*, pages 208–214\. JMLR Workshop and Conference Proceedings, 2011\.
- Bastani and Bayati \[2020\]Hamsa Bastani and Mohsen Bayati\.Online decision making with high\-dimensional covariates\.*Operations Research*, 68\(1\):276–294, 2020\.
- Kim et al\. \[2025\]Wonyoung Kim, Sungwoo Park, Garud Iyengar, Assaf Zeevi, and Min\-hwan Oh\.Linear bandits with partially observable features\.*arXiv preprint arXiv:2502\.06142*, 2025\.
- Park and Faradonbeh \[2022\]Hongju Park and Mohamad Kazem Shirani Faradonbeh\.A regret bound for greedy partially observed stochastic contextual bandits\.In*Proceedings of the 39th International Conference on Machine Learning*, volume 162, pages 1805–1812\. PMLR, 2022\.
- Lale et al\. \[2019\]Sahin Lale, Kamyar Azizzadenesheli, Anima Anandkumar, and Babak Hassibi\.Stochastic linear bandits with hidden low rank structure\.*arXiv preprint arXiv:1901\.09490*, 2019\.
- Abbasi\-Yadkori et al\. \[2012\]Y\. Abbasi\-Yadkori, D\. Pal, and C\. Szepesvari\.Online\-to\-confidence\-set conversions and application to sparse stochastic bandits\.In*Artificial Intelligence and Statistics*, pages 1–9, 2012\.
- Carpentier and Munos \[2012\]A\. Carpentier and R\. Munos\.Bandit theory meets compressed sensing for high dimensional stochastic linear bandit\.In*Artificial Intelligence and Statistics*, pages 190–198, 2012\.
- Kwon et al\. \[2017\]Joon Kwon, Vianney Perchet, and Claire Vernade\.Sparse stochastic bandits\.*arXiv preprint arXiv:1706\.01383*, 2017\.
- Jang et al\. \[2022\]Kyoungseok Jang, Chicheng Zhang, and Kwang\-Sung Jun\.Popart: Efficient sparse regression and experimental design for optimal sparse linear bandits\.*Advances in Neural Information Processing Systems*, 35:2102–2114, 2022\.
- Jun et al\. \[2019\]Kwang\-Sung Jun, Rebecca Willett, Stephen Wright, and Robert Nowak\.Bilinear bandits with low\-rank structure\.In*Proceedings of the 36th International Conference on Machine Learning*, volume 97 of*Proceedings of Machine Learning Research*, pages 3163–3172\. PMLR, 2019\.
- Lu et al\. \[2021\]Yangyi Lu, Amirhossein Meisami, and Ambuj Tewari\.Low\-rank generalized linear bandit problems\.In*Proceedings of the 24th International Conference on Artificial Intelligence and Statistics*, volume 130 of*Proceedings of Machine Learning Research*, pages 460–468\. PMLR, 2021\.
- Jang et al\. \[2021\]Young\-Hwan Jang, Kwang\-Sung Jun, and Se\-Young Yun\.Improved regret analysis for bilinear bandits\.In*Proceedings of the 38th International Conference on Machine Learning*, volume 139 of*Proceedings of Machine Learning Research*, pages 4753–4763\. PMLR, 2021\.
- Kang et al\. \[2022\]Yang Kang, Cho\-Jui Hsieh, and Thomas C\. M\. Lee\.Efficient frameworks for generalized low\-rank matrix bandit problems\.*Advances in Neural Information Processing Systems*, 35:19971–19983, 2022\.
- Valko et al\. \[2014\]Michal Valko, Rémi Munos, Branislav Kveton, and Tomáš Kocák\.Spectral bandits for smooth graph functions\.In*International Conference on Machine Learning*, 2014\.
- LeJeune et al\. \[2020\]Daniel LeJeune, Gautam Dasarathy, and Richard G\. Baraniuk\.Thresholding graph bandits with GrAPL\.In*International Conference on Artificial Intelligence and Statistics*, 2020\.
- Thaker et al\. \[2022\]Parth Thaker, Mohit Malu, Nikhil Rao, and Gautam Dasarathy\.Maximizing and satisficing in multi\-armed bandits with graph information\.In*Advances in Neural Information Processing Systems*, 2022\.
- Candès and Recht \[2009\]Emmanuel J Candès and Benjamin Recht\.Exact matrix completion via convex optimization\.*Foundations of Computational mathematics*, 9\(6\):717–772, 2009\.
- Candès et al\. \[2011\]Emmanuel J Candès, Xiaodong Li, Yi Ma, and John Wright\.Robust principal component analysis?*Journal of the ACM \(JACM\)*, 58\(3\):1–37, 2011\.
- Oja \[1982\]Erkki Oja\.Simplified neuron model as a principal component analyzer\.*Journal of mathematical biology*, 15\(3\):267–273, 1982\.
- Balzano et al\. \[2010\]Laura Balzano, Robert Nowak, and Benjamin Recht\.High\-dimensional robust subspace tracking under missing data and outliers\.In*2010 Conference Record of the Forty Fourth Asilomar Conference on Signals, Systems and Computers*, pages 2087–2091\. IEEE, 2010\.
- Chi et al\. \[2013\]Yuejie Chi, Yonina C Eldar, and Robert Calderbank\.Petrels: Parallel subspace estimation and tracking by recursive least squares from partial observations\.*IEEE Transactions on Signal Processing*, 61\(23\):5947–5959, 2013\.
- Liang et al\. \[2019\]Yingyu Liang, Zhuoran Xu, and Dale Schuurmans\.An exponential convergence rate for subspace estimation from partial observations\.*arXiv preprint arXiv:1905\.13595*, 2019\.
- Agrawal and Goyal \[2013\]Shipra Agrawal and Navin Goyal\.Thompson sampling for contextual bandits with linear payoffs\.In*International Conference on Machine Learning*, pages 127–135, 2013\.
- Abeille and Lazaric \[2017\]Marc Abeille and Alessandro Lazaric\.Linear Thompson sampling revisited\.In*Artificial Intelligence and Statistics*, pages 176–184, 2017\.
- Krishnamurthy et al\. \[2018\]Akshay Krishnamurthy, Zhiwei Steven Wu, and Vasilis Syrgkanis\.Semiparametric contextual bandits\.In*International Conference on Machine Learning*, pages 2776–2785, 2018\.
- Lounici \[2014\]Karim Lounici\.High\-dimensional covariance matrix estimation with missing observations\.*Bernoulli*, 20\(3\):1029 – 1058, 2014\.doi:10\.3150/12\-BEJ487\.URL[https://doi\.org/10\.3150/12\-BEJ487](https://doi.org/10.3150/12-BEJ487)\.
- Davis and Kahan \[1970\]Chandler Davis and William Morton Kahan\.The rotation of eigenvectors by a perturbation\. iii\.*SIAM Journal on Numerical Analysis*, 7\(1\):1–46, 1970\.
- LeCun et al\. \[1998\]Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner\.Gradient\-based learning applied to document recognition\.*Proceedings of the IEEE*, 86\(11\):2278–2324, 1998\.
- Alpaydin and Kaynak \[1998\]Ethem Alpaydin and Cenk Kaynak\.Optical recognition of handwritten digits\.UCI Machine Learning Repository, 1998\.DOI: 10\.24432/C50P49\.
- Lang \[1995\]Ken Lang\.Newsweeder: Learning to filter netnews\.In*Proceedings of the Twelfth International Conference on Machine Learning*, pages 331–339\. Morgan Kaufmann, 1995\.
- Dasarathy et al\. \[2016\]Gautam Dasarathy, Aarti Singh, Maria\-Florina Balcan, and Jong Hyuk Park\.Active learning algorithms for graphical model selection\.In*Artificial Intelligence and Statistics*, 2016\.
- Kandasamy et al\. \[2016a\]Kirthevasan Kandasamy, Gautam Dasarathy, Barnabás Póczos, and Jeff Schneider\.The multi\-fidelity multi\-armed bandit\.In*Advances in Neural Information Processing Systems*, 2016a\.
- Kandasamy et al\. \[2016b\]Kirthevasan Kandasamy, Gautam Dasarathy, Junier B\. Oliva, Jeff Schneider, and Barnabás Póczos\.Gaussian process bandit optimisation with multi\-fidelity evaluations\.In*Advances in Neural Information Processing Systems*, 2016b\.
- Kandasamy et al\. \[2017\]Kirthevasan Kandasamy, Gautam Dasarathy, Jeff Schneider, and Barnabás Póczos\.Multi\-fidelity Bayesian optimisation with continuous approximations\.In*International Conference on Machine Learning*, 2017\.
- Kandasamy et al\. \[2019\]Kirthevasan Kandasamy, Gautam Dasarathy, Junier B\. Oliva, Jeff Schneider, and Barnabás Póczos\.Multi\-fidelity Gaussian process bandit optimisation\.*Journal of Artificial Intelligence Research*, 66:151–196, 2019\.
- Tropp \[2015\]Joel A\. Tropp\.An introduction to matrix concentration inequalities, 2015\.URL[https://arxiv\.org/abs/1501\.01571](https://arxiv.org/abs/1501.01571)\.
- Björck and Golub \[1973\]Åke Björck and Gene H Golub\.Numerical methods for computing angles between linear subspaces\.*Mathematics of Computation*, 27\(123\):579–594, 1973\.
- Stewart and Sun \[1990\]Gilbert W Stewart and Ji\-guang Sun\.*Matrix Perturbation Theory*\.Academic Press, 1990\.
- Horn and Johnson \[2012\]Roger A\. Horn and Charles R\. Johnson\.*Matrix Analysis*\.Cambridge University Press, 2 edition, 2012\.

Stochastic Linear Bandits with Partially Observed Actions: Appendices

## Appendix ATechnical Comparison with PSLB and the Role of Epoch\-wise Freezing

The closest predecessor to our work is the projected stochastic linear bandit framework of\[[9](https://arxiv.org/html/2607.08971#bib.bib9)\]\. PSLB estimates a low\-dimensional subspace from the fully observed actions appearing in the decision sets and then uses projected confidence sets inside an optimistic linear bandit rule\. Our setting differs at the modeling level because the ideal action vectors are observed only through random coordinate masks, so the learner must recover the latent subspace and impute the currently available actions\. There is also a proof\-level distinction: a direct projected\-OFUL analysis with a projection matrix updated every round does not follow from the standard self\-normalized and elliptical\-potential arguments\. More specifically, the PSLB proof appears to rely on two steps that require additional justification: a self\-normalized martingale argument for a retrospectively projected noise sum, and a projected\-potential/minimum\-eigenvalue argument for the covariance of OFU\-selected actions under a time\-varying projection\.

Retrospective projections and self\-normalization\.The usual OFUL confidence analysis controlsStOFUL:=∑s=1txs​ηs,S\_\{t\}^\{\\rm OFUL\}:=\\sum\_\{s=1\}^\{t\}x\_\{s\}\\eta\_\{s\},wherexsx\_\{s\}is predictable before the reward noiseηs\\eta\_\{s\}is observed\. In a projected analysis with a time\-varying projection, the analogous object is

Stproj:=∑s=1t𝐏^t​xs​ηs=𝐏^t​∑s=1txs​ηs\.S\_\{t\}^\{\\rm proj\}:=\\sum\_\{s=1\}^\{t\}\\hat\{\\mathbf\{P\}\}\_\{t\}x\_\{s\}\\eta\_\{s\}=\\hat\{\\mathbf\{P\}\}\_\{t\}\\sum\_\{s=1\}^\{t\}x\_\{s\}\\eta\_\{s\}\.This is not the predictable martingale transform∑s=1t𝐏^s​xs​ηs\\sum\_\{s=1\}^\{t\}\\hat\{\\mathbf\{P\}\}\_\{s\}x\_\{s\}\\eta\_\{s\}: the projection at timettis applied retroactively to all previous noise terms\. Indeed,

St\+1proj−Stproj=𝐏^t\+1​xt\+1​ηt\+1\+\(𝐏^t\+1−𝐏^t\)​∑s=1txs​ηs\.S\_\{t\+1\}^\{\\rm proj\}\-S\_\{t\}^\{\\rm proj\}=\\hat\{\\mathbf\{P\}\}\_\{t\+1\}x\_\{t\+1\}\\eta\_\{t\+1\}\+\(\\hat\{\\mathbf\{P\}\}\_\{t\+1\}\-\\hat\{\\mathbf\{P\}\}\_\{t\}\)\\sum\_\{s=1\}^\{t\}x\_\{s\}\\eta\_\{s\}\.The second term reweights past noise and is not a standard martingale increment\. Thus the standard anytime OFUL confidence proof, which relies on a supermartingale/Ville argument, cannot be invoked simply by replacing each past feature with its projection under the latest estimated subspace\.

Projected potential and selected\-action covariance\.The usual elliptical\-potential lemma is an algebraic, pathwise statement\. For any realized sequence of features, if the design evolves by rank\-one updates, we get the following:

Vt\+1=Vt\+zt​zt⊤,∑t=1Tmin⁡\{1,‖zt‖Vt−12\}≤2​log⁡det\(VT\+1\)det\(V1\)\.V\_\{t\+1\}=V\_\{t\}\+z\_\{t\}z\_\{t\}^\{\\top\},\\qquad\\sum\_\{t=1\}^\{T\}\\min\\\{1,\\\|z\_\{t\}\\\|\_\{V\_\{t\}^\{\-1\}\}^\{2\}\\\}\\leq 2\\log\\frac\{\\det\(V\_\{T\+1\}\)\}\{\\det\(V\_\{1\}\)\}\.With a changing projection, the natural projected design

At=𝐏^t​\(λ​I\+∑s<txs​xs⊤\)​𝐏^tA\_\{t\}=\\hat\{\\mathbf\{P\}\}\_\{t\}\\left\(\\lambda I\+\\sum\_\{s<t\}x\_\{s\}x\_\{s\}^\{\\top\}\\right\)\\hat\{\\mathbf\{P\}\}\_\{t\}does not satisfy such a recursion\. Instead,

At\+1−At\\displaystyle A\_\{t\+1\}\-A\_\{t\}=𝐏^t\+1​\(λ​I\+∑s<txs​xs⊤\)​𝐏^t\+1−𝐏^t​\(λ​I\+∑s<txs​xs⊤\)​𝐏^t\\displaystyle=\\hat\{\\mathbf\{P\}\}\_\{t\+1\}\\left\(\\lambda I\+\\sum\_\{s<t\}x\_\{s\}x\_\{s\}^\{\\top\}\\right\)\\hat\{\\mathbf\{P\}\}\_\{t\+1\}\-\\hat\{\\mathbf\{P\}\}\_\{t\}\\left\(\\lambda I\+\\sum\_\{s<t\}x\_\{s\}x\_\{s\}^\{\\top\}\\right\)\\hat\{\\mathbf\{P\}\}\_\{t\}\+𝐏^t\+1​xt​xt⊤​𝐏^t\+1,\\displaystyle\\quad\+\\hat\{\\mathbf\{P\}\}\_\{t\+1\}x\_\{t\}x\_\{t\}^\{\\top\}\\hat\{\\mathbf\{P\}\}\_\{t\+1\},and the first two terms need not be positive semidefinite or low\-rank\. Hence the determinant\-telescoping proof does not directly apply to projected designs with a projection updated every round\.

One alternative, used in the PSLB analysis of\[[9](https://arxiv.org/html/2607.08971#bib.bib9)\], is to prove that the covariance of the selected actions has a linearly growing minimum eigenvalue\. The delicate point is that this is the covariance of actions chosen by the OFU rule, not the population covariance of a random arm from the offered decision set\. Population excitation of the offered arms does not, by itself, imply selected\-design excitation: if every offered set is\{e1,e2\}\\\{e\_\{1\},e\_\{2\}\\\}, then a uniformly sampled offered arm has covarianceI2/2I\_\{2\}/2, but the rule that always choosese1e\_\{1\}has selected covarianceT​e1​e1⊤Te\_\{1\}e\_\{1\}^\{\\top\}, whose minimum eigenvalue is zero\. A Matrix Chernoff argument for selected actions therefore requires an explicit policy\-specific excitation or stronger assumptions that guarantee this\.

Why epoch\-wise freezing avoids these issues\.We tackle both of these with our epoch\-wise algorithm construction \(Algorithm[1](https://arxiv.org/html/2607.08971#alg1)\)\. We estimate𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}using only pre\-epoch decision sets, then keep it fixed during𝔗e\\mathfrak\{T\}\_\{e\}\. Conditional on the epoch\-start sigma\-field, the induced featureszt,i=𝐔^e⊤​X^t,iz\_\{t,i\}=\\hat\{\\mathbf\{U\}\}\_\{e\}^\{\\top\}\\widehat\{X\}\_\{t,i\}live in a fixedmm\-dimensional coordinate system\. Thus the selected featureztz\_\{t\}is predictable,∑s∈𝔗e:s<tzs​ηs\\sum\_\{s\\in\\mathfrak\{T\}\_\{e\}:s<t\}z\_\{s\}\\eta\_\{s\}is the usual self\-normalized martingale transform, andVe,t:=λ​Im\+∑s∈𝔗e:s<tzs​zs⊤V\_\{e,t\}:=\\lambda I\_\{m\}\+\\sum\_\{s\\in\\mathfrak\{T\}\_\{e\}:s<t\}z\_\{s\}z\_\{s\}^\{\\top\}satisfies the rank\-one recursion\. Standard OFUL confidence and potential bounds therefore apply inside each epoch; this is the reduction used in Section[4\.3](https://arxiv.org/html/2607.08971#S4.SS3)\.

What remains is a controlled approximation error rather than a time\-varying\-projection issue\. In the frozen coordinates,rt=⟨zt,θe⋆⟩\+bt\+ηtr\_\{t\}=\\langle z\_\{t\},\\theta\_\{e\}^\{\\star\}\\rangle\+b\_\{t\}\+\\eta\_\{t\}with\|bt\|≤be\|b\_\{t\}\|\\leq b\_\{e\}, wherebeb\_\{e\}is determined by the epoch\-start subspace and imputation accuracy \(Lemma[3](https://arxiv.org/html/2607.08971#Thmlemma3)\)\. The main regret proof in Section[5](https://arxiv.org/html/2607.08971#S5)then sums the usual OFUL terms and this representation\-misspecification contribution over epochs; Lemma[4](https://arxiv.org/html/2607.08971#Thmlemma4)gives the corresponding misspecification\-control step, and Appendix[G](https://arxiv.org/html/2607.08971#A7)explains why this misspecification handling needs some care\.

## Appendix BComparison with a One\-shot Two\-phase Baseline

This section compares the epoch\-wise algorithm with a simpler one\-shot two\-phase strategy\. This natural baseline first usesτ\\taurounds to estimate the subspace, then freezes this estimate for the remainingT−τT\-\\taurounds and runs a standardmm\-dimensional OFUL algorithm in the frozen coordinates\. One could then ask if aT\\sqrt\{T\}regret is achievable by optimizingτ\\tau\. In what follows, we will show that the best one could hope for with such a strategy is aT2/3T^\{2/3\}regret\.

Let𝐔^τ\\hat\{\\mathbf\{U\}\}\_\{\\tau\}be the subspace estimate afterτ≥tb\\tau\\geq t\_\{b\}rounds, and let𝐏^τ=𝐔^τ​𝐔^τ⊤\\hat\{\\mathbf\{P\}\}\_\{\\tau\}=\\hat\{\\mathbf\{U\}\}\_\{\\tau\}\\hat\{\\mathbf\{U\}\}\_\{\\tau\}^\{\\top\}\. Fort\>τt\>\\tau, the two\-phase baseline imputes each arm using𝐔^τ\\hat\{\\mathbf\{U\}\}\_\{\\tau\}, formszt,i:=𝐔^τ⊤​X^t,i\(τ\)∈ℝm,z\_\{t,i\}:=\\hat\{\\mathbf\{U\}\}\_\{\\tau\}^\{\\top\}\\hat\{X\}\_\{t,i\}^\{\(\\tau\)\}\\in\\mathbb\{R\}^\{m\},and runs OFUL on these fixedmm\-dimensional features\. On the representation eventℰrep\\mathcal\{E\}\_\{\\rm rep\}, defined in Section[4\.1](https://arxiv.org/html/2607.08971#S4.SS1)by combining Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)and Lemma[2](https://arxiv.org/html/2607.08971#Thmlemma2), the one\-shot analogue of Lemma[3](https://arxiv.org/html/2607.08971#Thmlemma3)gives, for allt\>τt\>\\tauandi∈\[K\]i\\in\[K\],

\|⟨Xt,i,θ⋆⟩−zt,i⊤​ϑτ⋆\|≤bτ,ϑτ⋆:=𝐔^τ⊤​θ⋆,\\left\|\\langle X\_\{t,i\},\\theta^\{\\star\}\\rangle\-z\_\{t,i\}^\{\\top\}\\vartheta\_\{\\tau\}^\{\\star\}\\right\|\\leq b\_\{\\tau\},\\qquad\\vartheta\_\{\\tau\}^\{\\star\}:=\\hat\{\\mathbf\{U\}\}\_\{\\tau\}^\{\\top\}\\theta^\{\\star\},where

bτ:=S​BX​\(2\+2p\)​ϵτ=O~​\(S​BX​κ​mp2​K​τ\)\.b\_\{\\tau\}:=SB\_\{X\}\\left\(2\+\\frac\{2\}\{p\}\\right\)\\epsilon\_\{\\tau\}=\\widetilde\{O\}\\left\(SB\_\{X\}\\,\\kappa\\frac\{\\sqrt\{m\}\}\{p^\{2\}\\sqrt\{K\\tau\}\}\\right\)\.Thus, during the exploitation phase, the realized reward in each round satisfies

rt=zt⊤​ϑτ⋆\+ξt\+ηt,\|ξt\|≤bτ\.r\_\{t\}=z\_\{t\}^\{\\top\}\\vartheta\_\{\\tau\}^\{\\star\}\+\\xi\_\{t\}\+\\eta\_\{t\},\\qquad\|\\xi\_\{t\}\|\\leq b\_\{\\tau\}\.
Applying the same misspecified\-OFUL argument as in the epoch\-wise proof \(Theorem[1](https://arxiv.org/html/2607.08971#Thmtheorem1), Lemma[4](https://arxiv.org/html/2607.08971#Thmlemma4), and Lemma[16](https://arxiv.org/html/2607.08971#Thmlemma16), with the single frozen epoch beginning atτ\\tau\), we get

RT1\-shot​\(τ\)≤O​\(BX​S​τ\)\+O~​\(\(R\+S​BX\)​m​T\)\+O~​\(S​BX​κ​m​Tp2​K​τ\)\.R\_\{T\}^\{\\mbox\{\\footnotesize\\sc 1\-shot\}\}\(\\tau\)\\leq O\(B\_\{X\}S\\,\\tau\)\+\\widetilde\{O\}\\\!\\left\(\(R\+SB\_\{X\}\)\\,m\\sqrt\{T\}\\right\)\+\\widetilde\{O\}\\\!\\left\(SB\_\{X\}\\,\\kappa\\frac\{mT\}\{p^\{2\}\\sqrt\{K\\tau\}\}\\right\)\.
Ignoring problem\-dependent constants, the twoτ\\tau\-dependent terms have the formτ\+Tτ\.\\tau\+\\frac\{T\}\{\\sqrt\{\\tau\}\}\.Balancing them gives us a scaling ofτ≍T2/3,\\tau\\asymp T^\{2/3\},and hence the regret scales asO~​\(T2/3\)\\widetilde\{O\}\(T^\{2/3\}\)up to the standard bandit terms and problem\-dependent constants\.

## Appendix CSubspace Estimation Error: Proof of Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)

We first record two ingredients used in our subspace recovery argument: unbiasedness of the corrected covariance estimator and a high\-probability spectral\-norm concentration bound\.

###### Lemma 5\(Unbiased covariance estimator under Bernoulli masking\)

LetX˙=S⊙X\\dot\{X\}=S\\odot X, where coordinates ofSS,Sj∼iidBernoulli​\(p\)S\_\{j\}\\stackrel\{\{\\scriptstyle\\rm iid\}\}\{\{\\sim\}\}\\mathrm\{Bernoulli\}\(p\)and are independent ofXX\. Define

Y​\(X˙\):=1p2​X˙​X˙⊤\+\(1p−1p2\)​diag​\(X˙​X˙⊤\)\.Y\(\\dot\{X\}\):=\\frac\{1\}\{p^\{2\}\}\\dot\{X\}\\dot\{X\}^\{\\top\}\+\\left\(\\frac\{1\}\{p\}\-\\frac\{1\}\{p^\{2\}\}\\right\)\\mathrm\{diag\}\(\\dot\{X\}\\dot\{X\}^\{\\top\}\)\.Then𝔼​\[Y​\(X˙\)∣X\]=X​X⊤\.\\mathbb\{E\}\[Y\(\\dot\{X\}\)\\mid X\]=XX^\{\\top\}\.Consequently,Σ˙t\\dot\{\\Sigma\}\_\{t\}in Equation \([3](https://arxiv.org/html/2607.08971#S4.E3)\) is an unbiased estimator ofΣ=𝔼​\[X​X⊤\]\\Sigma=\\mathbb\{E\}\[XX^\{\\top\}\]\.

Proof:Forj≠kj\\neq k, observe that

𝔼​\[X˙j​X˙k∣X\]=𝔼​\[Sj​Sk\]​Xj​Xk=p2​Xj​Xk\.\\mathbb\{E\}\[\\dot\{X\}\_\{j\}\\dot\{X\}\_\{k\}\\mid X\]=\\mathbb\{E\}\[S\_\{j\}S\_\{k\}\]X\_\{j\}X\_\{k\}=p^\{2\}X\_\{j\}X\_\{k\}\.Therefore the off\-diagonal entries ofp−2​X˙​X˙⊤p^\{\-2\}\\dot\{X\}\\dot\{X\}^\{\\top\}are unbiased estimates of the off\-diagonal entries ofX​X⊤XX^\{\\top\}\.

Next, lets turn our attention to thejj\-th diagonal entry and observe that

𝔼\[\[Y\(X˙\)\]j​j\|X\]=1p2𝔼\[Sj2\]Xj2\+\(1p−1p2\)𝔼\[Sj2\]Xj2=1p×p×Xj2=Xj2\.\\mathbb\{E\}\\left\[\[Y\(\\dot\{X\}\)\]\_\{jj\}\\middle\|X\\right\]=\\frac\{1\}\{p^\{2\}\}\\mathbb\{E\}\\left\[S\_\{j\}^\{2\}\\right\]X\_\{j\}^\{2\}\+\\left\(\\frac\{1\}\{p\}\-\\frac\{1\}\{p^\{2\}\}\\right\)\\mathbb\{E\}\\left\[S\_\{j\}^\{2\}\\right\]X\_\{j\}^\{2\}=\\frac\{1\}\{p\}\\times p\\times X\_\{j\}^\{2\}=X\_\{j\}^\{2\}\.From this, the proof follows\.

Next, we compute the conditional second moment of a single corrected sample exactly\. This is the ingredient that allows us to determine the variance proxy in the matrix Bernstein bound\.

###### Lemma 6\(Conditional second moment of the corrected estimator\)

LetS1,…,SdS\_\{1\},\\dots,S\_\{d\}be i\.i\.d\.Bernoulli​\(p\)\\mathrm\{Bernoulli\}\(p\)random variables, independent ofXX, and letY​\(X˙\)Y\(\\dot\{X\}\)be as in Lemma[5](https://arxiv.org/html/2607.08971#Thmlemma5)\. Then

𝔼​\[Y​\(X˙\)2\|X\]=‖X‖22p​X​X⊤\+\(1p2−1p\)​diag​\(\(Xj2​\(‖X‖22−Xj2\)\)j∈\[d\]\)\.\\mathbb\{E\}\\big\[Y\(\\dot\{X\}\)^\{2\}\\,\\big\|\\,X\\big\]=\\frac\{\\\|X\\\|\_\{2\}^\{2\}\}\{p\}\\,XX^\{\\top\}\+\\Big\(\\frac\{1\}\{p^\{2\}\}\-\\frac\{1\}\{p\}\\Big\)\\,\\mathrm\{diag\}\\Big\(\\big\(X\_\{j\}^\{2\}\(\\\|X\\\|\_\{2\}^\{2\}\-X\_\{j\}^\{2\}\)\\big\)\_\{j\\in\[d\]\}\\Big\)\.

Proof:For notational ease, we writeY:=Y​\(X˙\)Y:=Y\(\\dot\{X\}\)in the sequel\. First, observe that forj≠kj\\neq k,Yj​k=p−2​Sj​Sk​Xj​XkY\_\{jk\}=p^\{\-2\}S\_\{j\}S\_\{k\}X\_\{j\}X\_\{k\}and thatYj​j=p−1​Sj​Xj2Y\_\{jj\}=p^\{\-1\}S\_\{j\}X\_\{j\}^\{2\}\.

Off\-diagonal entries\.Fixj≠kj\\neq kand expand\(Y2\)j​k=∑lYj​l​Yl​k\(Y^\{2\}\)\_\{jk\}=\\sum\_\{l\}Y\_\{jl\}Y\_\{lk\}\. Notice that the term corresponding tol=jl=jsimplifies top−3​Sj​Sk​Xj3​Xkp^\{\-3\}S\_\{j\}S\_\{k\}X\_\{j\}^\{3\}X\_\{k\}, and hence \(after applying𝔼\[⋅∣X\]\\mathbb\{E\}\\left\[\\cdot\\mid X\\right\]\) contributesp−1​Xj3​Xkp^\{\-1\}X\_\{j\}^\{3\}X\_\{k\}to the sum\. Thel=kl=kterm similarly contributesp−1​Xj​Xk3p^\{\-1\}X\_\{j\}X\_\{k\}^\{3\}\. Eachl∉\{j,k\}l\\notin\\\{j,k\\\}on the other hand contributesp−4​Sj​Sk​Sl​Xj​Xk​Xl2p^\{\-4\}S\_\{j\}S\_\{k\}S\_\{l\}X\_\{j\}X\_\{k\}X\_\{l\}^\{2\}, with conditional meanp−1​Xj​Xk​Xl2p^\{\-1\}X\_\{j\}X\_\{k\}X\_\{l\}^\{2\}, where we use the factSl2=SlS\_\{l\}^\{2\}=S\_\{l\}and𝔼​\[Sj​Sk​Sl\]=p3\\mathbb\{E\}\[S\_\{j\}S\_\{k\}S\_\{l\}\]=p^\{3\}for distinct indices\. Summing overll, we have

𝔼​\[\(Y2\)j​k\|X\]=Xj​Xk​‖X‖22p,\\mathbb\{E\}\\big\[\(Y^\{2\}\)\_\{jk\}\\,\\big\|\\,X\\big\]=\\frac\{X\_\{j\}X\_\{k\}\\\|X\\\|\_\{2\}^\{2\}\}\{p\},which, after some algebra, can be seen to match the\(j,k\)\(j,k\)entry of the stated matrix\.

Diagonal Entries\.Notice that\(Y2\)j​j=Yj​j2\+∑l≠jYj​l2\(Y^\{2\}\)\_\{jj\}=Y\_\{jj\}^\{2\}\+\\sum\_\{l\\neq j\}Y\_\{jl\}^\{2\}, which in turn equals

p−2​Sj​Xj4\+p−4​∑l≠jSj​Sl​Xj2​Xl2,p^\{\-2\}S\_\{j\}X\_\{j\}^\{4\}\+p^\{\-4\}\\sum\_\{l\\neq j\}S\_\{j\}S\_\{l\}X\_\{j\}^\{2\}X\_\{l\}^\{2\},by definition\. We therefore have

𝔼​\[\(Y2\)j​j\|X\]=Xj4p\+Xj2​\(‖X‖22−Xj2\)p2\.\\mathbb\{E\}\\big\[\(Y^\{2\}\)\_\{jj\}\\,\\big\|\\,X\\big\]=\\frac\{X\_\{j\}^\{4\}\}\{p\}\+\\frac\{X\_\{j\}^\{2\}\(\\\|X\\\|\_\{2\}^\{2\}\-X\_\{j\}^\{2\}\)\}\{p^\{2\}\}\.Indeed, this is the\(j,j\)\(j,j\)entry of the stated matrix\.

We can now use these to control the covariance estimation error using a matrix Bernstein argument\.

###### Lemma 7\(Covariance estimation error\)

Let Assumptions[1](https://arxiv.org/html/2607.08971#Thmassumption1)and[2](https://arxiv.org/html/2607.08971#Thmassumption2)hold, and let the coordinate masks be i\.i\.d\.Bernoulli​\(p\)\\mathrm\{Bernoulli\}\(p\), independent across coordinates, arms, and rounds, and independent of the actions\. For any fixedt∈\[T\]t\\in\[T\], with probability at least1−δ1\-\\delta,

‖Σ˙t−Σ‖2≤2​BX​λ¯p2​t​K​log⁡2​dδ\+2​BX2p2​t​K​log⁡2​dδ\.\\\|\\dot\{\\Sigma\}\_\{t\}\-\\Sigma\\\|\_\{2\}\\;\\leq\\;2B\_\{X\}\\sqrt\{\\frac\{\\bar\{\\lambda\}\}\{p^\{2\}\\,tK\}\\log\\frac\{2d\}\{\\delta\}\}\\;\+\\;\\frac\{2B\_\{X\}^\{2\}\}\{p^\{2\}\\,tK\}\\log\\frac\{2d\}\{\\delta\}\.\(10\)

Proof:In the following, we will letn:=t​Kn:=tKand index the i\.i\.d\. samples byℓ∈\[n\]\\ell\\in\[n\]\. Let’s setQℓ:=Yℓ−ΣQ\_\{\\ell\}:=Y\_\{\\ell\}\-\\Sigma, soΣ˙t−Σ=n−1​∑ℓQℓ\\dot\{\\Sigma\}\_\{t\}\-\\Sigma=n^\{\-1\}\\sum\_\{\\ell\}Q\_\{\\ell\}and𝔼​\[Qℓ\]=0\\mathbb\{E\}\[Q\_\{\\ell\}\]=0by Lemma[5](https://arxiv.org/html/2607.08971#Thmlemma5)\.

Uniform bound\.Since the mask cannot increase Euclidean norms, we have that‖X˙ℓ‖2≤BX\\\|\\dot\{X\}\_\{\\ell\}\\\|\_\{2\}\\leq B\_\{X\}andmaxj⁡X˙ℓ,j2≤BX2\\max\_\{j\}\\dot\{X\}\_\{\\ell,j\}^\{2\}\\leq B\_\{X\}^\{2\}\. This gives us the following uniform norm bound on theQℓQ\_\{\\ell\}’s

∥Yℓ∥2≤‖X˙ℓ‖22p2\+1−pp2maxjX˙ℓ,j2≤2​BX2p2,∥Qℓ∥2≤2​BX2p2\+λ1≤3​BX2p2=:LQ,\\\|Y\_\{\\ell\}\\\|\_\{2\}\\leq\\frac\{\\\|\\dot\{X\}\_\{\\ell\}\\\|\_\{2\}^\{2\}\}\{p^\{2\}\}\+\\frac\{1\-p\}\{p^\{2\}\}\\max\_\{j\}\\dot\{X\}\_\{\\ell,j\}^\{2\}\\leq\\frac\{2B\_\{X\}^\{2\}\}\{p^\{2\}\},\\qquad\\\|Q\_\{\\ell\}\\\|\_\{2\}\\leq\\frac\{2B\_\{X\}^\{2\}\}\{p^\{2\}\}\+\\lambda\_\{1\}\\leq\\frac\{3B\_\{X\}^\{2\}\}\{p^\{2\}\}=:L\_\{Q\},where we of course useλ1≤𝔼​‖X‖22≤BX2\\lambda\_\{1\}\\leq\\mathbb\{E\}\\\|X\\\|\_\{2\}^\{2\}\\leq B\_\{X\}^\{2\}\.

Variance\.Since𝔼​Yℓ=Σ\\mathbb\{E\}Y\_\{\\ell\}=\\Sigma, we have that𝔼​Qℓ2=𝔼​Yℓ2−Σ2⪯𝔼​Yℓ2\\mathbb\{E\}Q\_\{\\ell\}^\{2\}=\\mathbb\{E\}Y\_\{\\ell\}^\{2\}\-\\Sigma^\{2\}\\preceq\\mathbb\{E\}Y\_\{\\ell\}^\{2\}\. We will use the fact thatXj2​\(‖X‖22−Xj2\)≤BX2​Xj2X\_\{j\}^\{2\}\(\\\|X\\\|\_\{2\}^\{2\}\-X\_\{j\}^\{2\}\)\\leq B\_\{X\}^\{2\}X\_\{j\}^\{2\}and0≤p−2−p−1≤p−20\\leq p^\{\-2\}\-p^\{\-1\}\\leq p^\{\-2\}in Lemma[6](https://arxiv.org/html/2607.08971#Thmlemma6)to get

𝔼​\[Yℓ2\|Xℓ\]⪯BX2p​Xℓ​Xℓ⊤\+BX2p2​diag​\(Xℓ​Xℓ⊤\)\.\\mathbb\{E\}\\big\[Y\_\{\\ell\}^\{2\}\\,\\big\|\\,X\_\{\\ell\}\\big\]\\preceq\\frac\{B\_\{X\}^\{2\}\}\{p\}X\_\{\\ell\}X\_\{\\ell\}^\{\\top\}\+\\frac\{B\_\{X\}^\{2\}\}\{p^\{2\}\}\\,\\mathrm\{diag\}\(X\_\{\\ell\}X\_\{\\ell\}^\{\\top\}\)\.Now, taking expectations overXℓX\_\{\\ell\}and writingΣD:=diag​\(Σ\)\\Sigma^\{D\}:=\\mathrm\{diag\}\(\\Sigma\), whose operator norm ismaxj⁡Σj​j≤λ1≤λ¯\\max\_\{j\}\\Sigma\_\{jj\}\\leq\\lambda\_\{1\}\\leq\\bar\{\\lambda\}, we have the following bound on the variance term

∥𝔼Qℓ2∥2≤BX2​λ1p\+BX2​λ¯p2≤2​BX2​λ¯p2=:v,∥∑ℓ=1n𝔼Qℓ2∥2≤nv\.\\big\\\|\\mathbb\{E\}Q\_\{\\ell\}^\{2\}\\big\\\|\_\{2\}\\leq\\frac\{B\_\{X\}^\{2\}\\lambda\_\{1\}\}\{p\}\+\\frac\{B\_\{X\}^\{2\}\\bar\{\\lambda\}\}\{p^\{2\}\}\\leq\\frac\{2B\_\{X\}^\{2\}\\bar\{\\lambda\}\}\{p^\{2\}\}=:v,\\qquad\\Big\\\|\\sum\_\{\\ell=1\}^\{n\}\\mathbb\{E\}Q\_\{\\ell\}^\{2\}\\Big\\\|\_\{2\}\\leq nv\.
Concentration\.The matricesQ1,…,QnQ\_\{1\},\\ldots,Q\_\{n\}are independent \(the offered arms and their masks are i\.i\.d\. and independent of the learner’s policy\), symmetric, and zero\-mean\. Moreover, by the two preceding steps, they satisfy the almost\-sure bound‖Qℓ‖2≤LQ\\\|Q\_\{\\ell\}\\\|\_\{2\}\\leq L\_\{Q\}and the variance bound‖∑ℓ𝔼​Qℓ2‖2≤σ2:=n​v\\big\\\|\\sum\_\{\\ell\}\\mathbb\{E\}Q\_\{\\ell\}^\{2\}\\big\\\|\_\{2\}\\leq\\sigma^\{2\}:=nv\. For such a family of matrices, recall that the Matrix Bernstein’s inequality\[[40](https://arxiv.org/html/2607.08971#bib.bib40)\]says, for everys≥0s\\geq 0,

Pr⁡\(‖∑ℓ=1nQℓ‖2≥s\)≤2​d​exp⁡\(−s2/2σ2\+LQ​s/3\)\.\\Pr\\Big\(\\Big\\\|\\sum\_\{\\ell=1\}^\{n\}Q\_\{\\ell\}\\Big\\\|\_\{2\}\\geq s\\Big\)\\leq 2d\\exp\\\!\\left\(\\frac\{\-s^\{2\}/2\}\{\\sigma^\{2\}\+L\_\{Q\}s/3\}\\right\)\.Equivalently, we may use the following form\. Fixu\>0u\>0, and observe that the right\-hand side above is at most2​d​e−u2d\\,e^\{\-u\}whenevers2/2≥u​\(σ2\+LQ​s/3\)s^\{2\}/2\\geq u\\left\(\\sigma^\{2\}\+L\_\{Q\}s/3\\right\), that is, wheneverssexceeds the larger root

s\+:=LQ​u3\+LQ2​u29\+2​σ2​us\_\{\+\}:=\\frac\{L\_\{Q\}u\}\{3\}\+\\sqrt\{\\frac\{L\_\{Q\}^\{2\}u^\{2\}\}\{9\}\+2\\sigma^\{2\}u\}of the corresponding quadratic equation\. Bya\+b≤a\+b\\sqrt\{a\+b\}\\leq\\sqrt\{a\}\+\\sqrt\{b\}, this root satisfiess\+≤23​LQ​u\+2​σ2​us\_\{\+\}\\leq\\tfrac\{2\}\{3\}L\_\{Q\}u\+\\sqrt\{2\\sigma^\{2\}u\}, so the choices:=2​σ2​u\+23​LQ​us:=\\sqrt\{2\\sigma^\{2\}u\}\+\\tfrac\{2\}\{3\}L\_\{Q\}usuffices, and we get

Pr⁡\(‖∑ℓ=1nQℓ‖2≥2​n​v​u\+23​LQ​u\)≤2​d​e−u\.\\Pr\\Big\(\\Big\\\|\\sum\_\{\\ell=1\}^\{n\}Q\_\{\\ell\}\\Big\\\|\_\{2\}\\geq\\sqrt\{2nvu\}\+\\tfrac\{2\}\{3\}L\_\{Q\}u\\Big\)\\leq 2d\\,e^\{\-u\}\.Settingu=log⁡\(2​d/δ\)u=\\log\(2d/\\delta\)and dividing byn=t​Kn=tKgives us the desired \([10](https://arxiv.org/html/2607.08971#A3.E10)\), since2​v=2​BX​λ¯/p\\sqrt\{2v\}=2B\_\{X\}\\sqrt\{\\bar\{\\lambda\}\}/pand23​LQ=2​BX2/p2\\tfrac\{2\}\{3\}L\_\{Q\}=2B\_\{X\}^\{2\}/p^\{2\}\.

We are now ready to prove the subspace recovery guarantee\.

Proof of Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)\.Recall from Section[3](https://arxiv.org/html/2607.08971#S3)thatκ​m=BX​λ¯/λm\\kappa\\sqrt\{m\}=B\_\{X\}\\sqrt\{\\bar\{\\lambda\}\}/\\lambda\_\{m\}, so that

ϵt=Csub​BX​λ¯λm​p​ut​K,u:=log⁡2​d​Tδ\.\\epsilon\_\{t\}=C\_\{\\mathrm\{sub\}\}\\,\\frac\{B\_\{X\}\\sqrt\{\\bar\{\\lambda\}\}\}\{\\lambda\_\{m\}\\,p\}\\sqrt\{\\frac\{u\}\{tK\}\},\\qquad u:=\\log\\frac\{2dT\}\{\\delta\}\.In the sequel, we will fixCsub=12C\_\{\\mathrm\{sub\}\}=12for the sake of convenience\. Applying Lemma[7](https://arxiv.org/html/2607.08971#Thmlemma7)at levelδ/T\\delta/Tand taking a union bound overt∈\[T\]t\\in\[T\], we get that, with probability at least1−δ1\-\\delta, for allt∈\[T\]t\\in\[T\],

‖Σ˙t−Σ‖2≤Et:=2​BX​λ¯​up2​t​K\+2​BX2​up2​t​K\.\\\|\\dot\{\\Sigma\}\_\{t\}\-\\Sigma\\\|\_\{2\}\\leq E\_\{t\}:=2B\_\{X\}\\sqrt\{\\frac\{\\bar\{\\lambda\}\\,u\}\{p^\{2\}\\,tK\}\}\+\\frac\{2B\_\{X\}^\{2\}u\}\{p^\{2\}\\,tK\}\.For any fixedt∈\[T\]t\\in\[T\], consider the following two cases\.

*Case 1:ϵt≥2\\epsilon\_\{t\}\\geq\\sqrt\{2\}\.*The difference of two orthogonal projectors always has operator norm at most one, so‖𝐏^t−𝐏‖2≤1≤ϵt\\\|\\hat\{\\mathbf\{P\}\}\_\{t\}\-\\mathbf\{P\}\\\|\_\{2\}\\leq 1\\leq\\epsilon\_\{t\}, and the claim holds trivially\.

*Case 2:ϵt<2\\epsilon\_\{t\}<\\sqrt\{2\}\.*By the definition ofϵt\\epsilon\_\{t\}, this means

t​K\>Csub22⋅BX2​λ¯​uλm2​p2=72​BX2​λ¯​uλm2​p2\.tK\>\\frac\{C\_\{\\mathrm\{sub\}\}^\{2\}\}\{2\}\\cdot\\frac\{B\_\{X\}^\{2\}\\bar\{\\lambda\}\\,u\}\{\\lambda\_\{m\}^\{2\}\\,p^\{2\}\}=72\\,\\frac\{B\_\{X\}^\{2\}\\bar\{\\lambda\}\\,u\}\{\\lambda\_\{m\}^\{2\}\\,p^\{2\}\}\.We use this lower bound ont​KtKtwice\. Sinceλm≤λ¯\\lambda\_\{m\}\\leq\\bar\{\\lambda\}, it impliest​K≥BX2​u/\(λ¯​p2\)tK\\geq B\_\{X\}^\{2\}u/\(\\bar\{\\lambda\}p^\{2\}\), which is exactly the condition under which the linear term ofEtE\_\{t\}is at most its square\-root term; henceEt≤4​BX​λ¯​u/\(p2​t​K\)E\_\{t\}\\leq 4B\_\{X\}\\sqrt\{\\bar\{\\lambda\}u/\(p^\{2\}tK\)\}\. Substituting the lower bound ont​KtKinto this expression then givesEt≤4​λm/72=\(2/3\)​λm≤λm/2\.E\_\{t\}\\leq 4\\lambda\_\{m\}/\\sqrt\{72\}=\(\\sqrt\{2\}/3\)\\,\\lambda\_\{m\}\\leq\\lambda\_\{m\}/2\.The Davis–Kahansin⁡Θ\\sin\\Thetatheorem\[[31](https://arxiv.org/html/2607.08971#bib.bib31)\]applied toΣ\\Sigma, whosemm\-th eigenvalue isλm\\lambda\_\{m\}and whose\(m\+1\)\(m\{\+\}1\)\-st is0, then gives

‖𝐏^t−𝐏‖2≤2​Etλm≤8​BX​λ¯λm​p​ut​K=8Csub​ϵt≤ϵt\.\\\|\\hat\{\\mathbf\{P\}\}\_\{t\}\-\\mathbf\{P\}\\\|\_\{2\}\\leq\\frac\{2E\_\{t\}\}\{\\lambda\_\{m\}\}\\leq\\frac\{8B\_\{X\}\\sqrt\{\\bar\{\\lambda\}\}\}\{\\lambda\_\{m\}\\,p\}\\sqrt\{\\frac\{u\}\{tK\}\}=\\frac\{8\}\{C\_\{\\mathrm\{sub\}\}\}\\,\\epsilon\_\{t\}\\leq\\epsilon\_\{t\}\.□\\square

In the imputation analysis of Appendix[D](https://arxiv.org/html/2607.08971#A4), we need a version of this guarantee for the basis matrices rather than the projectors\. The following standard argument shows us how one can obtain the former from the latter\.

###### Lemma 8\(Basis alignment\)

Let𝐔^,𝐔∈ℝd×m\\hat\{\\mathbf\{U\}\},\\mathbf\{U\}\\in\\mathbb\{R\}^\{d\\times m\}have orthonormal columns, with projectors𝐏^=𝐔^​𝐔^⊤\\hat\{\\mathbf\{P\}\}=\\hat\{\\mathbf\{U\}\}\\hat\{\\mathbf\{U\}\}^\{\\top\}and𝐏=𝐔𝐔⊤\\mathbf\{P\}=\\mathbf\{U\}\\mathbf\{U\}^\{\\top\}\. Then there is an orthogonalO∈ℝm×mO\\in\\mathbb\{R\}^\{m\\times m\}with‖𝐔^​O−𝐔‖2≤2​‖𝐏^−𝐏‖2\\\|\\hat\{\\mathbf\{U\}\}O\-\\mathbf\{U\}\\\|\_\{2\}\\leq\\sqrt\{2\}\\,\\\|\\hat\{\\mathbf\{P\}\}\-\\mathbf\{P\}\\\|\_\{2\}\.

Proof:Consider an SVD of them×mm\\times mmatrix𝐔^⊤​𝐔\\hat\{\\mathbf\{U\}\}^\{\\top\}\\mathbf\{U\}\. Every singular value of this matrix is of the form⟨𝐔^​x,𝐔​y⟩\\langle\\hat\{\\mathbf\{U\}\}x,\\mathbf\{U\}y\\ranglefor unit vectorsx,y∈ℝmx,y\\in\\mathbb\{R\}^\{m\}\. Since𝐔^\\hat\{\\mathbf\{U\}\}and𝐔\\mathbf\{U\}have orthonormal columns,𝐔^​x\\hat\{\\mathbf\{U\}\}xand𝐔​y\\mathbf\{U\}yare also unit vectors, and therefore, by Cauchy–Schwarz all singular values lie in\[0,1\]\[0,1\]\. We may therefore parametrize them ascos⁡θ1≥⋯≥cos⁡θm\\cos\\theta\_\{1\}\\geq\\cdots\\geq\\cos\\theta\_\{m\}for anglesθ1≤⋯≤θm=:θmax\\theta\_\{1\}\\leq\\cdots\\leq\\theta\_\{m\}=:\\theta\_\{\\max\}in\[0,π/2\]\[0,\\pi/2\]\. These are, by definition, the principal angles between the two subspaces\[[41](https://arxiv.org/html/2607.08971#bib.bib41)\]\. So, if we write the SVD of𝐔^⊤​𝐔^\\hat\{\\mathbf\{U\}\}^\{\\top\}\\hat\{\\mathbf\{U\}\}as

𝐔^⊤​𝐔=A​cos⁡Θ​B⊤,cos⁡Θ:=diag​\(cos⁡θ1,…,cos⁡θm\),\\hat\{\\mathbf\{U\}\}^\{\\top\}\\mathbf\{U\}=A\\cos\\Theta\\,B^\{\\top\},\\qquad\\cos\\Theta:=\\mathrm\{diag\}\(\\cos\\theta\_\{1\},\\ldots,\\cos\\theta\_\{m\}\),and setO:=A​B⊤O:=AB^\{\\top\}, which is orthogonal, we get the following:

\(𝐔^​O−𝐔\)⊤​\(𝐔^​O−𝐔\)=O⊤​𝐔^⊤​𝐔^​O−O⊤​𝐔^⊤​𝐔−𝐔⊤​𝐔^​O\+𝐔⊤​𝐔\.\(\\hat\{\\mathbf\{U\}\}O\-\\mathbf\{U\}\)^\{\\top\}\(\\hat\{\\mathbf\{U\}\}O\-\\mathbf\{U\}\)=O^\{\\top\}\\hat\{\\mathbf\{U\}\}^\{\\top\}\\hat\{\\mathbf\{U\}\}O\-O^\{\\top\}\\hat\{\\mathbf\{U\}\}^\{\\top\}\\mathbf\{U\}\-\\mathbf\{U\}^\{\\top\}\\hat\{\\mathbf\{U\}\}O\+\\mathbf\{U\}^\{\\top\}\\mathbf\{U\}\.The first and last terms each equalImI\_\{m\}\. For the cross terms,O⊤​𝐔^⊤​𝐔=B​A⊤⋅A​cos⁡Θ​B⊤=B​cos⁡Θ​B⊤O^\{\\top\}\\hat\{\\mathbf\{U\}\}^\{\\top\}\\mathbf\{U\}=BA^\{\\top\}\\cdot A\\cos\\Theta\\,B^\{\\top\}=B\\cos\\Theta B^\{\\top\}, and𝐔⊤​𝐔^​O\\mathbf\{U\}^\{\\top\}\\hat\{\\mathbf\{U\}\}Ois its transpose, which is the same symmetric matrix\. Therefore

\(𝐔^​O−𝐔\)⊤​\(𝐔^​O−𝐔\)=2​I−2​B​cos⁡Θ​B⊤=B​\(2​I−2​cos⁡Θ\)​B⊤\.\(\\hat\{\\mathbf\{U\}\}O\-\\mathbf\{U\}\)^\{\\top\}\(\\hat\{\\mathbf\{U\}\}O\-\\mathbf\{U\}\)=2I\-2B\\cos\\Theta B^\{\\top\}=B\\left\(2I\-2\\cos\\Theta\\right\)B^\{\\top\}\.Notice that this is an eigendecomposition with eigenvalues2​\(1−cos⁡θj\)2\(1\-\\cos\\theta\_\{j\}\), the largest value being attained atθmax\\theta\_\{\\max\}\. This implies the following inequality:

‖𝐔^​O−𝐔‖22=2​\(1−cos⁡θmax\)≤2​\(1−cos⁡θmax\)​\(1\+cos⁡θmax\)=2​sin2⁡θmax,\\\|\\hat\{\\mathbf\{U\}\}O\-\\mathbf\{U\}\\\|\_\{2\}^\{2\}=2\(1\-\\cos\\theta\_\{\\max\}\)\\leq 2\(1\-\\cos\\theta\_\{\\max\}\)\(1\+\\cos\\theta\_\{\\max\}\)=2\\sin^\{2\}\\theta\_\{\\max\},where we use the fact thatcos⁡θmax∈\[0,1\]\\cos\\theta\_\{\\max\}\\in\[0,1\]\. Since‖𝐏^−𝐏‖2=sin⁡θmax\\\|\\hat\{\\mathbf\{P\}\}\-\\mathbf\{P\}\\\|\_\{2\}=\\sin\\theta\_\{\\max\}for equal\-rank projectors \(see, e\.g\.,\[[42](https://arxiv.org/html/2607.08971#bib.bib42)\]\), the claim follows\.

###### Corollary 1

On the event of Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1), simultaneously for allt∈\[T\]t\\in\[T\], we have the following

minO∈𝕆​\(m\)⁡‖𝐔^t​O−𝐔‖2≤ϵt,\\min\_\{O\\in\\mathbb\{O\}\(m\)\}\\\|\\hat\{\\mathbf\{U\}\}\_\{t\}O\-\\mathbf\{U\}\\\|\_\{2\}\\leq\\epsilon\_\{t\},where𝕆​\(m\)\\mathbb\{O\}\(m\)is the set ofm×mm\\times morthogonal matrices\.

Proof:We consider two cases: Case 1:ϵt≥2\\epsilon\_\{t\}\\geq\\sqrt\{2\}\. In this situation, notice that Lemma[8](https://arxiv.org/html/2607.08971#Thmlemma8)gives us

minO⁡‖𝐔^t​O−𝐔‖2≤2​‖𝐏^t−𝐏‖2≤2≤ϵt\.\\min\_\{O\}\\\|\\hat\{\\mathbf\{U\}\}\_\{t\}O\-\\mathbf\{U\}\\\|\_\{2\}\\leq\\sqrt\{2\}\\,\\\|\\hat\{\\mathbf\{P\}\}\_\{t\}\-\\mathbf\{P\}\\\|\_\{2\}\\leq\\sqrt\{2\}\\leq\\epsilon\_\{t\}\.Case 2:ϵt<2\\epsilon\_\{t\}<\\sqrt\{2\}Here, Case 2 of the preceding proof gives‖𝐏^t−𝐏‖2≤\(8/Csub\)​ϵt\\\|\\hat\{\\mathbf\{P\}\}\_\{t\}\-\\mathbf\{P\}\\\|\_\{2\}\\leq\(8/C\_\{\\mathrm\{sub\}\}\)\\epsilon\_\{t\}\. This, along with Lemma[8](https://arxiv.org/html/2607.08971#Thmlemma8), implies that

minO⁡‖𝐔^t​O−𝐔‖2≤\(8​2/Csub\)​ϵt≤ϵt,\\min\_\{O\}\\\|\\hat\{\\mathbf\{U\}\}\_\{t\}O\-\\mathbf\{U\}\\\|\_\{2\}\\leq\(8\\sqrt\{2\}/C\_\{\\mathrm\{sub\}\}\)\\epsilon\_\{t\}\\leq\\epsilon\_\{t\},where we used the fact thatCsub=12≥8​2C\_\{\\mathrm\{sub\}\}=12\\geq 8\\sqrt\{2\}\.

## Appendix DImputation Error: Proof of Lemma[2](https://arxiv.org/html/2607.08971#Thmlemma2)

In Section[C](https://arxiv.org/html/2607.08971#A3), we showed that the projection matrices onto the estimated subspaces converge to the true projector\. That is, on a single high\-probability event,‖𝐏^t−𝐏‖2≤ϵt\\\|\\hat\{\\mathbf\{P\}\}\_\{t\}\-\\mathbf\{P\}\\\|\_\{2\}\\leq\\epsilon\_\{t\}simultaneously for allt∈\[T\]t\\in\[T\]\(Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)\), and the same bound holds for the best\-aligned bases \(Corollary[1](https://arxiv.org/html/2607.08971#Thmcorollary1)\)\. In this section, we prove Lemma[2](https://arxiv.org/html/2607.08971#Thmlemma2), which converts these subspace guarantees into a uniform bound on the error of the least\-squares imputation in Equation \([6](https://arxiv.org/html/2607.08971#S4.E6)\), over every epoch, every round within it, and every offered arm\. Our strategy is guided by the observation that the imputation solves a least\-squares problem on the*observed rows*of the frozen basis𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}, so its stability is governed by the smallest eigenvalue of the observed Gram matrix𝐔^e,Ωt,i⊤​𝐔^e,Ωt,i\\hat\{\\mathbf\{U\}\}\_\{e,\\Omega\_\{t,i\}\}^\{\\top\}\\hat\{\\mathbf\{U\}\}\_\{e,\\Omega\_\{t,i\}\}\. We first prove that the observed rows of the true subspace are well\-conditioned under Bernoulli masking\. We then show that this conditioning transfers to the estimated subspace after sufficient burning in\. Finally, we combine this conditioning with the aligned\-basis guarantee to control the imputation error itself\.

###### Lemma 9\(Conditioning of the true observed subspace\)

Assume𝐔\\mathbf\{U\}isμ\\mu\-incoherent \(i\.e\.,maxj∈\[d\]⁡‖ej⊤​𝐔‖22≤μ2​md\.\\max\_\{j\\in\[d\]\}\\\|e\_\{j\}^\{\\top\}\\mathbf\{U\}\\\|\_\{2\}^\{2\}\\leq\\frac\{\\mu^\{2\}m\}\{d\}\.\) If Equation \([7](https://arxiv.org/html/2607.08971#S4.E7)\) holds withCμC\_\{\\mu\}sufficiently large, then with probability at least1−δ1\-\\delta, simultaneously for allt∈\[T\]t\\in\[T\]and alli∈\[K\]i\\in\[K\],

λmin​\(𝐔Ωt,i⊤​𝐔Ωt,i\)≥3​p4\.\\displaystyle\\lambda\_\{\\min\}\(\\mathbf\{U\}\_\{\\Omega\_\{t,i\}\}^\{\\top\}\\mathbf\{U\}\_\{\\Omega\_\{t,i\}\}\)\\geq\\frac\{3p\}\{4\}\.\(11\)

Proof:For a fixed maskΩ\\Omega, notice that we can write

𝐔Ω⊤​𝐔Ω=∑j=1dSj​uj​uj⊤,\\mathbf\{U\}\_\{\\Omega\}^\{\\top\}\\mathbf\{U\}\_\{\\Omega\}=\\sum\_\{j=1\}^\{d\}S\_\{j\}u\_\{j\}u\_\{j\}^\{\\top\},whereSj∼Bernoulli​\(p\)S\_\{j\}\\sim\\mathrm\{Bernoulli\}\(p\)anduj⊤=ej⊤​𝐔u\_\{j\}^\{\\top\}=e\_\{j\}^\{\\top\}\\mathbf\{U\}is thejj\-th row of𝐔\\mathbf\{U\}\. Indeed, each summand is positive semidefinite and satisfies

0⪯Sj​uj​uj⊤⪯μ2​md​Im\.0\\preceq S\_\{j\}u\_\{j\}u\_\{j\}^\{\\top\}\\preceq\\frac\{\\mu^\{2\}m\}\{d\}I\_\{m\}\.Moreover,𝔼​\[𝐔Ω⊤​𝐔Ω\]=p​∑j=1duj​uj⊤=p​Im\.\\mathbb\{E\}\[\\mathbf\{U\}\_\{\\Omega\}^\{\\top\}\\mathbf\{U\}\_\{\\Omega\}\]=p\\sum\_\{j=1\}^\{d\}u\_\{j\}u\_\{j\}^\{\\top\}=pI\_\{m\}\.Now, we can use the Matrix Chernoff bound\[[40](https://arxiv.org/html/2607.08971#bib.bib40)\]and observe that

Pr⁡\(λmin​\(𝐔Ω⊤​𝐔Ω\)≤3​p4\)≤m​exp⁡\(−c​p​dμ2​m\)\.\\Pr\\\!\\left\(\\lambda\_\{\\min\}\(\\mathbf\{U\}\_\{\\Omega\}^\{\\top\}\\mathbf\{U\}\_\{\\Omega\}\)\\leq\\frac\{3p\}\{4\}\\right\)\\leq m\\exp\\\!\\left\(\-c\\frac\{pd\}\{\\mu^\{2\}m\}\\right\)\.Choosingp≥Cμ​\(μ2​m/d\)​log⁡\(m​T​K/δ\)p\\geq C\_\{\\mu\}\(\\mu^\{2\}m/d\)\\log\(mTK/\\delta\)and taking a union bound over allT​KTKmasks gives us the desired result\.

###### Lemma 10\(Conditioning of the estimated observed subspace\)

Fix an epochee, and write𝐏^e:=𝐔^e​𝐔^e⊤\\hat\{\\mathbf\{P\}\}\_\{e\}:=\\hat\{\\mathbf\{U\}\}\_\{e\}\\hat\{\\mathbf\{U\}\}\_\{e\}^\{\\top\}\. On the event of Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1), since𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}is computed from theτe−1≥tb\\tau\_\{e\}\-1\\geq t\_\{b\}rounds preceding the epoch, we have‖𝐏^e−𝐏‖2≤ϵe≤p/32\\\|\\hat\{\\mathbf\{P\}\}\_\{e\}\-\\mathbf\{P\}\\\|\_\{2\}\\leq\\epsilon\_\{e\}\\leq p/32\. On the event of Lemma[9](https://arxiv.org/html/2607.08971#Thmlemma9), for allt∈𝔗et\\in\\mathfrak\{T\}\_\{e\}andi∈\[K\]i\\in\[K\],

λmin​\(𝐔^e,Ωt,i⊤​𝐔^e,Ωt,i\)≥p2\.\\displaystyle\\lambda\_\{\\min\}\(\\hat\{\\mathbf\{U\}\}\_\{e,\\Omega\_\{t,i\}\}^\{\\top\}\\hat\{\\mathbf\{U\}\}\_\{e,\\Omega\_\{t,i\}\}\)\\geq\\frac\{p\}\{2\}\.\(12\)

Proof:LetRΩR\_\{\\Omega\}be the coordinate\-selection matrix forΩ=Ωt,i\\Omega=\\Omega\_\{t,i\}\. SinceRΩ​𝐏​RΩ⊤=𝐔Ω​𝐔Ω⊤R\_\{\\Omega\}\\mathbf\{P\}R\_\{\\Omega\}^\{\\top\}=\\mathbf\{U\}\_\{\\Omega\}\\mathbf\{U\}\_\{\\Omega\}^\{\\top\}, the non\-zero eigenvalues ofRΩ​𝐏​RΩ⊤R\_\{\\Omega\}\\mathbf\{P\}R\_\{\\Omega\}^\{\\top\}coincide with the eigenvalues of𝐔Ω⊤​𝐔Ω\\mathbf\{U\}\_\{\\Omega\}^\{\\top\}\\mathbf\{U\}\_\{\\Omega\}\. Similarly, the nonzero eigenvalues ofRΩ​𝐏^e​RΩ⊤R\_\{\\Omega\}\\hat\{\\mathbf\{P\}\}\_\{e\}R\_\{\\Omega\}^\{\\top\}coincide with those of𝐔^e,Ω⊤​𝐔^e,Ω\\hat\{\\mathbf\{U\}\}\_\{e,\\Omega\}^\{\\top\}\\hat\{\\mathbf\{U\}\}\_\{e,\\Omega\}\.

Note that on the event of Lemma[9](https://arxiv.org/html/2607.08971#Thmlemma9)we haveλmin​\(𝐔Ω⊤​𝐔Ω\)≥3​p/4\>0\\lambda\_\{\\min\}\(\\mathbf\{U\}\_\{\\Omega\}^\{\\top\}\\mathbf\{U\}\_\{\\Omega\}\)\\geq 3p/4\>0, which forcesrank​\(𝐔Ω\)=m\\mathrm\{rank\}\(\\mathbf\{U\}\_\{\\Omega\}\)=mand in particular\|Ω\|≥m\|\\Omega\|\\geq m; hence the spectra of them×mm\\times mGram matrices are exactly the topmmeigenvalues of the corresponding restricted projectors, and Weyl’s inequality may be applied to themm\-th eigenvalue\. Therefore, by Weyl’s inequality\[[43](https://arxiv.org/html/2607.08971#bib.bib43)\],

λmin​\(𝐔^e,Ω⊤​𝐔^e,Ω\)\\displaystyle\\lambda\_\{\\min\}\(\\hat\{\\mathbf\{U\}\}\_\{e,\\Omega\}^\{\\top\}\\hat\{\\mathbf\{U\}\}\_\{e,\\Omega\}\)≥λmin​\(𝐔Ω⊤​𝐔Ω\)−‖RΩ​\(𝐏^e−𝐏\)​RΩ⊤‖2\\displaystyle\\geq\\lambda\_\{\\min\}\(\\mathbf\{U\}\_\{\\Omega\}^\{\\top\}\\mathbf\{U\}\_\{\\Omega\}\)\-\\\|R\_\{\\Omega\}\(\\hat\{\\mathbf\{P\}\}\_\{e\}\-\\mathbf\{P\}\)R\_\{\\Omega\}^\{\\top\}\\\|\_\{2\}≥3​p4−‖𝐏^e−𝐏‖2≥3​p4−p32≥p2\.\\displaystyle\\geq\\frac\{3p\}\{4\}\-\\\|\\hat\{\\mathbf\{P\}\}\_\{e\}\-\\mathbf\{P\}\\\|\_\{2\}\\geq\\frac\{3p\}\{4\}\-\\frac\{p\}\{32\}\\geq\\frac\{p\}\{2\}\.This proves the claim\.

We are now ready to prove Lemma[2](https://arxiv.org/html/2607.08971#Thmlemma2)\. At a high level, the imputation error has two components\. First, even if the latent coefficient were known, reconstructing with the frozen basis𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}instead of𝐔\\mathbf\{U\}creates an error proportional to the subspace error\. Second, the coefficient estimated from the observed entries is itself perturbed by the subspace error, and this perturbation is amplified by the inverse observed Gram matrix; since the smallest eigenvalue of that Gram matrix is at leastp/2p/2, the amplification is at most2/p2/p\.

Proof:We invoke Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)and Lemma[9](https://arxiv.org/html/2607.08971#Thmlemma9), each at failure probabilityδ\\delta, and work on the intersection of the two events\. By a union bound, this intersection has probability at least1−2​δ≥1−3​δ1\-2\\delta\\geq 1\-3\\delta\. On it, the conclusions of Lemma[10](https://arxiv.org/html/2607.08971#Thmlemma10)and Corollary[1](https://arxiv.org/html/2607.08971#Thmcorollary1)hold deterministically\.

Fix an epochee, a roundt∈𝔗et\\in\\mathfrak\{T\}\_\{e\}, and an armi∈\[K\]i\\in\[K\], and abbreviateΩ=Ωt,i\\Omega=\\Omega\_\{t,i\}and𝐔^=𝐔^e\\hat\{\\mathbf\{U\}\}=\\hat\{\\mathbf\{U\}\}\_\{e\}\. Recall thatϵe=ϵτe−1\\epsilon\_\{e\}=\\epsilon\_\{\\tau\_\{e\}\-1\}, and that Corollary[1](https://arxiv.org/html/2607.08971#Thmcorollary1), applied at the epoch start, controls the aligned distance of𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}at this level\. SinceXt,i∈span​\(𝐔\)X\_\{t,i\}\\in\\mathrm\{span\}\(\\mathbf\{U\}\), there existsa∈ℝma\\in\\mathbb\{R\}^\{m\}such that

Xt,i=𝐔​a,‖a‖2=‖Xt,i‖2≤BX\.X\_\{t,i\}=\\mathbf\{U\}a,\\qquad\\\|a\\\|\_\{2\}=\\\|X\_\{t,i\}\\\|\_\{2\}\\leq B\_\{X\}\.The imputed coefficient is

a^=\(𝐔^Ω⊤​𝐔^Ω\)−1​𝐔^Ω⊤​𝐔Ω​a\.\\hat\{a\}=\(\\hat\{\\mathbf\{U\}\}\_\{\\Omega\}^\{\\top\}\\hat\{\\mathbf\{U\}\}\_\{\\Omega\}\)^\{\-1\}\\hat\{\\mathbf\{U\}\}\_\{\\Omega\}^\{\\top\}\\mathbf\{U\}\_\{\\Omega\}a\.Notice that the imputed vectorX^t,i\\hat\{X\}\_\{t,i\}is invariant under right\-orthogonal rotations𝐔^↦𝐔^​O\\hat\{\\mathbf\{U\}\}\\mapsto\\hat\{\\mathbf\{U\}\}Osince the coefficient transforms asa^↦O⊤​a^\\hat\{a\}\\mapsto O^\{\\top\}\\hat\{a\}, and the reconstruction𝐔^Ωc​a^\\hat\{\\mathbf\{U\}\}\_\{\\Omega^\{c\}\}\\hat\{a\}remains unchanged\. Similarly,λmin​\(𝐔^Ω⊤​𝐔^Ω\)\\lambda\_\{\\min\}\(\\hat\{\\mathbf\{U\}\}\_\{\\Omega\}^\{\\top\}\\hat\{\\mathbf\{U\}\}\_\{\\Omega\}\)is rotation\-invariant, so Lemma[10](https://arxiv.org/html/2607.08971#Thmlemma10)is unaffected\. Therefore, in what follows, we assume that𝐔^\\hat\{\\mathbf\{U\}\}is the aligned basis of Corollary[1](https://arxiv.org/html/2607.08971#Thmcorollary1), so that we actually have

‖𝐔^−𝐔‖2≤ϵe\.\\\|\\hat\{\\mathbf\{U\}\}\-\\mathbf\{U\}\\\|\_\{2\}\\leq\\epsilon\_\{e\}\.
Next, we letG^:=𝐔^Ω⊤​𝐔^Ω\\hat\{G\}:=\\hat\{\\mathbf\{U\}\}\_\{\\Omega\}^\{\\top\}\\hat\{\\mathbf\{U\}\}\_\{\\Omega\}\. And, sinceG^​\(a^−a\)=𝐔^Ω⊤​\(𝐔Ω−𝐔^Ω\)​a,\\hat\{G\}\(\\hat\{a\}\-a\)=\\hat\{\\mathbf\{U\}\}\_\{\\Omega\}^\{\\top\}\(\\mathbf\{U\}\_\{\\Omega\}\-\\hat\{\\mathbf\{U\}\}\_\{\\Omega\}\)a,Lemma[10](https://arxiv.org/html/2607.08971#Thmlemma10)gives us the following ineqiality:

‖a^−a‖2\\displaystyle\\\|\\hat\{a\}\-a\\\|\_\{2\}≤‖G^−1‖2​‖𝐔^Ω‖2​‖𝐔Ω−𝐔^Ω‖2​‖a‖2\\displaystyle\\leq\\\|\\hat\{G\}^\{\-1\}\\\|\_\{2\}\\,\\\|\\hat\{\\mathbf\{U\}\}\_\{\\Omega\}\\\|\_\{2\}\\,\\\|\\mathbf\{U\}\_\{\\Omega\}\-\\hat\{\\mathbf\{U\}\}\_\{\\Omega\}\\\|\_\{2\}\\,\\\|a\\\|\_\{2\}≤2p⋅1⋅ϵe⋅BX=2​BX​ϵep\.\\displaystyle\\leq\\frac\{2\}\{p\}\\cdot 1\\cdot\\epsilon\_\{e\}\\cdot B\_\{X\}=\\frac\{2B\_\{X\}\\epsilon\_\{e\}\}\{p\}\.
Indeed, the imputation error is zero on the observed coordinates and, on the missing coordinates, we have

‖Xt,i\(Ωc\)−X^t,i\(Ωc\)‖2\\displaystyle\\\|X\_\{t,i\}^\{\(\\Omega^\{c\}\)\}\-\\hat\{X\}\_\{t,i\}^\{\(\\Omega^\{c\}\)\}\\\|\_\{2\}=‖𝐔Ωc​a−𝐔^Ωc​a^‖2\\displaystyle=\\\|\\mathbf\{U\}\_\{\\Omega^\{c\}\}a\-\\hat\{\\mathbf\{U\}\}\_\{\\Omega^\{c\}\}\\hat\{a\}\\\|\_\{2\}≤‖\(𝐔Ωc−𝐔^Ωc\)​a‖2\+‖𝐔^Ωc​\(a−a^\)‖2\\displaystyle\\leq\\\|\(\\mathbf\{U\}\_\{\\Omega^\{c\}\}\-\\hat\{\\mathbf\{U\}\}\_\{\\Omega^\{c\}\}\)a\\\|\_\{2\}\+\\\|\\hat\{\\mathbf\{U\}\}\_\{\\Omega^\{c\}\}\(a\-\\hat\{a\}\)\\\|\_\{2\}≤BX​ϵe\+‖a−a^‖2\\displaystyle\\leq B\_\{X\}\\epsilon\_\{e\}\+\\\|a\-\\hat\{a\}\\\|\_\{2\}≤\(1\+2p\)​BX​ϵe\.\\displaystyle\\leq\\left\(1\+\\frac\{2\}\{p\}\\right\)B\_\{X\}\\epsilon\_\{e\}\.Since the observed coordinates are copied exactly, the same bound holds for the full vector:

‖Xt,i−X^t,i‖2≤\(1\+2p\)​BX​ϵe\.\\\|X\_\{t,i\}\-\\hat\{X\}\_\{t,i\}\\\|\_\{2\}\\leq\\left\(1\+\\frac\{2\}\{p\}\\right\)B\_\{X\}\\epsilon\_\{e\}\.The argument is uniform over all epochsee, roundst∈𝔗et\\in\\mathfrak\{T\}\_\{e\}, and armsi∈\[K\]i\\in\[K\]on the same good event, which proves the lemma\.

## Appendix EEpoch\-wise Surrogate Model and Estimation Error Proofs

This appendix proves the confidence set bounds of Theorem[1](https://arxiv.org/html/2607.08971#Thmtheorem1), together with the supporting lemmas stated in Section[4\.3](https://arxiv.org/html/2607.08971#S4.SS3)\. Throughout, we fix an epocheeand recall that the representation𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}is frozen for its duration\. The proof proceeds in three stages, mirroring the error decomposition

Ve,t​\(ϑ^e,t−ϑe⋆\)=−λ​ϑe⋆\+∑s∈𝔗es<tzs​ξs\+∑s∈𝔗es<tzs​ηsV\_\{e,t\}\\left\(\\hat\{\\vartheta\}\_\{e,t\}\-\\vartheta\_\{e\}^\{\\star\}\\right\)=\-\\lambda\\vartheta\_\{e\}^\{\\star\}\+\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}\\xi\_\{s\}\+\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}\\eta\_\{s\}from Section[4\.3](https://arxiv.org/html/2607.08971#S4.SS3)\. First, we prove the surrogate approximation guarantee \(Lemma[3](https://arxiv.org/html/2607.08971#Thmlemma3)\): on the representation event, the frozen\-coordinate surrogate mean is withinbeb\_\{e\}of the true mean reward, which justifies the reward decompositionrt=⟨zt,ϑe⋆⟩\+ξt\+ηtr\_\{t\}=\\langle z\_\{t\},\\vartheta\_\{e\}^\{\\star\}\\rangle\+\\xi\_\{t\}\+\\eta\_\{t\}with\|ξt\|≤be\|\\xi\_\{t\}\|\\leq b\_\{e\}\. This is where the subspace and imputation guarantees of Appendices[C](https://arxiv.org/html/2607.08971#A3)and[D](https://arxiv.org/html/2607.08971#A4)enter the bandit analysis\. Second, we control the aggregated misspecification term∑s<tzs​ξs\\sum\_\{s<t\}z\_\{s\}\\xi\_\{s\}\(Lemma[4](https://arxiv.org/html/2607.08971#Thmlemma4)\)\. Theorem[1](https://arxiv.org/html/2607.08971#Thmtheorem1)then follows by takingVe,t−1V\_\{e,t\}^\{\-1\}\-weighted norms in the error decomposition above, and combining with a self\-normalized inequality argument that controls the noise term\.

### E\.1Surrogate approximation

We first reintroduce the fixed\-epoch objects used in the proof\. Fix an epochee\. Fort∈𝔗et\\in\\mathfrak\{T\}\_\{e\}andi∈\[K\]i\\in\[K\], letzt,i:=𝐔^e⊤​X^t,iz\_\{t,i\}:=\\hat\{\\mathbf\{U\}\}\_\{e\}^\{\\top\}\\hat\{X\}\_\{t,i\},ϑe⋆:=𝐔^e⊤​θ⋆\\vartheta\_\{e\}^\{\\star\}:=\\hat\{\\mathbf\{U\}\}\_\{e\}^\{\\top\}\\theta^\{\\star\}, andμ¯t,i:=⟨zt,i,ϑe⋆⟩\\bar\{\\mu\}\_\{t,i\}:=\\langle z\_\{t,i\},\\vartheta\_\{e\}^\{\\star\}\\rangle\. The purpose of this section is to show that, after conditioning on the representation event, the epoch behaves like an ordinarymm\-dimensional linear bandit with a controlled misspecification term\. The first step is to compare the true reward mean with its frozen\-coordinate surrogate\. Notice that the error has exactly two sources: the estimated projection is not exactly the true projection, and the current partially observed arm must be imputed before it can be projected\.

###### Lemma 11\(Restatement of Lemma[3](https://arxiv.org/html/2607.08971#Thmlemma3)\)

On the eventℰrep\\mathcal\{E\}\_\{\\rm rep\}defined in Section[2](https://arxiv.org/html/2607.08971#Thmlemma2), for every epochee, every roundt∈𝔗et\\in\\mathfrak\{T\}\_\{e\}, and every armi∈\[K\]i\\in\[K\],

\|⟨Xt,i,θ⋆⟩−μ¯t,i\|≤be,be:=S​BX​\(2\+2p\)​ϵe\.\\left\|\\langle X\_\{t,i\},\\theta^\{\\star\}\\rangle\-\\bar\{\\mu\}\_\{t,i\}\\right\|\\leq b\_\{e\},\\qquad b\_\{e\}:=SB\_\{X\}\\left\(2\+\\frac\{2\}\{p\}\\right\)\\epsilon\_\{e\}\.

Proof:

First we begin by observing that

μ¯t,i=zt,i⊤​ϑe∗=\(𝐔^e⊤​X^t,i\)⊤​\(𝐔^e⊤​θ⋆\)=⟨𝐏^e​X^t,i,θ⋆⟩\.\\bar\{\\mu\}\_\{t,i\}=z\_\{t,i\}^\{\\top\}\\vartheta^\{\\ast\}\_\{e\}=\(\\hat\{\\mathbf\{U\}\}\_\{e\}^\{\\top\}\\hat\{X\}\_\{t,i\}\)^\{\\top\}\(\\hat\{\\mathbf\{U\}\}\_\{e\}^\{\\top\}\\theta^\{\\star\}\)=\\left\\langle\\hat\{\\mathbf\{P\}\}\_\{e\}\\hat\{X\}\_\{t,i\},\\theta^\{\\star\}\\right\\rangle\.Therefore, onℰrep\\mathcal\{E\}\_\{\\rm rep\}, we have the following inequality:

\|⟨Xt,i,θ⋆⟩−μ¯t,i\|=\|⟨Xt,i−𝐏^e​X^t,i,θ⋆⟩\|≤S​‖Xt,i−𝐏^e​X^t,i‖2,\|\\left\\langle X\_\{t,i\},\\theta^\{\\star\}\\right\\rangle\-\\bar\{\\mu\}\_\{t,i\}\|=\\left\|\\left\\langle X\_\{t,i\}\-\\hat\{\\mathbf\{P\}\}\_\{e\}\\hat\{X\}\_\{t,i\},\\theta^\{\\star\}\\right\\rangle\\right\|\\leq S\\left\\\|\{X\_\{t,i\}\-\\hat\{\\mathbf\{P\}\}\_\{e\}\\hat\{X\}\_\{t,i\}\}\\right\\\|\_\{2\},\(13\)which follows from the Cauchy\-Schwarz inequality\. Next, we observe that

Xt,i−𝐏^e​X^t,i\\displaystyle X\_\{t,i\}\-\\hat\{\\mathbf\{P\}\}\_\{e\}\\hat\{X\}\_\{t,i\}=\(I−𝐏^e\)​Xt,i\+𝐏^e​\(Xt,i−X^t,i\)\.\\displaystyle=\(I\-\\hat\{\\mathbf\{P\}\}\_\{e\}\)X\_\{t,i\}\+\\hat\{\\mathbf\{P\}\}\_\{e\}\(X\_\{t,i\}\-\\hat\{X\}\_\{t,i\}\)\.Therefore, by triangle inequality,

‖Xt,i−𝐏^e​X^t,i‖2\\displaystyle\\left\\\|\{X\_\{t,i\}\-\\hat\{\\mathbf\{P\}\}\_\{e\}\\hat\{X\}\_\{t,i\}\}\\right\\\|\_\{2\}≤‖\(I−𝐏^e\)​Xt,i‖2\+‖Xt,i−X^t,i‖2\\displaystyle\\leq\\left\\\|\{\(I\-\\hat\{\\mathbf\{P\}\}\_\{e\}\)X\_\{t,i\}\}\\right\\\|\_\{2\}\+\\left\\\|\{X\_\{t,i\}\-\\hat\{X\}\_\{t,i\}\}\\right\\\|\_\{2\}\(14\)We will now bound these two terms\. For the first term, begin by observing that𝐏​Xt,i=Xt,i\\mathbf\{P\}X\_\{t,i\}=X\_\{t,i\},\(I−𝐏^e\)​Xt,i=\(𝐏−𝐏^e\)​Xt,i\.\(I\-\\hat\{\\mathbf\{P\}\}\_\{e\}\)X\_\{t,i\}=\(\\mathbf\{P\}\-\\hat\{\\mathbf\{P\}\}\_\{e\}\)X\_\{t,i\}\.Onℰrep\\mathcal\{E\}\_\{\\rm rep\}, Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1), invoked at the epoch start \(recall thatϵe:=ϵτe−1\\epsilon\_\{e\}:=\\epsilon\_\{\\tau\_\{e\}\-1\}, matching theτe−1\\tau\_\{e\}\-1rounds from which𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}is computed\), gives‖𝐏−𝐏^e‖2≤ϵe\\\|\\mathbf\{P\}\-\\hat\{\\mathbf\{P\}\}\_\{e\}\\\|\_\{2\}\\leq\\epsilon\_\{e\}, and we therefore have:

‖\(I−𝐏^e\)​Xt,i‖2≤‖𝐏−𝐏^e‖2​‖Xt,i‖2≤BX​ϵe\.\\displaystyle\\left\\\|\{\(I\-\\hat\{\\mathbf\{P\}\}\_\{e\}\)X\_\{t,i\}\}\\right\\\|\_\{2\}\\leq\\left\\\|\{\\mathbf\{P\}\-\\hat\{\\mathbf\{P\}\}\_\{e\}\}\\right\\\|\_\{2\}\\left\\\|\{X\_\{t,i\}\}\\right\\\|\_\{2\}\\leq B\_\{X\}\\epsilon\_\{e\}\.\(15\)
Finally, using the imputation bound in Lemma[2](https://arxiv.org/html/2607.08971#Thmlemma2), we have

‖Xt,i−X^t,i‖2≤\(1\+2p\)​BX​ϵe\.\\displaystyle\\left\\\|\{X\_\{t,i\}\-\\hat\{X\}\_\{t,i\}\}\\right\\\|\_\{2\}\\leq\\left\(1\+\\frac\{2\}\{p\}\\right\)B\_\{X\}\\epsilon\_\{e\}\.\(16\)Combining \([13](https://arxiv.org/html/2607.08971#A5.E13)\), \([14](https://arxiv.org/html/2607.08971#A5.E14)\), \([15](https://arxiv.org/html/2607.08971#A5.E15)\), and \([16](https://arxiv.org/html/2607.08971#A5.E16)\) gives

\|⟨Xt,i,θ⋆⟩−μ¯t,i\|≤S​BX​\(2\+2p\)​ϵe=be\.\|\\left\\langle X\_\{t,i\},\\theta^\{\\star\}\\right\\rangle\-\\bar\{\\mu\}\_\{t,i\}\|\\leq SB\_\{X\}\\left\(2\+\\frac\{2\}\{p\}\\right\)\\epsilon\_\{e\}=b\_\{e\}\.
Thus all representation error inside epocheeis controlled by the scalar radiusbeb\_\{e\}\. Once this approximation is in place, the confidence analysis can be carried out in the frozen feature space, provided we account for how the accumulatedbeb\_\{e\}\-misspecification enters the confidence radius\. We do this next\.

### E\.2Proof of Lemma[4](https://arxiv.org/html/2607.08971#Thmlemma4)

We now control the contribution of the bounded misspecification termsξs\\xi\_\{s\}\. A term\-by\-term triangle inequality would be valid but loose; Appendix[G](https://arxiv.org/html/2607.08971#A7)shows this loss explicitly\. The sharper argument keeps the misspecification vector aggregated and uses the fact that the same feature matrix that multiplies the misspecification also appears in the ridge design matrix\.

Proof:The proof proceeds in two steps\. First, we rewrite the misspecification sum in matrix form and express its squaredVe,t−1V\_\{e,t\}^\{\-1\}\-norm as a quadratic form in a leverage matrix\. Second, we show that this leverage matrix is a contraction, so that the quadratic form is bounded by‖ξt‖22\\\|\\xi\_\{t\}\\\|\_\{2\}^\{2\}, which in turn is at mostbe2​\(t−τe\)b\_\{e\}^\{2\}\(t\-\\tau\_\{e\}\)\.

We begin by setting up the matrix form\. Letq:=t−τeq:=t\-\\tau\_\{e\}, and form the feature matrixZtZ\_\{t\}and the misspecification error vectorξt\\xi\_\{t\}as follows:

Zt:=\[zτezτe\+1⋯zt−1\]∈ℝm×q,ξt:=\[ξτeξτe\+1⋯ξt−1\]⊤∈ℝq\.Z\_\{t\}:=\\begin\{bmatrix\}z\_\{\\tau\_\{e\}\}&z\_\{\\tau\_\{e\}\+1\}&\\cdots&z\_\{t\-1\}\\end\{bmatrix\}\\in\\mathbb\{R\}^\{m\\times q\},\\qquad\\xi\_\{t\}:=\\begin\{bmatrix\}\\xi\_\{\\tau\_\{e\}\}&\\xi\_\{\\tau\_\{e\}\+1\}&\\cdots&\\xi\_\{t\-1\}\\end\{bmatrix\}^\{\\top\}\\in\\mathbb\{R\}^\{q\}\.By construction, we have that∑s∈𝔗es<tzs​ξs=Zt​ξt\.\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}\\xi\_\{s\}=Z\_\{t\}\\xi\_\{t\}\.Moreover, sinceVe,tV\_\{e,t\}is the within\-epoch design matrix before roundtt, we may write it as

Ve,t=λ​Im\+∑s∈𝔗es<tzs​zs⊤=λ​Im\+Zt​Zt⊤\.V\_\{e,t\}=\\lambda I\_\{m\}\+\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}z\_\{s\}^\{\\top\}=\\lambda I\_\{m\}\+Z\_\{t\}Z\_\{t\}^\{\\top\}\.Combining the two preceding facts, we have the following identity:

‖∑s∈𝔗es<tzs​ξs‖Ve,t−12\\displaystyle\\left\\\|\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}\\xi\_\{s\}\\right\\\|\_\{V\_\{e,t\}^\{\-1\}\}^\{2\}=‖Zt​ξt‖Ve,t−12\\displaystyle=\\\|Z\_\{t\}\\xi\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}^\{2\}=\(Zt​ξt\)⊤​Ve,t−1​\(Zt​ξt\)\\displaystyle=\(Z\_\{t\}\\xi\_\{t\}\)^\{\\top\}V\_\{e,t\}^\{\-1\}\(Z\_\{t\}\\xi\_\{t\}\)=ξt⊤​Zt⊤​\(λ​Im\+Zt​Zt⊤\)−1​Zt​ξt\.\\displaystyle=\\xi\_\{t\}^\{\\top\}Z\_\{t\}^\{\\top\}\(\\lambda I\_\{m\}\+Z\_\{t\}Z\_\{t\}^\{\\top\}\)^\{\-1\}Z\_\{t\}\\xi\_\{t\}\.
We will next show that the leverage matrix appearing in this quadratic form is a contraction:

Zt⊤​\(λ​Im\+Zt​Zt⊤\)−1​Zt⪯Iq\.Z\_\{t\}^\{\\top\}\(\\lambda I\_\{m\}\+Z\_\{t\}Z\_\{t\}^\{\\top\}\)^\{\-1\}Z\_\{t\}\\preceq I\_\{q\}\.Towards this, we write a thin singular value decompositionZt=A​Σ​B⊤Z\_\{t\}=A\\Sigma B^\{\\top\}, wherer=rank⁡\(Zt\)r=\\operatorname\{rank\}\(Z\_\{t\}\),A∈ℝm×rA\\in\\mathbb\{R\}^\{m\\times r\}andB∈ℝq×rB\\in\\mathbb\{R\}^\{q\\times r\}have orthonormal columns, andΣ=diag⁡\(σ1,…,σr\)\\Sigma=\\operatorname\{diag\}\(\\sigma\_\{1\},\\ldots,\\sigma\_\{r\}\)contains the positive singular values ofZtZ\_\{t\}\. We then have thatZt​Zt⊤=A​Σ2​A⊤\.Z\_\{t\}Z\_\{t\}^\{\\top\}=A\\Sigma^\{2\}A^\{\\top\}\.

Therefore, the matrixλ​Im\+Zt​Zt⊤\\lambda I\_\{m\}\+Z\_\{t\}Z\_\{t\}^\{\\top\}has eigenvalueλ\+σj2\\lambda\+\\sigma\_\{j\}^\{2\}in the direction of thejj\-th column ofAA, and eigenvalueλ\\lambdaon the orthogonal complement ofspan⁡\(A\)\\operatorname\{span\}\(A\)\. This means, we can write:

A⊤​\(λ​Im\+Zt​Zt⊤\)−1​A=diag⁡\(1λ\+σ12,…,1λ\+σr2\)\.A^\{\\top\}\(\\lambda I\_\{m\}\+Z\_\{t\}Z\_\{t\}^\{\\top\}\)^\{\-1\}A=\\operatorname\{diag\}\\left\(\\frac\{1\}\{\\lambda\+\\sigma\_\{1\}^\{2\}\},\\ldots,\\frac\{1\}\{\\lambda\+\\sigma\_\{r\}^\{2\}\}\\right\)\.Now, substituting the singular value decomposition into the leverage matrix, we have:

Zt⊤​\(λ​Im\+Zt​Zt⊤\)−1​Zt\\displaystyle Z\_\{t\}^\{\\top\}\(\\lambda I\_\{m\}\+Z\_\{t\}Z\_\{t\}^\{\\top\}\)^\{\-1\}Z\_\{t\}=B​Σ​A⊤​\(λ​Im\+Zt​Zt⊤\)−1​A​Σ​B⊤\\displaystyle=B\\Sigma A^\{\\top\}\(\\lambda I\_\{m\}\+Z\_\{t\}Z\_\{t\}^\{\\top\}\)^\{\-1\}A\\Sigma B^\{\\top\}=B​diag⁡\(σ12λ\+σ12,…,σr2λ\+σr2\)​B⊤\.\\displaystyle=B\\operatorname\{diag\}\\left\(\\frac\{\\sigma\_\{1\}^\{2\}\}\{\\lambda\+\\sigma\_\{1\}^\{2\}\},\\ldots,\\frac\{\\sigma\_\{r\}^\{2\}\}\{\\lambda\+\\sigma\_\{r\}^\{2\}\}\\right\)B^\{\\top\}\.We observe that the nonzero eigenvalues of this matrix areσj2λ\+σj2,j=1,…,r,\\frac\{\\sigma\_\{j\}^\{2\}\}\{\\lambda\+\\sigma\_\{j\}^\{2\}\},j=1,\\ldots,r,and sinceλ\>0\\lambda\>0, each of these lies in\[0,1\]\[0,1\]\. On the other hand, on the orthogonal complement ofspan⁡\(B\)\\operatorname\{span\}\(B\), the matrix has eigenvalue zero\. This immediately allows us to conclude the claimed contraction property:0⪯Zt⊤​\(λ​Im\+Zt​Zt⊤\)−1​Zt⪯Iq\.0\\preceq Z\_\{t\}^\{\\top\}\(\\lambda I\_\{m\}\+Z\_\{t\}Z\_\{t\}^\{\\top\}\)^\{\-1\}Z\_\{t\}\\preceq I\_\{q\}\.

Indeed, applying this contraction to the quadratic form above, we observe that:

‖Zt​ξt‖Ve,t−12≤‖ξt‖22\.\\\|Z\_\{t\}\\xi\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}^\{2\}\\leq\\\|\\xi\_\{t\}\\\|\_\{2\}^\{2\}\.Finally, since the hypothesis of the lemma gives\|ξs\|≤be\|\\xi\_\{s\}\|\\leq b\_\{e\}for everys∈𝔗es\\in\\mathfrak\{T\}\_\{e\}withs<ts<t, we have:

‖ξt‖22=∑s=τet−1ξs2≤∑s=τet−1be2=be2​\(t−τe\)\.\\\|\\xi\_\{t\}\\\|\_\{2\}^\{2\}=\\sum\_\{s=\\tau\_\{e\}\}^\{t\-1\}\\xi\_\{s\}^\{2\}\\leq\\sum\_\{s=\\tau\_\{e\}\}^\{t\-1\}b\_\{e\}^\{2\}=b\_\{e\}^\{2\}\(t\-\\tau\_\{e\}\)\.Putting everything together, we conclude that

‖∑s∈𝔗es<tzs​ξs‖Ve,t−12≤be2​\(t−τe\)\.\\left\\\|\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}\\xi\_\{s\}\\right\\\|\_\{V\_\{e,t\}^\{\-1\}\}^\{2\}\\leq b\_\{e\}^\{2\}\(t\-\\tau\_\{e\}\)\.Taking square roots gives us the claim\.

The preceding contraction argument shows that the misspecification contribution scales asbe​t−τeb\_\{e\}\\sqrt\{t\-\\tau\_\{e\}\}\. Combining this controlled term with the ridge regularization term and the standard self\-normalized reward\-noise term gives the frozen\-epoch confidence radius\.

### E\.3Proof of Theorem[1](https://arxiv.org/html/2607.08971#Thmtheorem1)

We finish the section by deriving the confidence set for the surrogate parameterϑe⋆\\vartheta\_\{e\}^\{\\star\}\. The calculation is the usual ridge\-regression decomposition, with the additional bounded misspecification term controlled by Lemma[4](https://arxiv.org/html/2607.08971#Thmlemma4)\.

Proof:Recall from Algorithm[1](https://arxiv.org/html/2607.08971#alg1)thatϑ^e,t=Ve,t−1​∑s∈𝔗e,s<tzs​rs\\hat\{\\vartheta\}\_\{e,t\}=V\_\{e,t\}^\{\-1\}\\sum\_\{s\\in\\mathfrak\{T\}\_\{e\},\\,s<t\}z\_\{s\}r\_\{s\}is the epoch\-eeridge estimator\. On the eventℰrep\\mathcal\{E\}\_\{\\rm rep\}, Lemma[3](https://arxiv.org/html/2607.08971#Thmlemma3)allows us to write each within\-epoch reward asrs=zs⊤​ϑe⋆\+ξs\+ηsr\_\{s\}=z\_\{s\}^\{\\top\}\\vartheta\_\{e\}^\{\\star\}\+\\xi\_\{s\}\+\\eta\_\{s\}with\|ξs\|≤be\|\\xi\_\{s\}\|\\leq b\_\{e\}\. Substituting this decomposition into the definition of the estimator, we have:

Ve,t​θ^e,t=∑s∈𝔗e,s<tzs​rs\\displaystyle V\_\{e,t\}\\hat\{\\theta\}\_\{e,t\}=\\sum\_\{s\\in\\mathfrak\{T\}\_\{e\},\\,s<t\}z\_\{s\}r\_\{s\}=∑s<tzs​\(zs⊤​ϑe⋆\+ξs\+ηs\)\\displaystyle=\\sum\_\{s<t\}z\_\{s\}\(z\_\{s\}^\{\\top\}\\vartheta\_\{e\}^\{\\star\}\+\\xi\_\{s\}\+\\eta\_\{s\}\)=\(Ve,t−λ​Im\)​ϑe⋆\+∑s<tzs​ξs\+∑s<tzs​ηs,\\displaystyle=\(V\_\{e,t\}\-\\lambda I\_\{m\}\)\\vartheta\_\{e\}^\{\\star\}\+\\sum\_\{s<t\}z\_\{s\}\\xi\_\{s\}\+\\sum\_\{s<t\}z\_\{s\}\\eta\_\{s\},where all sums are overs∈𝔗es\\in\\mathfrak\{T\}\_\{e\}withs<ts<t\. Therefore

Ve,t​\(θ^e,t−ϑe⋆\)=−λ​ϑe⋆\+∑s<tzs​ξs\+∑s<tzs​ηs\.V\_\{e,t\}\(\\hat\{\\theta\}\_\{e,t\}\-\\vartheta\_\{e\}^\{\\star\}\)=\-\\lambda\\vartheta\_\{e\}^\{\\star\}\+\\sum\_\{s<t\}z\_\{s\}\\xi\_\{s\}\+\\sum\_\{s<t\}z\_\{s\}\\eta\_\{s\}\.Taking theVe,t−1V\_\{e,t\}^\{\-1\}norm of both sides and applying the triangle inequality \(the first term usesVe,t⪰λ​ImV\_\{e,t\}\\succeq\\lambda I\_\{m\}, so that‖λ​ϑe⋆‖Ve,t−1≤λ​‖ϑe⋆‖2\\\|\\lambda\\vartheta\_\{e\}^\{\\star\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}\\leq\\sqrt\{\\lambda\}\\\|\\vartheta\_\{e\}^\{\\star\}\\\|\_\{2\}\), we have:

‖θ^e,t−ϑe⋆‖Ve,t\\displaystyle\\left\\\|\{\\hat\{\\theta\}\_\{e,t\}\-\\vartheta\_\{e\}^\{\\star\}\}\\right\\\|\_\{V\_\{e,t\}\}≤λ​‖ϑe⋆‖2\+‖∑s<tzs​ξs‖Ve,t−1\+‖∑s<tzs​ηs‖Ve,t−1\.\\displaystyle\\leq\\sqrt\{\\lambda\}\\left\\\|\{\\vartheta\_\{e\}^\{\\star\}\}\\right\\\|\_\{2\}\+\\left\\\|\\sum\_\{s<t\}z\_\{s\}\\xi\_\{s\}\\right\\\|\_\{V\_\{e,t\}^\{\-1\}\}\+\\left\\\|\\sum\_\{s<t\}z\_\{s\}\\eta\_\{s\}\\right\\\|\_\{V\_\{e,t\}^\{\-1\}\}\.Since‖ϑe⋆‖2≤‖θ∗‖2≤S\\left\\\|\{\\vartheta\_\{e\}^\{\\star\}\}\\right\\\|\_\{2\}\\leq\\left\\\|\{\\theta^\{\\ast\}\}\\right\\\|\_\{2\}\\leq S, the first term is at mostλ​S\\sqrt\{\\lambda\}S\. By Lemma[4](https://arxiv.org/html/2607.08971#Thmlemma4), the misspecification term is at mostbe​t−τeb\_\{e\}\\sqrt\{t\-\\tau\_\{e\}\}\. Finally, we control the noise term with the self\-normalized inequality ofAbbasi\-Yadkori et al\. \[[1](https://arxiv.org/html/2607.08971#bib.bib1)\]\. This applies here because, conditional on the epoch\-startσ\\sigma\-field, the frozen feature map makes eachzsz\_\{s\}predictable \(measurable with respect to the history available before the rewardrsr\_\{s\}is revealed\), whileηs\\eta\_\{s\}remains conditionallyRR\-sub\-Gaussian; thus∑s<tzs​ηs\\sum\_\{s<t\}z\_\{s\}\\eta\_\{s\}is the standard martingale transform\. We therefore have, with probability at least1−δe1\-\\delta\_\{e\}, simultaneously for allt∈𝔗et\\in\\mathfrak\{T\}\_\{e\},

‖∑s<tzs​ηs‖Ve,t−1≤R​2​log⁡\(det\(Ve,t\)1/2det\(λ​Im\)1/2​δe\)\.\\left\\\|\\sum\_\{s<t\}z\_\{s\}\\eta\_\{s\}\\right\\\|\_\{V\_\{e,t\}^\{\-1\}\}\\leq R\\sqrt\{2\\log\\left\(\\frac\{\\det\(V\_\{e,t\}\)^\{1/2\}\}\{\\det\(\\lambda I\_\{m\}\)^\{1/2\}\\delta\_\{e\}\}\\right\)\}\.Combining these three bounds proves the theorem\.

## Appendix FMain Regret Analysis

This section proves the regret bound for the epoch\-wise version ofTOFU\-POV\. We use the representation eventℰrep\\mathcal\{E\}\_\{\\rm rep\}from Section[4\.1](https://arxiv.org/html/2607.08971#S4.SS1)and the frozen\-epoch confidence sets from Section[4\.3](https://arxiv.org/html/2607.08971#S4.SS3); the only remaining task is to convert these ingredients into cumulative regret\.

Let

τ0:=tb\+1,τe\+1:=2​τe,𝔗e:=\{τe,τe\+1,…,min⁡\(τe\+1−1,T\)\},\\tau\_\{0\}:=t\_\{b\}\+1,\\qquad\\tau\_\{e\+1\}:=2\\tau\_\{e\},\\qquad\\mathfrak\{T\}\_\{e\}:=\\\{\\tau\_\{e\},\\tau\_\{e\}\+1,\\ldots,\\min\(\\tau\_\{e\+1\}\-1,T\)\\\},and writene:=\|𝔗e\|n\_\{e\}:=\|\\mathfrak\{T\}\_\{e\}\|\. We also writeϵe:=ϵτe−1\\epsilon\_\{e\}:=\\epsilon\_\{\\tau\_\{e\}\-1\}, as in the frozen\-epoch representation bounds\. LetEEdenote the final epoch index, soE≤⌈log2⁡T⌉\+1E\\leq\\lceil\\log\_\{2\}T\\rceil\+1\. Throughout this section, we define

RT:=∑t=1T⟨Xt,it⋆−Xt,it,θ⋆⟩,it⋆∈arg⁡maxi∈\[K\]⁡⟨Xt,i,θ⋆⟩\.\\displaystyle R\_\{T\}:=\\sum\_\{t=1\}^\{T\}\\langle X\_\{t,i\_\{t\}^\{\\star\}\}\-X\_\{t,i\_\{t\}\},\\theta^\{\\star\}\\rangle,\\qquad i\_\{t\}^\{\\star\}\\in\\arg\\max\_\{i\\in\[K\]\}\\langle X\_\{t,i\},\\theta^\{\\star\}\\rangle\.\(17\)For each epochee, letℰconf,e\\mathcal\{E\}\_\{\{\\rm conf\},e\}denote the epoch\-wise confidence event defined after Theorem[1](https://arxiv.org/html/2607.08971#Thmtheorem1), and let

ℰconf:=⋂e=0Eℰconf,e\.\\mathcal\{E\}\_\{\\rm conf\}:=\\bigcap\_\{e=0\}^\{E\}\\mathcal\{E\}\_\{\{\\rm conf\},e\}\.
We begin by recording the elliptical\-potential control for the frozen epochs\. This is the ingredient that lets the regret summation proceed as in an ordinarymm\-dimensional linear bandit\. Recall thatzt:=zt,itz\_\{t\}:=z\_\{t,i\_\{t\}\}denotes the reduced feature of the arm played at roundtt\.

###### Lemma 12\(Potential control inside an epoch\)

Assumeℰrep\\mathcal\{E\}\_\{\\rm rep\}holds, fix an epochee, and suppose thatλ≥Be2,\\lambda\\geq B\_\{e\}^\{2\},whereBe:=BX​\(1\+\(1\+2p\)​ϵe\)B\_\{e\}:=B\_\{X\}\\left\(1\+\\left\(1\+\\frac\{2\}\{p\}\\right\)\\epsilon\_\{e\}\\right\)\. Then, we have

∑t∈𝔗e‖zt‖Ve,t−12≤2​Γeand∑t∈𝔗e‖zt‖Ve,t−1≤2​\|𝔗e\|​Γe,\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}^\{2\}\\leq 2\\Gamma\_\{e\}\\qquad\\text\{and\}\\qquad\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}\\leq\\sqrt\{2\|\\mathfrak\{T\}\_\{e\}\|\\Gamma\_\{e\}\},where

Γe:=log⁡det\(Ve,end\)det\(λ​Im\)≤m​log⁡\(1\+\|𝔗e\|​Be2m​λ\),Ve,end:=λ​Im\+∑s∈𝔗ezs​zs⊤\.\\Gamma\_\{e\}:=\\log\\frac\{\\det\(V\_\{e,\\mathrm\{end\}\}\)\}\{\\det\(\\lambda I\_\{m\}\)\}\\leq m\\log\\left\(1\+\\frac\{\|\\mathfrak\{T\}\_\{e\}\|B\_\{e\}^\{2\}\}\{m\\lambda\}\\right\),\\qquad V\_\{e,\\mathrm\{end\}\}:=\\lambda I\_\{m\}\+\\sum\_\{s\\in\\mathfrak\{T\}\_\{e\}\}z\_\{s\}z\_\{s\}^\{\\top\}\.

Proof:We first begin by showing that the features are uniformly bounded\. Since𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}has orthonormal columns,

‖zt,i‖2≤‖X^t,i‖2≤‖Xt,i‖2\+‖X^t,i−Xt,i‖2≤Be,\\\|z\_\{t,i\}\\\|\_\{2\}\\leq\\\|\\hat\{X\}\_\{t,i\}\\\|\_\{2\}\\leq\\\|X\_\{t,i\}\\\|\_\{2\}\+\\\|\\hat\{X\}\_\{t,i\}\-X\_\{t,i\}\\\|\_\{2\}\\leq B\_\{e\},where we both used the norm bound onXt,iX\_\{t,i\}and the imputation error bound from Lemma[2](https://arxiv.org/html/2607.08971#Thmlemma2)\. For the potential bound, we proceed by fixing the selected features inside epocheeand writingqt:=‖zt‖Ve,t−12\.q\_\{t\}:=\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}^\{2\}\.Indeed what follows is the standard elliptical\-potential/determinant\-telescoping argument used in linear bandit analyses; see, for example,Abbasi\-Yadkori et al\. \[[1](https://arxiv.org/html/2607.08971#bib.bib1)\]orLattimore and Szepesvári \[[3](https://arxiv.org/html/2607.08971#bib.bib3)\]\. Because the representation is frozen, the design matrices satisfy

Ve,t\+1=Ve,t\+zt​zt⊤,V\_\{e,t\+1\}=V\_\{e,t\}\+z\_\{t\}z\_\{t\}^\{\\top\},and the matrix determinant lemma gives

log⁡det\(Ve,t\+1\)det\(Ve,t\)=log⁡\(1\+qt\)\.\\log\\frac\{\\det\(V\_\{e,t\+1\}\)\}\{\\det\(V\_\{e,t\}\)\}=\\log\(1\+q\_\{t\}\)\.If one takesλ≥Be2\\lambda\\geq B\_\{e\}^\{2\}, then by the norm bound, we haveqt≤1q\_\{t\}\\leq 1, henceqt≤2​log⁡\(1\+qt\)q\_\{t\}\\leq 2\\log\(1\+q\_\{t\}\)\. Summing over the epoch yields

∑t∈𝔗eqt≤2​log⁡det\(Ve,end\)det\(λ​Im\)=2​Γe\.\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}q\_\{t\}\\leq 2\\log\\frac\{\\det\(V\_\{e,\\mathrm\{end\}\}\)\}\{\\det\(\\lambda I\_\{m\}\)\}=2\\Gamma\_\{e\}\.The second inequality follows from Cauchy–Schwarz\. Finally,

det\(Ve,end\)≤\(tr⁡\(Ve,end\)m\)m≤\(λ\+\|𝔗e\|​Be2m\)m,\\det\(V\_\{e,\\mathrm\{end\}\}\)\\leq\\left\(\\frac\{\\operatorname\{tr\}\(V\_\{e,\\mathrm\{end\}\}\)\}\{m\}\\right\)^\{m\}\\leq\\left\(\\lambda\+\\frac\{\|\\mathfrak\{T\}\_\{e\}\|B\_\{e\}^\{2\}\}\{m\}\\right\)^\{m\},which gives the stated upper bound onΓe\\Gamma\_\{e\}\.

This lemma is where epoch\-wise freezing enters the proof algebraically\. Because𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}is fixed throughout the epoch, the design matrices evolve by the standard rank\-one recursion, so the determinant telescope is the usual OFUL one\. The only remaining departure from ordinary OFUL is the controlled misspecification term created by the surrogate approximation\.

### F\.1From optimism to epoch regret

We now use the estimation guarantee of Theorem[1](https://arxiv.org/html/2607.08971#Thmtheorem1)to bound the regret accumulated inside a single epoch\. First, on the confidence eventℰconf,e\\mathcal\{E\}\_\{\{\\rm conf\},e\}, the optimistic action selection rule controls the*surrogate*regret of each played arm by the usual OFUL width2​βe,t​‖zt‖Ve,t−12\\beta\_\{e,t\}\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}\(Lemma[13](https://arxiv.org/html/2607.08971#Thmlemma13)\)\. This is precisely where freezing the representation makes the argument identical in form to a standardmm\-dimensional linear bandit calculation\. Second, the surrogate approximation guarantee lets us pass from surrogate regret back to true regret at an additive cost of2​be2b\_\{e\}per round \(Lemma[14](https://arxiv.org/html/2607.08971#Thmlemma14)\)\. Combining these with the potential control of Lemma[12](https://arxiv.org/html/2607.08971#Thmlemma12)yields the epoch regret bound \(Lemma[16](https://arxiv.org/html/2607.08971#Thmlemma16)\), which the proof of Theorem[2](https://arxiv.org/html/2607.08971#Thmtheorem2)then sums over the doubling epoch schedule\. Along the way, Lemma[15](https://arxiv.org/html/2607.08971#Thmlemma15)records bounds onBeB\_\{e\},Γe\\Gamma\_\{e\}, andbeb\_\{e\}that hold uniformly over the post\-burn\-in epochs; this technical step simplifies the statement of Lemma[16](https://arxiv.org/html/2607.08971#Thmlemma16)and the final summation\.

###### Lemma 13\(Surrogate optimism\)

Fix an epochee\. Onℰconf,e\\mathcal\{E\}\_\{\{\\rm conf\},e\}, for everyt∈𝔗et\\in\\mathfrak\{T\}\_\{e\},

μ¯t,i¯t⋆−μ¯t,it≤2​βe,t​‖zt‖Ve,t−1,\\bar\{\\mu\}\_\{t,\\bar\{i\}\_\{t\}^\{\\star\}\}\-\\bar\{\\mu\}\_\{t,i\_\{t\}\}\\leq 2\\beta\_\{e,t\}\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\},wherezt:=zt,itz\_\{t\}:=z\_\{t,i\_\{t\}\}and

i¯t⋆∈arg⁡maxi∈\[K\]⁡μ¯t,i\.\\bar\{i\}\_\{t\}^\{\\star\}\\in\\arg\\max\_\{i\\in\[K\]\}\\bar\{\\mu\}\_\{t,i\}\.

Proof:We start by observing that, onℰconf,e\\mathcal\{E\}\_\{\{\\rm conf\},e\}, Theorem[1](https://arxiv.org/html/2607.08971#Thmtheorem1)gives‖ϑ^e,t−ϑe⋆‖Ve,t≤βe,t\\\|\\hat\{\\vartheta\}\_\{e,t\}\-\\vartheta\_\{e\}^\{\\star\}\\\|\_\{V\_\{e,t\}\}\\leq\\beta\_\{e,t\}\. Therefore, by the Cauchy–Schwarz inequality, for any armii, we have

μ¯t,i=zt,i⊤​ϑe⋆≤zt,i⊤​ϑ^e,t\+βe,t​‖zt,i‖Ve,t−1\.\\bar\{\\mu\}\_\{t,i\}=z\_\{t,i\}^\{\\top\}\\vartheta\_\{e\}^\{\\star\}\\leq z\_\{t,i\}^\{\\top\}\\widehat\{\\vartheta\}\_\{e,t\}\+\\beta\_\{e,t\}\\\|z\_\{t,i\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}\.Similarly, we also have

μ¯t,it≥zt⊤​ϑ^e,t−βe,t​‖zt‖Ve,t−1\.\\bar\{\\mu\}\_\{t,i\_\{t\}\}\\geq z\_\{t\}^\{\\top\}\\widehat\{\\vartheta\}\_\{e,t\}\-\\beta\_\{e,t\}\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}\.By the optimistic action selection rule in Algorithm[1](https://arxiv.org/html/2607.08971#alg1)

zt,i¯t⋆⊤​ϑ^e,t\+βe,t​‖zt,i¯t⋆‖Ve,t−1≤zt⊤​ϑ^e,t\+βe,t​‖zt‖Ve,t−1\.z\_\{t,\\bar\{i\}\_\{t\}^\{\\star\}\}^\{\\top\}\\widehat\{\\vartheta\}\_\{e,t\}\+\\beta\_\{e,t\}\\\|z\_\{t,\\bar\{i\}\_\{t\}^\{\\star\}\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}\\leq z\_\{t\}^\{\\top\}\\widehat\{\\vartheta\}\_\{e,t\}\+\\beta\_\{e,t\}\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}\.Combining the three equations above proves the claim\.

The previous lemma only controls regret in the surrogate model\. To return to the original problem, we use the epoch\-wise approximation lemma: every true arm value and its surrogate value differ by at mostbeb\_\{e\}\. This converts surrogate optimism into a one\-step true regret bound, at the cost of the controlled misspecification term\.

###### Lemma 14\(True regret versus surrogate regret\)

Onℰrep∩ℰconf,e\\mathcal\{E\}\_\{\\rm rep\}\\cap\\mathcal\{E\}\_\{\{\\rm conf\},e\}, for everyt∈𝔗et\\in\\mathfrak\{T\}\_\{e\},

⟨Xt,it⋆−Xt,it,θ⋆⟩≤2​βe,t​‖zt‖Ve,t−1\+2​be\.\\langle X\_\{t,i\_\{t\}^\{\\star\}\}\-X\_\{t,i\_\{t\}\},\\theta^\{\\star\}\\rangle\\leq 2\\beta\_\{e,t\}\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}\+2b\_\{e\}\.

Proof:We first show that, onℰrep\\mathcal\{E\}\_\{\\rm rep\}, the true one\-step regret exceeds the surrogate one\-step regret by at most2​be2b\_\{e\}:

⟨Xt,it⋆−Xt,it,θ⋆⟩≤μ¯t,i¯t⋆−μ¯t,it\+2​be\.\\langle X\_\{t,i\_\{t\}^\{\\star\}\}\-X\_\{t,i\_\{t\}\},\\theta^\{\\star\}\\rangle\\leq\\bar\{\\mu\}\_\{t,\\bar\{i\}\_\{t\}^\{\\star\}\}\-\\bar\{\\mu\}\_\{t,i\_\{t\}\}\+2b\_\{e\}\.Towards this, notice that, by Lemma[3](https://arxiv.org/html/2607.08971#Thmlemma3), we have

⟨Xt,it⋆,θ⋆⟩≤μ¯t,it⋆\+be≤μ¯t,i¯t⋆\+be,\\langle X\_\{t,i\_\{t\}^\{\\star\}\},\\theta^\{\\star\}\\rangle\\leq\\bar\{\\mu\}\_\{t,i\_\{t\}^\{\\star\}\}\+b\_\{e\}\\leq\\bar\{\\mu\}\_\{t,\\bar\{i\}\_\{t\}^\{\\star\}\}\+b\_\{e\},where the second inequality holds sincei¯t⋆\\bar\{i\}\_\{t\}^\{\\star\}maximizes the surrogate mean\. Similarly, we also have

⟨Xt,it,θ⋆⟩≥μ¯t,it−be\.\\langle X\_\{t,i\_\{t\}\},\\theta^\{\\star\}\\rangle\\geq\\bar\{\\mu\}\_\{t,i\_\{t\}\}\-b\_\{e\}\.Subtracting the two inequalities above gives the claimed comparison\. Finally, onℰconf,e\\mathcal\{E\}\_\{\{\\rm conf\},e\}, Lemma[13](https://arxiv.org/html/2607.08971#Thmlemma13)bounds the surrogate regret by2​βe,t​‖zt‖Ve,t−12\\beta\_\{e,t\}\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}, which completes the proof\.

Lemma[14](https://arxiv.org/html/2607.08971#Thmlemma14)is the bridge between the two scales at which the algorithm operates: decisions are made, and the confidence set lives, in the frozen surrogate coordinates, while regret is charged against the true means\. The lemma shows that each round of this translation costs two prices:2​βe,t​‖zt‖Ve,t−12\\beta\_\{e,t\}\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}is the familiar OFUL price of parameter uncertainty inmmdimensions, and2​be2b\_\{e\}is the price of acting through an estimated and imputed representation\. It now remains to sum this one\-step bound over the epoch\. The OFUL term will be controlled by the elliptical potential of Lemma[12](https://arxiv.org/html/2607.08971#Thmlemma12), while thebeb\_\{e\}\-dependent contributions \(both the additive2​be2b\_\{e\}and thebe​t−τeb\_\{e\}\\sqrt\{t\-\\tau\_\{e\}\}part insideβe,t\\beta\_\{e,t\}\) will be summed separately\. Before carrying this out, we record bounds on certain epoch\-level quantities that hold uniformly across all epochs\. As we will see below, this allows the epoch regret bound to be stated with epoch\-independent constants\.

###### Lemma 15\(Uniform bounds across epochs\)

Setλ:=4​BX2\\lambda:=4B\_\{X\}^\{2\}, and define

GT:=m​log⁡\(1\+Tm\),He:=2​log⁡\(1/δe\),Arep:=4​2​Csub​S​BX​κp2​mK​log⁡\(8​d​Tδrep\),G\_\{T\}:=m\\log\\\!\\left\(1\+\\frac\{T\}\{m\}\\right\),\\qquad H\_\{e\}:=2\\log\(1/\\delta\_\{e\}\),\\qquad A\_\{\\rm rep\}:=4\\sqrt\{2\}\\,C\_\{\\rm sub\}SB\_\{X\}\\frac\{\\kappa\}\{p^\{2\}\}\\sqrt\{\\frac\{m\}\{K\}\\log\\\!\\left\(\\frac\{8dT\}\{\\delta\_\{\\rm rep\}\}\\right\)\},whereδe\\delta\_\{e\}is the confidence level at which Theorem[1](https://arxiv.org/html/2607.08971#Thmtheorem1)is invoked in epochee, andδrep\\delta\_\{\\rm rep\}is the failure probability of the representation eventℰrep\\mathcal\{E\}\_\{\\rm rep\}\. On the representation eventℰrep\\mathcal\{E\}\_\{\\rm rep\}, the following hold in every epochee:

Be≤B⋆:=2​BX\(so that​λ≥Be2​\),Γe≤GT,be≤Arepτe\.B\_\{e\}\\leq B\_\{\\star\}:=2B\_\{X\}\\quad\\text\{\(so that \}\\lambda\\geq B\_\{e\}^\{2\}\\text\{\)\},\\qquad\\Gamma\_\{e\}\\leq G\_\{T\},\\qquad b\_\{e\}\\leq\\frac\{A\_\{\\rm rep\}\}\{\\sqrt\{\\tau\_\{e\}\}\}\.

Proof:We first boundBeB\_\{e\}\. Sinceϵt\\epsilon\_\{t\}is nonincreasing andτe−1≥tb\\tau\_\{e\}\-1\\geq t\_\{b\}\(recall thatϵe:=ϵτe−1\\epsilon\_\{e\}:=\\epsilon\_\{\\tau\_\{e\}\-1\}\), we haveϵe≤ϵtb\\epsilon\_\{e\}\\leq\\epsilon\_\{t\_\{b\}\}, and the choice oftbt\_\{b\}in Equation \([5](https://arxiv.org/html/2607.08971#S4.E5)\) \(withCb=\(32​Csub\)2C\_\{b\}=\(32C\_\{\\rm sub\}\)^\{2\}\) guaranteesϵtb≤p/32\\epsilon\_\{t\_\{b\}\}\\leq p/32\. Sincep≤1p\\leq 1, this gives\(1\+2p\)​ϵtb≤p32\+232≤332<1\\left\(1\+\\frac\{2\}\{p\}\\right\)\\epsilon\_\{t\_\{b\}\}\\leq\\frac\{p\}\{32\}\+\\frac\{2\}\{32\}\\leq\\frac\{3\}\{32\}<1, and therefore

Be=BX​\(1\+\(1\+2p\)​ϵe\)≤2​BX=B⋆\.B\_\{e\}=B\_\{X\}\\left\(1\+\\left\(1\+\\frac\{2\}\{p\}\\right\)\\epsilon\_\{e\}\\right\)\\leq 2B\_\{X\}=B\_\{\\star\}\.Next, since we takeλ=4​BX2≥Be2\\lambda=4B\_\{X\}^\{2\}\\geq B\_\{e\}^\{2\}, Lemma[12](https://arxiv.org/html/2607.08971#Thmlemma12)applies and gives us the bound onΓe\\Gamma\_\{e\}:

Γe≤m​log⁡\(1\+ne​Be2m​λ\)≤m​log⁡\(1\+T⋅4​BX2m⋅4​BX2\)=GT\.\\Gamma\_\{e\}\\leq m\\log\\\!\\left\(1\+\\frac\{n\_\{e\}B\_\{e\}^\{2\}\}\{m\\lambda\}\\right\)\\leq m\\log\\\!\\left\(1\+\\frac\{T\\cdot 4B\_\{X\}^\{2\}\}\{m\\cdot 4B\_\{X\}^\{2\}\}\\right\)=G\_\{T\}\.Finally, we boundbeb\_\{e\}\. Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1), invoked at confidence levelδrep/4\\delta\_\{\\rm rep\}/4, gives

ϵe=Csub​κp​m\(τe−1\)​K​log⁡\(8​d​Tδrep\)\.\\epsilon\_\{e\}=C\_\{\\rm sub\}\\frac\{\\kappa\}\{p\}\\sqrt\{\\frac\{m\}\{\(\\tau\_\{e\}\-1\)K\}\\log\\\!\\left\(\\frac\{8dT\}\{\\delta\_\{\\rm rep\}\}\\right\)\}\.Sinceτe≥2\\tau\_\{e\}\\geq 2, we haveτe−1≥τe/2\\tau\_\{e\}\-1\\geq\\tau\_\{e\}/2, and sincep≤1p\\leq 1, we have2\+2/p≤4/p2\+2/p\\leq 4/p; together these give

be=S​BX​\(2\+2p\)​ϵe≤Arepτe,b\_\{e\}=SB\_\{X\}\\left\(2\+\\frac\{2\}\{p\}\\right\)\\epsilon\_\{e\}\\leq\\frac\{A\_\{\\rm rep\}\}\{\\sqrt\{\\tau\_\{e\}\}\},as claimed\.

With these uniform bounds in hand, we can now sum the one\-step regret bound of Lemma[14](https://arxiv.org/html/2607.08971#Thmlemma14)over a single epoch\.

###### Lemma 16\(Epoch regret\)

Setλ:=4​BX2\\lambda:=4B\_\{X\}^\{2\}, and letGTG\_\{T\},HeH\_\{e\}, andArepA\_\{\\rm rep\}be as defined in Lemma[15](https://arxiv.org/html/2607.08971#Thmlemma15)\. Onℰrep∩ℰconf,e\\mathcal\{E\}\_\{\\rm rep\}\\cap\\mathcal\{E\}\_\{\{\\rm conf\},e\},

Re:=∑t∈𝔗e⟨Xt,it⋆−Xt,it,θ⋆⟩\\displaystyle R\_\{e\}:=\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}\\langle X\_\{t,i\_\{t\}^\{\\star\}\}\-X\_\{t,i\_\{t\}\},\\theta^\{\\star\}\\rangle\(18\)satisfies

Re≤2​2​S​λ​GT​ne\+2​2​R​GT​\(GT\+He\)​ne\+2​Arep​\(GT\+1\)​neτe\.\\displaystyle R\_\{e\}\\leq 2\\sqrt\{2\}S\\sqrt\{\\lambda G\_\{T\}n\_\{e\}\}\+2\\sqrt\{2\}R\\sqrt\{G\_\{T\}\(G\_\{T\}\+H\_\{e\}\)n\_\{e\}\}\+2A\_\{\\rm rep\}\(\\sqrt\{G\_\{T\}\}\+1\)\\frac\{n\_\{e\}\}\{\\sqrt\{\\tau\_\{e\}\}\}\.\(19\)

Proof:Summing the one\-step bound of Lemma[14](https://arxiv.org/html/2607.08971#Thmlemma14)overt∈𝔗et\\in\\mathfrak\{T\}\_\{e\}\(recall thatne:=\|𝔗e\|n\_\{e\}:=\|\\mathfrak\{T\}\_\{e\}\|\), we have:

Re\\displaystyle R\_\{e\}≤2​∑t∈𝔗eβe,t​‖zt‖Ve,t−1\+2​be​ne\.\\displaystyle\\leq 2\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}\\beta\_\{e,t\}\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}\+2b\_\{e\}n\_\{e\}\.\(20\)Now, recalling the definition of the confidence radiusβe,t\\beta\_\{e,t\}from Theorem[1](https://arxiv.org/html/2607.08971#Thmtheorem1), we write

βe,t=wt\+be​t−τe,where​wt:=λ​S\+R​2​log⁡\(det\(Ve,t\)1/2det\(λ​Im\)1/2​δe\)\.\\beta\_\{e,t\}=w\_\{t\}\+b\_\{e\}\\sqrt\{t\-\\tau\_\{e\}\},\\qquad\{\\rm where\}\\;w\_\{t\}:=\\sqrt\{\\lambda\}S\+R\\sqrt\{2\\log\\left\(\\frac\{\\det\(V\_\{e,t\}\)^\{1/2\}\}\{\\det\(\\lambda I\_\{m\}\)^\{1/2\}\\delta\_\{e\}\}\\right\)\}\.In what follows, we abbreviateqt:=‖zt‖Ve,t−1q\_\{t\}:=\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}and split the sum in \([20](https://arxiv.org/html/2607.08971#A6.E20)\) using the notation above:

∑t∈𝔗eβe,t​qt=∑t∈𝔗ewt​qt\+∑t∈𝔗ebe​t−τe​qt\.\\displaystyle\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}\\beta\_\{e,t\}\\,q\_\{t\}=\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}w\_\{t\}\\,q\_\{t\}\+\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}b\_\{e\}\\sqrt\{t\-\\tau\_\{e\}\}\\,q\_\{t\}\.\(21\)We will bound the two sums in \([21](https://arxiv.org/html/2607.08971#A6.E21)\) in turn\. Both bounds rely on the following consequence of Lemma[12](https://arxiv.org/html/2607.08971#Thmlemma12), which applies since our choice ofλ\\lambda\(and Lemma[15](https://arxiv.org/html/2607.08971#Thmlemma15)\) guaranteesλ≥Be2\\lambda\\geq B\_\{e\}^\{2\}, together with the Cauchy–Schwarz inequality:

∑t∈𝔗eqt2≤2​Γeand∑t∈𝔗eqt≤2​ne​Γe\.\\displaystyle\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}q\_\{t\}^\{2\}\\leq 2\\Gamma\_\{e\}\\qquad\\text\{and\}\\qquad\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}q\_\{t\}\\leq\\sqrt\{2n\_\{e\}\\Gamma\_\{e\}\}\.\(22\)
We first bound the sum involvingwtw\_\{t\}, which is the part of the confidence radius that does not grow within the epoch\. SinceVe,t⪯Ve,endV\_\{e,t\}\\preceq V\_\{e,\\mathrm\{end\}\}, we havelog⁡det\(Ve,t\)det\(λ​Im\)≤Γe\\log\\frac\{\\det\(V\_\{e,t\}\)\}\{\\det\(\\lambda I\_\{m\}\)\}\\leq\\Gamma\_\{e\}, and thereforewt≤λ​S\+R​Γe\+2​log⁡\(1/δe\)w\_\{t\}\\leq\\sqrt\{\\lambda\}S\+R\\sqrt\{\\Gamma\_\{e\}\+2\\log\(1/\\delta\_\{e\}\)\}for everyt∈𝔗et\\in\\mathfrak\{T\}\_\{e\}\. Combining this with the second bound in \([22](https://arxiv.org/html/2607.08971#A6.E22)\), we have:

∑t∈𝔗ewt​qt≤\(λ​S\+R​Γe\+2​log⁡\(1/δe\)\)​2​ne​Γe\.\\displaystyle\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}w\_\{t\}\\,q\_\{t\}\\leq\\left\(\\sqrt\{\\lambda\}S\+R\\sqrt\{\\Gamma\_\{e\}\+2\\log\(1/\\delta\_\{e\}\)\}\\right\)\\sqrt\{2n\_\{e\}\\Gamma\_\{e\}\}\.\(23\)
We next bound the second sum in \([21](https://arxiv.org/html/2607.08971#A6.E21)\), which collects the growing misspecification part of the radius\. By the Cauchy–Schwarz inequality, the first bound in \([22](https://arxiv.org/html/2607.08971#A6.E22)\), and the fact that∑t∈𝔗e\(t−τe\)≤ne2/2\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}\(t\-\\tau\_\{e\}\)\\leq n\_\{e\}^\{2\}/2, we have:

∑t∈𝔗ebe​t−τe​qt≤be​∑t∈𝔗e\(t−τe\)​∑t∈𝔗eqt2≤be​ne​Γe\.\\displaystyle\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}b\_\{e\}\\sqrt\{t\-\\tau\_\{e\}\}\\,q\_\{t\}\\leq b\_\{e\}\\sqrt\{\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}\(t\-\\tau\_\{e\}\)\}\\sqrt\{\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}q\_\{t\}^\{2\}\}\\leq b\_\{e\}n\_\{e\}\\sqrt\{\\Gamma\_\{e\}\}\.\(24\)Substituting \([23](https://arxiv.org/html/2607.08971#A6.E23)\) and \([24](https://arxiv.org/html/2607.08971#A6.E24)\) into \([21](https://arxiv.org/html/2607.08971#A6.E21)\), and the result into \([20](https://arxiv.org/html/2607.08971#A6.E20)\), we have:

Re\\displaystyle R\_\{e\}≤2​2​S​λ​Γe​ne\+2​2​R​Γe​\(Γe\+2​log⁡\(1/δe\)\)​ne\+2​be​ne​\(Γe\+1\)\.\\displaystyle\\leq 2\\sqrt\{2\}S\\sqrt\{\\lambda\\Gamma\_\{e\}n\_\{e\}\}\+2\\sqrt\{2\}R\\sqrt\{\\Gamma\_\{e\}\\left\(\\Gamma\_\{e\}\+2\\log\(1/\\delta\_\{e\}\)\\right\)n\_\{e\}\}\+2b\_\{e\}n\_\{e\}\\left\(\\sqrt\{\\Gamma\_\{e\}\}\+1\\right\)\.\(25\)Finally, Lemma[15](https://arxiv.org/html/2607.08971#Thmlemma15)givesΓe≤GT\\Gamma\_\{e\}\\leq G\_\{T\},2​log⁡\(1/δe\)=He2\\log\(1/\\delta\_\{e\}\)=H\_\{e\}, andbe≤Arep/τeb\_\{e\}\\leq A\_\{\\rm rep\}/\\sqrt\{\\tau\_\{e\}\}; substituting these three bounds into \([25](https://arxiv.org/html/2607.08971#A6.E25)\) yields \([19](https://arxiv.org/html/2607.08971#A6.E19)\)\.

### F\.2Main regret theorem

We are now ready to prove Theorem[2](https://arxiv.org/html/2607.08971#Thmtheorem2)\. The proof splits the horizon into the burn\-in rounds, which we charge at the worst\-case rate, and the post\-burn\-in epochs, to each of which we apply Lemma[16](https://arxiv.org/html/2607.08971#Thmlemma16); the doubling schedule then makes the epoch bounds summable at theT\\sqrt\{T\}scale\.

Proof:Let𝒢:=ℰrep∩ℰconf\\mathcal\{G\}:=\\mathcal\{E\}\_\{\\rm rep\}\\cap\\mathcal\{E\}\_\{\\rm conf\}be the event on which the representation guarantees and all epoch\-wise confidence sets hold\. We first verify that𝒢\\mathcal\{G\}has the claimed probability\. The representation eventℰrep\\mathcal\{E\}\_\{\\rm rep\}is defined in Section[4\.1](https://arxiv.org/html/2607.08971#S4.SS1)and fails with probability at mostδrep=δ/2\\delta\_\{\\rm rep\}=\\delta/2\. Conditional onℰrep\\mathcal\{E\}\_\{\\rm rep\}, Theorem[1](https://arxiv.org/html/2607.08971#Thmtheorem1)shows that each epoch\-wise confidence eventℰconf,e\\mathcal\{E\}\_\{\{\\rm conf\},e\}fails with probability at mostδe=δ/\(2​\(E\+1\)\)\\delta\_\{e\}=\\delta/\(2\(E\+1\)\)\. Since there are at mostE\+1E\+1epochs, a union bound gives:

Pr⁡\(𝒢\)≥1−δrep−∑e=0Eδe≥1−δ\.\\Pr\(\\mathcal\{G\}\)\\geq 1\-\\delta\_\{\\rm rep\}\-\\sum\_\{e=0\}^\{E\}\\delta\_\{e\}\\geq 1\-\\delta\.It is therefore enough to prove the claimed regret bound on𝒢\\mathcal\{G\}\.

We now decompose the cumulative regret\. Recalling the definition ofRTR\_\{T\}in \([17](https://arxiv.org/html/2607.08971#A6.E17)\), we split the sum over rounds into the burn\-in roundst≤tbt\\leq t\_\{b\}and the post\-burn\-in epochs𝔗0,…,𝔗E\\mathfrak\{T\}\_\{0\},\\ldots,\\mathfrak\{T\}\_\{E\}, which partition the remaining rounds\{tb\+1,…,T\}\\\{t\_\{b\}\+1,\\ldots,T\\\}; the regret accumulated in epocheeis exactly the quantityReR\_\{e\}defined in \([18](https://arxiv.org/html/2607.08971#A6.E18)\)\. We handle the burn\-in part conservatively, charging every round the worst\-case regret: since‖Xt,i‖2≤BX\\\|X\_\{t,i\}\\\|\_\{2\}\\leq B\_\{X\}and‖θ⋆‖2≤S\\\|\\theta^\{\\star\}\\\|\_\{2\}\\leq S, the Cauchy–Schwarz inequality bounds each instantaneous regret by2​BX​S2B\_\{X\}S\. We therefore have:

RT≤2​BX​S​tb\+∑e=0ERe\.R\_\{T\}\\leq 2B\_\{X\}S\\,t\_\{b\}\+\\sum\_\{e=0\}^\{E\}R\_\{e\}\.We next bound eachReR\_\{e\}\. Recall that we set the regularization toλ=4​BX2\\lambda=4B\_\{X\}^\{2\}in Theorem[2](https://arxiv.org/html/2607.08971#Thmtheorem2), and since we are on𝒢\\mathcal\{G\}, Lemma[16](https://arxiv.org/html/2607.08971#Thmlemma16)applies to every epoch\. With the choice ofδe=δ/\(2​\(E\+1\)\)\\delta\_\{e\}=\\delta/\(2\(E\+1\)\), the quantityHeH\_\{e\}in Lemma[16](https://arxiv.org/html/2607.08971#Thmlemma16)is the same for every epoch; write

HT:=2​log⁡\(2​\(E\+1\)δ\)\.H\_\{T\}:=2\\log\\\!\\left\(\\frac\{2\(E\+1\)\}\{\\delta\}\\right\)\.Thus, on𝒢\\mathcal\{G\}, Lemma[16](https://arxiv.org/html/2607.08971#Thmlemma16)allows us to bound the epoch regret for each epocheeas follows:

Re\\displaystyle R\_\{e\}≤2​2​S​λ​GT​ne\+2​2​R​GT​\(GT\+HT\)​ne\+2​Arep​\(GT\+1\)​neτe\.\\displaystyle\\leq 2\\sqrt\{2\}S\\sqrt\{\\lambda G\_\{T\}n\_\{e\}\}\+2\\sqrt\{2\}R\\sqrt\{G\_\{T\}\(G\_\{T\}\+H\_\{T\}\)n\_\{e\}\}\+2A\_\{\\rm rep\}\(\\sqrt\{G\_\{T\}\}\+1\)\\frac\{n\_\{e\}\}\{\\sqrt\{\\tau\_\{e\}\}\}\.\(26\)
Next, we notice that the doubling schedule implies thatτe=2e​τ0\\tau\_\{e\}=2^\{e\}\\tau\_\{0\}\. Since𝔗E≠∅\\mathfrak\{T\}\_\{E\}\\neq\\emptyset,τE≤T\\tau\_\{E\}\\leq T, and sincene≤τen\_\{e\}\\leq\\tau\_\{e\},

∑e=0Ene≤∑e=0Eτe=τE​∑j=0E2−j/2≤τE1−2−1/2≤4​T,\\sum\_\{e=0\}^\{E\}\\sqrt\{n\_\{e\}\}\\leq\\sum\_\{e=0\}^\{E\}\\sqrt\{\\tau\_\{e\}\}=\\sqrt\{\\tau\_\{E\}\}\\sum\_\{j=0\}^\{E\}2^\{\-j/2\}\\leq\\frac\{\\sqrt\{\\tau\_\{E\}\}\}\{1\-2^\{\-1/2\}\}\\leq 4\\sqrt\{T\},and

∑e=0Eneτe≤∑e=0Eτe≤4​T\.\\sum\_\{e=0\}^\{E\}\\frac\{n\_\{e\}\}\{\\sqrt\{\\tau\_\{e\}\}\}\\leq\\sum\_\{e=0\}^\{E\}\\sqrt\{\\tau\_\{e\}\}\\leq 4\\sqrt\{T\}\.Summing \([26](https://arxiv.org/html/2607.08971#A6.E26)\) overe=0,…,Ee=0,\\ldots,E, we therefore have:

∑e=0ERe\\displaystyle\\sum\_\{e=0\}^\{E\}R\_\{e\}≤8​2​S​λ​GT​T\+8​2​R​GT​\(GT\+HT\)​T\+8​Arep​\(GT\+1\)​T\.\\displaystyle\\leq 8\\sqrt\{2\}S\\sqrt\{\\lambda G\_\{T\}T\}\+8\\sqrt\{2\}R\\sqrt\{G\_\{T\}\(G\_\{T\}\+H\_\{T\}\)T\}\+8A\_\{\\rm rep\}\(\\sqrt\{G\_\{T\}\}\+1\)\\sqrt\{T\}\.\(27\)Combining \([27](https://arxiv.org/html/2607.08971#A6.E27)\) with the burn\-in split, we conclude that

RT≤2​BX​S​tb\+8​2​S​λ​GT​T\+8​2​R​GT​\(GT\+HT\)​T\+8​Arep​\(GT\+1\)​T\.R\_\{T\}\\leq 2B\_\{X\}S\\,t\_\{b\}\+8\\sqrt\{2\}S\\sqrt\{\\lambda G\_\{T\}T\}\+8\\sqrt\{2\}R\\sqrt\{G\_\{T\}\(G\_\{T\}\+H\_\{T\}\)T\}\+8A\_\{\\rm rep\}\(\\sqrt\{G\_\{T\}\}\+1\)\\sqrt\{T\}\.Finally, we substitute the definitions of the constants involved\. By Equation \([5](https://arxiv.org/html/2607.08971#S4.E5)\), the burn\-in term satisfies2​BX​S​tb=O~​\(S​BX​κ2​m/\(p4​K\)\)2B\_\{X\}S\\,t\_\{b\}=\\widetilde\{O\}\\big\(SB\_\{X\}\\,\\kappa^\{2\}m/\(p^\{4\}K\)\\big\)\. Sinceλ=4​BX2\\lambda=4B\_\{X\}^\{2\}andGT=m​log⁡\(1\+T/m\)G\_\{T\}=m\\log\(1\+T/m\), the second term is8​2​S​λ​GT​T=O~​\(S​BX​m​T\)≤O~​\(S​BX​m​T\)8\\sqrt\{2\}\\,S\\sqrt\{\\lambda G\_\{T\}T\}=\\widetilde\{O\}\\big\(SB\_\{X\}\\sqrt\{mT\}\\big\)\\leq\\widetilde\{O\}\\big\(SB\_\{X\}\\,m\\sqrt\{T\}\\big\)\(usingm≥1m\\geq 1\), and the third term is8​2​R​GT​\(GT\+HT\)​T=O~​\(R​m​T\)8\\sqrt\{2\}\\,R\\sqrt\{G\_\{T\}\(G\_\{T\}\+H\_\{T\}\)T\}=\\widetilde\{O\}\\big\(Rm\\sqrt\{T\}\\big\), sinceGT\+HT=O~​\(m\)G\_\{T\}\+H\_\{T\}=\\widetilde\{O\}\(m\)\. For the last term, substituting the definition ofArepA\_\{\\rm rep\}from Lemma[15](https://arxiv.org/html/2607.08971#Thmlemma15), we have:

8​Arep​\(GT\+1\)​T=O~​\(S​BX​κp2​mK⋅m⋅T\)=O~​\(S​BX​κ​m​Tp2​K\)\.8A\_\{\\rm rep\}\(\\sqrt\{G\_\{T\}\}\+1\)\\sqrt\{T\}=\\widetilde\{O\}\\left\(SB\_\{X\}\\frac\{\\kappa\}\{p^\{2\}\}\\sqrt\{\\frac\{m\}\{K\}\}\\cdot\\sqrt\{m\}\\cdot\\sqrt\{T\}\\right\)=\\widetilde\{O\}\\left\(SB\_\{X\}\\,\\frac\{\\kappa m\\sqrt\{T\}\}\{p^\{2\}\\sqrt\{K\}\}\\right\)\.Collecting these contributions gives the three terms stated in Theorem[2](https://arxiv.org/html/2607.08971#Thmtheorem2)\.

## Appendix GMisspecification Control in Frozen Epochs: Why the naive bound loses a factor ofm\\sqrt\{m\}

This appendix section explains why a direct triangle\-inequality bound on the epoch\-wise misspecification error leads to a suboptimal dependence on the intrinsic dimensionmm\.

Fix an epochee, and let𝔗e=\{τe,…,τe\+ne−1\}\.\\mathfrak\{T\}\_\{e\}=\\\{\\tau\_\{e\},\\ldots,\\tau\_\{e\}\+n\_\{e\}\-1\\\}\.denote the time stamps inside this epoch\. Letne:=\|𝔗e\|n\_\{e\}:=\|\\mathfrak\{T\}\_\{e\}\|denote its length\.

Within this epoch the representation is frozen, and the learner usesmm\-dimensional featureszt∈ℝmz\_\{t\}\\in\\mathbb\{R\}^\{m\}\. Define

Ve,t:=λ​Im\+∑s∈𝔗es<tzs​zs⊤\.V\_\{e,t\}:=\\lambda I\_\{m\}\+\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}z\_\{s\}^\{\\top\}\.As in Section[4\.3](https://arxiv.org/html/2607.08971#S4.SS3), we suppose that the reward model in the epoch is

rt=zt⊤​ϑe⋆\+ξt\+ηt,\|ξt\|≤be,r\_\{t\}=z\_\{t\}^\{\\top\}\\vartheta\_\{e\}^\{\\star\}\+\\xi\_\{t\}\+\\eta\_\{t\},\\qquad\|\\xi\_\{t\}\|\\leq b\_\{e\},whereξt\\xi\_\{t\}is the controlled representation/imputation misspecification andηt\\eta\_\{t\}is the stochastic reward noise\. Then, the contribution of this misspecification to the self\-normalized confidence radius is given by

𝖡e,t:=‖∑s∈𝔗es<tzs​ξs‖Ve,t−1\.\\mathsf\{B\}\_\{e,t\}:=\\left\\\|\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}z\_\{s\}\\xi\_\{s\}\\right\\\|\_\{V\_\{e,t\}^\{\-1\}\}\.
Naive triangle\-inequality control\.The most direct bound is

𝖡e,t\\displaystyle\\mathsf\{B\}\_\{e,t\}≤∑s∈𝔗es<t\|ξs\|​‖zs‖Ve,t−1≤be​∑s∈𝔗es<t‖zs‖Ve,t−1≤be​∑s∈𝔗es<t‖zs‖Ve,s−1,\\displaystyle\\leq\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}\|\\xi\_\{s\}\|\\,\\\|z\_\{s\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}\\leq b\_\{e\}\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}\\\|z\_\{s\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}\\leq b\_\{e\}\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}\\\|z\_\{s\}\\\|\_\{V\_\{e,s\}^\{\-1\}\},\(28\)where the last inequality usesVe,t⪰Ve,sV\_\{e,t\}\\succeq V\_\{e,s\}fors<ts<t, henceVe,t−1⪯Ve,s−1V\_\{e,t\}^\{\-1\}\\preceq V\_\{e,s\}^\{\-1\}\.

We know that‖zt‖2≤Be\\\|z\_\{t\}\\\|\_\{2\}\\leq B\_\{e\}\(the exact constant is calculated in the proof of Lemma[12](https://arxiv.org/html/2607.08971#Thmlemma12)\), and we arrange thatλ≥Be2\\lambda\\geq B\_\{e\}^\{2\}\. This implies that‖zt‖Ve,t−12≤1\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}^\{2\}\\leq 1, and the standard elliptical\-potential lemma then gives

∑t∈𝔗e‖zt‖Ve,t−12≤2​Γe,Γe:=log⁡det\(Ve,end\)det\(λ​Im\)\.\\sum\_\{t\\in\\mathfrak\{T\}\_\{e\}\}\\\|z\_\{t\}\\\|\_\{V\_\{e,t\}^\{\-1\}\}^\{2\}\\leq 2\\Gamma\_\{e\},\\qquad\\Gamma\_\{e\}:=\\log\\frac\{\\det\(V\_\{e,\\mathrm\{end\}\}\)\}\{\\det\(\\lambda I\_\{m\}\)\}\.By Cauchy–Schwarz and \([28](https://arxiv.org/html/2607.08971#A7.E28)\), for anyt∈𝔗et\\in\\mathfrak\{T\}\_\{e\},

𝖡e,t≤be​\(t−τe\)​∑s∈𝔗es<t‖zs‖Ve,s−12≤be​2​\(t−τe\)​Γe\.\\mathsf\{B\}\_\{e,t\}\\leq b\_\{e\}\\sqrt\{\(t\-\\tau\_\{e\}\)\\sum\_\{\\begin\{subarray\}\{c\}s\\in\\mathfrak\{T\}\_\{e\}\\\\ s<t\\end\{subarray\}\}\\\|z\_\{s\}\\\|\_\{V\_\{e,s\}^\{\-1\}\}^\{2\}\}\\leq b\_\{e\}\\sqrt\{2\(t\-\\tau\_\{e\}\)\\Gamma\_\{e\}\}\.In particular, at the end of the epoch,𝖡e,end≤be​2​ne​Γe\.\\mathsf\{B\}\_\{e,\\mathrm\{end\}\}\\leq b\_\{e\}\\sqrt\{2n\_\{e\}\\Gamma\_\{e\}\}\.

SinceΓe≤m​log⁡\(1\+ne​Be2m​λ\)=O~​\(m\),\\Gamma\_\{e\}\\leq m\\log\\left\(1\+\\frac\{n\_\{e\}B\_\{e\}^\{2\}\}\{m\\lambda\}\\right\)=\\widetilde\{O\}\(m\),it can be seen that the naive misspecification bound causes the regret to scale like𝒪~​\(κ​m3/2​T/\(p2​K\)\)\\widetilde\{\\mathcal\{O\}\}\\left\(\\kappa\\,m^\{3/2\}\\sqrt\{T\}/\(p^\{2\}\\sqrt\{K\}\)\\right\)\. This is worse by a factor ofm\\sqrt\{m\}than the corresponding termO~​\(κ​m​T/\(p2​K\)\)\\widetilde\{O\}\\big\(\\kappa m\\sqrt\{T\}/\(p^\{2\}\\sqrt\{K\}\)\\big\)in Theorem[2](https://arxiv.org/html/2607.08971#Thmtheorem2)\. Our analysis instead treats the whole sum together and leverages the contractionZt⊤​\(λ​Im\+Zt​Zt⊤\)−1​Zt⪯I,Z\_\{t\}^\{\\top\}\(\\lambda I\_\{m\}\+Z\_\{t\}Z\_\{t\}^\{\\top\}\)^\{\-1\}Z\_\{t\}\\preceq I,as shown in the proof of Lemma[4](https://arxiv.org/html/2607.08971#Thmlemma4), which gives

‖∑s<tzs​ξs‖Ve,t−1≤be​t−τe\\left\\\|\\sum\_\{s<t\}z\_\{s\}\\xi\_\{s\}\\right\\\|\_\{V\_\{e,t\}^\{\-1\}\}\\leq b\_\{e\}\\sqrt\{t\-\\tau\_\{e\}\}and avoids the extraΓe∼m\\sqrt\{\\Gamma\_\{e\}\}\\sim\\sqrt\{m\}factor\.

## Appendix HProof of the Lower Bound

We prove Theorem[4](https://arxiv.org/html/2607.08971#Thmtheorem4)\. The proof is organized as a sequence of elementary reductions\. First we build a four\-instance product family\. Then we isolate the two pieces of regret: one piece comes from an ordinary reward\-learning sign, and the other from a hidden completion sign\. Finally, two two\-point testing arguments lower bound the lifetime of these two uncertainties\. Throughout this sectionc,c0,c1,…c,c\_\{0\},c\_\{1\},\\ldotsdenote positive universal constants whose values may change\.

We begin by describing the lower bound model\.

Model\.At each roundt∈\[T\]t\\in\[T\], the learner is presented withKKaction vectorsXt,1,…,Xt,K∈ℝd,X\_\{t,1\},\\ldots,X\_\{t,K\}\\in\\mathbb\{R\}^\{d\},drawn independently from a fixed environment\-dependent distribution\. Unless otherwise specified in the oracle\-augmented setting below, each coordinate of each action is observed independently with probabilitypp\. The learner then chooses an armit∈\[K\]i\_\{t\}\\in\[K\]and receives

Yt=⟨Xt,it,θ⋆⟩\+ηt,ηt∼N​\(0,R2\),Y\_\{t\}=\\langle X\_\{t,i\_\{t\}\},\\theta^\{\\star\}\\rangle\+\\eta\_\{t\},\\qquad\\eta\_\{t\}\\sim N\(0,R^\{2\}\),with independent reward noise\. Regret is measured against the full\-information oracle \(which only makes our lower bound stronger\):

RT:=∑t=1T\(maxi∈\[K\]⁡⟨Xt,i,θ⋆⟩−⟨Xt,it,θ⋆⟩\)\.R\_\{T\}:=\\sum\_\{t=1\}^\{T\}\\left\(\\max\_\{i\\in\[K\]\}\\langle X\_\{t,i\},\\theta^\{\\star\}\\rangle\-\\langle X\_\{t,i\_\{t\}\},\\theta^\{\\star\}\\rangle\\right\)\.
Hard i\.i\.d\. action distribution\.AssumeK≥4K\\geq 4\. The construction uses dimensiond=4d=4, with orthonormal basis\{es,e0,e1,e2\}\.\\\{e\_\{s\},e\_\{0\},e\_\{1\},e\_\{2\}\\\}\.We further suppose that there are two hidden signs,ν∈\{\+1,−1\},σ∈\{\+1,−1\}\.\\nu\\in\\\{\+1,\-1\\\},\\;\\sigma\\in\\\{\+1,\-1\\\}\.The signν\\nuenforces the usual noisy reward\-learning difficulty, while the signσ\\sigmaenforces the missingness\-discovery difficulty\. Forσ∈\{\+1,−1\}\\sigma\\in\\\{\+1,\-1\\\}, definevσ:=12​\(e1\+σ​e2\),v\_\{\\sigma\}:=\\frac\{1\}\{\\sqrt\{2\}\}\(e\_\{1\}\+\\sigma e\_\{2\}\),and the rank\-three action subspace𝒰σ:=span⁡\{es,e0,vσ\}\.\\mathcal\{U\}\_\{\\sigma\}:=\\operatorname\{span\}\\\{e\_\{s\},e\_\{0\},v\_\{\\sigma\}\\\}\.Further, suppose thatγ,Δ\>0\\gamma,\\Delta\>0satisfy

γ2\+Δ2≤S2​BX24\.\\gamma^\{2\}\+\\Delta^\{2\}\\leq\\frac\{S^\{2\}B\_\{X\}^\{2\}\}\{4\}\.\(29\)Then, for environment\(ν,σ\)\(\\nu,\\sigma\), if we set

θν,σ:=2​γBX​ν​es\+2​ΔBX​σ​e0,\\theta\_\{\\nu,\\sigma\}:=\\frac\{2\\gamma\}\{B\_\{X\}\}\\nu e\_\{s\}\+\\frac\{2\\Delta\}\{B\_\{X\}\}\\sigma e\_\{0\},then we have‖θν,σ‖2≤S\\\|\\theta\_\{\\nu,\\sigma\}\\\|\_\{2\}\\leq S\. Now, we are ready to define the arm distribution\. With probability1/81/8for each\(a,b\)∈\{\+1,−1\}2\(a,b\)\\in\\\{\+1,\-1\\\}^\{2\}, the arm is a decision arm

Xa,bdec:=BX2​\(a​es\+b​e0\)\.X^\{\\rm dec\}\_\{a,b\}:=\\frac\{B\_\{X\}\}\{2\}\(ae\_\{s\}\+be\_\{0\}\)\.With the remaining probability1/21/2, the arm is a side\-information arm

Xside:=BX2​W​vσ,X^\{\\rm side\}:=\\frac\{B\_\{X\}\}\{2\}Wv\_\{\\sigma\},whereW∈\{\+1,−1\}W\\in\\\{\+1,\-1\\\}is an independent Rademacher random variable\. Given our choices, notice that all actions have norm at mostBXB\_\{X\}and lie in𝒰σ\\mathcal\{U\}\_\{\\sigma\}\.

We prove the lower bound in an oracle\-augmented observation model\. The learner is told whether each arm is a decision arm or a side\-information arm\. If an arm is a decision arm, its label\(a,b\)\(a,b\)and its full vectorXa,bdecX^\{\\rm dec\}\_\{a,b\}are revealed without masking\. Thus, for decision arms, the Bernoulli coordinate mask is suspended\. If an arm is a side\-information arm, its type is revealed, but its coordinates are observed through the usual independent Bernoulli\(p\)\(p\)coordinate mask\. This augmentation can only make the learner stronger, so any lower bound in the augmented experiment also holds in the original partially observed model\.

The side\-information arms have zero expected reward because they lie inspan⁡\{e1,e2\}\\operatorname\{span\}\\\{e\_\{1\},e\_\{2\}\\\}, whileθν,σ∈span⁡\{es,e0\}\\theta\_\{\\nu,\\sigma\}\\in\\operatorname\{span\}\\\{e\_\{s\},e\_\{0\}\\\}\. The learner may know this fact\.The side arms are useful only because their partially observed coordinates may reveal the hidden signσ\\sigma\.

Let𝒟t\\mathcal\{D\}\_\{t\}be the event that, among theKKarms shown in roundtt, all four decision labels\(\+,\+\),\(\+,−\)\(\+,\+\),\(\+,\-\),\(−,\+\),\(−,−\)\(\-,\+\),\(\-,\-\)are present at least once\. Since the arms are drawn i\.i\.d\., the inclusion–exclusion principle gives us the following expression for this probability:

πK:=ℙ​\(𝒟t\)=∑j=04\(−1\)j​\(4j\)​\(1−j8\)K\.\\pi\_\{K\}:=\\mathbb\{P\}\(\\mathcal\{D\}\_\{t\}\)=\\sum\_\{j=0\}^\{4\}\(\-1\)^\{j\}\\binom\{4\}\{j\}\\left\(1\-\\frac\{j\}\{8\}\\right\)^\{K\}\.\(30\)In particular, since it suffices that the first four offered arms carry the four distinct labels, for everyK≥4K\\geq 4we have:

πK≥4\!​\(18\)4=3512\.\\pi\_\{K\}\\geq 4\!\\left\(\\frac\{1\}\{8\}\\right\)^\{4\}=\\frac\{3\}\{512\}\.\(31\)On𝒟t\\mathcal\{D\}\_\{t\}, the full\-information oracle has access to the decision arm with label\(ν,σ\)\(\\nu,\\sigma\), whose mean reward isγ\+Δ\\gamma\+\\Delta\.

For the learner’s chosen arm at timett, definea^t,b^t∈\{−1,0,\+1\}\\widehat\{a\}\_\{t\},\\widehat\{b\}\_\{t\}\\in\\\{\-1,0,\+1\\\}as follows\. If the learner chooses a decision arm with label\(a,b\)\(a,b\), we seta^t:=a\\widehat\{a\}\_\{t\}:=aandb^t:=b\\widehat\{b\}\_\{t\}:=b; if the learner chooses a side\-information arm, we seta^t:=b^t:=0\\widehat\{a\}\_\{t\}:=\\widehat\{b\}\_\{t\}:=0\.

###### Lemma 17\(Regret on a complete decision round\)

For every environment\(ν,σ\)\(\\nu,\\sigma\)and every roundtt, on the event𝒟t\\mathcal\{D\}\_\{t\},

maxi∈\[K\]⁡⟨Xt,i,θν,σ⟩−⟨Xt,it,θν,σ⟩≥γ​𝟏​\{a^t≠ν\}\+Δ​𝟏​\{b^t≠σ\}\.\\max\_\{i\\in\[K\]\}\\langle X\_\{t,i\},\\theta\_\{\\nu,\\sigma\}\\rangle\-\\langle X\_\{t,i\_\{t\}\},\\theta\_\{\\nu,\\sigma\}\\rangle\\geq\\gamma\\mathbf\{1\}\\\{\\widehat\{a\}\_\{t\}\\neq\\nu\\\}\+\\Delta\\mathbf\{1\}\\\{\\widehat\{b\}\_\{t\}\\neq\\sigma\\\}\.

Proof:The intuition here is that on𝒟t\\mathcal\{D\}\_\{t\}, all four decision labels are available\. Therefore the oracle can choose the label matching both hidden signs,\(ν,σ\)\(\\nu,\\sigma\)\. If the learner chooses a decision arm with the wrongν\\nu\-label, it loses at leastγ\\gamma; if it chooses one with the wrongσ\\sigma\-label, it loses at leastΔ\\Delta\.

For a decision arm with label\(a,b\)\(a,b\), we have⟨Xa,bdec,θν,σ⟩=a​ν​γ\+b​σ​Δ\\left\\langle X^\{\\rm dec\}\_\{a,b\},\\theta\_\{\\nu,\\sigma\}\\right\\rangle=a\\nu\\gamma\+b\\sigma\\Delta\. On𝒟t\\mathcal\{D\}\_\{t\}, the arm with label\(ν,σ\)\(\\nu,\\sigma\)is present and has mean rewardγ\+Δ\\gamma\+\\Delta\. If the learner chooses a decision arm\(a,b\)\(a,b\), its regret is

γ\+Δ−a​ν​γ−b​σ​Δ=γ​\(1−a​ν\)\+Δ​\(1−b​σ\)\.\\gamma\+\\Delta\-a\\nu\\gamma\-b\\sigma\\Delta=\\gamma\(1\-a\\nu\)\+\\Delta\(1\-b\\sigma\)\.Since1−a​ν1\-a\\nuequals0whena=νa=\\nuand22whena≠νa\\neq\\nu, we haveγ​\(1−a​ν\)≥γ​𝟏​\{a≠ν\}\\gamma\(1\-a\\nu\)\\geq\\gamma\\mathbf\{1\}\\\{a\\neq\\nu\\\}, and similarlyΔ​\(1−b​σ\)≥Δ​𝟏​\{b≠σ\}\\Delta\(1\-b\\sigma\)\\geq\\Delta\\mathbf\{1\}\\\{b\\neq\\sigma\\\}\. This proves the claim if the learner chooses a decision arm\. If the learner chooses a side\-information arm, then its mean reward is zero, while the oracle obtainsγ\+Δ\\gamma\+\\Delta\. Since thena^t=b^t=0\\widehat\{a\}\_\{t\}=\\widehat\{b\}\_\{t\}=0, both indicators equal one, and the same bound holds\.

###### Lemma 18\(Testing the reward bit\)

Fixσ∈\{\+1,−1\}\\sigma\\in\\\{\+1,\-1\\\}\. At any roundtt, conditional on𝒟t\\mathcal\{D\}\_\{t\}and under the uniform prior onν∈\{\+1,−1\}\\nu\\in\\\{\+1,\-1\\\},

ℙ​\(a^t≠ν∣𝒟t,σ\)≥14​exp⁡\(−2​\(t−1\)​γ2R2\)\.\\mathbb\{P\}\(\\widehat\{a\}\_\{t\}\\neq\\nu\\mid\\mathcal\{D\}\_\{t\},\\sigma\)\\geq\\frac\{1\}\{4\}\\exp\\left\(\-\\frac\{2\(t\-1\)\\gamma^\{2\}\}\{R^\{2\}\}\\right\)\.

Proof:The intuition here is that the signν\\nuaffects only the reward means through theese\_\{s\}coordinate\. All contexts, labels, and side\-information observations have the same law underν=\+1\\nu=\+1andν=−1\\nu=\-1\. Thus information aboutν\\nucan only accumulate through noisy rewards, and each reward mean changes by at most2​γ2\\gammabetween the two alternatives\.

Letℙ\+\\mathbb\{P\}\_\{\+\}andℙ−\\mathbb\{P\}\_\{\-\}denote the conditional laws, given𝒟t\\mathcal\{D\}\_\{t\}, of the learner’s information before choosing at roundttunderν=\+1\\nu=\+1andν=−1\\nu=\-1, respectively, withσ\\sigmafixed\. Since𝒟t\\mathcal\{D\}\_\{t\}depends only on the current arm labels, and these labels have the same law under the two values ofν\\nu, conditioning on𝒟t\\mathcal\{D\}\_\{t\}introduces no information aboutν\\nu\.

By the chain rule for KL divergence and the Gaussian reward model, we have the following bound:

KL​\(ℙ\+∥ℙ−\)≤∑s=1t−112​R2​\(μs\+−μs−\)2≤2​\(t−1\)​γ2R2,\\mathrm\{KL\}\(\\mathbb\{P\}\_\{\+\}\\\|\\mathbb\{P\}\_\{\-\}\)\\leq\\sum\_\{s=1\}^\{t\-1\}\\frac\{1\}\{2R^\{2\}\}\(\\mu\_\{s\}^\{\+\}\-\\mu\_\{s\}^\{\-\}\)^\{2\}\\leq\\frac\{2\(t\-1\)\\gamma^\{2\}\}\{R^\{2\}\},\(32\)whereμs\+\\mu\_\{s\}^\{\+\}andμs−\\mu\_\{s\}^\{\-\}are the conditional mean rewards of the arm selected at timessunderν=\+1\\nu=\+1andν=−1\\nu=\-1, and the second inequality follows since every possible selected arm satisfies\|μs\+−μs−\|≤2​γ\|\\mu\_\{s\}^\{\+\}\-\\mu\_\{s\}^\{\-\}\|\\leq 2\\gamma\. LetAAbe the event\{a^t=\+1\}\\\{\\widehat\{a\}\_\{t\}=\+1\\\}\. Now, we can use the Bretagnolle–Huber inequality\[[3](https://arxiv.org/html/2607.08971#bib.bib3)\]and get

ℙ\+​\(Ac\)\+ℙ−​\(A\)≥12​exp⁡\{−KL​\(ℙ\+∥ℙ−\)\}\.\\mathbb\{P\}\_\{\+\}\(A^\{c\}\)\+\\mathbb\{P\}\_\{\-\}\(A\)\\geq\\frac\{1\}\{2\}\\exp\\\{\-\\mathrm\{KL\}\(\\mathbb\{P\}\_\{\+\}\\\|\\mathbb\{P\}\_\{\-\}\)\\\}\.Sinceℙ\+​\(a^t≠\+1\)≥ℙ\+​\(Ac\)\\mathbb\{P\}\_\{\+\}\(\\widehat\{a\}\_\{t\}\\neq\+1\)\\geq\\mathbb\{P\}\_\{\+\}\(A^\{c\}\)andℙ−​\(a^t≠−1\)≥ℙ−​\(A\)\\mathbb\{P\}\_\{\-\}\(\\widehat\{a\}\_\{t\}\\neq\-1\)\\geq\\mathbb\{P\}\_\{\-\}\(A\), averaging the two errors under the uniform prior onν\\nuand applying the bound in \([32](https://arxiv.org/html/2607.08971#A8.E32)\) gives

ℙ​\(a^t≠ν∣𝒟t,σ\)≥14​exp⁡\(−2​\(t−1\)​γ2R2\)\.\\mathbb\{P\}\(\\widehat\{a\}\_\{t\}\\neq\\nu\\mid\\mathcal\{D\}\_\{t\},\\sigma\)\\geq\\frac\{1\}\{4\}\\exp\\left\(\-\\frac\{2\(t\-1\)\\gamma^\{2\}\}\{R^\{2\}\}\\right\)\.
###### Lemma 19\(Testing the missingness bit\)

Assumep≤1/2p\\leq 1/2\. Fixν∈\{\+1,−1\}\\nu\\in\\\{\+1,\-1\\\}\. At any roundtt, conditional on𝒟t\\mathcal\{D\}\_\{t\}and under the uniform prior onσ∈\{\+1,−1\}\\sigma\\in\\\{\+1,\-1\\\},

ℙ​\(b^t≠σ∣𝒟t,ν\)≥14​exp⁡\(−2​K​p2​t−2​\(t−1\)​Δ2R2\)\.\\mathbb\{P\}\(\\widehat\{b\}\_\{t\}\\neq\\sigma\\mid\\mathcal\{D\}\_\{t\},\\nu\)\\geq\\frac\{1\}\{4\}\\exp\\left\(\-2Kp^\{2\}t\-\\frac\{2\(t\-1\)\\Delta^\{2\}\}\{R^\{2\}\}\\right\)\.

Proof:

Notice that the signσ\\sigmacan be learned in two ways\. First, a side\-information arm may reveal both coordinatese1e\_\{1\}ande2e\_\{2\}; then the relative sign of the two observed entries revealsσ\\sigma\. This occurs at rate proportional toK​p2Kp^\{2\}\. Second, rewards from decision arms carry information aboutσ\\sigma, at rate proportional toΔ2/R2\\Delta^\{2\}/R^\{2\}\. If neither source has provided enough information, the learner cannot reliably choose the correctσ\\sigma\-label\. To leverage this intuition, we proceed as follows\. We first lower bound the probability that no side\-information arm reveals both coordinates\. Then, we argue that, on this event, only the rewards carry information aboutσ\\sigma\. This then allows us to finally use a hypothesis\-testing argument to conclude the proof\.

Letℙ\+\\mathbb\{P\}\_\{\+\}andℙ−\\mathbb\{P\}\_\{\-\}denote the conditional laws, given𝒟t\\mathcal\{D\}\_\{t\}, of the learner’s information before choosing at roundttunderσ=\+1\\sigma=\+1andσ=−1\\sigma=\-1, respectively, withν\\nufixed\.

Let𝒩t\\mathcal\{N\}\_\{t\}be the event that, throughout rounds1,…,t1,\\ldots,t, no side\-information arm has both coordinatese1e\_\{1\}ande2e\_\{2\}observed\. This includes side\-information arms in the current round, because current action observations are available to the learner before it chooses\. Conditional on𝒟t\\mathcal\{D\}\_\{t\}, the probability of𝒩t\\mathcal\{N\}\_\{t\}is bounded below as follows\. In each of the firstt−1t\-1rounds, each arm is a side\-information arm with probability1/21/2, and conditional on being a side\-information arm, it reveals both coordinatese1e\_\{1\}ande2e\_\{2\}with probabilityp2p^\{2\}\. Hence no revealing side\-information arm occurs in the firstt−1t\-1rounds with probability\(1−p2/2\)K​\(t−1\)\(1\-p^\{2\}/2\)^\{K\(t\-1\)\}\. In the current round, conditional on𝒟t\\mathcal\{D\}\_\{t\}, there are at mostKKside\-information arms, and each such arm reveals both coordinates with probabilityp2p^\{2\}\. We therefore have:

ℙ​\(𝒩t∣𝒟t\)\\displaystyle\\mathbb\{P\}\(\\mathcal\{N\}\_\{t\}\\mid\\mathcal\{D\}\_\{t\}\)≥\(1−p2/2\)K​\(t−1\)​\(1−p2\)K≥exp⁡\(−2​K​p2​t\)\.\\displaystyle\\geq\(1\-p^\{2\}/2\)^\{K\(t\-1\)\}\(1\-p^\{2\}\)^\{K\}\\geq\\exp\(\-2Kp^\{2\}t\)\.\(33\)Here, the second inequality follows becausep≤1/2p\\leq 1/2: bothp2/2p^\{2\}/2andp2p^\{2\}are then at most1/21/2, so that1−x≥exp⁡\(−2​x\)1\-x\\geq\\exp\(\-2x\)applies withx∈\[0,1/2\]x\\in\[0,1/2\]\.

On the event𝒩t\\mathcal\{N\}\_\{t\}, the non\-reward observations have the same distribution underσ=\+1\\sigma=\+1andσ=−1\\sigma=\-1\. Indeed, if a side\-information arm reveals neither or only one of the coordinatese1,e2e\_\{1\},e\_\{2\}, then the observed value has the same distribution under both signs because the side arm contains an independent Rademacher multiplierWW\. Decision\-arm labels and vectors are independent ofσ\\sigmain the oracle\-augmented observation model\.

Now condition on𝒟t∩𝒩t\\mathcal\{D\}\_\{t\}\\cap\\mathcal\{N\}\_\{t\}\. Under this conditioning, the only remaining difference between the two signs comes from reward observations\. By the chain rule for KL divergence and the Gaussian reward model, we have the following bound:

KL​\(ℙ\+𝒩∥ℙ−𝒩\)≤∑s=1t−112​R2​\(μs\+−μs−\)2≤2​\(t−1\)​Δ2R2,\\mathrm\{KL\}\(\\mathbb\{P\}\_\{\+\}^\{\\mathcal\{N\}\}\\\|\\mathbb\{P\}\_\{\-\}^\{\\mathcal\{N\}\}\)\\leq\\sum\_\{s=1\}^\{t\-1\}\\frac\{1\}\{2R^\{2\}\}\(\\mu\_\{s\}^\{\+\}\-\\mu\_\{s\}^\{\-\}\)^\{2\}\\leq\\frac\{2\(t\-1\)\\Delta^\{2\}\}\{R^\{2\}\},\(34\)whereℙ\+𝒩\\mathbb\{P\}\_\{\+\}^\{\\mathcal\{N\}\}andℙ−𝒩\\mathbb\{P\}\_\{\-\}^\{\\mathcal\{N\}\}denote the conditional laws given𝒟t∩𝒩t\\mathcal\{D\}\_\{t\}\\cap\\mathcal\{N\}\_\{t\}, and the second inequality follows since every possible selected arm satisfies\|μs\+−μs−\|≤2​Δ\|\\mu\_\{s\}^\{\+\}\-\\mu\_\{s\}^\{\-\}\|\\leq 2\\Delta\. LetAAbe the event\{b^t=\+1\}\\\{\\widehat\{b\}\_\{t\}=\+1\\\}\. Again, we invoke the Bretagnolle–Huber inequality\[[3](https://arxiv.org/html/2607.08971#bib.bib3)\], applied conditionally on𝒟t∩𝒩t\\mathcal\{D\}\_\{t\}\\cap\\mathcal\{N\}\_\{t\}together with the bound in \([34](https://arxiv.org/html/2607.08971#A8.E34)\), to get

ℙ\+𝒩​\(Ac\)\+ℙ−𝒩​\(A\)≥12​exp⁡\(−2​\(t−1\)​Δ2R2\)\.\\mathbb\{P\}\_\{\+\}^\{\\mathcal\{N\}\}\(A^\{c\}\)\+\\mathbb\{P\}\_\{\-\}^\{\\mathcal\{N\}\}\(A\)\\geq\\frac\{1\}\{2\}\\exp\\left\(\-\\frac\{2\(t\-1\)\\Delta^\{2\}\}\{R^\{2\}\}\\right\)\.Multiplying by the common lower bound \([33](https://arxiv.org/html/2607.08971#A8.E33)\) for the probability of𝒩t\\mathcal\{N\}\_\{t\}conditional on𝒟t\\mathcal\{D\}\_\{t\}, and then averaging the two signs under the uniform prior onσ\\sigma, yields

ℙ​\(b^t≠σ∣𝒟t,ν\)≥14​exp⁡\(−2​K​p2​t−2​\(t−1\)​Δ2R2\)\.\\mathbb\{P\}\(\\widehat\{b\}\_\{t\}\\neq\\sigma\\mid\\mathcal\{D\}\_\{t\},\\nu\)\\geq\\frac\{1\}\{4\}\\exp\\left\(\-2Kp^\{2\}t\-\\frac\{2\(t\-1\)\\Delta^\{2\}\}\{R^\{2\}\}\\right\)\.
###### Theorem 5\(i\.i\.d\. action\-set lower bound\)

AssumeK≥4K\\geq 4,p≤1/2p\\leq 1/2, and letγ,Δ\>0\\gamma,\\Delta\>0satisfy \([29](https://arxiv.org/html/2607.08971#A8.E29)\)\. For the i\.i\.d\. action\-set instance above, every algorithm satisfies

maxν,σ∈\{±1\}⁡𝔼ν,σ​RT≥πK​γ4​∑t=1Texp⁡\(−2​\(t−1\)​γ2R2\)\+πK​Δ4​∑t=1Texp⁡\(−2​K​p2​t−2​\(t−1\)​Δ2R2\),\\max\_\{\\nu,\\sigma\\in\\\{\\pm 1\\\}\}\\mathbb\{E\}\_\{\\nu,\\sigma\}R\_\{T\}\\geq\\frac\{\\pi\_\{K\}\\gamma\}\{4\}\\sum\_\{t=1\}^\{T\}\\exp\\left\(\-\\frac\{2\(t\-1\)\\gamma^\{2\}\}\{R^\{2\}\}\\right\)\+\\frac\{\\pi\_\{K\}\\Delta\}\{4\}\\sum\_\{t=1\}^\{T\}\\exp\\left\(\-2Kp^\{2\}t\-\\frac\{2\(t\-1\)\\Delta^\{2\}\}\{R^\{2\}\}\\right\),whereπK\\pi\_\{K\}is defined in \([30](https://arxiv.org/html/2607.08971#A8.E30)\)\. Consequently, there is a universal constantc\>0c\>0such that, wheneverK​p2\+S2​BX2/R2≤c−1Kp^\{2\}\+S^\{2\}B\_\{X\}^\{2\}/R^\{2\}\\leq c^\{\-1\}, the following optimized bound holds:

supvalid instances𝔼​RT≥c​min⁡\{S​BX​T,R​T\}\+c​S​BX​min⁡\{T,1K​p2\+S2​BX2/R2\}\.\\sup\_\{\\textnormal\{valid instances\}\}\\mathbb\{E\}R\_\{T\}\\geq c\\min\\\{SB\_\{X\}T,\\,R\\sqrt\{T\}\\\}\+cSB\_\{X\}\\min\\left\\\{T,\\frac\{1\}\{Kp^\{2\}\+S^\{2\}B\_\{X\}^\{2\}/R^\{2\}\}\\right\\\}\.In particular, in the context\-limited regimeR≥S​BX/\(p​K\)R\\geq SB\_\{X\}/\(p\\sqrt\{K\}\), one obtains

supvalid instances𝔼​RT≥c​min⁡\{S​BX​T,R​T\}\+c​S​BX​min⁡\{T,1K​p2\}\.\\sup\_\{\\textnormal\{valid instances\}\}\\mathbb\{E\}R\_\{T\}\\geq c\\min\\\{SB\_\{X\}T,\\,R\\sqrt\{T\}\\\}\+cSB\_\{X\}\\min\\left\\\{T,\\frac\{1\}\{Kp^\{2\}\}\\right\\\}\.

Proof:The proof of this theorem puts the above pieces together\. Notice that our construction allows us to add the contributions because of the two independent hidden bits\. The bitν\\nuis hard to learn only through noisy rewards, producing the standard stochastic\-bandit term\. The bitσ\\sigmais hard to learn until a side\-information arm reveals both relevant coordinates, which happens at rateK​p2Kp^\{2\}, or until rewards reveal it at rateΔ2/R2\\Delta^\{2\}/R^\{2\}\. The proof has two stages: we first establish a per\-round regret bound and sum it overtt, and we then optimize over the parametersγ\\gammaandΔ\\Delta\.

We begin by placing a uniform prior on\(ν,σ\)∈\{±1\}2\(\\nu,\\sigma\)\\in\\\{\\pm 1\\\}^\{2\}\. By Lemma[17](https://arxiv.org/html/2607.08971#Thmlemma17), on𝒟t\\mathcal\{D\}\_\{t\}, we have the following:

regrett≥γ​𝟏​\{a^t≠ν\}\+Δ​𝟏​\{b^t≠σ\}\.\\textnormal\{regret\}\_\{t\}\\geq\\gamma\\mathbf\{1\}\\\{\\widehat\{a\}\_\{t\}\\neq\\nu\\\}\+\\Delta\\mathbf\{1\}\\\{\\widehat\{b\}\_\{t\}\\neq\\sigma\\\}\.Notice that the event𝒟t\\mathcal\{D\}\_\{t\}depends only on the current arm labels and is independent of the hidden signs\. It has probabilityπK\\pi\_\{K\}\(as defined in \([30](https://arxiv.org/html/2607.08971#A8.E30)\)\)\. Moreover, the instantaneous regret is nonnegative on every round \(the oracle maximizes over the offered set\), so restricting attention to the rounds on which𝒟t\\mathcal\{D\}\_\{t\}holds can only decrease the total\. Taking expectation under the uniform prior and using Lemmas[18](https://arxiv.org/html/2607.08971#Thmlemma18)and[19](https://arxiv.org/html/2607.08971#Thmlemma19)gives us the following:

𝔼​\[regrett\]≥πK​γ4​exp⁡\(−2​\(t−1\)​γ2R2\)\+πK​Δ4​exp⁡\(−2​K​p2​t−2​\(t−1\)​Δ2R2\)\.\\mathbb\{E\}\[\\textnormal\{regret\}\_\{t\}\]\\geq\\frac\{\\pi\_\{K\}\\gamma\}\{4\}\\exp\\left\(\-\\frac\{2\(t\-1\)\\gamma^\{2\}\}\{R^\{2\}\}\\right\)\+\\frac\{\\pi\_\{K\}\\Delta\}\{4\}\\exp\\left\(\-2Kp^\{2\}t\-\\frac\{2\(t\-1\)\\Delta^\{2\}\}\{R^\{2\}\}\\right\)\.\(35\)We now sum \([35](https://arxiv.org/html/2607.08971#A8.E35)\) overt=1,…,Tt=1,\\ldots,Tto obtain the first claim\. Since the maximum over environments is at least the Bayes average, the same lower bound holds formaxν,σ⁡𝔼ν,σ​RT\\max\_\{\\nu,\\sigma\}\\mathbb\{E\}\_\{\\nu,\\sigma\}R\_\{T\}\.

Next, for the optimized form, we choose

Δ=c0​S​BX,γ=c0​min⁡\{S​BX,R/T\},\\Delta=c\_\{0\}SB\_\{X\},\\qquad\\gamma=c\_\{0\}\\min\\\{SB\_\{X\},R/\\sqrt\{T\}\\\},withc0\>0c\_\{0\}\>0sufficiently small so that \([29](https://arxiv.org/html/2607.08971#A8.E29)\) holds\. SinceπK≥3/512\\pi\_\{K\}\\geq 3/512by \([31](https://arxiv.org/html/2607.08971#A8.E31)\), the first exponential sum may be bounded as follows:

γ​∑t=1Texp⁡\(−2​\(t−1\)​γ2R2\)≥c​min⁡\{S​BX​T,R​T\}\.\\gamma\\sum\_\{t=1\}^\{T\}\\exp\\left\(\-\\frac\{2\(t\-1\)\\gamma^\{2\}\}\{R^\{2\}\}\\right\)\\geq c\\min\\\{SB\_\{X\}T,R\\sqrt\{T\}\\\}\.For the second sum, leta:=2​K​p2\+2​Δ2/R2\.a:=2Kp^\{2\}\+2\\Delta^\{2\}/R^\{2\}\.Notice that, ifa≤1a\\leq 1, we have∑t=1Te−a​t≥c​min⁡\{T,1/a\}\.\\sum\_\{t=1\}^\{T\}e^\{\-at\}\\geq c\\min\\\{T,1/a\\\}\.Therefore, usingΔ=c0​S​BX\\Delta=c\_\{0\}SB\_\{X\}, we obtain the following bound:

Δ​∑t=1Texp⁡\(−2​K​p2​t−2​\(t−1\)​Δ2R2\)≥c​S​BX​min⁡\{T,1K​p2\+S2​BX2/R2\}\.\\Delta\\sum\_\{t=1\}^\{T\}\\exp\\left\(\-2Kp^\{2\}t\-\\frac\{2\(t\-1\)\\Delta^\{2\}\}\{R^\{2\}\}\\right\)\\geq cSB\_\{X\}\\min\\left\\\{T,\\frac\{1\}\{Kp^\{2\}\+S^\{2\}B\_\{X\}^\{2\}/R^\{2\}\}\\right\\\}\.\(36\)This proves the optimized bound under the stated nontriviality condition\. IfR≥S​BX/\(p​K\)R\\geq SB\_\{X\}/\(p\\sqrt\{K\}\), thenS2​BX2/R2≤K​p2S^\{2\}B\_\{X\}^\{2\}/R^\{2\}\\leq Kp^\{2\}, and Equation \([36](https://arxiv.org/html/2607.08971#A8.E36)\) implies the “context\-limited” corollary\.

## Appendix IAdaptivity to Unknown Subspace Dimensionality

We now give the full argument for the rank\-adaptive version ofTOFU\-POV\. The known\-rank proof assumes that the learner is told the latent dimensionmm, so that each epoch uses the topmmdirections of the corrected covariance estimate\. Whenmmis unknown, our rank\-adaptive algorithm first estimates the spectrum of the corrected covariance and keeps only eigenvalues that are separated from the noise floor\. The proof below shows that, after a finite “rank\-identification” time, this thresholding rule selects exactly the signal subspace\. From that epoch onward the algorithm is identical to the known\-rank procedure, and all regret before this point is charged by a worst\-case bound\.

The argument below has three parts\. First, we define a uniform covariance perturbation event that controls the empirical spectrum at all epoch starts\. Second, we show that a simple spectral threshold recovers the correct rank once the perturbation radius is below the population eigengap\. Third, we combine rank identification with the burn\-in condition needed for imputation, and then reuse the known\-rank epoch regret template\. We first begin with the spectral event used to separate signal eigenvalues from null directions\.

Covariance perturbation event\.LetΣ:=𝔼​\[Xt,i​Xt,i⊤\]\\Sigma:=\\mathbb\{E\}\[X\_\{t,i\}X\_\{t,i\}^\{\\top\}\]denote the population covariance of the ideal action vectors\. Let its eigenvalues be

λ1​\(Σ\)≥λ2​\(Σ\)≥⋯≥λd​\(Σ\)\.\\lambda\_\{1\}\(\\Sigma\)\\geq\\lambda\_\{2\}\(\\Sigma\)\\geq\\cdots\\geq\\lambda\_\{d\}\(\\Sigma\)\.Recall that we suppose that the rank ofΣ\\Sigmaismm, and therefore, we have thatλm​\(Σ\)\>0\\lambda\_\{m\}\(\\Sigma\)\>0, whileλm\+1​\(Σ\)=0\.\\lambda\_\{m\+1\}\(\\Sigma\)=0\.To avoid overloading notation, define the population eigengap

Δm:=λm​\(Σ\)−λm\+1​\(Σ\)=λm​\(Σ\)\.\\Delta\_\{m\}:=\\lambda\_\{m\}\(\\Sigma\)\-\\lambda\_\{m\+1\}\(\\Sigma\)=\\lambda\_\{m\}\(\\Sigma\)\.
LetΣ˙t\\dot\{\\Sigma\}\_\{t\}be the unbiased covariance estimator in Equation \([3](https://arxiv.org/html/2607.08971#S4.E3)\), and let

λ^t,1≥λ^t,2≥⋯≥λ^t,d\\hat\{\\lambda\}\_\{t,1\}\\geq\\hat\{\\lambda\}\_\{t,2\}\\geq\\cdots\\geq\\hat\{\\lambda\}\_\{t,d\}
be its eigenvalues\. For a target failure probabilityδrank\\delta\_\{\\rm rank\}, define

ρt:=2​BX​λ¯p2​t​K​log⁡2​d​Tδrank\+2​BX2p2​t​K​log⁡2​d​Tδrank,\\rho\_\{t\}:=2B\_\{X\}\\sqrt\{\\frac\{\\bar\{\\lambda\}\}\{p^\{2\}\\,tK\}\\log\\frac\{2dT\}\{\\delta\_\{\\rm rank\}\}\}\+\\frac\{2B\_\{X\}^\{2\}\}\{p^\{2\}\\,tK\}\\log\\frac\{2dT\}\{\\delta\_\{\\rm rank\}\},\(37\)which is exactly the high\-probability bound of Lemma[7](https://arxiv.org/html/2607.08971#Thmlemma7)at levelδrank/T\\delta\_\{\\rm rank\}/T\. Note thatρt\\rho\_\{t\}is computable from\(BX,p,K\)\(B\_\{X\},p,K\)alone \(recall that one may always takeλ¯=BX2\\bar\{\\lambda\}=B\_\{X\}^\{2\}\); in particular, no knowledge of upper bound on the rankmmis required\. In what follows, we work on the event

𝒢rank:=\{‖Σ˙t−Σ‖2≤ρtfor all​t∈\[T\]\}\.\\mathcal\{G\}\_\{\\rm rank\}:=\\left\\\{\\\|\\dot\{\\Sigma\}\_\{t\}\-\\Sigma\\\|\_\{2\}\\leq\\rho\_\{t\}\\quad\\text\{for all \}t\\in\[T\]\\right\\\}\.\(38\)Indeed, applying Lemma[7](https://arxiv.org/html/2607.08971#Thmlemma7)at levelδrank/T\\delta\_\{\\rm rank\}/Tand taking a union bound overt∈\[T\]t\\in\[T\], we have that

Pr⁡\(𝒢rank\)≥1−δrank\.\\Pr\(\\mathcal\{G\}\_\{\\rm rank\}\)\\geq 1\-\\delta\_\{\\rm rank\}\.
On this event, the empirical eigenvalues are uniformly close to the population eigenvalues\. The rank selector below keeps precisely those empirical directions whose eigenvalues exceed twice this perturbation radius\.

Rank selector\.At the beginning of epochee, define

m^e:=\#​\{j∈\[d\]:λ^τe−1,j≥2​ρτe−1\}\.\\hat\{m\}\_\{e\}:=\\\#\\left\\\{j\\in\[d\]:\\hat\{\\lambda\}\_\{\\tau\_\{e\}\-1,j\}\\geq 2\\rho\_\{\\tau\_\{e\}\-1\}\\right\\\}\.\(39\)The rank\-adaptive algorithm uses the topm^e\\hat\{m\}\_\{e\}eigenvectors ofΣ˙τe−1\\dot\{\\Sigma\}\_\{\\tau\_\{e\}\-1\}to form𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}\. Ifm^e=0\\hat\{m\}\_\{e\}=0, the algorithm may use any admissible fallback policy in that epoch; the regret beforemmwill be bounded conservatively\.

Define the rank\-identification time

trank:=⌈512​BX2​λ¯Δm2​p2​K​log⁡\(2​d​Tδrank\)⌉=⌈512​κ2​mp2​K​log⁡\(2​d​Tδrank\)⌉,t\_\{\\rm rank\}:=\\left\\lceil\\frac\{512\\,B\_\{X\}^\{2\}\\bar\{\\lambda\}\}\{\\Delta\_\{m\}^\{2\}p^\{2\}K\}\\log\\\!\\left\(\\frac\{2dT\}\{\\delta\_\{\\rm rank\}\}\\right\)\\right\\rceil=\\left\\lceil\\frac\{512\\,\\kappa^\{2\}m\}\{p^\{2\}K\}\\log\\\!\\left\(\\frac\{2dT\}\{\\delta\_\{\\rm rank\}\}\\right\)\\right\\rceil,\(40\)where the second expression follows sinceΔm=λm\\Delta\_\{m\}=\\lambda\_\{m\}andκ2​m=BX2​λ¯/λm2\\kappa^\{2\}m=B\_\{X\}^\{2\}\\bar\{\\lambda\}/\\lambda\_\{m\}^\{2\}\. This is the form quoted in Theorem[3](https://arxiv.org/html/2607.08971#Thmtheorem3)\. We now show that, for everyt≥trankt\\geq t\_\{\\rm rank\},

ρt≤Δm4\.\\rho\_\{t\}\\leq\\frac\{\\Delta\_\{m\}\}\{4\}\.\(41\)Sincet​K≥512​BX2​λ¯​log⁡\(2​d​T/δrank\)/\(Δm2​p2\)tK\\geq 512\\,B\_\{X\}^\{2\}\\bar\{\\lambda\}\\log\(2dT/\\delta\_\{\\rm rank\}\)/\(\\Delta\_\{m\}^\{2\}p^\{2\}\), the square\-root term of \([37](https://arxiv.org/html/2607.08971#A9.E37)\) is at most2​Δm/512≤Δm/82\\Delta\_\{m\}/\\sqrt\{512\}\\leq\\Delta\_\{m\}/8, and the linear term is at mostΔm2/\(256​λ¯\)≤Δm/8\\Delta\_\{m\}^\{2\}/\(256\\,\\bar\{\\lambda\}\)\\leq\\Delta\_\{m\}/8, where we usedΔm=λm≤λ¯\\Delta\_\{m\}=\\lambda\_\{m\}\\leq\\bar\{\\lambda\}\. Adding the two terms gives \([41](https://arxiv.org/html/2607.08971#A9.E41)\)\.

The next lemma formalizes the separation argument\. Onceρτe−1≤Δm/4\\rho\_\{\\tau\_\{e\}\-1\}\\leq\\Delta\_\{m\}/4, every true signal eigenvalue remains above the threshold2​ρτe−12\\rho\_\{\\tau\_\{e\}\-1\}, while every null eigenvalue remains below it\.

###### Lemma 20\(Rank identification by spectral thresholding\)

On the event𝒢rank\\mathcal\{G\}\_\{\\rm rank\}, for every epocheewithτe−1≥trank\\tau\_\{e\}\-1\\geq t\_\{\\rm rank\}, the selector in Equation \([39](https://arxiv.org/html/2607.08971#A9.E39)\) recovers the true rank\. That ism^e=m\.\\hat\{m\}\_\{e\}=m\.

Proof:Fix an epocheewithτe−1≥trank\\tau\_\{e\}\-1\\geq t\_\{\\rm rank\}\. We will show that, on𝒢rank\\mathcal\{G\}\_\{\\rm rank\}, every signal eigenvalue \(j≤mj\\leq m\) clears the threshold2​ρτe−12\\rho\_\{\\tau\_\{e\}\-1\}, while every null eigenvalue \(j\>mj\>m\) falls below it, so that the selector in Equation \([39](https://arxiv.org/html/2607.08971#A9.E39)\) counts exactlymmdirections\.

We begin by transferring the covariance perturbation to the eigenvalues\. On𝒢rank\\mathcal\{G\}\_\{\\rm rank\}, we have‖Σ˙τe−1−Σ‖2≤ρτe−1\\\|\\dot\{\\Sigma\}\_\{\\tau\_\{e\}\-1\}\-\\Sigma\\\|\_\{2\}\\leq\\rho\_\{\\tau\_\{e\}\-1\}, and therefore, by Weyl’s inequality\[[43](https://arxiv.org/html/2607.08971#bib.bib43)\], we have the following inequality for allj∈\[d\]j\\in\[d\]:

\|λ^τe−1,j−λj​\(Σ\)\|≤ρτe−1\.\|\\hat\{\\lambda\}\_\{\\tau\_\{e\}\-1,j\}\-\\lambda\_\{j\}\(\\Sigma\)\|\\leq\\rho\_\{\\tau\_\{e\}\-1\}\.Moreover, sinceτe−1≥trank\\tau\_\{e\}\-1\\geq t\_\{\\rm rank\}, Equation \([41](https://arxiv.org/html/2607.08971#A9.E41)\) givesρτe−1≤Δm/4\\rho\_\{\\tau\_\{e\}\-1\}\\leq\\Delta\_\{m\}/4\.

First, we consider the signal eigenvalues,j≤mj\\leq m\. Sinceλj​\(Σ\)≥λm​\(Σ\)=Δm\\lambda\_\{j\}\(\\Sigma\)\\geq\\lambda\_\{m\}\(\\Sigma\)=\\Delta\_\{m\}, we may chain the two preceding bounds to obtain:

λ^τe−1,j≥λj​\(Σ\)−ρτe−1≥Δm−Δm4=3​Δm4\>Δm2≥2​ρτe−1\.\\hat\{\\lambda\}\_\{\\tau\_\{e\}\-1,j\}\\geq\\lambda\_\{j\}\(\\Sigma\)\-\\rho\_\{\\tau\_\{e\}\-1\}\\geq\\Delta\_\{m\}\-\\frac\{\\Delta\_\{m\}\}\{4\}=\\frac\{3\\Delta\_\{m\}\}\{4\}\>\\frac\{\\Delta\_\{m\}\}\{2\}\\geq 2\\rho\_\{\\tau\_\{e\}\-1\}\.Therefore, every signal eigenvalue is selected\.

Next, we consider the null eigenvalues,j\>mj\>m\. Sinceλj​\(Σ\)=0\\lambda\_\{j\}\(\\Sigma\)=0, the same perturbation bound gives:

λ^τe−1,j≤ρτe−1<2​ρτe−1,\\hat\{\\lambda\}\_\{\\tau\_\{e\}\-1,j\}\\leq\\rho\_\{\\tau\_\{e\}\-1\}<2\\rho\_\{\\tau\_\{e\}\-1\},so no null eigenvalue is selected\. Putting the two cases together, exactlymmempirical eigenvalues exceed the threshold2​ρτe−12\\rho\_\{\\tau\_\{e\}\-1\}, and hencem^e=m\\hat\{m\}\_\{e\}=m\.

Once Lemma[20](https://arxiv.org/html/2607.08971#Thmlemma20)has identified the rank, the remaining representation and imputation guarantees exactly follow the known\-rank ones \(conditional on the event identified above\)\. The only bookkeeping is to wait until both prerequisites hold: the rank must be identified, and the burn\-in condition for stable imputation must have passed\. This is why we introduce the synchronization timetid:=max⁡\{tb,trank\}t\_\{\\rm id\}:=\\max\\\{t\_\{b\},t\_\{\\rm rank\}\\\}\.

###### Lemma 21\(Representation event after rank identification\)

Lettid:=max⁡\{tb,trank\},t\_\{\\rm id\}:=\\max\\\{t\_\{b\},t\_\{\\rm rank\}\\\},and let𝒢rep\\mathcal\{G\}\_\{\\rm rep\}denote the subspace and imputation good event of Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)and Lemma[2](https://arxiv.org/html/2607.08971#Thmlemma2)\. On the event𝒢rank∩𝒢rep,\\mathcal\{G\}\_\{\\rm rank\}\\cap\\mathcal\{G\}\_\{\\rm rep\},every epocheewithτe−1≥tid\\tau\_\{e\}\-1\\geq t\_\{\\rm id\}satisfiesm^e=m\\hat\{m\}\_\{e\}=m\. Consequently, the adaptive basis𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}coincides with the known\-rank epoch basis, and the subspace and imputation guarantees of Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)and Lemma[2](https://arxiv.org/html/2607.08971#Thmlemma2)hold for epocheeas stated\.

Proof:Fix an epocheewithτe−1≥tid\\tau\_\{e\}\-1\\geq t\_\{\\rm id\}\. Sincetid≥trankt\_\{\\rm id\}\\geq t\_\{\\rm rank\}, Lemma[20](https://arxiv.org/html/2607.08971#Thmlemma20)givesm^e=m,\\hat\{m\}\_\{e\}=m,so the adaptive basis𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}is the top\-mmeigenspace ofΣ˙τe−1\\dot\{\\Sigma\}\_\{\\tau\_\{e\}\-1\}; this is exactly the basis that the known\-rank algorithm would use at the start of the epoch\. Sincetid≥tbt\_\{\\rm id\}\\geq t\_\{b\}, we also haveτe−1≥tb\\tau\_\{e\}\-1\\geq t\_\{b\}, and therefore, on𝒢rep\\mathcal\{G\}\_\{\\rm rep\}, the conclusions of Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1)and of Lemma[2](https://arxiv.org/html/2607.08971#Thmlemma2)apply verbatim to𝐔^e\\hat\{\\mathbf\{U\}\}\_\{e\}\.

It remains to account for the few epochs before both conditions hold\. We isolate the first epoch whose start time is beyondtidt\_\{\\rm id\}; all earlier rounds will be charged directly, and all later epochs can use the known\-rank analysis\.

First correctly ranked epoch\.Let

e⋆:=min⁡\{e:τe−1≥tid\},τ⋆:=τe⋆\.e\_\{\\star\}:=\\min\\\{e:\\tau\_\{e\}\-1\\geq t\_\{\\rm id\}\\\},\\qquad\\tau\_\{\\star\}:=\\tau\_\{e\_\{\\star\}\}\.Since the epoch starts are on a doubling schedule, we have that

τ⋆≤2​\(tid\+1\)\.\\tau\_\{\\star\}\\leq 2\(t\_\{\\rm id\}\+1\)\.\(42\)Indeed, ifτ⋆\\tau\_\{\\star\}is the first epoch start at leasttidt\_\{\\rm id\}, then the previous epoch start, if it exists, is smaller thantidt\_\{\\rm id\}, and the next epoch start is twice the previous one\. The additive11covers the case wheretidt\_\{\\rm id\}is below the first epoch start\. We can now state the explicit regret theorem\. Let

ET:=⌈log2⁡T⌉\+1,δe:=δofulET\.E\_\{T\}:=\\lceil\\log\_\{2\}T\\rceil\+1,\\qquad\\delta\_\{e\}:=\\frac\{\\delta\_\{\\rm oful\}\}\{E\_\{T\}\}\.As in the known\-rank analysis, we set the regularization toλ:=4​BX2\\lambda:=4B\_\{X\}^\{2\}and define

GT:=m​log⁡\(1\+Tm\),HT:=2​log⁡\(ETδoful\),Arep:=4​2​Csub​S​BX​κp2​mK​log⁡\(8​d​Tδrep\),G\_\{T\}:=m\\log\\\!\\left\(1\+\\frac\{T\}\{m\}\\right\),\\qquad H\_\{T\}:=2\\log\\\!\\left\(\\frac\{E\_\{T\}\}\{\\delta\_\{\\rm oful\}\}\\right\),\\qquad A\_\{\\rm rep\}:=4\\sqrt\{2\}\\,C\_\{\\rm sub\}SB\_\{X\}\\frac\{\\kappa\}\{p^\{2\}\}\\sqrt\{\\frac\{m\}\{K\}\\log\\\!\\left\(\\frac\{8dT\}\{\\delta\_\{\\rm rep\}\}\\right\)\},matching Lemma[15](https://arxiv.org/html/2607.08971#Thmlemma15)\. By Lemma[1](https://arxiv.org/html/2607.08971#Thmlemma1), for every correctly ranked epochee,

be:=S​BX​\(2\+2p\)​ϵτe−1≤Arepτe\.b\_\{e\}:=SB\_\{X\}\\left\(2\+\\frac\{2\}\{p\}\\right\)\\epsilon\_\{\\tau\_\{e\}\-1\}\\leq\\frac\{A\_\{\\rm rep\}\}\{\\sqrt\{\\tau\_\{e\}\}\}\.\(43\)The inequality usesp≤1p\\leq 1, so2\+2/p≤4/p2\+2/p\\leq 4/p, together withτe−1≥τe/2\\tau\_\{e\}\-1\\geq\\tau\_\{e\}/2, exactly as in the proof of Lemma[15](https://arxiv.org/html/2607.08971#Thmlemma15)\.

We can now state the regret bound\. The theorem is the same known\-rank epoch summation, with one additional cost for the rounds before the first correctly ranked and stably imputable epoch\.

###### Theorem 6\(Restatement of Theorem[3](https://arxiv.org/html/2607.08971#Thmtheorem3), with explicit constants\)

Assume the hypotheses of Theorem[2](https://arxiv.org/html/2607.08971#Thmtheorem2), except thatmmis not known to the algorithm\. Let the rank\-adaptive algorithm use the selector in Equation \([39](https://arxiv.org/html/2607.08971#A9.E39)\)\. Suppose

Pr⁡\(𝒢rank\)≥1−δrank,Pr⁡\(𝒢rep\)≥1−δrep,\\Pr\(\\mathcal\{G\}\_\{\\rm rank\}\)\\geq 1\-\\delta\_\{\\rm rank\},\\qquad\\Pr\(\\mathcal\{G\}\_\{\\rm rep\}\)\\geq 1\-\\delta\_\{\\rm rep\},and allocate the OFUL confidence probabilities as above\. Then, with probability at least1−δrank−δrep−δoful,1\-\\delta\_\{\\rm rank\}\-\\delta\_\{\\rm rep\}\-\\delta\_\{\\rm oful\},the regret of the rank\-adaptive epoch\-wise algorithm satisfies

RT\\displaystyle R\_\{T\}≤4​BX​S​\(tid\+1\)\+8​2​S​λ​GT​T\+8​2​R​GT​\(GT\+HT\)​T\+8​Arep​\(GT\+1\)​T\.\\displaystyle\\leq 4B\_\{X\}S\(t\_\{\\rm id\}\+1\)\+8\\sqrt\{2\}S\\sqrt\{\\lambda G\_\{T\}T\}\+8\\sqrt\{2\}R\\sqrt\{G\_\{T\}\(G\_\{T\}\+H\_\{T\}\)T\}\+8A\_\{\\rm rep\}\(\\sqrt\{G\_\{T\}\}\+1\)\\sqrt\{T\}\.\(44\)Consequently, up to logarithmic factors,

RT≤O​\(BX​S​tid\)\+O~​\(\(R\+S​BX\)​m​T\)\+O~​\(S​BX​κ​m​Tp2​K\),R\_\{T\}\\leq O\(B\_\{X\}S\\,t\_\{\\rm id\}\)\+\\widetilde\{O\}\\\!\\left\(\(R\+SB\_\{X\}\)\\,m\\sqrt\{T\}\\right\)\+\\widetilde\{O\}\\\!\\left\(SB\_\{X\}\\frac\{\\kappa m\\sqrt\{T\}\}\{p^\{2\}\\sqrt\{K\}\}\\right\),wheretid=max⁡\{tb,trank\}t\_\{\\rm id\}=\\max\\\{t\_\{b\},\\,t\_\{\\rm rank\}\\\}, withtrankt\_\{\\rm rank\}as in Equation \([40](https://arxiv.org/html/2607.08971#A9.E40)\)\.

Proof:The proof follows that of Theorem[2](https://arxiv.org/html/2607.08971#Thmtheorem2)in Appendix[F](https://arxiv.org/html/2607.08971#A6); we describe the two modifications\. First, the good event additionally includes the rank event: we work on𝒢rank∩𝒢rep∩𝒢oful,\\mathcal\{G\}\_\{\\rm rank\}\\cap\\mathcal\{G\}\_\{\\rm rep\}\\cap\\mathcal\{G\}\_\{\\rm oful\},where𝒢oful\\mathcal\{G\}\_\{\\rm oful\}is the intersection of all epoch\-wise OFUL confidence events\. Since∑e=0ET−1δe=δoful,\\sum\_\{e=0\}^\{E\_\{T\}\-1\}\\delta\_\{e\}=\\delta\_\{\\rm oful\},the same union bound as before gives

Pr⁡\(𝒢rank∩𝒢rep∩𝒢oful\)≥1−δrank−δrep−δoful\.\\Pr\(\\mathcal\{G\}\_\{\\rm rank\}\\cap\\mathcal\{G\}\_\{\\rm rep\}\\cap\\mathcal\{G\}\_\{\\rm oful\}\)\\geq 1\-\\delta\_\{\\rm rank\}\-\\delta\_\{\\rm rep\}\-\\delta\_\{\\rm oful\}\.
Second, the worst\-case portion of the horizon is potentially longer: rather than only the burn\-in rounds, we charge every round before the first correctly ranked and stably imputable epoch at the worst\-case rate\. Let

e⋆:=min⁡\{e:τe≥tid\},τ⋆:=τe⋆\.e\_\{\\star\}:=\\min\\\{e:\\tau\_\{e\}\\geq t\_\{\\rm id\}\\\},\\qquad\\tau\_\{\\star\}:=\\tau\_\{e\_\{\\star\}\}\.Beforeτ⋆\\tau\_\{\\star\}, the rank may be wrong or imputation may not yet be stable\. Since each one\-step regret is at most2​BX​S2B\_\{X\}S\(by the Cauchy–Schwarz inequality, as before\), Equation \([42](https://arxiv.org/html/2607.08971#A9.E42)\) gives

∑t<τ⋆\(μt,it⋆−μt,it\)≤2​BX​S​τ⋆≤4​BX​S​\(tid\+1\)\.\\sum\_\{t<\\tau\_\{\\star\}\}\(\\mu\_\{t,i\_\{t\}^\{\\star\}\}\-\\mu\_\{t,i\_\{t\}\}\)\\leq 2B\_\{X\}S\\,\\tau\_\{\\star\}\\leq 4B\_\{X\}S\(t\_\{\\rm id\}\+1\)\.
Fromτ⋆\\tau\_\{\\star\}onward, the analysis is identical to the known\-rank case\. For every epoche≥e⋆e\\geq e\_\{\\star\}, Lemma[21](https://arxiv.org/html/2607.08971#Thmlemma21)shows that the adaptive algorithm uses the correctmm\-dimensional representation with the same subspace and imputation guarantees as in the known\-rank proof, so Lemma[16](https://arxiv.org/html/2607.08971#Thmlemma16)applies withHe=HTH\_\{e\}=H\_\{T\}and yields exactly the epoch bound \([26](https://arxiv.org/html/2607.08971#A6.E26)\) from the proof of Theorem[2](https://arxiv.org/html/2607.08971#Thmtheorem2)\. The summation over epochs is also unchanged: the doubling\-schedule bounds∑e≥e⋆ne≤4​T\\sum\_\{e\\geq e\_\{\\star\}\}\\sqrt\{n\_\{e\}\}\\leq 4\\sqrt\{T\}and∑e≥e⋆ne/τe≤4​T\\sum\_\{e\\geq e\_\{\\star\}\}n\_\{e\}/\\sqrt\{\\tau\_\{e\}\}\\leq 4\\sqrt\{T\}, established in that proof, hold verbatim here since the sums run over a subset of the epochs\. We therefore have:

∑e≥e⋆Re\\displaystyle\\sum\_\{e\\geq e\_\{\\star\}\}R\_\{e\}≤8​2​S​λ​GT​T\+8​2​R​GT​\(GT\+HT\)​T\+8​Arep​\(GT\+1\)​T\.\\displaystyle\\leq 8\\sqrt\{2\}S\\sqrt\{\\lambda G\_\{T\}T\}\+8\\sqrt\{2\}R\\sqrt\{G\_\{T\}\(G\_\{T\}\+H\_\{T\}\)T\}\+8A\_\{\\rm rep\}\(\\sqrt\{G\_\{T\}\}\+1\)\\sqrt\{T\}\.Adding the pre\-identification regret4​BX​S​\(tid\+1\)4B\_\{X\}S\(t\_\{\\rm id\}\+1\)establishes Equation \([44](https://arxiv.org/html/2607.08971#A9.E44)\)\.

Remark\.The theorem makes the additional cost of unknown rank explicit\. Rank identification requires

ρt≤Δm/4,\\rho\_\{t\}\\leq\\Delta\_\{m\}/4,which yields the timetrankt\_\{\\rm rank\}in Equation \([40](https://arxiv.org/html/2607.08971#A9.E40)\)\. After the first epoch beginning aftertid=max⁡\{tb,trank\}t\_\{\\rm id\}=\\max\\\{t\_\{b\},t\_\{\\rm rank\}\\\}, the algorithm is identical to the known\-rank epoch\-wise method\. Moreover, comparing Equations \([40](https://arxiv.org/html/2607.08971#A9.E40)\) and \([5](https://arxiv.org/html/2607.08971#S4.E5)\), we havetrank=O~​\(κ2​m/\(p2​K\)\)t\_\{\\rm rank\}=\\widetilde\{O\}\\big\(\\kappa^\{2\}m/\(p^\{2\}K\)\\big\)whiletb=O~​\(κ2​m/\(p4​K\)\)t\_\{b\}=\\widetilde\{O\}\\big\(\\kappa^\{2\}m/\(p^\{4\}K\)\\big\), sotrank≤tbt\_\{\\rm rank\}\\leq t\_\{b\}up to constants wheneverδrank\\delta\_\{\\rm rank\}andδ\\deltaare of the same order\. Hencetid=tbt\_\{\\rm id\}=t\_\{b\}up to constants, and the identification cost is absorbed by the imputation burn\-in: stable imputation requires a stronger representation condition \(accuracy at scalepp\) than merely separating the nonzero and zero eigenvalues of the covariance matrix\.

## Appendix JAdditional Experimental Details

### J\.1Additional Experimental Diagnostics

This appendix collects additional experiments supporting Section[8](https://arxiv.org/html/2607.08971#S8): the full\-history synthetic counterpart to the main synthetic experiment, real\-feature synthetic tasks using optical digit covariates\[[33](https://arxiv.org/html/2607.08971#bib.bib33)\], rank recovery and rank\-misspecification diagnostics, warm\-start comparisons, MNIST product\-context diagnostics\[[32](https://arxiv.org/html/2607.08971#bib.bib32)\], and a text product\-context experiment using 20 Newsgroups\[[34](https://arxiv.org/html/2607.08971#bib.bib34)\]with an approximately low\-rank nuisance tail\.

Figure[3](https://arxiv.org/html/2607.08971#A10.F3)repeats the controlled synthetic experiment from Figure[1](https://arxiv.org/html/2607.08971#S8.F1)using the full\-history replay variants\. These are the practical variants used in the real\-feature experiments in the main text\. As expected, replaying past rewards after each frozen representation update lowers regret relative to the restart version while preserving the same missingness trend\.

![Refer to caption](https://arxiv.org/html/2607.08971v1/x4.png)\(a\)Regret over time\.
![Refer to caption](https://arxiv.org/html/2607.08971v1/x5.png)\(b\)Final regret vs\.pp\.

Figure 3:Full\-history synthetic counterpart\. TOFU\-FH and RA\-TOFU\-FH use the same subspace\-estimation mechanism as TOFU and RA\-TOFU, but replay previously observed rewards after each representation update\.Figure[4](https://arxiv.org/html/2607.08971#A10.F4)reports a “quasi\-synthetic” digit experiment\. The raw covariates are optical digit features\[[33](https://arxiv.org/html/2607.08971#bib.bib33)\], so the candidate arms are no longer drawn from the Gaussian latent model used in the main synthetic study\. At the same time, we keep the reward geometry controlled: the arm latents are constructed from these real covariates, embedded into a rank\-m⋆m^\{\\star\}subspace, and then masked coordinatewise using the same Bernoulli observation model as in the theory\. This lets us test whether the corrected low\-rank mechanism remains useful when the feature distribution is less idealized, while still retaining a known reward\-relevant subspace\. The qualitative pattern matches the fully synthetic experiment\. Whenppis large, zero\-imputed OFUL is competitive; asppdecreases, the ambient zero\-filled representation becomes increasingly distorted, and TOFU separates from the baselines by exploiting the recovered low\-rank structure\.

![Refer to caption](https://arxiv.org/html/2607.08971v1/x6.png)\(a\)Final regret vs\.pp\.
![Refer to caption](https://arxiv.org/html/2607.08971v1/x7.png)\(b\)Regret trajectories\.

Figure 4:Real\-feature synthetic experiment\. Real covariates replace Gaussian arms, but the reward\-relevant geometry remains low\-rank; TOFU benefits most when missingness is substantial\.Figures[5](https://arxiv.org/html/2607.08971#A10.F5)and[6](https://arxiv.org/html/2607.08971#A10.F6)probe the two implementation choices that are suppressed in the main synthetic figure: how the rank is selected, and how much benefit comes from reusing past reward data\. Figure[5](https://arxiv.org/html/2607.08971#A10.F5)shows the adaptive\-rank full\-history method together with the final selected ranks, making visible whether the thresholding rule is stabilizing near the intended dimension\. Figure[6](https://arxiv.org/html/2607.08971#A10.F6)then separates the diagnostics\. The fixed\-rank misspecification panel shows the cost of choosing a rank below or above the true value; the rank\-recovery panel checks that the corrected\-covariance spectrum contains a usable eigengap; and the warm\-start panel isolates the finite\-sample gain from re\-imputing previously selected arms and replaying their rewards after the first learned subspace is formed\.

![Refer to caption](https://arxiv.org/html/2607.08971v1/x8.png)Figure 5:Synthetic adaptive\-rank summary\. The final selected ranks are shown here rather than in the main paper; RA\-TOFU\-FH tracks the known\-rank TOFU\-FH method closely when the eigenspectrum separates the signal directions from the null directions\.![Refer to caption](https://arxiv.org/html/2607.08971v1/x9.png)\(a\)Fixed\-rank misspecification\.
![Refer to caption](https://arxiv.org/html/2607.08971v1/x10.png)\(b\)Rank recovery\.
![Refer to caption](https://arxiv.org/html/2607.08971v1/x11.png)\(c\)Warm\-start variants\.

Figure 6:Synthetic diagnostics\. Adaptive TOFU reduces sensitivity to rank choice, the corrected\-covariance spectrum identifies the relevant rank, and warm\-starting improves finite\-sample performance\.Figure[7](https://arxiv.org/html/2607.08971#A10.F7)gives additional diagnostics for the MNIST product\-context experiment in Figure[2](https://arxiv.org/html/2607.08971#S9.F2)\. This experiment starts from a supervised image model rather than a synthetic latent distribution\. We train a small CNN on MNIST\[[32](https://arxiv.org/html/2607.08971#bib.bib32)\], freeze it, and use itsm=4m=4\-dimensional penultimate representationh​\(x\)h\(x\)together with the final classification head\. Ifwk∈ℝ4w\_\{k\}\\in\\mathbb\{R\}^\{4\}is the class\-kkweight vector, the bandit arm for labelkkis the product contexth​\(x\)⊙wkh\(x\)\\odot w\_\{k\}, so the linear bandit reward preserves the classifier score through⟨h​\(x\)⊙wk,𝟏⟩=wk⊤​h​\(x\)\\langle h\(x\)\\odot w\_\{k\},\\mathbf\{1\}\\rangle=w\_\{k\}^\{\\top\}h\(x\)\. These ten class arms are then lifted intoℝ100\\mathbb\{R\}^\{100\}by a fixed orthonormal embedding and masked coordinatewise\. Thus the experiment uses real image\-derived representations, but the low\-rank bandit geometry is known by construction\. The diagnostics check that this construction is behaving as intended: the adaptive\-rank estimates concentrate near the construction rankm=4m=4, and the fixed\-rank validation sweep shows that the fixed\-rank baselines used in the main comparison were chosen on held\-out validation seeds rather than tuned on the reporting seeds\.

![Refer to caption](https://arxiv.org/html/2607.08971v1/x12.png)\(a\)Adaptive\-rank diagnostics\.
![Refer to caption](https://arxiv.org/html/2607.08971v1/x13.png)\(b\)Fixed\-rank validation\.

Figure 7:MNIST product\-context diagnostics\. The rank behavior is consistent with the constructed low\-rank feature map, and fixed\-rank comparisons use validation choices disjoint from the reporting seeds\.
### J\.2Text Product\-Context Experiment

As a second real\-data problem, we construct a text product\-context bandit from a four\-class 20 Newsgroups classification task\[[34](https://arxiv.org/html/2607.08971#bib.bib34)\]\. TF\-IDF features are compressed by TruncatedSVD and fit with a no\-intercept multinomial logistic\-regression classifier\. For documentxxand classkk, with document embeddingh​\(x\)∈ℝmh\(x\)\\in\\mathbb\{R\}^\{m\}and class weightwk∈ℝmw\_\{k\}\\in\\mathbb\{R\}^\{m\}, the bandit arm is the coordinatewise product contextXk​\(x\)=h​\(x\)⊙wkX\_\{k\}\(x\)=h\(x\)\\odot w\_\{k\}\. This preserves the classifier score since⟨Xk​\(x\),𝟏m⟩=wk⊤​h​\(x\)\\langle X\_\{k\}\(x\),\\mathbf\{1\}\_\{m\}\\rangle=w\_\{k\}^\{\\top\}h\(x\)\. The low\-dimensional product\-context arms are lifted into ambient dimensiond=1000d=1000, and we add a reward\-irrelevant orthogonal nuisance tail whose top empirical eigenvalue is0\.250\.25times the smallest retained signal eigenvalue\. Thus the instance is approximately low\-rank: the reward\-relevant subspace is recoverable, but ambient methods must learn through many irrelevant masked coordinates\.

The experiment usesm=20m=20,K=4K=4, horizonT=8000T=8000, five reporting seeds, and observation probabilitiesp∈\{0\.4,0\.3,0\.2\}p\\in\\\{0\.4,0\.3,0\.2\\\}\. The underlying text classifier has held\-out accuracy about0\.8600\.860\. Figure[8](https://arxiv.org/html/2607.08971#A10.F8)and Table[1](https://arxiv.org/html/2607.08971#A10.T1)show the same qualitative behavior as the image product\-context experiment\. Fixed\-rank full\-historyTOFU\-POVhas the lowest final regret at all tested missingness levels, adaptiveTOFU\-POVremains close, and masked PSLB is worse, especially atp=0\.2p=0\.2\. The gains over zero\-imputed OFUL are smaller than in MNIST but consistent across the sweep, giving a second real\-data modality in which the corrected low\-rank representation helps under coordinate missingness\.

![Refer to caption](https://arxiv.org/html/2607.08971v1/x14.png)\(a\)Final regret vs\.pp\.
![Refer to caption](https://arxiv.org/html/2607.08971v1/x15.png)\(b\)Regret trajectories atp=0\.4p=0\.4\.
![Refer to caption](https://arxiv.org/html/2607.08971v1/x16.png)\(c\)Adaptive\-rank diagnostics\.

Figure 8:Text product\-context experiment from four\-class 20 Newsgroups\. Fixed\-rank full\-historyTOFU\-POVis consistently best, while adaptiveTOFU\-POVremains competitive and masked PSLB is worse at heavier missingness\.Table 1:Final cumulative regret in the text product\-context experiment\. Entries are mean±\\pmstandard error over five reporting seeds\.The rank diagnostic explains the small gap between fixed\-rank and adaptiveTOFU\-POV\. The fixed\-rank methods use the construction rankm=20m=20, while adaptiveTOFU\-POVselects mean final ranks8\.2,7\.4,8\.2,7\.4,and5\.65\.6forp=0\.4,0\.3,p=0\.4,0\.3,and0\.20\.2, respectively\. Thus the adaptive selector is conservative on this approximately low\-rank text instance, especially when missingness is heavier\. Even with this lower selected rank, adaptiveTOFU\-POVremains close to the fixed\-rank method and improves over the zero\-imputed and masked\-PSLB baselines, which is the main point of the diagnostic\.

### J\.3Computational Resources

The reported experiments were run on an Exxact TensorEX 2U rackmount machine with two AMD EPYC Rome 7542 processors \(32 cores and 64 threads each\), 1 TB DDR4 ECC memory, four NVIDIA A100 SXM4 GPUs with 40 GB memory each, a 2 TB NVMe OS drive, a 15\.36 TB NVMe data drive, and Ubuntu 18\.04\. Some development and pilot runs were also executed on a 2021 Apple M1 Macbook with 64GB RAM, but the server configuration above is the conservative compute environment for reproducing the reported results\. We do not report exact runtimes, since the experiments are small\-scale validations of the theory rather than exhaustive compute benchmarks\.