Hierarchical Grading in Large Language Models

# Hierarchical Grading in Large Language Models
Source: [https://arxiv.org/html/2607.22757](https://arxiv.org/html/2607.22757)
T\. ShaskaDepartment of Computer Science and Engineering, Oakland University, Rochester, MI 48309[shaska@oakland\.edu](https://arxiv.org/html/2607.22757v1/mailto:[email protected])

###### Abstract\.

We introduce Graded Large Language Models \(GLLMs\), an algebraic framework that equips the representation space of a transformer with a grading and propagates the induced weighted scalar action through embeddings, self\-attention, and the training objective\. The construction extends the theory of graded neural networks and graded transformers to autoregressive language models while preserving expressive power, asymptotic computational complexity, and inference cost\.

The governing geometric picture is that of geometric invariant theory\. The benefit of a grading is expressed by a Kempf–Ness functional on the grading torus; the grades that improve upon the uniform architecture form an open convex cone whose membership is decided by a Hilbert–Mumford\-type criterion pairing a grade direction against two measurable profiles of the target and the data; the optimal grades are the coincidence point of two moment maps, given in closed form; and the ordinary transformer appears as a semistable isotropic point on the boundary of the cone: one member of a larger graded family rather than a distinguished optimum\.

Separately, for level\-stratified targets we prove a minimax separation between the graded prior and its absence: over all estimators the risks of the graded and uniform target classes separate throughout an explicit window of sample sizes, by a factor that decays exponentially in the number of levels under geometric stratification\. Both profiles are estimable offline, so the optimal grades solve a convex program certified before training begins\. Because the grading is absorbed into the learned parameters after training, every GLLM compiles to a standard transformer of identical architecture and inference complexity\.

We conclude with the grade\-selection procedure for domains without canonical gradings, instantiated for natural language, and a pre\-registered validation program whose empirical results will be reported in a companion manuscript\.

###### Key words and phrases:

graded vector spaces, graded transformers, large language models, hierarchical attention, sample complexity, minimax lower bounds, inductive bias

###### 2020 Mathematics Subject Classification:

68T07, 68T50, 68Q32, 62C20, 16W50

## 1\.Introduction

Large Language Models \(LLMs\) have moved from statisticalnn\-gram estimators to transformer systems ofO​\(1011\)O\(10^\{11\}\)parameters trained onO​\(1013\)O\(10^\{13\}\)tokens, and they now generate relatively fluent text, perform multi\-step tasks, and support code synthesis and scientific work; see\[radford2019language\]among many other sources\. This progress came almost entirely from scaling; see\[kaplan2020scaling\]\. The architecture behind it is uniform: every token occupies the same space, every coordinate of that space carries the same weight, and every attention head computes over a flat, isotropic geometry in which no direction is special\.

Language, in contrast, is hierarchical\. Subword units compose into words, words into phrases and clauses, clauses into sentences, sentences into discourse, and discourse into pragmatic inference\. A model with no representation of this hierarchy must recover it from data\. The cost of that recovery is the question that[Sec\.2](https://arxiv.org/html/2607.22757#S2)is organised around\. The answer is given mainly in two statements\. The first is the classical conjecture that an architecture without a structural prior must discriminate among theΩ​\(nL\)\\Omega\(n^\{L\}\)parse trees a depth\-LLgrammar admits over inputs of lengthnn, at a sample cost exponential inLL\([Conj\.1](https://arxiv.org/html/2607.22757#Thmcon1)\)\. This conjecture motivated the graded program, but no result below depends on it\. The second is weaker, can be measured on an annotated corpus without training a model, and is the hypothesis the framework rests on: in language, the features that decide hierarchical targets are not the features that carry the corpus variance \([Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)\)\.

This work introduces Graded Large Language Models \(GLLMs\), which supply the missing prior through the algebra of the representation space rather than through new architectural components\. The construction builds on a line of work on graded vector spaces in machine learning: artificial neural networks over graded spaces\[2024\-02\], graded neural networks with graded neurons, activations, and losses\[sh\-89\], and the Graded Transformer framework comprising the Linearly Graded Transformer \(LGT\) and the Exponentially Graded Transformer \(EGT\)\[sh\-95\]\. The object underlying all of them is a graded vector spaceℝ𝐪n\\mathbb\{R\}^\{n\}\_\{\\mathbf\{q\}\}: a copy ofℝn\\mathbb\{R\}^\{n\}equipped with a grade tuple𝐪=\(q0,…,qn−1\)\\mathbf\{q\}=\(q\_\{0\},\\dots,q\_\{n\-1\}\)and the induced scalar action

λ⋆x=\(λq0​x0,…,λqn−1​xn−1\),λ\>1\.\\lambda\\star x=\(\\lambda^\{q\_\{0\}\}x\_\{0\},\\dots,\\lambda^\{q\_\{n\-1\}\}x\_\{n\-1\}\),\\qquad\\lambda\>1\.A grade tuple states which coordinates matter and by how much\. Assigning grades to the dimensions of an embedding space and propagating the action through attention, embeddings, and the loss amplifies the subspaces that carry syntactic heads, semantic nuclei, and pragmatic signal, and reduces the weight of the rest\. Nothing is removed: the grading changes the metric of the representation space and leaves the computation over it intact, so expressive power, parallelisability, and autoregressive decoding are all preserved\. What changes is the geometry the model searches over\.

The construction has three parts and a cost analysis\.[Sec\.3](https://arxiv.org/html/2607.22757#S3)develops exponentially graded multi\-head self\-attention \(EG\-MHSA\) within causal decoder\-only stacks and establishes three facts that locate the advantage\. Grading leaves the asymptotic cost of attention unchanged atO​\(n2​d\+n​d2\)O\(n^\{2\}d\+nd^\{2\}\)per layer \([Prop\.4](https://arxiv.org/html/2607.22757#Thmprop4)\)\. The grading matrix is absorbed into the projection weights by an invertible substitution \([Rem\.1](https://arxiv.org/html/2607.22757#Thmrem1)\), so a trained GLLM compiles to a standard transformer of identical architecture and carries its prior at zero deployment cost \([Cor\.1](https://arxiv.org/html/2607.22757#Thmcor1)\); no variant in[Tab\.1](https://arxiv.org/html/2607.22757#S2.T1)has this property, since recurrence and sparsity encode their bias in the computation and pay for it at every forward pass\. At a fixed norm budget the graded constraint set is an ellipsoid whose axes are aligned with the grades and which strictly contains the uniform ball \([Prop\.5](https://arxiv.org/html/2607.22757#Thmprop5)\), so the advantage cannot come from representability: a larger class at a common budget hurts generalization rather than helping it\. Together these facts place the advantage in the norm required to represent a given target, and hence in the data required to learn it\.[Sec\.4](https://arxiv.org/html/2607.22757#S4)introduces Multi\-Level Graded Embeddings \(MLGE\), decomposingℝd=⨁l=0L−1Vl\\mathbb\{R\}^\{d\}=\\bigoplus\_\{l=0\}^\{L\-1\}V\_\{l\}into level\-specific subspaces, and carries out that analysis\.[Sec\.5](https://arxiv.org/html/2607.22757#S5)extends the grading to the training objective and characterizes exactly when it remains consistent for the corpus conditional \([Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\)\.[Sec\.8](https://arxiv.org/html/2607.22757#S8)states the program that will test the resulting predictions, and[Sec\.6](https://arxiv.org/html/2607.22757#S6)assembles the results into the framework’s main purpose: a procedure that selects, certifies, and prices grades in domains that supply none\.

The analysis yields the paper’s central quantity\. Writeα\\alphafor the profile of the target’s energy across the graded basis andτ\\taufor the profile of the data’s variance \([Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2)\)\. Then the Rademacher\-based sample complexity of the graded class stands to that of the uniform class in the ratio

Λ​\(g\)\\displaystyle\\Lambda\(g\)=\(∑j=1dαj​gj−2\)​\(∑j=1dτj​gj2\),\\displaystyle=\\Bigl\(\\sum\_\{j=1\}^\{d\}\\alpha\_\{j\}\\,g\_\{j\}^\{\-2\}\\Bigr\)\\Bigl\(\\sum\_\{j=1\}^\{d\}\\tau\_\{j\}\\,g\_\{j\}^\{2\}\\Bigr\),ming∈ℝ\>0d⁡Λ​\(g\)\\displaystyle\\min\_\{g\\in\\mathbb\{R\}\_\{\>0\}^\{d\}\}\\Lambda\(g\)=\(∑j=1dαj​τj\)2=BC​\(α,τ\)2≤1,\\displaystyle=\\Bigl\(\\sum\_\{j=1\}^\{d\}\\sqrt\{\\alpha\_\{j\}\\tau\_\{j\}\}\\Bigr\)^\{2\}=\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\}\\leq 1,with equality exactly whenα=τ\\alpha=\\tau\([Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\)\. The entire benefit of grading is therefore the Bhattacharyya affinity between where the target places its weight and where the data places its variance\. Grading creates no information: it converts a correct prior about the target into reduced sample complexity at this rate, and an incorrect prior into increased sample complexity at the same rate\. Under geometric stratification of the profiles the ratio decays exponentially in the number of levels \([Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)\)\. This is the only route by which any exponential\-in\-LLstatement enters this work, and it enters as a hypothesis, not as a theorem\.

[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)reads that quantity as a condition on the grades themselves, and it is the paper’s main positive result\. The grades at which the graded bound is strictly smaller form an open convex set, invariant under translation along𝟏\\mathbf\{1\}, so restricting to non\-negative rationals costs nothing\. The set is entered, to first order, exactly when

⟨𝐪,α−τ⟩\>0,\\langle\\mathbf\{q\},\\alpha\-\\tau\\rangle\>0,that is, high grades on coordinates where the target’s energy exceeds the data’s variance and low grades elsewhere\. The set is nonempty precisely whenα≠τ\\alpha\\neq\\tau, and the standard transformer, which setsg=𝟏g=\\mathbf\{1\}, lies on its boundary: the gain there is exactly11and the gradient is2​\(log⁡λ\)​\(τ−α\)≠02\(\\log\\lambda\)\(\\tau\-\\alpha\)\\neq 0\. Isotropy is therefore not a local optimum of the sample\-complexity landscape\. Its gauge orbit is the unique set of grades at which the gain equals11for every corpus and every target—the choice that ignores the profiles—and a descending direction is available from it in the direction the two profiles determine\. Both profiles can be estimated offline, so the sign of that inner product is checkable before any pretraining begins \([Rem\.17](https://arxiv.org/html/2607.22757#Thmrem17)\)\. The optimal grades within the region are then the solution of a convex program rather than the result of a search \([Cor\.4](https://arxiv.org/html/2607.22757#Thmcor4)\), projected to the clipped region admitted by[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\. The gain attained at grades computed from estimated profiles degrades only to second order in the estimation error \([Lem\.2](https://arxiv.org/html/2607.22757#Thmlem2)\): the gradient of the gain vanishes at the optimum, so the framework is protected exactly where protection is needed\.

The geometry of this selection problem admits a natural interpretation in geometric invariant theory, developed in[Sec\.6\.7](https://arxiv.org/html/2607.22757#S6.SS7)\. The admissible set is an open convex cone; the criterion⟨𝐪,α−τ⟩\>0\\langle\\mathbf\{q\},\\alpha\-\\tau\\rangle\>0is a Hilbert–Mumford\-type pairing that decides which directions descend from the uniform point; and the ordinary transformer sits on the boundary as a semistable point of the grading action\. The detailed development is deferred to that subsection so that the statistical content can be stated first in elementary language\.

Three facts fix the location of the claim\. The advantage is not one of expressivity: by[Remarks1](https://arxiv.org/html/2607.22757#Thmrem1)and[3](https://arxiv.org/html/2607.22757#Thmrem3)graded and ungraded models realise the same function class, and the content of grading lies in parameterization, initialisation, and implicit bias\. It is also not one of optimisation: exponential rescaling degrades the smoothness constant by a factorλqmax≥1\\lambda^\{q\_\{\\max\}\}\\geq 1, so the stationarity guarantee available for a graded model is weaker than for its ungraded counterpart \([Lemmas5](https://arxiv.org/html/2607.22757#Thmlem5)and[5\.1](https://arxiv.org/html/2607.22757#S5.Thmthm1)\)\. This asymmetry is the substance of the claim, and both of its sides are priced by the same constant: at clipλqmax≤C\\lambda^\{q\_\{\\max\}\}\\leq Cthe penalty in optimisation steps is at mostCC, uniformly inLLand in the profiles, while the saving in tokens is at mostC2C^\{2\}\([Rem\.16](https://arxiv.org/html/2607.22757#Thmrem16),[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\)\. The price is linear in the clip and the purchase quadratic, so the trade is favourable at every setting, and the exponential regime of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)is reached as the clip opens\. Grading spends a cheap resource—compute per step—to save an expensive one—high\-quality tokens\. The advantage lives in the norm required to represent a given target, and hence in the data required to learn it, which is where[Propositions7](https://arxiv.org/html/2607.22757#Thmprop7)and[8](https://arxiv.org/html/2607.22757#Thmprop8)place it in closed form\.

[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)compares two upper bounds, for a graded embedding composed with a linear read\-out\. Since graded and ungraded models realise the same functions, the separation the framework claims is between the graded prior and its absence rather than between two hypothesis classes\. What converts the comparison of bounds into that separation is a matching minimax lower bound over the uniform target class, with the estimator unrestricted\.[Thm\.4\.1](https://arxiv.org/html/2607.22757#S4.Thmthm1)proves it in the level\-stratified regime of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3): at squared loss and over all estimators, the ratio of minimax risks isΘ​\(Λ​\(g\)\)\\Theta\(\\Lambda\(g\)\)throughout an explicit window of sample sizes, so in the regime the planned configurations occupy the separation is established\. The general case is[Conj\.2](https://arxiv.org/html/2607.22757#Thmcon2), and[Sec\.8\.4](https://arxiv.org/html/2607.22757#S8.SS4)records what its proof requires;[Rem\.11](https://arxiv.org/html/2607.22757#Thmrem11)records the scope\. Two limitations are resolved in this manuscript—the adaptivity cost of estimated grades by[Lem\.2](https://arxiv.org/html/2607.22757#Thmlem2)and the lower bound in the stratified regime by[Thm\.4\.1](https://arxiv.org/html/2607.22757#S4.Thmthm1)—and each of the remaining ones names the tool its repair would require\.

The costs are small and are computed rather than asserted\. Grading preserves the asymptotic complexity of attention exactly \([Prop\.4](https://arxiv.org/html/2607.22757#Thmprop4)\)\. The training overhead is dominated by the level projections atd2d^\{2\}parameters, giving0\.33%0\.33\\%at 345M and0\.24%0\.24\\%at 7B for the configurations of[Sec\.8\.1\.1](https://arxiv.org/html/2607.22757#S8.SS1.SSS1); this follows fromd2≪Nd^\{2\}\\ll Nat those scales and must be recomputed at other scales\. At deployment the overhead is zero: by[Corollaries1](https://arxiv.org/html/2607.22757#Thmcor1)and[3](https://arxiv.org/html/2607.22757#Thmrem3)a trained GLLM compiles to a standard transformer of identical architecture and identical inference cost, so the training figure is an upper bound on the total lifetime cost of the prior\.

The framework’s governing hypothesis is settled before any model is trained, and the predictions are staged so that the cheapest test comes first\. Both profiles can be estimated offline, soBC​\(α,τ\)\\mathrm\{BC\}\(\\alpha,\\tau\)is reported per task in advance; a value near11on the hierarchical tasks falsifies[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1), and with it every conditional prediction, at the cost of two estimates\. The same two estimates certify every candidate grading against the criterion above, one inner product apiece\. Configurations, baselines, benchmarks, and the selection ladder of[Tab\.2](https://arxiv.org/html/2607.22757#S6.T2)follow, with arms of identical architecture and budget distinguished only by what their grades consulted and ordered in advance by their certified inner products; each prediction is marked as established or as conditional on named hypotheses\. The ordering is fixed before training, which makes it a prediction rather than a fit, and[Sec\.8](https://arxiv.org/html/2607.22757#S8)specifies the runs that test it\.

The framework extends in a direction the present construction does not reach\. The grading of[Sec\.4](https://arxiv.org/html/2607.22757#S4)is block\-diagonal with respect to⨁lVl\\bigoplus\_\{l\}V\_\{l\}and therefore transports nothing between levels; directed level\-to\-level transport requires block\-triangular morphismsϕl′←l:Vl→Vl′\\phi\_\{l^\{\\prime\}\\leftarrow l\}:V\_\{l\}\\to V\_\{l^\{\\prime\}\}internal to the graded hidden space\. The morphic structure of\[sh\-111\]supplies them, and subsumes external\-tool paradigms as a special case via functorial internalization; MLGE does not\.

## 2\.Background: Language Models and the Uniform Transformer

In this section we fix the notation and review the classical results of the graded constructions introduced in\[2024\-02,sh\-89,sh\-95,sh\-111\]\. The central question, made precise in[Conj\.1](https://arxiv.org/html/2607.22757#Thmcon1)and[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1), is what the transformer’s uniform treatment of positions and feature dimensions costs and where that cost is located\.

### 2\.1\.Language Models and the Autoregressive Objective

###### Definition 1\.

Let𝒱\\mathcal\{V\}be a finite vocabulary\. A*language model*is a parameterized probability distributionpθp\_\{\\theta\}on𝒱∗\\mathcal\{V\}^\{\*\}\.

Forw=\(w1,…,wT\)∈𝒱Tw=\(w\_\{1\},\\dots,w\_\{T\}\)\\in\\mathcal\{V\}^\{T\}the chain rule gives the autoregressive factorization

\(1\)pθ​\(w\)=∏t=1Tpθ​\(wt∣w<t\),w<t:=\(w1,…,wt−1\),p\_\{\\theta\}\(w\)=\\prod\_\{t=1\}^\{T\}p\_\{\\theta\}\(w\_\{t\}\\mid w\_\{<t\}\),\\qquad w\_\{<t\}:=\(w\_\{1\},\\dots,w\_\{t\-1\}\),reducing the task to estimation of the conditionals\[shannon1951prediction,bengio2003neural\]\. Quality is reported as perplexity, the exponentiated per\-token cross\-entropy

\(2\)PPL​\(w\)=exp⁡\(−1T​∑t=1Tlog⁡pθ​\(wt∣w<t\)\)\.\\mathrm\{PPL\}\(w\)=\\exp\\\!\\left\(\-\\tfrac\{1\}\{T\}\\textstyle\\sum\_\{t=1\}^\{T\}\\log p\_\{\\theta\}\(w\_\{t\}\\mid w\_\{<t\}\)\\right\)\.Every architecture in this paper, graded or not, is a parameterization of[Eq\.1](https://arxiv.org/html/2607.22757#S2.E1); grading acts on the representation ofw<tw\_\{<t\}and on the weight each conditional carries in the loss, and leaves the factorization itself intact\.

The direct empirical objective for[Eq\.1](https://arxiv.org/html/2607.22757#S2.E1)is causal language modeling,

\(3\)ℒCLM​\(θ\)=−1n​∑t=1nlog⁡pθ​\(xt∣x<t\)\.\\mathcal\{L\}\_\{\\mathrm\{CLM\}\}\(\\theta\)=\-\\frac\{1\}\{n\}\\sum\_\{t=1\}^\{n\}\\log p\_\{\\theta\}\(x\_\{t\}\\mid x\_\{<t\}\)\.
###### Lemma 1\.

Writept:=p\(⋅∣x<t\)p\_\{t\}:=p\(\\cdot\\mid x\_\{<t\}\)for the data conditional andp^t:=pθ\(⋅∣x<t\)\\hat\{p\}\_\{t\}:=p\_\{\\theta\}\(\\cdot\\mid x\_\{<t\}\)for the model’s\. The population form of[Eq\.3](https://arxiv.org/html/2607.22757#S2.E3)decomposes as

\(4\)𝔼​\[ℒCLM​\(θ\)\]=1n​∑t=1n\(H​\(pt\)\+DKL​\(pt∥p^t\)\),\\mathbb\{E\}\\\!\\left\[\\mathcal\{L\}\_\{\\mathrm\{CLM\}\}\(\\theta\)\\right\]=\\frac\{1\}\{n\}\\sum\_\{t=1\}^\{n\}\\Bigl\(H\(p\_\{t\}\)\+D\_\{\\mathrm\{KL\}\}\(p\_\{t\}\\,\\\|\\,\\hat\{p\}\_\{t\}\)\\Bigr\),so the objective is bounded below by the conditional entropy of the data, attained exactly whenp^t=pt\\hat\{p\}\_\{t\}=p\_\{t\}for alltt\.

###### Proof\.

Take expectations in[Eq\.3](https://arxiv.org/html/2607.22757#S2.E3)overxt∼ptx\_\{t\}\\sim p\_\{t\}and add and subtractH​\(pt\)H\(p\_\{t\}\)term by term\. ∎

[Eq\.4](https://arxiv.org/html/2607.22757#S2.E4)is the object the graded loss of[Sec\.5](https://arxiv.org/html/2607.22757#S5)modifies: per\-token weightsλqt\\lambda^\{q\_\{t\}\}re\-weight its summands, leaving the form of the decomposition intact while changing which positions dominate the gradient\. Which re\-weightings preserve the population minimizer is exactly the consistency question answered by[Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\.

[Eq\.3](https://arxiv.org/html/2607.22757#S2.E3)is non\-convex inθ\\theta, and no convergence guarantee to a global minimizer is known for transformer parameterizations; standard non\-convex SGD analysis controls stationarity,

mint≤T⁡𝔼​‖∇ℒ‖2=O​\(1/T\),\\min\_\{t\\leq T\}\\mathbb\{E\}\\\|\\nabla\\mathcal\{L\}\\\|^\{2\}=O\(1/\\sqrt\{T\}\),not the value of[Eq\.4](https://arxiv.org/html/2607.22757#S2.E4)\. This gap is inherited, not repaired, by grading, and[Thm\.5\.1](https://arxiv.org/html/2607.22757#S5.Thmthm1)quantifies the direction in which grading moves it\.

### 2\.2\.The Transformer

Letnnbe the sequence length,ddthe model dimension,HHthe number of heads,dk=dv=d/Hd\_\{k\}=d\_\{v\}=d/Hthe per\-head dimension, andX∈ℝn×dX\\in\\mathbb\{R\}^\{n\\times d\}a layer input; see\[vaswani2017attention\]for the standard transformer and\[sh\-95\]for the graded transformer\.

Tokens enter through an embedding matrixWe∈ℝ\|𝒱\|×dW\_\{e\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\\times d\}; subword vocabularies are built by byte\-pair encoding or unigram segmentation\[sennrich2016neural,kudo2018sentencepiece\], and tying the output projection toWe⊤W\_\{e\}^\{\\top\}couples the generation and representation geometries; the graded softmax of[Sec\.5](https://arxiv.org/html/2607.22757#S5)acts on precisely this tied projection\. Segmentation is chosen by corpus frequency and carries no record of morphological or syntactic role\. Tokenisation is thus the first place hierarchy is discarded, and the level\-0subspaceV0V\_\{0\}of[Sec\.4](https://arxiv.org/html/2607.22757#S4)is the first point at which it is reintroduced\. Since attention is permutation\-equivariant in the rows ofXX, position is supplied explicitly, classically by the sinusoidal encoding

\(5\)PE​\(t,j\)=\{sin⁡\(t/100002​⌊j/2⌋/d\),j​even,cos⁡\(t/100002​⌊j/2⌋/d\),j​odd,\\mathrm\{PE\}\(t,j\)=\\begin\{cases\}\\sin\\\!\\left\(t\\,/\\,10000^\{\\,2\\lfloor j/2\\rfloor/d\}\\right\),&j\\text\{ even\},\\\\\[4\.30554pt\] \\cos\\\!\\left\(t\\,/\\,10000^\{\\,2\\lfloor j/2\\rfloor/d\}\\right\),&j\\text\{ odd\},\\end\{cases\}added to the embeddings;[Eq\.16](https://arxiv.org/html/2607.22757#S4.E16)is its graded refinement\.

With projectionsWQh,WKh∈ℝd×dkW\_\{Q\_\{h\}\},W\_\{K\_\{h\}\}\\in\\mathbb\{R\}^\{d\\times d\_\{k\}\},WVh∈ℝd×dvW\_\{V\_\{h\}\}\\in\\mathbb\{R\}^\{d\\times d\_\{v\}\}, andWO∈ℝH​dv×dW\_\{O\}\\in\\mathbb\{R\}^\{Hd\_\{v\}\\times d\}, multi\-head self\-attention is

\(6\)MHSA​\(X\)=Concat​\(Head1,…,HeadH\)​WO,Headh=Attention​\(X​WQh,X​WKh,X​WVh\),\\begin\{split\}\\mathrm\{MHSA\}\(X\)&=\\mathrm\{Concat\}\(\\mathrm\{Head\}\_\{1\},\\dots,\\mathrm\{Head\}\_\{H\}\)\\,W\_\{O\},\\\\ \\mathrm\{Head\}\_\{h\}&=\\mathrm\{Attention\}\(XW\_\{Q\_\{h\}\},XW\_\{K\_\{h\}\},XW\_\{V\_\{h\}\}\),\\end\{split\}where, with the causal maskMcausal∈\{0,−∞\}n×nM\_\{\\mathrm\{causal\}\}\\in\\\{0,\-\\infty\\\}^\{n\\times n\},\(Mcausal\)i​j=−∞\(M\_\{\\mathrm\{causal\}\}\)\_\{ij\}=\-\\inftyforj\>ij\>i, enforcing[Eq\.1](https://arxiv.org/html/2607.22757#S2.E1),

\(7\)Attention​\(Q,K,V\)=softmax​\(Q​K⊤dk\+Mcausal\)​V\.\\mathrm\{Attention\}\(Q,K,V\)=\\mathrm\{softmax\}\\\!\\left\(\\frac\{QK^\{\\top\}\}\{\\sqrt\{d\_\{k\}\}\}\+M\_\{\\mathrm\{causal\}\}\\right\)V\.
The scalingdk−1/2d\_\{k\}^\{\-1/2\}is dictated by the variance of the logits: forq,kq,kwith independent unit\-variance entries,Var​\(q⊤​k\)=dk\\mathrm\{Var\}\(q^\{\\top\}k\)=d\_\{k\}, so unscaled logits drive the softmax toward a one\-hot distribution with vanishing gradient asdkd\_\{k\}grows\.[Eq\.7](https://arxiv.org/html/2607.22757#S2.E7)computes the bilinear formq⊤​kq^\{\\top\}k, the Euclidean inner product onℝdk\\mathbb\{R\}^\{d\_\{k\}\}, in which no coordinate is distinguished\. Replacing this form by a graded one is the single modification of[Sec\.3\.1](https://arxiv.org/html/2607.22757#S3.SS1)\.

Each position independently passes through a feed\-forward network,

\(8\)FFN​\(x\)=max⁡\(0,x​W1\+b1\)​W2\+b2,W1∈ℝd×df​f,W2∈ℝdf​f×d,\\mathrm\{FFN\}\(x\)=\\max\(0,\\,xW\_\{1\}\+b\_\{1\}\)W\_\{2\}\+b\_\{2\},\\qquad W\_\{1\}\\in\\mathbb\{R\}^\{d\\times d\_\{ff\}\},\\ W\_\{2\}\\in\\mathbb\{R\}^\{d\_\{ff\}\\times d\},conventionally withdf​f=4​dd\_\{ff\}=4d; gated variants such as SwiGLU adjustdf​fd\_\{ff\}to hold the parameter count fixed\[shazeer2020glu\]\. Sublayers are wrapped in residual connections and normalisation, in the pre\-norm orderingx′=x\+Sublayer​\(LN​\(x\)\)x^\{\\prime\}=x\+\\mathrm\{Sublayer\}\(\\mathrm\{LN\}\(x\)\), which bounds gradients at initialisation independently of depth\[xiong2020pre\]\. Normalisation is by LayerNorm\[ba2016layer\]or, standard in current open models, RMSNorm\[zhang2022rms\],

\(9\)RMSNorm​\(x\)=γ⊙x1d​∑jxj2\+ϵ\.\\mathrm\{RMSNorm\}\(x\)=\\gamma\\odot\\frac\{x\}\{\\sqrt\{\\tfrac\{1\}\{d\}\\sum\_\{j\}x\_\{j\}^\{2\}\+\\epsilon\}\}\.
###### Proposition 1\.

In a pre\-norm block withdf​f=4​dd\_\{ff\}=4danddk=dv=d/Hd\_\{k\}=d\_\{v\}=d/H, the FFN accounts for2/32/3of the block’s parameters, up toO​\(d\)O\(d\)terms\.

###### Proof\.

Attention contributesWQ,WK,WV,WOW\_\{Q\},W\_\{K\},W\_\{V\},W\_\{O\}, eachd×dd\\times dafter concatenating heads, hence4​d24d^\{2\}; the FFN contributesW1∈ℝd×4​dW\_\{1\}\\in\\mathbb\{R\}^\{d\\times 4d\}andW2∈ℝ4​d×dW\_\{2\}\\in\\mathbb\{R\}^\{4d\\times d\}, hence8​d28d^\{2\}\. Biases and normalisation parameters areO​\(d\)O\(d\), and8​d2/12​d2=2/38d^\{2\}/12d^\{2\}=2/3\. ∎

[Prop\.1](https://arxiv.org/html/2607.22757#Thmprop1)calibrates every overhead claim in this paper: a graded parameter budget ofO​\(d\+H​dk\)O\(d\+Hd\_\{k\}\)per block is negligible against the12​d212d^\{2\}established here, and the full accounting for the proposed configurations appears in[Sec\.8\.1](https://arxiv.org/html/2607.22757#S8.SS1)\.

###### Proposition 2\.

One layer of[Eq\.6](https://arxiv.org/html/2607.22757#S2.E6)together with[Eq\.8](https://arxiv.org/html/2607.22757#S2.E8)costsO​\(n2​d\+n​d2\)O\(n^\{2\}d\+nd^\{2\}\)time\.

###### Proof\.

Per head, logits and value weighting costO​\(n2​dk\)O\(n^\{2\}d\_\{k\}\); overHHheads withdk=d/Hd\_\{k\}=d/Hthis isO​\(n2​d\)O\(n^\{2\}d\)\. Projections, concatenation, and[Eq\.8](https://arxiv.org/html/2607.22757#S2.E8)contributeO​\(n​d2\)O\(nd^\{2\}\)\. ∎

[Prop\.4](https://arxiv.org/html/2607.22757#Thmprop4)shows grading preserves this bound exactly\.

### 2\.3\.Scaling Laws

Cross\-entropy loss under[Eq\.3](https://arxiv.org/html/2607.22757#S2.E3)follows fitted power laws in non\-embedding parametersNN, dataDD, and computeC≈6​N​DC\\approx 6ND\[kaplan2020scaling\]; the compute budgetCCof this subsection is standard notation and is distinct from the grade clipCCof[Sec\.8\.1\.1](https://arxiv.org/html/2607.22757#S8.SS1.SSS1)\. The parameterization of Hoffmann et al\.\[hoff\],

\(10\)L​\(N,D\)=E\+ANα\+BDβ,E≈1\.69,α≈0\.34,β≈0\.28,L\(N,D\)=E\+\\frac\{A\}\{N^\{\\alpha\}\}\+\\frac\{B\}\{D^\{\\beta\}\},\\qquad E\\approx 1\.69,\\ \\alpha\\approx 0\.34,\\ \\beta\\approx 0\.28,yields the compute\-optimal allocation below\.

###### Proposition 3\.

Minimising[Eq\.10](https://arxiv.org/html/2607.22757#S2.E10)subject to6​N​D=C6ND=CgivesN∗∝Cβ/\(α\+β\)N^\{\*\}\\propto C^\{\\beta/\(\\alpha\+\\beta\)\}andD∗∝Cα/\(α\+β\)D^\{\*\}\\propto C^\{\\alpha/\(\\alpha\+\\beta\)\}; forα≈β\\alpha\\approx\\betaboth exponents approach1/21/2, realised empirically at roughly2020tokens per parameter\.

###### Proof\.

SubstitutingD=C/\(6​N\)D=C/\(6N\)into[Eq\.10](https://arxiv.org/html/2607.22757#S2.E10)and differentiating inNNgives−α​A​N−α−1\+β​B​\(6​N/C\)β​N−1=0\-\\alpha AN^\{\-\\alpha\-1\}\+\\beta B\(6N/C\)^\{\\beta\}N^\{\-1\}=0, whenceNα\+β∝CβN^\{\\alpha\+\\beta\}\\propto C^\{\\beta\}\. ∎

Two qualifications govern the use made of[Eq\.10](https://arxiv.org/html/2607.22757#S2.E10)here\. First, it is a fitted regularity with no derivation from[Eq\.3](https://arxiv.org/html/2607.22757#S2.E3); it constrains the loss, not the capabilities the loss induces, and whether capability changes smoothly or abruptly with scale is disputed\[wei2022emergent,schaeffer2023mirage\]\. Second,[Eq\.10](https://arxiv.org/html/2607.22757#S2.E10)aggregates all positions into one scalar: a token carrying a clause boundary and a token carrying a function word contribute identically\. The graded loss of[Sec\.5](https://arxiv.org/html/2607.22757#S5)reweights the summands of[Eq\.4](https://arxiv.org/html/2607.22757#S2.E4)by token grade, which plausibly changes the constants of[Eq\.10](https://arxiv.org/html/2607.22757#S2.E10)rather than its form; whether it changes the exponents is an empirical question this paper does not settle\.

### 2\.4\.The Uniformity Principle and Its Cost

Every component above treats its inputs uniformly\. Attention[Eq\.7](https://arxiv.org/html/2607.22757#S2.E7)scores all pairs through one Euclidean form; the FFN[Eq\.8](https://arxiv.org/html/2607.22757#S2.E8)applies one map to all positions; the loss[Eq\.3](https://arxiv.org/html/2607.22757#S2.E3)weights all positions equally; and the scaling law[Eq\.10](https://arxiv.org/html/2607.22757#S2.E10)aggregates them into one scalar\. We call this the*uniformity principle*: all tokens, parameters, and feature dimensions carry equal structural status\. Its geometric expression is that the representation spaceℝd\\mathbb\{R\}^\{d\}is isotropic: no coordinate direction is distinguished, so no direction can carry structural meaning that the architecture must respect\. Formally,[Eq\.7](https://arxiv.org/html/2607.22757#S2.E7)is equivariant under the orthogonal group acting onℝdk\\mathbb\{R\}^\{d\_\{k\}\}jointly through the projections, and in particular under the unweighted scalar actionλ⋅x\\lambda\\cdot x; it is equivariant under the weighted actionλ⋆x=\(λqj​xj\)\\lambda\\star x=\(\\lambda^\{q\_\{j\}\}x\_\{j\}\)only in the degenerate caseq=\(1,…,1\)q=\(1,\\dots,1\)\. When data carries a weighted structure withq≠\(1,…,1\)q\\neq\(1,\\dots,1\), the architecture is committed to the wrong symmetry, and the structure must be recovered from data if at all\.

Existing architectural variants relax uniformity only as a function of positional offset\.[Tab\.1](https://arxiv.org/html/2607.22757#S2.T1)classifies them by the structural information their inductive bias can express\.

Table 1\.Architecture variants by structural inductive bias\[gu2023mamba,gu2024mamba2,peng2023rwkv,raffel2020exploring,radford2018improving\]\. “Recency” and “locality” denote biases that are functions of positional offset\|i−j\|\|i\-j\|alone; no existing variant conditions on structural role\. Grading acts on the metric of the representation space, sparsity and recurrence on the mask, and the two compose\.The cost of uniformity that motivates this paper is compositional, and we record it in two statements with different levels of support\. The first is the classical conjecture, which this paper does not prove\.

###### Conjecture 1\.

AnLL\-level context\-free grammar admitsΩ​\(nL\)\\Omega\(n^\{L\}\)parse trees over inputs of lengthnn\. The sample complexity of an architecture with no structural prior, trained by[Eq\.3](https://arxiv.org/html/2607.22757#S2.E3)to a fixed error on structures of depthLL, grows exponentially inLL\.

[Conj\.1](https://arxiv.org/html/2607.22757#Thmcon1)is consistent with the observed accuracy decay onkk\-nested arithmetic,kk\-hop inference, andkk\-deep call stacks, and it motivated the graded program; but the results of this paper neither prove it nor depend on it\. What the theory developed in[Sec\.4](https://arxiv.org/html/2607.22757#S4)requires is strictly weaker, and unlike[Conj\.1](https://arxiv.org/html/2607.22757#Thmcon1)it is directly measurable on annotated corpora\.

###### Assumption 1\(Profile Divergence\)\.

For hierarchical prediction targets in natural language, the target energy profileα\\alphaand the data energy profileτ\\tauof[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2)are far apart in Bhattacharyya affinity: structurally decisive features carry little of the corpus variance, and high\-variance lexical features carry little of the decision\.

[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)is the empirical hypothesis on which the benefit of grading rests\.[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)converts it into an exact sample\-complexity ratio,Λ⋆=BC​\(α,τ\)2\\Lambda^\{\\star\}=\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\}, and[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)shows that under geometric stratification of the profiles the ratio decays exponentially in the number of levels; this is the only route by which any exponential\-in\-LLstatement enters this work, and it enters as a hypothesis, not as a theorem\. Both profiles can be estimated offline, so[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)is testable before any model is trained;[Sec\.7](https://arxiv.org/html/2607.22757#S7)carries out the construction of the estimators and[Sec\.8](https://arxiv.org/html/2607.22757#S8)stages the measurement first\.

## 3\.Graded Attention in Decoder\-Only Architectures

[Sec\.2](https://arxiv.org/html/2607.22757#S2)closed with a deficit stated in two forms:[Conj\.1](https://arxiv.org/html/2607.22757#Thmcon1), which this paper does not prove, and[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1), which it makes precise and quantifies\. The transformer’s representation space is isotropic, so hierarchical competence must be recovered from data\. The remainder of the paper addresses the deficit by changing the metric on the representation space rather than the computation performed over it\.

This section develops exponentially graded multi\-head self\-attention within causal decoder\-only stacks and establishes three facts\. Grading leaves the asymptotic cost of attention unchanged \([Prop\.4](https://arxiv.org/html/2607.22757#Thmprop4)\)\. It acts by an invertible change of parameters \([Rem\.1](https://arxiv.org/html/2607.22757#Thmrem1)\), so a trained GLLM compiles to a standard transformer and carries its prior at zero deployment cost \([Cor\.1](https://arxiv.org/html/2607.22757#Thmcor1)\), a property none of the variants in[Tab\.1](https://arxiv.org/html/2607.22757#S2.T1)shares\. And at a fixed norm budget the graded constraint set is an*ellipsoid*whose axes are aligned with the grades \([Prop\.5](https://arxiv.org/html/2607.22757#Thmprop5)\), strictly containing the uniform ball; the advantage therefore cannot come from representability, since a larger class at fixed budget hurts generalisation\. Together these facts place the advantage in the*norm required to represent a given target*, and hence in the data required to learn it\.[Sec\.4](https://arxiv.org/html/2607.22757#S4)decomposes the embedding space into level\-specific subspaces via Multi\-Level Graded Embeddings and computes that quantity exactly \([Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\), characterises in closed form the grades at which the graded bound is the smaller one \([Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\), and gives the behaviour under stratified profiles \([Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)\)\.[Sec\.5](https://arxiv.org/html/2607.22757#S5)extends the grading to the training objective and characterises its consistency \([Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\)\.[Sec\.8](https://arxiv.org/html/2607.22757#S8)states the program that will test the resulting predictions, and[Sec\.6](https://arxiv.org/html/2607.22757#S6)assembles the results into a procedure that selects, certifies, and prices the grades in domains where they are not given\.

Three distinct grade objects appear in what follows and should not be conflated: per\-head feature gradesqh∈ℚ≥0dkq\_\{h\}\\in\\mathbb\{Q\}\_\{\\geq 0\}^\{d\_\{k\}\}\(this section\), per\-level embedding gradesql∈ℚ≥0dlq\_\{l\}\\in\\mathbb\{Q\}\_\{\\geq 0\}^\{d\_\{l\}\}\([Sec\.4](https://arxiv.org/html/2607.22757#S4)\), and per\-token loss gradesqt∈ℚ≥0q\_\{t\}\\in\\mathbb\{Q\}\_\{\\geq 0\}\([Sec\.5](https://arxiv.org/html/2607.22757#S5)\)\. All take values inℚ≥0\\mathbb\{Q\}\_\{\\geq 0\}, and all act through the same scalar actionλ⋆x=\(λqi​xi\)\\lambda\\star x=\(\\lambda^\{q\_\{i\}\}x\_\{i\}\)with a common baseλ\>1\\lambda\>1\.

We work in the decoder\-only paradigm of the GPT series\[radford2019language\], with multi\-head self\-attention and the causal mask as fixed in[Eqs\.6](https://arxiv.org/html/2607.22757#S2.E6)and[7](https://arxiv.org/html/2607.22757#S2.E7): queriesQh=X​WQhQ\_\{h\}=XW\_\{Q\_\{h\}\}, keysKh=X​WKhK\_\{h\}=XW\_\{K\_\{h\}\}, and valuesVh=X​WVhV\_\{h\}=XW\_\{V\_\{h\}\}inℝn×dk\\mathbb\{R\}^\{n\\times d\_\{k\}\}for each headh∈\{1,…,H\}h\\in\\\{1,\\dots,H\\\}, withdk=dv=d/Hd\_\{k\}=d\_\{v\}=d/Hand the maskMcausalM\_\{\\mathrm\{causal\}\}enforcing[Eq\.1](https://arxiv.org/html/2607.22757#S2.E1)\. Every position and every feature dimension enters[Eq\.7](https://arxiv.org/html/2607.22757#S2.E7)identically: the cost isO​\(n2​d\+n​d2\)O\(n^\{2\}d\+nd^\{2\}\)per layer by[Prop\.2](https://arxiv.org/html/2607.22757#Thmprop2), and no coordinate ofℝdk\\mathbb\{R\}^\{d\_\{k\}\}is distinguished from any other\.

### 3\.1\.Exponentially Graded Multi\-Head Self\-Attention

In GLLMs we replace MHSA by*Exponentially Graded Multi\-Head Self\-Attention*\(EG\-MHSA\), built from the Exponentially Graded Transformer of\[sh\-95\]\. Each head carries its own grading of the per\-head spaceℝdk=⨁j=1dkℝ​ej\\mathbb\{R\}^\{d\_\{k\}\}=\\bigoplus\_\{j=1\}^\{d\_\{k\}\}\\mathbb\{R\}e\_\{j\}, assigning to the basis vectoreje\_\{j\}a gradeqh,j∈ℚ≥0q\_\{h,j\}\\in\\mathbb\{Q\}\_\{\\geq 0\}\. The grading transformation is the diagonal matrix

Mqh,λ=diag⁡\(λqh,1,…,λqh,dk\)∈ℝdk×dk,M\_\{q\_\{h\},\\lambda\}=\\operatorname\{diag\}\\\!\\left\(\\lambda^\{q\_\{h,1\}\},\\dots,\\lambda^\{q\_\{h,d\_\{k\}\}\}\\right\)\\in\\mathbb\{R\}^\{d\_\{k\}\\times d\_\{k\}\},with fixed baseλ\>1\\lambda\>1\. Takingλ=e1/dk\\lambda=e^\{1/d\_\{k\}\}givesλ≈1\.008\\lambda\\approx 1\.008atdk=128d\_\{k\}=128andλ≈1\.016\\lambda\\approx 1\.016atdk=64d\_\{k\}=64, so that the clipped grade rangeqmax=dk​ln⁡Cq\_\{\\max\}=d\_\{k\}\\ln Cof[Sec\.8\.1\.1](https://arxiv.org/html/2607.22757#S8.SS1.SSS1)spans a spread of orderdkd\_\{k\}\. The base is chosen small precisely because the grades are not, and the clipCC, not the base, is the constant that prices the construction \([Rem\.18](https://arxiv.org/html/2607.22757#Thmrem18)\)\.

The graded queries, keys, and values are obtained by right\-multiplication:

\(11\)Qh′=Qh​Mqh,λ,Kh′=Kh​Mqh,λ,Vh′=Vh​Mqh,λ\.Q\_\{h\}^\{\\prime\}=Q\_\{h\}M\_\{q\_\{h\},\\lambda\},\\qquad K\_\{h\}^\{\\prime\}=K\_\{h\}M\_\{q\_\{h\},\\lambda\},\\qquad V\_\{h\}^\{\\prime\}=V\_\{h\}M\_\{q\_\{h\},\\lambda\}\.
The graded head is

\(12\)Headh′=softmax​\(Qh′​\(Kh′\)⊤dk\+Mcausal\)​Vh′\.\\mathrm\{Head\}\_\{h\}^\{\\prime\}=\\mathrm\{softmax\}\\\!\\left\(\\frac\{Q\_\{h\}^\{\\prime\}\(K\_\{h\}^\{\\prime\}\)^\{\\top\}\}\{\\sqrt\{d\_\{k\}\}\}\+M\_\{\\mathrm\{causal\}\}\\right\)V\_\{h\}^\{\\prime\}\.
SinceMqh,λM\_\{q\_\{h\},\\lambda\}is diagonal and hence symmetric,\(Kh′\)⊤=Mqh,λ​Kh⊤\(K\_\{h\}^\{\\prime\}\)^\{\\top\}=M\_\{q\_\{h\},\\lambda\}K\_\{h\}^\{\\top\}, so the pre\-softmax logits are

Qh′​\(Kh′\)⊤dk=Qh​Mqh,λ2​Kh⊤dk,Mqh,λ2=diag⁡\(λ2​qh,1,…,λ2​qh,dk\),\\frac\{Q\_\{h\}^\{\\prime\}\(K\_\{h\}^\{\\prime\}\)^\{\\top\}\}\{\\sqrt\{d\_\{k\}\}\}=\\frac\{Q\_\{h\}M\_\{q\_\{h\},\\lambda\}^\{2\}K\_\{h\}^\{\\top\}\}\{\\sqrt\{d\_\{k\}\}\},\\qquad M\_\{q\_\{h\},\\lambda\}^\{2\}=\\operatorname\{diag\}\\\!\\left\(\\lambda^\{2q\_\{h,1\}\},\\dots,\\lambda^\{2q\_\{h,d\_\{k\}\}\}\\right\),so that agreement between query and key alongeje\_\{j\}is weighted byλ2​qh,j\\lambda^\{2q\_\{h,j\}\}: the grading acts on the bilinear form defining attention, replacing the Euclidean inner product of[Eq\.7](https://arxiv.org/html/2607.22757#S2.E7)onℝdk\\mathbb\{R\}^\{d\_\{k\}\}by the graded form⟨u,v⟩qh=u⊤​Mqh,λ2​v\\langle u,v\\rangle\_\{q\_\{h\}\}=u^\{\\top\}M\_\{q\_\{h\},\\lambda\}^\{2\}v\. The multi\-head output is

EG​\-​MHSA​\(X\)=Concat​\(Head1′,…,HeadH′\)​WO\.\\mathrm\{EG\\text\{\-\}MHSA\}\(X\)=\\mathrm\{Concat\}\(\\mathrm\{Head\}\_\{1\}^\{\\prime\},\\dots,\\mathrm\{Head\}\_\{H\}^\{\\prime\}\)\\,W\_\{O\}\.
The grading induces a filtration of the per\-head space by grade,

V≤q\(h\)=⨁j:qh,j≤qℝ​ej,V≤q\(h\)⊆V≤q′\(h\)​for​q≤q′,V\_\{\\leq q\}^\{\(h\)\}=\\bigoplus\_\{j\\,:\\,q\_\{h,j\}\\leq q\}\\mathbb\{R\}e\_\{j\},\\qquad V\_\{\\leq q\}^\{\(h\)\}\\subseteq V\_\{\\leq q^\{\\prime\}\}^\{\(h\)\}\\text\{ for \}q\\leq q^\{\\prime\},which is the algebraic object that linguistic stratification is to be mapped onto: low grades for lexical and morphological features, high grades for phrasal and discourse\-level ones\[2024\-02,sh\-89\]\.

###### Proposition 4\.

EG\-MHSA has the same asymptotic time and space complexity as standard MHSA, namelyO​\(n2​d\+n​d2\)O\(n^\{2\}d\+nd^\{2\}\)per layer including feed\-forward components\.

###### Proof\.

The grading step[Eq\.11](https://arxiv.org/html/2607.22757#S3.E11)applies3​H3Hdiagonal multiplications, each costingO​\(n​dk\)O\(nd\_\{k\}\), for a total ofO​\(n​H​dk\)=O​\(n​d\)O\(nHd\_\{k\}\)=O\(nd\)\. The matrixMqh,λ2M\_\{q\_\{h\},\\lambda\}^\{2\}is precomputable per head inO​\(dk\)O\(d\_\{k\}\)\. The attention computation[Eq\.12](https://arxiv.org/html/2607.22757#S3.E12)is structurally identical to standard MHSA:O​\(n2​dk\)O\(n^\{2\}d\_\{k\}\)per head for the logits andO​\(n2​dk\)O\(n^\{2\}d\_\{k\}\)per head for the softmax and value weighting, givingO​\(n2​d\)O\(n^\{2\}d\)overHHheads\. Concatenation and the output projection addO​\(n​d2\)O\(nd^\{2\}\), and the feed\-forward blockO​\(n​d2\)O\(nd^\{2\}\)\. Summing givesO​\(n2​d\+n​d2\)O\(n^\{2\}d\+nd^\{2\}\), matching[Prop\.2](https://arxiv.org/html/2607.22757#Thmprop2)\. ∎

###### Remark 1\.

BecauseMqh,λM\_\{q\_\{h\},\\lambda\}is invertible andWQh,WKh,WVhW\_\{Q\_\{h\}\},W\_\{K\_\{h\}\},W\_\{V\_\{h\}\}are unconstrained, the substitutionW~∙h=W∙h​Mqh,λ\\tilde\{W\}\_\{\\bullet h\}=W\_\{\\bullet h\}M\_\{q\_\{h\},\\lambda\}is a bijection of the parameter space carrying[Eq\.12](https://arxiv.org/html/2607.22757#S3.E12)to a standard head\. EG\-MHSA and MHSA therefore realise the same set of functions\. In this precise sense the grading changes the*metric*of the representation space and leaves the computation over it intact, and two consequences follow, running in opposite directions\. On one side, no separation is available from representability\. On the other, an invertible reparameterization can be applied once and then discarded, which gives[Cor\.1](https://arxiv.org/html/2607.22757#Thmcor1)\. What grading fixes is a parameterization, and with it an initialisation, an implicit bias under gradient descent, and, once the projection weights are norm\-constrained, an effective hypothesis class whose geometry is computed in[Prop\.5](https://arxiv.org/html/2607.22757#Thmprop5)\. This is a statement about which functions are*reachable and preferred*, not about which are*representable*, and the informative comparison fixes the target rather than the budget \([Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\)\.

###### Corollary 1\.

Letθ\\thetabe the parameters of a trained GLLM whose attention sublayers are EG\-MHSA with grades\{qh\}\\\{q\_\{h\}\\\}\. The mapθ↦θ~\\theta\\mapsto\\tilde\{\\theta\}given byW~∙h=W∙h​Mqh,λ\\tilde\{W\}\_\{\\bullet h\}=W\_\{\\bullet h\}M\_\{q\_\{h\},\\lambda\}for∙∈\{Q,K,V\}\\bullet\\in\\\{Q,K,V\\\}yields a standard decoder\-only transformer of identical architecture computing the identical function on every input\. The grades need not be stored at inference, and the diagonal multiplications of[Eq\.11](https://arxiv.org/html/2607.22757#S3.E11)need not be performed\.

###### Proof\.

Immediate from the bijection of[Rem\.1](https://arxiv.org/html/2607.22757#Thmrem1), applied once and offline\. SinceMqh,λM\_\{q\_\{h\},\\lambda\}is diagonal with strictly positive entries,W~∙h\\tilde\{W\}\_\{\\bullet h\}is well defined and the substitution violates no constraint onθ\\theta\. ∎

###### Remark 2\.

The graded prior is paid for once, during training, and costs nothing afterwards\. This separates grading from every variant classified in[Tab\.1](https://arxiv.org/html/2607.22757#S2.T1): selective recurrence, restricted mask support, and hybrid schemes encode their inductive bias in the computation, pay for it at every forward pass, and do not compile to the architecture they modify\. A metric can be absorbed into the weights, while a mask cannot\. The training overhead accounted in[Sec\.8\.1\.3](https://arxiv.org/html/2607.22757#S8.SS1.SSS3)is therefore an upper bound on the total lifetime cost of the prior, and[Sec\.8](https://arxiv.org/html/2607.22757#S8)reports zero inference overhead as an established consequence of the construction rather than an experimental outcome\.

###### Proposition 5\.

Fix grades\{qh\}h=1H\\\{q\_\{h\}\\\}\_\{h=1\}^\{H\}and a budgetB\>0B\>0, and let

ℱBgraded=\{X↦EG\-MHSA\(X\):∥W∙h∥F≤B∀∙∈\{Q,K,V\},∀h\},ℱBunif=\{X↦MHSA\(X\):∥W∙h∥F≤B∀∙∈\{Q,K,V\},∀h\}\.\\begin\{split\}\\mathcal\{F\}\_\{B\}^\{\\mathrm\{graded\}\}&=\\left\\\{X\\mapsto\\mathrm\{EG\\text\{\-\}MHSA\}\(X\)\\;:\\;\\\|W\_\{\\bullet h\}\\\|\_\{F\}\\leq B\\ \\ \\forall\\,\\bullet\\in\\\{Q,K,V\\\},\\ \\forall h\\right\\\},\\\\ \\mathcal\{F\}\_\{B\}^\{\\mathrm\{unif\}\}&=\\left\\\{X\\mapsto\\mathrm\{MHSA\}\(X\)\\;:\\;\\\|W\_\{\\bullet h\}\\\|\_\{F\}\\leq B\\ \\ \\forall\\,\\bullet\\in\\\{Q,K,V\\\},\\ \\forall h\\right\\\}\.\\end\{split\}In the effective coordinatesW~∙h=W∙h​Mqh,λ\\tilde\{W\}\_\{\\bullet h\}=W\_\{\\bullet h\}M\_\{q\_\{h\},\\lambda\}of[Rem\.1](https://arxiv.org/html/2607.22757#Thmrem1), the budget constraint definingℱBgraded\\mathcal\{F\}\_\{B\}^\{\\mathrm\{graded\}\}is the ellipsoid

∑j=1dkλ−2​qh,j​‖W~∙h​ej‖22≤B2,\\sum\_\{j=1\}^\{d\_\{k\}\}\\lambda^\{\-2q\_\{h,j\}\}\\big\\\|\\tilde\{W\}\_\{\\bullet h\}\\,e\_\{j\}\\big\\\|\_\{2\}^\{2\}\\;\\leq\\;B^\{2\},with semi\-axisB​λqh,jB\\lambda^\{q\_\{h,j\}\}along thejj\-th column, whereas that definingℱBunif\\mathcal\{F\}\_\{B\}^\{\\mathrm\{unif\}\}is the Frobenius ball of radiusBB\. Consequently

ℱBunif⊆ℱBgraded,\\mathcal\{F\}\_\{B\}^\{\\mathrm\{unif\}\}\\subseteq\\mathcal\{F\}\_\{B\}^\{\\mathrm\{graded\}\},with strict inclusion whenever someqh,j\>0q\_\{h,j\}\>0, and the Rademacher complexity of the graded class is weakly greater at every fixedBB\.

###### Proof\.

By the substitution of[Rem\.1](https://arxiv.org/html/2607.22757#Thmrem1), a graded head with weightsW∙hW\_\{\\bullet h\}computes the same function as a standard head with effective weightsW~∙h=W∙h​Mqh,λ\\tilde\{W\}\_\{\\bullet h\}=W\_\{\\bullet h\}M\_\{q\_\{h\},\\lambda\}\. Under this bijection the constraint‖W∙h‖F≤B\\\|W\_\{\\bullet h\}\\\|\_\{F\}\\leq Bbecomes‖W~∙h​Mqh,λ−1‖F≤B\\\|\\tilde\{W\}\_\{\\bullet h\}M\_\{q\_\{h\},\\lambda\}^\{\-1\}\\\|\_\{F\}\\leq B, which is the displayed ellipsoid; its semi\-axis alongeje\_\{j\}isB​λqh,jB\\lambda^\{q\_\{h,j\}\}\. Sinceqh,j≥0q\_\{h,j\}\\geq 0andλ\>1\\lambda\>1, every semi\-axis is at leastBB, so the ellipsoid contains the Frobenius ball of radiusBB, strictly in thejj\-th direction wheneverqh,j\>0q\_\{h,j\}\>0\. Passing from weight sets to function classes preserves the inclusion, and anyW~∙h\\tilde\{W\}\_\{\\bullet h\}with‖W~∙h‖F=B\\\|\\tilde\{W\}\_\{\\bullet h\}\\\|\_\{F\}=Bsupported on a coordinate withqh,j\>0q\_\{h,j\}\>0can be scaled byλqh,j\\lambda^\{q\_\{h,j\}\}to lie in the graded class but not the uniform one, giving strictness\. Monotonicity of Rademacher complexity under inclusion of hypothesis classes gives the final claim\. ∎

###### Corollary 2\.

No capacity bound favouring the graded model is available at fixedBB\.

[Prop\.5](https://arxiv.org/html/2607.22757#Thmprop5)identifies the geometric object grading produces: an*ellipsoidal*constraint set, its axes aligned with the grades and its eccentricity alongeje\_\{j\}equal toλqh,j\\lambda^\{q\_\{h,j\}\}\. Reading the proposition as[Cor\.2](https://arxiv.org/html/2607.22757#Thmcor2)alone understates it\. The informative comparison fixes the target rather than the budget, asks each class for the smallest budget at which it represents that target, and evaluates the capacity bounds at those budgets;[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)carries this out and identifies the sample\-complexity ratio in closed form, and[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)characterises exactly the grades at which that ratio favours the graded bound\. The ellipsoid is also what makes the comparison sharp: it is the geometry on which a matching minimax lower bound is posed, and on it the separation between the graded prior and its absence, rather than a ratio of upper bounds, is obtained\.[Thm\.4\.1](https://arxiv.org/html/2607.22757#S4.Thmthm1)establishes that separation in the level\-stratified regime, over all estimators, throughout the window of[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4);[Conj\.2](https://arxiv.org/html/2607.22757#Thmcon2)states the general case, and[Sec\.8\.4](https://arxiv.org/html/2607.22757#S8.SS4)records what its proof requires\.

## 4\.Hierarchical Grading for Language Modeling

Linguistic structure is layered\. Subword units aggregate into words, words compose into phrases and clauses, clauses into sentences, sentences into discourse, and discourse into pragmatic inference; the observation is common to the generative and model\-theoretic traditions in linguistics, and we take it as given rather than argue it here\. Classical LLMs approximate this stratification implicitly, through patterns emergent in unannotated corpora\[radford2019language\]\. Their embedding space is isotropic: no coordinate direction is distinguished, so no direction can be reserved for a linguistic level\. We propose*Multi\-Level Graded Embeddings*\(MLGE\), which give the stratification an explicit home in the geometry of the embedding space, extending the graded vector space formalism of\[2024\-02,sh\-89,sh\-95\]to autoregressive language modeling\.[Sec\.4\.2](https://arxiv.org/html/2607.22757#S4.SS2)then quantifies the construction: it establishes the exact sample\-complexity ratio obtained by grading \([Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\), characterises in closed form the grades at which the graded model is favoured and locates the standard transformer on the boundary of that set \([Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\), and shows the ratio decays exponentially in depth under stratified profiles \([Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)\), converting[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)into a computable, falsifiable quantity\.

### 4\.1\.The Graded Embedding Space

LetE=\(ε1,…,εn\)⊤∈ℝn×dE=\(\\varepsilon\_\{1\},\\dots,\\varepsilon\_\{n\}\)^\{\\top\}\\in\\mathbb\{R\}^\{n\\times d\}be the input embedding matrix for a token sequencey=\(y1,…,yn\)∈𝒱ny=\(y\_\{1\},\\dots,y\_\{n\}\)\\in\\mathcal\{V\}^\{n\}, withεt=We​\[yt\]∈ℝd\\varepsilon\_\{t\}=W\_\{e\}\[y\_\{t\}\]\\in\\mathbb\{R\}^\{d\}and learnable weightsWe∈ℝ\|𝒱\|×dW\_\{e\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\\times d\}as in[Sec\.2\.2](https://arxiv.org/html/2607.22757#S2.SS2)\. We decompose the embedding space as an orthogonal direct sum

ℝd=⨁l=0L−1Vl,Vl≅ℝdl,∑l=0L−1dl=d,\\mathbb\{R\}^\{d\}=\\bigoplus\_\{l=0\}^\{L\-1\}V\_\{l\},\\qquad V\_\{l\}\\cong\\mathbb\{R\}^\{d\_\{l\}\},\\qquad\\sum\_\{l=0\}^\{L\-1\}d\_\{l\}=d,withdl=d/Ld\_\{l\}=d/Lunder uniform allocation, whereVlV\_\{l\}is reserved for features salient to linguistic levelll:V0V\_\{0\}for subword and morphological structure,V1V\_\{1\}for syntactic constituency,V2V\_\{2\}for sentential semantics,VL−1V\_\{L\-1\}for discourse and pragmatics\. Level\-specific components areEl=E​Pl∈ℝn×dlE\_\{l\}=EP\_\{l\}\\in\\mathbb\{R\}^\{n\\times d\_\{l\}\}, wherePl∈ℝd×dlP\_\{l\}\\in\\mathbb\{R\}^\{d\\times d\_\{l\}\}has orthonormal columns spanningVlV\_\{l\}, initialised by principal component analysis of features annotated for levelllby an offline constituency parser\.

When thePlP\_\{l\}are exactly orthonormal with mutually orthogonal ranges,∑lPl​Pl⊤=Id\\sum\_\{l\}P\_\{l\}P\_\{l\}^\{\\top\}=I\_\{d\}and the reconstructionE=∑lEl​Pl⊤E=\\sum\_\{l\}E\_\{l\}P\_\{l\}^\{\\top\}is exact\. During training the columns are only approximately orthonormal, and we penalise

Ωorth​\(P\)=∑l,l′‖Pl⊤​Pl′−δl​l′​Idl‖F2,\\Omega\_\{\\mathrm\{orth\}\}\(P\)=\\sum\_\{l,l^\{\\prime\}\}\\big\\\|P\_\{l\}^\{\\top\}P\_\{l^\{\\prime\}\}\-\\delta\_\{ll^\{\\prime\}\}I\_\{d\_\{l\}\}\\big\\\|\_\{F\}^\{2\},so that the reconstruction error is controlled byΩorth​\(P\)\\Omega\_\{\\mathrm\{orth\}\}\(P\)and hence by the penalty weight\. The error is a function of optimisation, not ofLL, and no rate inLLis claimed\.

EachVlV\_\{l\}carries a grading tupleql=\(ql,1,…,ql,dl\)∈ℚ≥0dlq\_\{l\}=\(q\_\{l,1\},\\dots,q\_\{l,d\_\{l\}\}\)\\in\\mathbb\{Q\}\_\{\\geq 0\}^\{d\_\{l\}\}, whose entries are rational so that they may record discrete structural depths exactly\. The associated grading transformation is

Mql,λ=diag⁡\(λql,1,…,λql,dl\)∈ℝdl×dl,M\_\{q\_\{l\},\\lambda\}=\\operatorname\{diag\}\\\!\\left\(\\lambda^\{q\_\{l,1\}\},\\dots,\\lambda^\{q\_\{l,d\_\{l\}\}\}\\right\)\\in\\mathbb\{R\}^\{d\_\{l\}\\times d\_\{l\}\},with the baseλ\>1\\lambda\>1fixed once for the whole architecture, as in[Sec\.3](https://arxiv.org/html/2607.22757#S3)\. The graded embeddings are

\(13\)E′=∑l=0L−1El​Mql,λ​Pl⊤=E​G,G:=∑l=0L−1Pl​Mql,λ​Pl⊤,E^\{\\prime\}=\\sum\_\{l=0\}^\{L\-1\}E\_\{l\}\\,M\_\{q\_\{l\},\\lambda\}\\,P\_\{l\}^\{\\top\}=E\\,G,\\qquad G:=\\sum\_\{l=0\}^\{L\-1\}P\_\{l\}M\_\{q\_\{l\},\\lambda\}P\_\{l\}^\{\\top\},which in the graded basis is the block\-diagonal map⨁lMql,λ\\bigoplus\_\{l\}M\_\{q\_\{l\},\\lambda\}\. Evaluating[Eq\.13](https://arxiv.org/html/2607.22757#S4.E13)costsO​\(n​d​L\)O\(ndL\)\.

###### Remark 3\.

GGis symmetric positive definite, hence invertible, andWeW\_\{e\}is unconstrained\. The substitutionW~e=We​G\\tilde\{W\}\_\{e\}=W\_\{e\}Gis therefore a bijection of the parameter space carrying[Eq\.13](https://arxiv.org/html/2607.22757#S4.E13)to an ungraded embedding, and MLGE realises exactly the function class of a standard embedding layer\. As in[Rem\.1](https://arxiv.org/html/2607.22757#Thmrem1), two consequences follow in opposite directions\. On one side, the construction adds no capacity, so no separation is available from representability\. On the other, the substitution extends[Cor\.1](https://arxiv.org/html/2607.22757#Thmcor1)from the attention sublayers to the embedding layer, so the whole of the graded construction, attention, embeddings, and positional encoding alike, is folded into the weights at the end of training and costs nothing at inference\. What the construction contributes is its parameterization:GGfixes an initialisation aligned with annotated linguistic levels, and constrains the geometry along which gradient descent moves\.

###### Proposition 6\.

Letη:=maxl≠l′⁡‖Pl⊤​Pl′‖o​p\\eta:=\\max\_\{l\\neq l^\{\\prime\}\}\\\|P\_\{l\}^\{\\top\}P\_\{l^\{\\prime\}\}\\\|\_\{op\}andλqmax:=maxl,j⁡λql,j\\lambda^\{q\_\{\\max\}\}:=\\max\_\{l,j\}\\lambda^\{q\_\{l,j\}\}\. ThenGGof[Eq\.13](https://arxiv.org/html/2607.22757#S4.E13)satisfiesG​\(Vl\)⊆VlG\(V\_\{l\}\)\\subseteq V\_\{l\}exactly whenη=0\\eta=0, and in general

‖G−⨁l=0L−1Mql,λ‖o​p≤L​λqmax​η\.\\Big\\\|G\-\\bigoplus\_\{l=0\}^\{L\-1\}M\_\{q\_\{l\},\\lambda\}\\Big\\\|\_\{op\}\\;\\leq\\;L\\,\\lambda^\{q\_\{\\max\}\}\\,\\eta\.

###### Proof\.

Whenη=0\\eta=0the ranges of thePlP\_\{l\}are mutually orthogonal, soPl′⊤​G​Pl′=Mql′,λP\_\{l^\{\\prime\}\}^\{\\top\}GP\_\{l^\{\\prime\}\}=M\_\{q\_\{l^\{\\prime\}\},\\lambda\}andPl′⊤​G​Pl=0P\_\{l^\{\\prime\}\}^\{\\top\}GP\_\{l\}=0forl≠l′l\\neq l^\{\\prime\}; thusGGis block\-diagonal in the graded basis and preserves eachVlV\_\{l\}\. In general, the off\-diagonal block indexed by\(l′,l\)\(l^\{\\prime\},l\)isPl′⊤​Pl​Mql,λ​Pl⊤​Pl′P\_\{l^\{\\prime\}\}^\{\\top\}P\_\{l\}M\_\{q\_\{l\},\\lambda\}P\_\{l\}^\{\\top\}P\_\{l^\{\\prime\}\}, of operator norm at mostλql,max​η2≤λqmax​η\\lambda^\{q\_\{l,\\max\}\}\\eta^\{2\}\\leq\\lambda^\{q\_\{\\max\}\}\\etaforη≤1\\eta\\leq 1, and there are at mostLLsuch blocks in any row\. Summing gives the bound\. ∎

###### Remark 4\.

[Prop\.6](https://arxiv.org/html/2607.22757#Thmprop6)is a consistency check, not a capability claim: it states that the intended block structure survives approximate orthogonality of thePlP\_\{l\}, with degradation linear inLLand in the grading amplitudeλqmax\\lambda^\{q\_\{\\max\}\}\. It is the first place where large grades are seen to cost something\.

### 4\.2\.Sample Complexity Under Grading

Two routes to a separation are closed before we begin, and closing them fixes the location of the third\. By[Rem\.3](https://arxiv.org/html/2607.22757#Thmrem3)graded and ungraded embeddings realise the same functions, so nothing follows from representability\. By[Prop\.5](https://arxiv.org/html/2607.22757#Thmprop5)the graded constraint set at a common norm budget is an ellipsoid strictly containing the uniform ball, with weakly greater Rademacher complexity, so nothing favouring the graded model follows at fixedBBeither\.

The informative comparison fixes the target rather than the budget\. Each class is asked for the smallest norm at which it represents a givenf∗f^\{\*\}, and the resulting bound is evaluated at that norm\. The grading then acts on two quantities at once: it lowers the norm required for targets aligned with it, and it raises the effective radius of the data\. The theorem below is the exact trade\-off between them\.[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)then reads that trade\-off as a condition on the grades themselves, and identifies the standard transformer as the point at which the trade\-off is declined\.

Throughout this subsection the analysis is carried out for the graded linear mapx↦⟨u,G​x⟩x\\mapsto\\langle u,Gx\\ranglerealised by MLGE composed with a single linear read\-out\. It is a statement about the embedding layer and its immediate consumer, not about the full stack; the extension is discussed in[Rem\.11](https://arxiv.org/html/2607.22757#Thmrem11)\.

###### Definition 2\(Energy Profiles\)\.

Let𝒫\\mathcal\{P\}be a distribution onℝd×ℝ\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}withσj2:=𝔼​\[xj2\]∈\(0,∞\)\\sigma\_\{j\}^\{2\}:=\\mathbb\{E\}\[x\_\{j\}^\{2\}\]\\in\(0,\\infty\)for everyjj, and letf∗​\(x\)=⟨w∗,x⟩f^\{\*\}\(x\)=\\langle w^\{\*\},x\\ranglewithwj∗≠0w\_\{j\}^\{\*\}\\neq 0for everyjj\. WritingΔd−1\\Delta^\{d\-1\}for the probability simplex inℝd\\mathbb\{R\}^\{d\}, the*target profile*α∈Δd−1\\alpha\\in\\Delta^\{d\-1\}and the*data profile*τ∈Δd−1\\tau\\in\\Delta^\{d\-1\}are

αj=\(wj∗\)2‖w∗‖22,τj=σj2∑k=1dσk2\.\\alpha\_\{j\}=\\frac\{\(w\_\{j\}^\{\*\}\)^\{2\}\}\{\\\|w^\{\*\}\\\|\_\{2\}^\{2\}\},\\qquad\\tau\_\{j\}=\\frac\{\\sigma\_\{j\}^\{2\}\}\{\\sum\_\{k=1\}^\{d\}\\sigma\_\{k\}^\{2\}\}\.Both are supported on the graded basis of[Eq\.13](https://arxiv.org/html/2607.22757#S4.E13), both are probability vectors, and both are strictly positive in every coordinate\. A coordinate on whichw∗w^\{\*\}vanishes contributes nothing toα\\alphaand is optimally suppressed bygj→0g\_\{j\}\\to 0; such coordinates are deleted from the analysis rather than carried through it\.

###### Definition 3\(Graded and Uniform Classes at a Target\)\.

ForB\>0B\>0set

ℋBgraded=\{x↦⟨u,G​x⟩:u∈ℝd,‖u‖2≤B\},ℋBunif=\{x↦⟨w,x⟩:w∈ℝd,‖w‖2≤B\}\.\\begin\{split\}\\mathcal\{H\}\_\{B\}^\{\\mathrm\{graded\}\}&=\\left\\\{x\\mapsto\\langle u,Gx\\rangle\\;:\\;u\\in\\mathbb\{R\}^\{d\},\\ \\\|u\\\|\_\{2\}\\leq B\\right\\\},\\\\ \\mathcal\{H\}\_\{B\}^\{\\mathrm\{unif\}\}&=\\left\\\{x\\mapsto\\langle w,x\\rangle\\;:\\;w\\in\\mathbb\{R\}^\{d\},\\ \\\|w\\\|\_\{2\}\\leq B\\right\\\}\.\\end\{split\}The*representation budgets*off∗f^\{\*\}are

Bgraded​\(f∗\)=min⁡\{‖u‖2:G​u=w∗\}=‖G−1​w∗‖2,Bunif​\(f∗\)=‖w∗‖2,B^\{\\mathrm\{graded\}\}\(f^\{\*\}\)=\\min\\left\\\{\\\|u\\\|\_\{2\}:Gu=w^\{\*\}\\right\\\}=\\big\\\|G^\{\-1\}w^\{\*\}\\big\\\|\_\{2\},\\qquad B^\{\\mathrm\{unif\}\}\(f^\{\*\}\)=\\\|w^\{\*\}\\\|\_\{2\},the smallest budgets at which each family containsf∗f^\{\*\}\.

###### Proposition 7\.

Letα\\alphaandτ\\taube as in[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2)and letℓ\\ellbe a loss that isLℓL\_\{\\ell\}\-Lipschitz in its first argument\. Define the*grading gain*

Λ​\(g\)=\(∑j=1dαj​gj−2\)​\(∑j=1dτj​gj2\),g∈ℝ\>0d\.\\Lambda\(g\)=\\left\(\\sum\_\{j=1\}^\{d\}\\alpha\_\{j\}\\,g\_\{j\}^\{\-2\}\\right\)\\left\(\\sum\_\{j=1\}^\{d\}\\tau\_\{j\}\\,g\_\{j\}^\{2\}\\right\),\\qquad g\\in\\mathbb\{R\}\_\{\>0\}^\{d\}\.Then:

1. \(i\)Λ\\Lambdais invariant underg↦c​gg\\mapsto cgfor everyc\>0c\>0, andΛ​\(𝟏\)=1\\Lambda\(\\mathbf\{1\}\)=1;
2. \(ii\)the Rademacher\-based sample complexity of reaching excess riskε\\varepsilonoverℋBgraded​\(f∗\)graded\\mathcal\{H\}^\{\\mathrm\{graded\}\}\_\{B^\{\\mathrm\{graded\}\}\(f^\{\*\}\)\}stands to that overℋBunif​\(f∗\)unif\\mathcal\{H\}^\{\\mathrm\{unif\}\}\_\{B^\{\\mathrm\{unif\}\}\(f^\{\*\}\)\}in the ratioΛ​\(g\)\\Lambda\(g\), uniformly inε\\varepsilonandf∗f^\{\*\};
3. \(iii\)the gain is minimised at gj⋆∝αj1/4​τj−1/4,g\_\{j\}^\{\\star\}\\;\\propto\\;\\alpha\_\{j\}^\{1/4\}\\,\\tau\_\{j\}^\{\-1/4\},and the optimal value is the squared Bhattacharyya affinity of the two profiles, \(14\)Λ⋆:=ming∈ℝ\>0d⁡Λ​\(g\)=\(∑j=1dαj​τj\)2=BC​\(α,τ\)2;\\Lambda^\{\\star\}:=\\min\_\{g\\in\\mathbb\{R\}\_\{\>0\}^\{d\}\}\\Lambda\(g\)=\\left\(\\sum\_\{j=1\}^\{d\}\\sqrt\{\\alpha\_\{j\}\\tau\_\{j\}\}\\right\)^\{2\}=\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\};
4. \(iv\)Λ⋆≤1\\Lambda^\{\\star\}\\leq 1, with equality if and only ifα=τ\\alpha=\\tau\.

###### Proof\.

\(i\) Replacingggbyc​gcgmultiplies the first factor byc−2c^\{\-2\}and the second byc2c^\{2\}\. Atg=𝟏g=\\mathbf\{1\}the factors are∑jαj=1\\sum\_\{j\}\\alpha\_\{j\}=1and∑jτj=1\\sum\_\{j\}\\tau\_\{j\}=1\.

\(ii\) FixB\>0B\>0and a samplex1,…,xmx\_\{1\},\\dots,x\_\{m\}\. Writingzi=G​xiz\_\{i\}=Gx\_\{i\}, the classℋBgraded\\mathcal\{H\}\_\{B\}^\{\\mathrm\{graded\}\}is theBB\-norm\-bounded linear class in the variablezz, so its empirical Rademacher complexity is

ℜ^m​\(ℋBgraded\)=1m​𝔼ϵ​sup‖u‖2≤B⟨u,∑i=1mϵi​zi⟩=Bm​𝔼ϵ​‖∑i=1mϵi​G​xi‖2≤Bm​\(𝔼ϵ​‖∑i=1mϵi​G​xi‖22\)1/2=Bm​\(∑i=1m‖G​xi‖22\)1/2,\\begin\{split\}\\widehat\{\\mathfrak\{R\}\}\_\{m\}\\big\(\\mathcal\{H\}\_\{B\}^\{\\mathrm\{graded\}\}\\big\)&=\\frac\{1\}\{m\}\\,\\mathbb\{E\}\_\{\\epsilon\}\\sup\_\{\\\|u\\\|\_\{2\}\\leq B\}\\Big\\langle u,\\sum\_\{i=1\}^\{m\}\\epsilon\_\{i\}z\_\{i\}\\Big\\rangle=\\frac\{B\}\{m\}\\,\\mathbb\{E\}\_\{\\epsilon\}\\Big\\\|\\sum\_\{i=1\}^\{m\}\\epsilon\_\{i\}Gx\_\{i\}\\Big\\\|\_\{2\}\\\\ &\\leq\\frac\{B\}\{m\}\\left\(\\mathbb\{E\}\_\{\\epsilon\}\\Big\\\|\\sum\_\{i=1\}^\{m\}\\epsilon\_\{i\}Gx\_\{i\}\\Big\\\|\_\{2\}^\{2\}\\right\)^\{1/2\}=\\frac\{B\}\{m\}\\left\(\\sum\_\{i=1\}^\{m\}\\\|Gx\_\{i\}\\\|\_\{2\}^\{2\}\\right\)^\{1/2\},\\end\{split\}the last equality because𝔼​\[ϵi​ϵi′\]=δi​i′\\mathbb\{E\}\[\\epsilon\_\{i\}\\epsilon\_\{i^\{\\prime\}\}\]=\\delta\_\{ii^\{\\prime\}\}\. Taking expectations over the sample and using𝔼​‖G​x‖22=∑jgj2​σj2\\mathbb\{E\}\\\|Gx\\\|\_\{2\}^\{2\}=\\sum\_\{j\}g\_\{j\}^\{2\}\\sigma\_\{j\}^\{2\},

ℜm​\(ℋBgraded\)≤Bm​\(∑j=1dgj2​σj2\)1/2\.\\mathfrak\{R\}\_\{m\}\\big\(\\mathcal\{H\}\_\{B\}^\{\\mathrm\{graded\}\}\\big\)\\leq\\frac\{B\}\{\\sqrt\{m\}\}\\left\(\\sum\_\{j=1\}^\{d\}g\_\{j\}^\{2\}\\sigma\_\{j\}^\{2\}\\right\)^\{1/2\}\.Settingg=𝟏g=\\mathbf\{1\}gives the uniform case\. Evaluating each bound at its own representation budget, and using‖G−1​w∗‖22=‖w∗‖22​∑jαj​gj−2\\\|G^\{\-1\}w^\{\*\}\\\|\_\{2\}^\{2\}=\\\|w^\{\*\}\\\|\_\{2\}^\{2\}\\sum\_\{j\}\\alpha\_\{j\}g\_\{j\}^\{\-2\}from[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2),

Bgraded​\(f∗\)​\(∑jgj2​σj2\)1/2=‖w∗‖2​\(∑kσk2\)1/2​\(∑jαj​gj−2\)1/2​\(∑jτj​gj2\)1/2=Bunif​\(f∗\)​\(∑kσk2\)1/2​Λ​\(g\)1/2\.\\begin\{split\}B^\{\\mathrm\{graded\}\}\(f^\{\*\}\)\\left\(\\sum\_\{j\}g\_\{j\}^\{2\}\\sigma\_\{j\}^\{2\}\\right\)^\{1/2\}&=\\\|w^\{\*\}\\\|\_\{2\}\\left\(\\sum\_\{k\}\\sigma\_\{k\}^\{2\}\\right\)^\{1/2\}\\left\(\\sum\_\{j\}\\alpha\_\{j\}g\_\{j\}^\{\-2\}\\right\)^\{1/2\}\\left\(\\sum\_\{j\}\\tau\_\{j\}g\_\{j\}^\{2\}\\right\)^\{1/2\}\\\\ &=B^\{\\mathrm\{unif\}\}\(f^\{\*\}\)\\left\(\\sum\_\{k\}\\sigma\_\{k\}^\{2\}\\right\)^\{1/2\}\\Lambda\(g\)^\{1/2\}\.\\end\{split\}For anLℓL\_\{\\ell\}\-Lipschitz loss the standard symmetrisation and contraction estimate of Bartlett and Mendelson\[bartlett2002rademacher\]bounds the excess risk of empirical risk minimisation over a classℋ\\mathcal\{H\}containingf∗f^\{\*\}by2​Lℓ​ℜm​\(ℋ\)\+O​\(log⁡\(1/δ\)/m\)2L\_\{\\ell\}\\mathfrak\{R\}\_\{m\}\(\\mathcal\{H\}\)\+O\(\\sqrt\{\\log\(1/\\delta\)/m\}\), so the sample size sufficient for excess riskε\\varepsilonscales as the square of the product of budget and radius\. That product carries the factorΛ​\(g\)1/2\\Lambda\(g\)^\{1/2\}in the graded case and11in the uniform case, and the ratio of sample sizes isΛ​\(g\)\\Lambda\(g\), independently ofε\\varepsilonand of‖w∗‖2\\\|w^\{\*\}\\\|\_\{2\}\.

\(iii\) Substitutetj=gj2\>0t\_\{j\}=g\_\{j\}^\{2\}\>0, so thatΛ=\(∑jαj​tj−1\)​\(∑jτj​tj\)\\Lambda=\\big\(\\sum\_\{j\}\\alpha\_\{j\}t\_\{j\}^\{\-1\}\\big\)\\big\(\\sum\_\{j\}\\tau\_\{j\}t\_\{j\}\\big\)\. By Cauchy–Schwarz applied to the vectors with entriesαj/tj\\sqrt\{\\alpha\_\{j\}/t\_\{j\}\}andτj​tj\\sqrt\{\\tau\_\{j\}t\_\{j\}\},

\(∑j=1dαj​τj\)2=\(∑j=1dαjtj⋅τj​tj\)2≤\(∑j=1dαjtj\)​\(∑j=1dτj​tj\)=Λ,\\left\(\\sum\_\{j=1\}^\{d\}\\sqrt\{\\alpha\_\{j\}\\tau\_\{j\}\}\\right\)^\{2\}=\\left\(\\sum\_\{j=1\}^\{d\}\\sqrt\{\\frac\{\\alpha\_\{j\}\}\{t\_\{j\}\}\}\\cdot\\sqrt\{\\tau\_\{j\}t\_\{j\}\}\\right\)^\{2\}\\leq\\left\(\\sum\_\{j=1\}^\{d\}\\frac\{\\alpha\_\{j\}\}\{t\_\{j\}\}\\right\)\\left\(\\sum\_\{j=1\}^\{d\}\\tau\_\{j\}t\_\{j\}\\right\)=\\Lambda,soBC​\(α,τ\)2\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\}is a lower bound\. Equality in Cauchy–Schwarz holds exactly when the two vectors are proportional, that isαj/tj=κ​τj​tj\\alpha\_\{j\}/t\_\{j\}=\\kappa\\,\\tau\_\{j\}t\_\{j\}for someκ\>0\\kappa\>0and alljj, givingtj=κ−1/2​\(αj/τj\)1/2t\_\{j\}=\\kappa^\{\-1/2\}\(\\alpha\_\{j\}/\\tau\_\{j\}\)^\{1/2\}and hencegj=tj1/2∝αj1/4​τj−1/4g\_\{j\}=t\_\{j\}^\{1/2\}\\propto\\alpha\_\{j\}^\{1/4\}\\tau\_\{j\}^\{\-1/4\}\. Thisgglies inℝ\>0d\\mathbb\{R\}\_\{\>0\}^\{d\}becauseαj\\alpha\_\{j\}andτj\\tau\_\{j\}are positive by[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2), so the bound is attained and[Eq\.14](https://arxiv.org/html/2607.22757#S4.E14)follows\.

\(iv\) Cauchy–Schwarz again, now onαj\\sqrt\{\\alpha\_\{j\}\}againstτj\\sqrt\{\\tau\_\{j\}\}, gives

∑jαj​τj≤\(∑jαj\)1/2​\(∑jτj\)1/2=1,\\sum\_\{j\}\\sqrt\{\\alpha\_\{j\}\\tau\_\{j\}\}\\leq\\big\(\\sum\_\{j\}\\alpha\_\{j\}\\big\)^\{1/2\}\\big\(\\sum\_\{j\}\\tau\_\{j\}\\big\)^\{1/2\}=1,soΛ⋆≤1\\Lambda^\{\\star\}\\leq 1\. Equality requiresαj=c​τj\\sqrt\{\\alpha\_\{j\}\}=c\\sqrt\{\\tau\_\{j\}\}for alljj; squaring and summing forcesc=1c=1, henceα=τ\\alpha=\\tau\. Converselyα=τ\\alpha=\\taugives

∑jαj​τj=∑jαj=1\.\\sum\_\{j\}\\sqrt\{\\alpha\_\{j\}\\tau\_\{j\}\}=\\sum\_\{j\}\\alpha\_\{j\}=1\.∎

###### Remark 5\.

Three points fix the standing of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(ii\)\. First, the two bounds are produced by the identical argument, the same symmetrisation, the same contraction, the same evaluation at the representation budget, with the grading entering only through the data radius and the budget\. The ratioΛ​\(g\)\\Lambda\(g\)therefore compares like with like, and is not an artefact of one bound being derived more loosely than the other\. Second, the confidence termO​\(log⁡\(1/δ\)/m\)O\(\\sqrt\{\\log\(1/\\delta\)/m\}\)of the excess\-risk bound is class\-independent and common to both sides, so at any fixedδ\\deltait cancels from the comparison of sufficient sample sizes to leading order; the ratio statement concerns the complexity\-dominated regime, which is the budget\-active regime made precise in[Rem\.12](https://arxiv.org/html/2607.22757#Thmrem12)\. Third, at its representation budget each class containsf∗f^\{\*\}on the boundary of its constraint set, so empirical risk minimisation over either class is well posed and the bound of\[bartlett2002rademacher\]applies to both without modification\.

###### Remark 6\.

[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)identifies the entire benefit of grading with a single scalar: the affinity between where the target places its weight and where the data places its variance\. Grading buys nothing when the two profiles agree, and its value grows as they separate\. In this exact sense grading does not create information\. It converts a correct prior about the target into a reduction in sample complexity at the computable rate[Eq\.14](https://arxiv.org/html/2607.22757#S4.E14), and an incorrect prior at the same rate in the opposite direction: anyggwithΛ​\(g\)\>1\\Lambda\(g\)\>1is admissible and costs data\.

[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)is exactly the assertion that these profiles diverge for language\. Deep structural features are decisive for the target and rare in the corpus, with highαj\\alpha\_\{j\}and lowτj\\tau\_\{j\}, while lexical features carry the bulk of the variance and comparatively little of the decision\. An isotropic architecture, which by construction setsg=𝟏g=\\mathbf\{1\}andΛ=1\\Lambda=1, forgoes the gain[Eq\.14](https://arxiv.org/html/2607.22757#S4.E14)whatever its size;[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)locates it precisely, on the boundary of the set of grades at which the graded bound is the smaller one\. Both profiles can be estimated from an annotated corpus without training a model, so[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)is testable in advance of any experiment;[Sec\.8\.2](https://arxiv.org/html/2607.22757#S8.SS2)includes this measurement in the program\.

###### Proposition 8\.

Letα,τ∈Δd−1\\alpha,\\tau\\in\\Delta^\{d\-1\}be the profiles of[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2), related to the grades bygj=λqjg\_\{j\}=\\lambda^\{q\_\{j\}\}, and write

Λ​\(q\)=\(∑j=1dαj​λ−2​qj\)​\(∑j=1dτj​λ2​qj\)\\Lambda\(q\)=\\Big\(\\sum\_\{j=1\}^\{d\}\\alpha\_\{j\}\\lambda^\{\-2q\_\{j\}\}\\Big\)\\Big\(\\sum\_\{j=1\}^\{d\}\\tau\_\{j\}\\lambda^\{2q\_\{j\}\}\\Big\)for the gain of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)expressed in the grades\. Define the*admissible set*

𝒬\+=\{q∈ℝd:Λ​\(q\)<1\},\\mathcal\{Q\}\_\{\+\}=\\left\\\{q\\in\\mathbb\{R\}^\{d\}\\;:\\;\\Lambda\(q\)<1\\right\\\},the grades at which the graded bound is strictly smaller than the uniform one\. Then:

1. \(i\)*\(Gauge invariance\)*Λ​\(q\+c​𝟏\)=Λ​\(q\)\\Lambda\(q\+c\\mathbf\{1\}\)=\\Lambda\(q\)for everyc∈ℝc\\in\\mathbb\{R\}\. Hence𝒬\+\\mathcal\{Q\}\_\{\+\}is invariant under translation along𝟏\\mathbf\{1\}, and𝒬\+≠∅\\mathcal\{Q\}\_\{\+\}\\neq\\emptysetimplies𝒬\+∩ℚ≥0d≠∅\\mathcal\{Q\}\_\{\+\}\\cap\\mathbb\{Q\}\_\{\\geq 0\}^\{d\}\\neq\\emptyset: restricting the grades to non\-negative rationals, as[Sec\.3](https://arxiv.org/html/2607.22757#S3)does throughout, costs nothing\.
2. \(ii\)*\(Convexity\)*q↦log⁡Λ​\(q\)q\\mapsto\\log\\Lambda\(q\)is convex onℝd\\mathbb\{R\}^\{d\}, and𝒬\+\\mathcal\{Q\}\_\{\+\}is open and convex\.
3. \(iii\)*\(Nonemptiness, and the position of the standard transformer\)*Λ​\(𝟎\)=1\\Lambda\(\\mathbf\{0\}\)=1, and𝒬\+≠∅\\mathcal\{Q\}\_\{\+\}\\neq\\emptysetif and only ifα≠τ\\alpha\\neq\\tau\. Whenα≠τ\\alpha\\neq\\tau, the uniform gradeq=𝟎q=\\mathbf\{0\}— which by[Remarks1](https://arxiv.org/html/2607.22757#Thmrem1)and[3](https://arxiv.org/html/2607.22757#Thmrem3)is the standard transformer — lies on∂𝒬\+\\partial\\mathcal\{Q\}\_\{\+\}\.
4. \(iv\)*\(First\-order criterion\)*∇qlog⁡Λ\|q=𝟎=2​\(log⁡λ\)​\(τ−α\)\\nabla\_\{q\}\\log\\Lambda\\big\|\_\{q=\\mathbf\{0\}\}=2\(\\log\\lambda\)\\,\(\\tau\-\\alpha\), and forv∈ℝdv\\in\\mathbb\{R\}^\{d\}the ray\{ε​v:ε\>0\}\\\{\\varepsilon v:\\varepsilon\>0\\\}meets𝒬\+\\mathcal\{Q\}\_\{\+\}if and only if \(15\)⟨v,α−τ⟩\>0\.\\langle v,\\alpha\-\\tau\\rangle\>0\.In that case\{ε\>0:ε​v∈𝒬\+\}\\\{\\varepsilon\>0:\\varepsilon v\\in\\mathcal\{Q\}\_\{\+\}\\\}is an interval\(0,εmax​\(v\)\)\(0,\\varepsilon\_\{\\max\}\(v\)\)\.
5. \(v\)*\(Gain along an admissible direction\)*Letvvsatisfy[Eq\.15](https://arxiv.org/html/2607.22757#S4.E15), and setD=⟨v,α−τ⟩D=\\langle v,\\alpha\-\\tau\\rangleandS=Varα​\(v\)\+Varτ​\(v\)S=\\mathrm\{Var\}\_\{\\alpha\}\(v\)\+\\mathrm\{Var\}\_\{\\tau\}\(v\), whereVarp​\(v\)=∑jpj​vj2−\(∑jpj​vj\)2\\mathrm\{Var\}\_\{p\}\(v\)=\\sum\_\{j\}p\_\{j\}v\_\{j\}^\{2\}\-\\big\(\\sum\_\{j\}p\_\{j\}v\_\{j\}\\big\)^\{2\}\. ThenS\>0S\>0and log⁡Λ​\(ε​v\)=−2​\(log⁡λ\)​D​ε\+2​\(log⁡λ\)2​S​ε2\+O​\(ε3\),\\log\\Lambda\(\\varepsilon v\)=\-2\(\\log\\lambda\)\\,D\\,\\varepsilon\\;\+\\;2\(\\log\\lambda\)^\{2\}S\\,\\varepsilon^\{2\}\\;\+\\;O\(\\varepsilon^\{3\}\),so the quadratic truncation is minimised atε⋆=D/\(2​S​log⁡λ\)\\varepsilon^\{\\star\}=D/\(2S\\log\\lambda\), with value−D2/\(2​S\)\-D^\{2\}/\(2S\)independent ofλ\\lambda\.
6. \(vi\)*\(Global optimum\)*minq∈ℝd⁡Λ​\(q\)=BC​\(α,τ\)2\\min\_\{q\\in\\mathbb\{R\}^\{d\}\}\\Lambda\(q\)=\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\}, attained exactly on the gauge orbit of theq⋆q^\{\\star\}of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\)\.

###### Proof\.

Setuj=2​qj​log⁡λu\_\{j\}=2q\_\{j\}\\log\\lambda, an invertible linear change of variable sinceλ\>1\\lambda\>1, and write

A​\(u\)=∑j=1dαj​e−uj,T​\(u\)=∑j=1dτj​euj,Φ​\(u\)=log⁡A​\(u\)\+log⁡T​\(u\)=log⁡Λ\.A\(u\)=\\sum\_\{j=1\}^\{d\}\\alpha\_\{j\}e^\{\-u\_\{j\}\},\\qquad T\(u\)=\\sum\_\{j=1\}^\{d\}\\tau\_\{j\}e^\{u\_\{j\}\},\\qquad\\Phi\(u\)=\\log A\(u\)\+\\log T\(u\)=\\log\\Lambda\.By[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2)bothα\\alphaandτ\\taulie inΔd−1\\Delta^\{d\-1\}and are strictly positive\. Foru∈ℝdu\\in\\mathbb\{R\}^\{d\}define the tilted vectors

αj\(u\)=αj​e−ujA​\(u\),τj\(u\)=τj​eujT​\(u\),\\alpha^\{\(u\)\}\_\{j\}=\\frac\{\\alpha\_\{j\}e^\{\-u\_\{j\}\}\}\{A\(u\)\},\\qquad\\tau^\{\(u\)\}\_\{j\}=\\frac\{\\tau\_\{j\}e^\{u\_\{j\}\}\}\{T\(u\)\},both inΔd−1\\Delta^\{d\-1\}, and forp∈Δd−1p\\in\\Delta^\{d\-1\}writeC​\(p\)=diag⁡\(p\)−p​p⊤\\mathrm\{C\}\(p\)=\\operatorname\{diag\}\(p\)\-pp^\{\\top\}, so thatv⊤​C​\(p\)​v=Varp​\(v\)≥0v^\{\\top\}\\mathrm\{C\}\(p\)\\,v=\\mathrm\{Var\}\_\{p\}\(v\)\\geq 0\. Differentiating,

∇uΦ​\(u\)=τ\(u\)−α\(u\),∇u2Φ​\(u\)=C​\(α\(u\)\)\+C​\(τ\(u\)\),\\begin\{split\}\\nabla\_\{u\}\\Phi\(u\)&=\\tau^\{\(u\)\}\-\\alpha^\{\(u\)\},\\\\ \\nabla^\{2\}\_\{u\}\\Phi\(u\)&=\\mathrm\{C\}\\big\(\\alpha^\{\(u\)\}\\big\)\+\\mathrm\{C\}\\big\(\\tau^\{\(u\)\}\\big\),\\end\{split\}the termlog⁡A\\log AcontributingC​\(α\(u\)\)\\mathrm\{C\}\(\\alpha^\{\(u\)\}\)to the Hessian because the two sign reversals arising fromu↦−uu\\mapsto\-ucancel\. Atu=𝟎u=\\mathbf\{0\}we haveα\(𝟎\)=α\\alpha^\{\(\\mathbf\{0\}\)\}=\\alphaandτ\(𝟎\)=τ\\tau^\{\(\\mathbf\{0\}\)\}=\\tau\.

\(i\) Underq↦q\+c​𝟏q\\mapsto q\+c\\mathbf\{1\}we getu↦u\+c′​𝟏u\\mapsto u\+c^\{\\prime\}\\mathbf\{1\}withc′=2​c​log⁡λc^\{\\prime\}=2c\\log\\lambda, soA↦e−c′​AA\\mapsto e^\{\-c^\{\\prime\}\}AandT↦ec′​TT\\mapsto e^\{c^\{\\prime\}\}T, leaving the productA​TATfixed\. For the second claim, letq∈𝒬\+q\\in\\mathcal\{Q\}\_\{\+\}and setq′=q−\(minj⁡qj\)​𝟏q^\{\\prime\}=q\-\(\\min\_\{j\}q\_\{j\}\)\\mathbf\{1\}, so thatq′≥0q^\{\\prime\}\\geq 0andΛ​\(q′\)=Λ​\(q\)<1\\Lambda\(q^\{\\prime\}\)=\\Lambda\(q\)<1\. SinceΛ\\Lambdais continuous andℚd\\mathbb\{Q\}^\{d\}is dense inℝd\\mathbb\{R\}^\{d\}, some rationalq′′≥0q^\{\\prime\\prime\}\\geq 0nearq′q^\{\\prime\}satisfiesΛ​\(q′′\)<1\\Lambda\(q^\{\\prime\\prime\}\)<1\.

\(ii\)∇u2Φ\\nabla^\{2\}\_\{u\}\\Phiis a sum of two matrices of the formC​\(p\)\\mathrm\{C\}\(p\)and is therefore positive semidefinite, soΦ\\Phiis convex inuuand hence inqq, the mapq↦uq\\mapsto ubeing linear\. Then𝒬\+=\{Φ<0\}\\mathcal\{Q\}\_\{\+\}=\\\{\\Phi<0\\\}is a strict sublevel set of a convex function, hence convex, and open by continuity\.

\(iii\)Λ​\(𝟎\)=\(∑jαj\)​\(∑jτj\)=1\\Lambda\(\\mathbf\{0\}\)=\\big\(\\sum\_\{j\}\\alpha\_\{j\}\\big\)\\big\(\\sum\_\{j\}\\tau\_\{j\}\\big\)=1\. By[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\) and \(iv\),

minq⁡Λ​\(q\)=BC​\(α,τ\)2≤1\\min\_\{q\}\\Lambda\(q\)=\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\}\\leq 1with equality if and only ifα=τ\\alpha=\\tau; hence𝒬\+≠∅\\mathcal\{Q\}\_\{\+\}\\neq\\emptysetexactly whenα≠τ\\alpha\\neq\\tau\. In that case𝟎∉𝒬\+\\mathbf\{0\}\\notin\\mathcal\{Q\}\_\{\+\}, while by \(iv\) the directionv=α−τv=\\alpha\-\\tausatisfies

⟨v,α−τ⟩=‖α−τ‖22\>0,\\langle v,\\alpha\-\\tau\\rangle=\\\|\\alpha\-\\tau\\\|\_\{2\}^\{2\}\>0,so every neighbourhood of𝟎\\mathbf\{0\}meets𝒬\+\\mathcal\{Q\}\_\{\+\}; thus𝟎∈∂𝒬\+\\mathbf\{0\}\\in\\partial\\mathcal\{Q\}\_\{\+\}\.

\(iv\) The gradient atu=𝟎u=\\mathbf\{0\}isτ−α\\tau\-\\alpha, and

∇q=2​\(log⁡λ\)​∇u,\\nabla\_\{q\}=2\(\\log\\lambda\)\\nabla\_\{u\},which gives the stated formula\. Putϕ​\(ε\)=log⁡Λ​\(ε​v\)\\phi\(\\varepsilon\)=\\log\\Lambda\(\\varepsilon v\), convex by \(ii\), withϕ​\(0\)=0\\phi\(0\)=0and

ϕ′​\(0\)=⟨v,∇qlog⁡Λ​\(𝟎\)⟩=−2​\(log⁡λ\)​⟨v,α−τ⟩\.\\phi^\{\\prime\}\(0\)=\\langle v,\\nabla\_\{q\}\\log\\Lambda\(\\mathbf\{0\}\)\\rangle=\-2\(\\log\\lambda\)\\langle v,\\alpha\-\\tau\\rangle\.If⟨v,α−τ⟩\>0\\langle v,\\alpha\-\\tau\\rangle\>0thenϕ′​\(0\)<0\\phi^\{\\prime\}\(0\)<0andϕ​\(ε\)<0\\phi\(\\varepsilon\)<0for all smallε\>0\\varepsilon\>0\. If⟨v,α−τ⟩≤0\\langle v,\\alpha\-\\tau\\rangle\\leq 0thenϕ′​\(0\)≥0\\phi^\{\\prime\}\(0\)\\geq 0, and convexity gives

ϕ​\(ε\)≥ϕ​\(0\)\+ϕ′​\(0\)​ε≥0\\phi\(\\varepsilon\)\\geq\\phi\(0\)\+\\phi^\{\\prime\}\(0\)\\varepsilon\\geq 0for everyε\>0\\varepsilon\>0, so the ray never meets𝒬\+\\mathcal\{Q\}\_\{\+\}\. This proves[Eq\.15](https://arxiv.org/html/2607.22757#S4.E15)\. The set\{ε\>0:ϕ​\(ε\)<0\}\\\{\\varepsilon\>0:\\phi\(\\varepsilon\)<0\\\}is the intersection of an interval, by convexity ofϕ\\phi, with\(0,∞\)\(0,\\infty\), and contains all smallε\>0\\varepsilon\>0; it is therefore of the form\(0,εmax​\(v\)\)\(0,\\varepsilon\_\{\\max\}\(v\)\)\.

\(v\) ThatS\>0S\>0follows fromD≠0D\\neq 0: sinceτj\>0\\tau\_\{j\}\>0for everyjj,Varτ​\(v\)=0\\mathrm\{Var\}\_\{\\tau\}\(v\)=0forcesv∈ℝ​𝟏v\\in\\mathbb\{R\}\\mathbf\{1\}, and thenD=⟨v,α−τ⟩=0D=\\langle v,\\alpha\-\\tau\\rangle=0becauseα\\alphaandτ\\tauare both probability vectors\. Taylor expansion ofϕ\\phiat0gives

ϕ​\(ε\)=ϕ′​\(0\)​ε\+12​ϕ′′​\(0\)​ε2\+O​\(ε3\)\\phi\(\\varepsilon\)=\\phi^\{\\prime\}\(0\)\\varepsilon\+\\tfrac\{1\}\{2\}\\phi^\{\\prime\\prime\}\(0\)\\varepsilon^\{2\}\+O\(\\varepsilon^\{3\}\)withϕ′​\(0\)=−2​\(log⁡λ\)​D\\phi^\{\\prime\}\(0\)=\-2\(\\log\\lambda\)Dand

ϕ′′​\(0\)=v⊤​∇q2log⁡Λ​\(𝟎\)​v=\(2​log⁡λ\)2​v⊤​\[C​\(α\)\+C​\(τ\)\]​v=4​\(log⁡λ\)2​S,\\phi^\{\\prime\\prime\}\(0\)=v^\{\\top\}\\nabla\_\{q\}^\{2\}\\log\\Lambda\(\\mathbf\{0\}\)\\,v=\(2\\log\\lambda\)^\{2\}\\,v^\{\\top\}\\big\[\\mathrm\{C\}\(\\alpha\)\+\\mathrm\{C\}\(\\tau\)\\big\]v=4\(\\log\\lambda\)^\{2\}S,which is the stated expansion\. Minimisinga​ε2\+b​εa\\varepsilon^\{2\}\+b\\varepsilonwitha=2​\(log⁡λ\)2​S\>0a=2\(\\log\\lambda\)^\{2\}S\>0andb=−2​\(log⁡λ\)​Db=\-2\(\\log\\lambda\)Dgives

ε⋆=−b/\(2​a\)=D/\(2​S​log⁡λ\)\\varepsilon^\{\\star\}=\-b/\(2a\)=D/\(2S\\log\\lambda\)and minimum value−b2/\(4​a\)=−D2/\(2​S\)\-b^\{2\}/\(4a\)=\-D^\{2\}/\(2S\), in whichλ\\lambdacancels\.

\(vi\) The value and the minimizer are[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\), transported throughgj=λqjg\_\{j\}=\\lambda^\{q\_\{j\}\}; the gauge orbit is the fibre of the scaling invariance recorded in \(i\)\. ∎

###### Remark 7\.

Four consequences are worth noting\.

First,[Eq\.15](https://arxiv.org/html/2607.22757#S4.E15)is the condition on the weights that the framework has needed\. Grading improves the bound, to first order, exactly when the grades correlate positively withα−τ\\alpha\-\\tau: high grades on coordinates where the target’s energy exceeds the data’s variance, low grades where it does not\.[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)thereby becomes an inner product one can compute\.

Second, the standard transformer is not a local optimum of the sample\-complexity landscape\. It sits atΛ=1\\Lambda=1on the boundary of𝒬\+\\mathcal\{Q\}\_\{\+\}with gradient2​\(log⁡λ\)​\(τ−α\)≠02\(\\log\\lambda\)\(\\tau\-\\alpha\)\\neq 0wheneverα≠τ\\alpha\\neq\\tau, so a strictly descending direction exists atq=𝟎q=\\mathbf\{0\}and[Eq\.15](https://arxiv.org/html/2607.22757#S4.E15)exhibits it\. The isotropic architecture forgoes the gain of[Eq\.14](https://arxiv.org/html/2607.22757#S4.E14)not because the gain is unavailable at its location, since the gradient there is nonzero, but because the gauge orbit ofq=𝟎q=\\mathbf\{0\}is the unique set of grades at whichΛ=1\\Lambda=1for*every*pair of profiles\. It is the choice that ignores the profiles\.

Third, only the*sign*of⟨q,α^−τ^⟩\\langle q,\\hat\{\\alpha\}\-\\hat\{\\tau\}\\rangleneed survive estimation error, not the magnitudes ofα^\\hat\{\\alpha\}andτ^\\hat\{\\tau\}\. This is what makes the coarse part\-of\-speech initialisation of[Sec\.8\.1\.2](https://arxiv.org/html/2607.22757#S8.SS1.SSS2)defensible: it does not attempt to locateq⋆q^\{\\star\}, only to enter𝒬\+\\mathcal\{Q\}\_\{\+\}\.

Fourth, by \(ii\) the offline selection of grades from estimated profiles is the minimisation of a convex function with no spurious stationary points, and by \(i\) the non\-negativity and rationality demanded of the grades throughout this paper are free\. The selection problem, on the full space and on the clipped region the configurations admit, is solved in[Cor\.4](https://arxiv.org/html/2607.22757#Thmcor4)and[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)respectively\.

Λ\\Lambdais a ratio of upper bounds, and[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)characterises when the graded bound beats the uniform bound\. In the level\-stratified regime of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)the comparison is a separation between the graded prior and its absence:[Thm\.4\.1](https://arxiv.org/html/2607.22757#S4.Thmthm1)establishes the two\-sided minimax ratio there, over all estimators\. For general profiles the corresponding statement is[Conj\.2](https://arxiv.org/html/2607.22757#Thmcon2), and[Rem\.11](https://arxiv.org/html/2607.22757#Thmrem11)records the scope\.

###### Corollary 3\.

Suppose both profiles are uniform within levels:αj=al/dl\\alpha\_\{j\}=a\_\{l\}/d\_\{l\}andτj=sl/dl\\tau\_\{j\}=s\_\{l\}/d\_\{l\}forj∈Vlj\\in V\_\{l\}, witha,s∈ΔL−1a,s\\in\\Delta^\{L\-1\}\. Then

Λ⋆=BC​\(a,s\)2=\(∑l=0L−1al​sl\)2,\\Lambda^\{\\star\}=\\mathrm\{BC\}\(a,s\)^\{2\}=\\left\(\\sum\_\{l=0\}^\{L\-1\}\\sqrt\{a\_\{l\}s\_\{l\}\}\\right\)^\{2\},independently of the allocation\{dl\}\\\{d\_\{l\}\\\}, and the optimal grade is constant on each level with

ql⋆​log⁡λ=14​log⁡al−14​log⁡sl\+const\.q\_\{l\}^\{\\star\}\\,\\log\\lambda=\\tfrac\{1\}\{4\}\\log a\_\{l\}\-\\tfrac\{1\}\{4\}\\log s\_\{l\}\+\\mathrm\{const\}\.If moreover the target concentrates geometrically towards the deepest level and the data decays geometrically away from the shallowest,al∝νL−1−la\_\{l\}\\propto\\nu^\{L\-1\-l\}andsl∝μls\_\{l\}\\propto\\mu^\{l\}with0<μ<ν<10<\\mu<\\nu<1, thenql⋆q\_\{l\}^\{\\star\}is affine inllwith positive slope, and

Λ⋆=Θ​\(νL\)\(L→∞\)\.\\Lambda^\{\\star\}=\\Theta\\\!\\left\(\\nu^\{L\}\\right\)\\qquad\(L\\to\\infty\)\.

###### Proof\.

For the first claim,∑j∈Vlαj​τj=dl⋅\(al/dl\)​\(sl/dl\)=al​sl\\sum\_\{j\\in V\_\{l\}\}\\sqrt\{\\alpha\_\{j\}\\tau\_\{j\}\}=d\_\{l\}\\cdot\\sqrt\{\(a\_\{l\}/d\_\{l\}\)\(s\_\{l\}/d\_\{l\}\)\}=\\sqrt\{a\_\{l\}s\_\{l\}\}, and summing overllgivesBC​\(α,τ\)=BC​\(a,s\)\\mathrm\{BC\}\(\\alpha,\\tau\)=\\mathrm\{BC\}\(a,s\); thedld\_\{l\}cancel\. The grade formula islog⁡gj⋆=14​log⁡αj−14​log⁡τj\+const\\log g\_\{j\}^\{\\star\}=\\tfrac\{1\}\{4\}\\log\\alpha\_\{j\}\-\\tfrac\{1\}\{4\}\\log\\tau\_\{j\}\+\\mathrm\{const\}from[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\), in which thelog⁡dl\\log d\_\{l\}terms cancel betweenαj\\alpha\_\{j\}andτj\\tau\_\{j\}\.

For the second, writeZa=∑l=0L−1νL−1−l=\(1−νL\)/\(1−ν\)Z\_\{a\}=\\sum\_\{l=0\}^\{L\-1\}\\nu^\{L\-1\-l\}=\(1\-\\nu^\{L\}\)/\(1\-\\nu\)andZs=\(1−μL\)/\(1−μ\)Z\_\{s\}=\(1\-\\mu^\{L\}\)/\(1\-\\mu\)\. Thenlog⁡al−log⁡sl=−l​\(log⁡ν\+log⁡μ\)\+const\\log a\_\{l\}\-\\log s\_\{l\}=\-l\(\\log\\nu\+\\log\\mu\)\+\\mathrm\{const\}, which is affine inllwith positive coefficient sincelog⁡ν,log⁡μ<0\\log\\nu,\\log\\mu<0; the grade claim follows\. Settingr=μ/ν∈\(0,1\)r=\\sqrt\{\\mu/\\nu\}\\in\(0,1\),

BC​\(a,s\)=1Za​Zs​∑l=0L−1ν\(L−1−l\)/2​μl/2=ν\(L−1\)/2Za​Zs⋅1−rL1−r\.\\mathrm\{BC\}\(a,s\)=\\frac\{1\}\{\\sqrt\{Z\_\{a\}Z\_\{s\}\}\}\\sum\_\{l=0\}^\{L\-1\}\\nu^\{\(L\-1\-l\)/2\}\\mu^\{l/2\}=\\frac\{\\nu^\{\(L\-1\)/2\}\}\{\\sqrt\{Z\_\{a\}Z\_\{s\}\}\}\\cdot\\frac\{1\-r^\{L\}\}\{1\-r\}\.AsL→∞L\\to\\inftywe haveZa→\(1−ν\)−1Z\_\{a\}\\to\(1\-\\nu\)^\{\-1\},Zs→\(1−μ\)−1Z\_\{s\}\\to\(1\-\\mu\)^\{\-1\}andrL→0r^\{L\}\\to 0, so

BC​\(a,s\)∼ν\(L−1\)/2⋅\(1−ν\)​\(1−μ\)1−μ/ν,\\mathrm\{BC\}\(a,s\)\\sim\\nu^\{\(L\-1\)/2\}\\cdot\\frac\{\\sqrt\{\(1\-\\nu\)\(1\-\\mu\)\}\}\{1\-\\sqrt\{\\mu/\\nu\}\},and squaring givesΛ⋆=Θ​\(νL\)\\Lambda^\{\\star\}=\\Theta\(\\nu^\{L\}\)\. ∎

###### Remark 8\.

[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)is an unconstrained statement: the optimal grade is affine inll, so its dynamic range grows linearly inLL, and realisingΛ⋆=Θ​\(νL\)\\Lambda^\{\\star\}=\\Theta\(\\nu^\{L\}\)requires grades that exceed any fixed clip onceLLis large\. At the clipped configurations of[Sec\.8\.1\.1](https://arxiv.org/html/2607.22757#S8.SS1.SSS1)the attainable gain is bounded byC2C^\{2\}\([Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\), and the exponential regime is reached as the clip opens \([Rem\.18](https://arxiv.org/html/2607.22757#Thmrem18)\)\. The corollary quantifies that trajectory; it does not describe the planned configurations\.

###### Remark 9\.

Under the hypotheses of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)the criterion[Eq\.15](https://arxiv.org/html/2607.22757#S4.E15)reduces to a condition onLLnumbers rather thandd: a level\-constant gradevj=vlv\_\{j\}=v\_\{l\}forj∈Vlj\\in V\_\{l\}is admissible exactly when∑l=0L−1vl​\(al−sl\)\>0\\sum\_\{l=0\}^\{L\-1\}v\_\{l\}\(a\_\{l\}\-s\_\{l\}\)\>0\. Under the geometric profiles this holds for everyvvaffine and increasing inll, so the linguistic ordering itself, with deeper levels receiving higher grades, is enough to enter𝒬\+\\mathcal\{Q\}\_\{\+\}without any estimate ofν\\nuorμ\\mu\.

###### Lemma 2\.

Letα^,τ^∈Δd−1\\hat\{\\alpha\},\\hat\{\\tau\}\\in\\Delta^\{d\-1\}be strictly positive estimates of the profiles of[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2), and letg^\\hat\{g\}be the plug\-in optimal gradesg^j∝α^j1/4​τ^j−1/4\\hat\{g\}\_\{j\}\\propto\\hat\{\\alpha\}\_\{j\}^\{1/4\}\\,\\hat\{\\tau\}\_\{j\}^\{\-1/4\}of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\)\. Set

δj=log⁡α^jαj−log⁡τ^jτj,δ¯=1d​∑j=1dδj\.\\delta\_\{j\}=\\log\\frac\{\\hat\{\\alpha\}\_\{j\}\}\{\\alpha\_\{j\}\}\-\\log\\frac\{\\hat\{\\tau\}\_\{j\}\}\{\\tau\_\{j\}\},\\qquad\\bar\{\\delta\}=\\frac\{1\}\{d\}\\sum\_\{j=1\}^\{d\}\\delta\_\{j\}\.Then

log⁡Λ​\(g^\)≤log⁡BC​\(α,τ\)2\+14​∑j=1d\(δj−δ¯\)2\.\\log\\Lambda\(\\hat\{g\}\)\\;\\leq\\;\\log\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\}\\;\+\\;\\frac\{1\}\{4\}\\sum\_\{j=1\}^\{d\}\\big\(\\delta\_\{j\}\-\\bar\{\\delta\}\\big\)^\{2\}\.The attained gain is therefore insensitive to first order to estimation error in the profiles: the degradation is quadratic in the centred log\-errors, and vanishes at rate the square of the profile error rather than the profile error itself\.

###### Proof\.

Work in the variablesuj=2​qj​log⁡λ=2​log⁡gju\_\{j\}=2q\_\{j\}\\log\\lambda=2\\log g\_\{j\}of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)and writeΦ​\(u\)=log⁡Λ\\Phi\(u\)=\\log\\Lambda\. The optimal grades of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\) correspond touj⋆=12​log⁡\(αj/τj\)\+c⋆u^\{\\star\}\_\{j\}=\\tfrac\{1\}\{2\}\\log\(\\alpha\_\{j\}/\\tau\_\{j\}\)\+c^\{\\star\}, and atu⋆u^\{\\star\}the tilted profiles of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)coincide,

αj\(u⋆\)=αj​τjBC​\(α,τ\)=τj\(u⋆\),\\alpha^\{\(u^\{\\star\}\)\}\_\{j\}=\\frac\{\\sqrt\{\\alpha\_\{j\}\\tau\_\{j\}\}\}\{\\mathrm\{BC\}\(\\alpha,\\tau\)\}=\\tau^\{\(u^\{\\star\}\)\}\_\{j\},so∇uΦ​\(u⋆\)=τ\(u⋆\)−α\(u⋆\)=0\\nabla\_\{u\}\\Phi\(u^\{\\star\}\)=\\tau^\{\(u^\{\\star\}\)\}\-\\alpha^\{\(u^\{\\star\}\)\}=0: the optimum is a stationary point\. For everyuuthe Hessian satisfies

∇u2Φ​\(u\)=C​\(α\(u\)\)\+C​\(τ\(u\)\)⪯2​I,\\nabla^\{2\}\_\{u\}\\Phi\(u\)=\\mathrm\{C\}\\big\(\\alpha^\{\(u\)\}\\big\)\+\\mathrm\{C\}\\big\(\\tau^\{\(u\)\}\\big\)\\preceq 2I,since for anyp∈Δd−1p\\in\\Delta^\{d\-1\}one hasC​\(p\)=diag⁡\(p\)−p​p⊤⪯diag⁡\(p\)⪯I\\mathrm\{C\}\(p\)=\\operatorname\{diag\}\(p\)\-pp^\{\\top\}\\preceq\\operatorname\{diag\}\(p\)\\preceq I\. Taylor expansion with integral remainder along the segment fromu⋆u^\{\\star\}tou^\\hat\{u\}, using the vanishing gradient and the uniform Hessian bound, gives

Φ​\(u^\)≤Φ​\(u⋆\)\+‖u^−u⋆‖22\.\\Phi\(\\hat\{u\}\)\\;\\leq\\;\\Phi\(u^\{\\star\}\)\+\\\|\\hat\{u\}\-u^\{\\star\}\\\|\_\{2\}^\{2\}\.The plug\-in grades correspond tou^j=12​log⁡\(α^j/τ^j\)\+c^\\hat\{u\}\_\{j\}=\\tfrac\{1\}\{2\}\\log\(\\hat\{\\alpha\}\_\{j\}/\\hat\{\\tau\}\_\{j\}\)\+\\hat\{c\}, sou^j−uj⋆=12​δj\+const\\hat\{u\}\_\{j\}\-u^\{\\star\}\_\{j\}=\\tfrac\{1\}\{2\}\\delta\_\{j\}\+\\mathrm\{const\}, and by the gauge invariance of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(i\) the representativeu⋆u^\{\\star\}on the optimal orbit may be chosen to minimise the distance, giving

minc∈ℝ⁡‖12​δ\+c​𝟏‖22=14​∑j=1d\(δj−δ¯\)2\.\\min\_\{c\\in\\mathbb\{R\}\}\\Big\\\|\\tfrac\{1\}\{2\}\\delta\+c\\mathbf\{1\}\\Big\\\|\_\{2\}^\{2\}=\\frac\{1\}\{4\}\\sum\_\{j=1\}^\{d\}\\big\(\\delta\_\{j\}\-\\bar\{\\delta\}\\big\)^\{2\}\.SinceΦ​\(u⋆\)=log⁡BC​\(α,τ\)2\\Phi\(u^\{\\star\}\)=\\log\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\}by[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(vi\), the claim follows\. ∎

###### Remark 10\.

The program estimates the profiles on an annotated corpus disjoint from the pretraining sample, and this disjointness is what closes the adaptivity question in the present setting\. Conditional ong^\\hat\{g\}, the grades are a fixed deterministic quantity with respect to the training sample, so[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(ii\) applies verbatim withΛ​\(g^\)\\Lambda\(\\hat\{g\}\)in place ofΛ​\(g\)\\Lambda\(g\)and no selection term arises\. The whole cost of not knowingα\\alphais therefore the degradation[Lem\.2](https://arxiv.org/html/2607.22757#Thmlem2)bounds, and that degradation is quadratic in the profile error because the gradient of the gain vanishes at the optimum: the framework is protected at exactly the point where protection is needed\. Certification asks for less still, since by[Rem\.7](https://arxiv.org/html/2607.22757#Thmrem7)only the sign of one inner product must survive the estimation\.

###### Remark 11\.

Four limitations bound what[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)establishes; two of them are resolved in this manuscript, one of them in the regime the program occupies, and the remaining ones name the tool their repair would require\.

First, the analysis is for the graded embedding composed with a linear read\-out\. The multi\-layer statement requires propagating the argument through a layerwise Lipschitz analysis of the graded stack, and the resulting bound compounds across depth; the extension is open\.

Second, both sides of \(ii\) are*upper*bounds, and a ratio of upper bounds becomes a separation only when a matching minimax lower bound over the uniform target class meets it\.[Thm\.4\.1](https://arxiv.org/html/2607.22757#S4.Thmthm1)supplies that bound in the level\-stratified regime of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3): at squared loss and over all estimators, the ratio of minimax risks isΘμ,ν​\(Λ​\(g\)\)\\Theta\_\{\\mu,\\nu\}\(\\Lambda\(g\)\)throughout the window of[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4), so in the regime the planned configurations occupy the separation between the graded prior and its absence is established\. The ellipsoidal constraint geometry of[Prop\.5](https://arxiv.org/html/2607.22757#Thmprop5)is the packing set on which that bound is posed\. What remains open is the general\-profile, Lipschitz\-loss case, stated as[Conj\.2](https://arxiv.org/html/2607.22757#Thmcon2)in the regime fixed by[Rem\.12](https://arxiv.org/html/2607.22757#Thmrem12)\.

Third,g⋆g^\{\\star\}depends onα\\alpha, which is unknown\.[Lem\.2](https://arxiv.org/html/2607.22757#Thmlem2)and[Rem\.10](https://arxiv.org/html/2607.22757#Thmrem10)resolve this for the present setting: profiles estimated on data disjoint from the training sample incur no adaptivity cost beyond the quadratic degradation of the lemma, which vanishes to first order at the optimum\. What remains open is only the fully adaptive variant in which grades and read\-out are fit on the same sample\.

Fourth,[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)assumes geometric profiles\. That assumption is an empirical refinement of[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1), not a theorem, and it is the assumption on which every exponential\-in\-LLstatement in this work rests;[Rem\.8](https://arxiv.org/html/2607.22757#Thmrem8)bounds what it delivers at fixed clip\.

###### Conjecture 2\.

Letx∼N​\(0,diag⁡\(σ12,…,σd2\)\)x\\sim N\\big\(0,\\operatorname\{diag\}\(\\sigma\_\{1\}^\{2\},\\dots,\\sigma\_\{d\}^\{2\}\)\\big\)andy=f∗​\(x\)\+ξy=f^\{\*\}\(x\)\+\\xiwithξ∼N​\(0,σξ2\)\\xi\\sim N\(0,\\sigma\_\{\\xi\}^\{2\}\)independent ofxx, in the setting of[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2), and letℓ\\ellbe a loss that isLℓL\_\{\\ell\}\-Lipschitz in its first argument, as in[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7); writeℰ​\(f^,f∗\)=𝔼​\[ℓ​\(f^​\(x\),y\)\]−𝔼​\[ℓ​\(f∗​\(x\),y\)\]\\mathcal\{E\}\(\\hat\{f\},f^\{\*\}\)=\\mathbb\{E\}\\big\[\\ell\(\\hat\{f\}\(x\),y\)\\big\]\-\\mathbb\{E\}\\big\[\\ell\(f^\{\*\}\(x\),y\)\\big\]for the excess risk\. Fix a targetf∗f^\{\*\}with profileα\\alphaand letg⋆g^\{\\star\}be the optimal grades of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\)\. Define the target classes carrying no prior and the graded prior respectively,

ℱunif=\{⟨w,⋅⟩:‖w‖2≤Bunif​\(f∗\)\},ℱgraded=\{⟨w,⋅⟩:‖G−1​w‖2≤Bgraded​\(f∗\)\},\\begin\{split\}\\mathcal\{F\}^\{\\mathrm\{unif\}\}&=\\left\\\{\\langle w,\\cdot\\rangle\\;:\\;\\\|w\\\|\_\{2\}\\leq B^\{\\mathrm\{unif\}\}\(f^\{\*\}\)\\right\\\},\\\\ \\mathcal\{F\}^\{\\mathrm\{graded\}\}&=\\left\\\{\\langle w,\\cdot\\rangle\\;:\\;\\big\\\|G^\{\-1\}w\\big\\\|\_\{2\}\\leq B^\{\\mathrm\{graded\}\}\(f^\{\*\}\)\\right\\\},\\end\{split\}withGGat gradesg⋆g^\{\\star\}and both budgets calibrated atf∗f^\{\*\}as in[Def\.3](https://arxiv.org/html/2607.22757#Thmdefn3), and the minimax excess risk of a target classℱ\\mathcal\{F\}at sample sizemm, over*all*estimators,

ℛm​\(ℱ\)=inff^supf∗∈ℱ𝔼​\[ℰ​\(f^,f∗\)\],mℱ​\(ε\)=min⁡\{m:ℛm​\(ℱ\)≤ε\}\.\\mathcal\{R\}\_\{m\}\(\\mathcal\{F\}\)=\\inf\_\{\\hat\{f\}\}\\,\\sup\_\{f^\{\*\}\\in\\mathcal\{F\}\}\\,\\mathbb\{E\}\\big\[\\mathcal\{E\}\(\\hat\{f\},f^\{\*\}\)\\big\],\\qquad m^\{\\mathcal\{F\}\}\(\\varepsilon\)=\\min\\left\\\{m\\;:\\;\\mathcal\{R\}\_\{m\}\(\\mathcal\{F\}\)\\leq\\varepsilon\\right\\\}\.Then there exist universal constants0<c1≤c20<c\_\{1\}\\leq c\_\{2\}such that, for everyε\\varepsilonin the budget\-active window of[Rem\.12](https://arxiv.org/html/2607.22757#Thmrem12),

c1⋅BC​\(α,τ\)2≤mℱgraded​\(ε\)mℱunif​\(ε\)≤c2⋅BC​\(α,τ\)2,c\_\{1\}\\cdot\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\}\\;\\leq\\;\\frac\{m^\{\\mathcal\{F\}^\{\\mathrm\{graded\}\}\}\(\\varepsilon\)\}\{m^\{\\mathcal\{F\}^\{\\mathrm\{unif\}\}\}\(\\varepsilon\)\}\\;\\leq\\;c\_\{2\}\\cdot\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\},so the gain[Eq\.14](https://arxiv.org/html/2607.22757#S4.E14)is attained up to universal constants and is a genuine separation between the graded prior and its absence, not an artefact of the upper bounds of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\.

[Conj\.2](https://arxiv.org/html/2607.22757#Thmcon2)is stated for general profiles and Lipschitz losses\. In the level\-stratified regime of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3), at squared loss, it is a theorem:[Thm\.4\.1](https://arxiv.org/html/2607.22757#S4.Thmthm1)establishes the two\-sided ratio, in risk form at each sample size of the window of[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4), and that regime is the one every planned configuration of[Sec\.8\.1\.1](https://arxiv.org/html/2607.22757#S8.SS1.SSS1)occupies\. What the conjecture asserts beyond the theorem is the general case, and the remark below fixes the regime in which it can hold\.

###### Remark 12\.

Two features of the statement are forced, and each replaces a formulation that cannot hold\. First, the infimum runs over all estimators and the two classes enter as classes of*targets*: by[Remarks1](https://arxiv.org/html/2607.22757#Thmrem1)and[3](https://arxiv.org/html/2607.22757#Thmrem3)the graded and uniform hypothesis classes coincide as sets of functions, so a comparison that restricted the estimator would compare a quantity with itself\. What differs between the two sides is what is known about the target in advance, which is exactly the content of a prior, and the conjecture quantifies that prior\. Second, the comparison cannot be asymptotic inε→0\\varepsilon\\to 0at fixeddd: below the parametric floor of orderσξ2​d/m\\sigma\_\{\\xi\}^\{2\}d/mthe norm constraint ceases to bind, both minimax problems reduce to the same unconstraineddd\-dimensional one, and the ratio tends to11for*every*pair of classes with nonempty interior\. Each class has a budget\-active interval ofε\\varepsilonon which its minimax risk exceeds its parametric floor by a fixed factor, equivalently sample sizes below the saturation scale at which its constraint stops binding, and the budget\-active window of the conjecture is the intersection of the two intervals, since the two\-sided ratio requires both constraints to bind\. Only in that window is the geometry of[Prop\.5](https://arxiv.org/html/2607.22757#Thmprop5)visible in the rates\. In the stratified regime the window is no longer a description but a set:[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4)exhibits it asWW, proves it nonempty atd≥3​L/σξd\\geq 3L/\\sigma\_\{\\xi\}, and proves that in the joint scalingd,m→∞d,m\\to\\inftyit widens rather than closes\. For general profiles the same two requirements, nonemptiness under[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)and widening under the joint scaling, are the conditions the conjecture’s window must satisfy\.

The route[Thm\.4\.1](https://arxiv.org/html/2607.22757#S4.Thmthm1)takes is the one the general case must extend\. The lower bound is an Assouad packing of sign patterns with the amplitude allocation chosen to hold the per\-coordinate testing error constant, run in the anisotropic Gaussian design directly and converted into the bound by Fano’s inequality \([Lem\.3](https://arxiv.org/html/2607.22757#Thmlem3)\); the window on which both budgets bind is exhibited and sized by[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4); and the upper bound in the matching form is carried by the envelope rather than imported from[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(ii\)\. Extending the argument beyond level\-homogeneous designs and squared loss is what the general case requires, and[Sec\.8\.4](https://arxiv.org/html/2607.22757#S8.SS4)records it as the remaining theoretical target alongside the layerwise extension\.

### 4\.3\.The Minimax Envelope

Throughout this subsection the design is Gaussian and level\-homogeneous:x∼N​\(0,Σ\)x\\sim N\(0,\\Sigma\)withΣ=diag⁡\(σ12,…,σd2\)\\Sigma=\\operatorname\{diag\}\(\\sigma\_\{1\}^\{2\},\\dots,\\sigma\_\{d\}^\{2\}\)constant on levels,σj2=σ\(l\)2\\sigma\_\{j\}^\{2\}=\\sigma\_\{\(l\)\}^\{2\}forj∈Vlj\\in V\_\{l\}anddimVl=dl\\dim V\_\{l\}=d\_\{l\}, andy=⟨w∗,x⟩\+ξy=\\langle w^\{\*\},x\\rangle\+\\xiwithξ∼N​\(0,σξ2\)\\xi\\sim N\(0,\\sigma\_\{\\xi\}^\{2\}\)independent ofxx\. The loss is squared, so the excess risk of a predictorf^\\hat\{f\}at the targetf∗=⟨w∗,⋅⟩f^\{\*\}=\\langle w^\{\*\},\\cdot\\rangleis‖f^−f∗‖L2​\(Px\)2\\\|\\hat\{f\}\-f^\{\*\}\\\|\_\{L^\{2\}\(P\_\{x\}\)\}^\{2\}, and for a classℱ\\mathcal\{F\}of targets the minimax excess risk at sample sizemmisℛm​\(ℱ\)=inff^supf∗∈ℱ𝔼​‖f^−f∗‖L2​\(Px\)2\\mathcal\{R\}\_\{m\}\(\\mathcal\{F\}\)=\\inf\_\{\\hat\{f\}\}\\sup\_\{f^\{\*\}\\in\\mathcal\{F\}\}\\mathbb\{E\}\\,\\\|\\hat\{f\}\-f^\{\*\}\\\|\_\{L^\{2\}\(P\_\{x\}\)\}^\{2\}, the infimum over all estimators, as in[Conj\.2](https://arxiv.org/html/2607.22757#Thmcon2)\. For a diagonal matrixD=diag⁡\(Dl​Idl\)D=\\operatorname\{diag\}\(D\_\{l\}I\_\{d\_\{l\}\}\)withDl\>0D\_\{l\}\>0and a budgetB\>0B\>0set

ℱ​\(B,D\)=\{⟨w,⋅⟩:w∈ℝd,‖D​w‖2≤B\},capl=σ\(l\)2​B2Dl2,ε2=σξ2m\.\\mathcal\{F\}\(B,D\)=\\bigl\\\{\\langle w,\\cdot\\rangle:w\\in\\mathbb\{R\}^\{d\},\\ \\\|Dw\\\|\_\{2\}\\leq B\\bigr\\\},\\qquad\\operatorname\{cap\}\_\{l\}=\\frac\{\\sigma\_\{\(l\)\}^\{2\}B^\{2\}\}\{D\_\{l\}^\{2\}\},\\qquad\\varepsilon^\{2\}=\\frac\{\\sigma\_\{\\xi\}^\{2\}\}\{m\}\.The choiceD=IdD=I\_\{d\},B=Bunif​\(f∗\)B=B^\{\\mathrm\{unif\}\}\(f^\{\*\}\)realises the uniform target class, andDl=gl−1D\_\{l\}=g\_\{l\}^\{\-1\},B=Bgraded​\(f∗\)B=B^\{\\mathrm\{graded\}\}\(f^\{\*\}\)the graded one;capl\\operatorname\{cap\}\_\{l\}is the prediction\-norm energy the budget can place at levelllif spent there entirely, and the two classes differ only through their caps\.

###### Lemma 3\.

LetS=\{l:dl​ε2≤capl\}S=\\\{l:d\_\{l\}\\varepsilon^\{2\}\\leq\\operatorname\{cap\}\_\{l\}\\\}\. There exist constants0<c​\(L\)≤C​\(L\)0<c\(L\)\\leq C\(L\)depending only onLLsuch that:

1. \(i\)for everym≥1m\\geq 1, ℛm​\(ℱ​\(B,D\)\)≥c​\(L\)​∑l=0L−1min⁡\{capl,dl​ε2\};\\mathcal\{R\}\_\{m\}\\bigl\(\\mathcal\{F\}\(B,D\)\\bigr\)\\ \\geq\\ c\(L\)\\sum\_\{l=0\}^\{L\-1\}\\min\\bigl\\\{\\operatorname\{cap\}\_\{l\},\\ d\_\{l\}\\varepsilon^\{2\}\\bigr\\\};
2. \(ii\)for everym≥2​∑l∈Sdl\+4m\\geq 2\\sum\_\{l\\in S\}d\_\{l\}\+4, ℛm​\(ℱ​\(B,D\)\)≤C​\(L\)​∑l=0L−1min⁡\{capl,dl​ε2\}\.\\mathcal\{R\}\_\{m\}\\bigl\(\\mathcal\{F\}\(B,D\)\\bigr\)\\ \\leq\\ C\(L\)\\sum\_\{l=0\}^\{L\-1\}\\min\\bigl\\\{\\operatorname\{cap\}\_\{l\},\\ d\_\{l\}\\varepsilon^\{2\}\\bigr\\\}\.

###### Proof\.

Writeμl=min⁡\{capl,dl​ε2\}\\mu\_\{l\}=\\min\\\{\\operatorname\{cap\}\_\{l\},d\_\{l\}\\varepsilon^\{2\}\\\}andM=∑lμlM=\\sum\_\{l\}\\mu\_\{l\}\.

\(i\) Fixc0=\(log⁡2\)/64c\_\{0\}=\(\\log 2\)/64and set, for each level,kl=min⁡\{dl,⌊capl/\(c0​L​ε2\)⌋\}k\_\{l\}=\\min\\bigl\\\{d\_\{l\},\\ \\lfloor\\operatorname\{cap\}\_\{l\}/\(c\_\{0\}L\\varepsilon^\{2\}\)\\rfloor\\bigr\\\}andδl2=c0​ε2/σ\(l\)2\\delta\_\{l\}^\{2\}=c\_\{0\}\\varepsilon^\{2\}/\\sigma\_\{\(l\)\}^\{2\}, and let𝒲⊆ℝd\\mathcal\{W\}\\subseteq\\mathbb\{R\}^\{d\}consist of the vectors supported on the firstklk\_\{l\}coordinates of eachVlV\_\{l\}with entries±δl\\pm\\delta\_\{l\}\. Everyw∈𝒲w\\in\\mathcal\{W\}is feasible:

‖D​w‖22=∑lkl​Dl2​δl2=c0​ε2​∑lkl​B2capl≤B2L​∑l1=B2\.\\\|Dw\\\|\_\{2\}^\{2\}=\\sum\_\{l\}k\_\{l\}D\_\{l\}^\{2\}\\delta\_\{l\}^\{2\}=c\_\{0\}\\varepsilon^\{2\}\\sum\_\{l\}k\_\{l\}\\frac\{B^\{2\}\}\{\\operatorname\{cap\}\_\{l\}\}\\leq\\frac\{B^\{2\}\}\{L\}\\sum\_\{l\}1=B^\{2\}\.Forw,w′∈𝒲w,w^\{\\prime\}\\in\\mathcal\{W\}with Hamming distanceρ​\(w,w′\)\\rho\(w,w^\{\\prime\}\)between their sign patterns, the level\-homogeneity ofΣ\\Sigmaand the constant\-testing\-error allocationσ\(l\)2​δl2=c0​ε2\\sigma\_\{\(l\)\}^\{2\}\\delta\_\{l\}^\{2\}=c\_\{0\}\\varepsilon^\{2\}give

‖Σ1/2​\(w−w′\)‖22=∑lσ\(l\)2​\(2​δl\)2​ρl​\(w,w′\)=4​c0​ε2​ρ​\(w,w′\),KL​\(Pw⊗m∥Pw′⊗m\)=m2​σξ2​‖Σ1/2​\(w−w′\)‖22≤2​c0​K,K:=∑lkl,\\begin\{split\}\\\|\\Sigma^\{1/2\}\(w\-w^\{\\prime\}\)\\\|\_\{2\}^\{2\}&=\\sum\_\{l\}\\sigma\_\{\(l\)\}^\{2\}\\,\(2\\delta\_\{l\}\)^\{2\}\\,\\rho\_\{l\}\(w,w^\{\\prime\}\)=4c\_\{0\}\\varepsilon^\{2\}\\,\\rho\(w,w^\{\\prime\}\),\\\\ \\mathrm\{KL\}\\bigl\(P\_\{w\}^\{\\otimes m\}\\,\\big\\\|\\,P\_\{w^\{\\prime\}\}^\{\\otimes m\}\\bigr\)&=\\frac\{m\}\{2\\sigma\_\{\\xi\}^\{2\}\}\\,\\\|\\Sigma^\{1/2\}\(w\-w^\{\\prime\}\)\\\|\_\{2\}^\{2\}\\leq 2c\_\{0\}K,\\qquad K:=\\sum\_\{l\}k\_\{l\},\\end\{split\}so the weighted packing problem collapses to an unweighted one on\{±1\}K\\\{\\pm 1\\\}^\{K\}\. Suppose firstK≥32K\\geq 32\. By the Varshamov–Gilbert bound there is𝒲′⊆𝒲\\mathcal\{W\}^\{\\prime\}\\subseteq\\mathcal\{W\}withlog⁡\|𝒲′\|≥\(K/8\)​log⁡2\\log\|\\mathcal\{W\}^\{\\prime\}\|\\geq\(K/8\)\\log 2and pairwise Hamming distance at leastK/8K/8, hence pairwiseL2​\(Px\)L^\{2\}\(P\_\{x\}\)\-separation at leastc0​K​ε2/2c\_\{0\}K\\varepsilon^\{2\}/2\. Fano’s inequality then yields

ℛm≥c0​K​ε28​\(1−2​c0​K\+log⁡2\(K/8\)​log⁡2\)≥c016​K​ε2,\\mathcal\{R\}\_\{m\}\\ \\geq\\ \\frac\{c\_\{0\}K\\varepsilon^\{2\}\}\{8\}\\left\(1\-\\frac\{2c\_\{0\}K\+\\log 2\}\{\(K/8\)\\log 2\}\\right\)\\ \\geq\\ \\frac\{c\_\{0\}\}\{16\}\\,K\\varepsilon^\{2\},the last step by2​c0​K=\(K/32\)​log⁡22c\_\{0\}K=\(K/32\)\\log 2andlog⁡2≤\(K/32\)​log⁡2\\log 2\\leq\(K/32\)\\log 2atK≥32K\\geq 32\. Whencapl≥c0​L​ε2\\operatorname\{cap\}\_\{l\}\\geq c\_\{0\}L\\varepsilon^\{2\}the floor inklk\_\{l\}is at least half its argument, so

kl​ε2≥12​min⁡\{dl​ε2,capl/\(c0​L\)\}≥μl/\(2​max⁡\{1,c0​L\}\);k\_\{l\}\\varepsilon^\{2\}\\geq\\tfrac\{1\}\{2\}\\min\\bigl\\\{d\_\{l\}\\varepsilon^\{2\},\\ \\operatorname\{cap\}\_\{l\}/\(c\_\{0\}L\)\\bigr\\\}\\geq\\mu\_\{l\}/\(2\\max\\\{1,c\_\{0\}L\\\}\);whencapl<c0​L​ε2\\operatorname\{cap\}\_\{l\}<c\_\{0\}L\\varepsilon^\{2\}the level contributesμl≤capl<c0​L​ε2≤\(c0​L/32\)​K​ε2\\mu\_\{l\}\\leq\\operatorname\{cap\}\_\{l\}<c\_\{0\}L\\varepsilon^\{2\}\\leq\(c\_\{0\}L/32\)\\,K\\varepsilon^\{2\}\. Summing over the two kinds of level,

M≤\(2max\{1,c0L\}\+c0L2/32\)Kε2=:C1\(L\)Kε2,M\\leq\\bigl\(2\\max\\\{1,c\_\{0\}L\\\}\+c\_\{0\}L^\{2\}/32\\bigr\)K\\varepsilon^\{2\}=:C\_\{1\}\(L\)\\,K\\varepsilon^\{2\},soℛm≥\(c0/\(16​C1​\(L\)\)\)​M\\mathcal\{R\}\_\{m\}\\geq\\bigl\(c\_\{0\}/\(16\\,C\_\{1\}\(L\)\)\\bigr\)Mand \(i\) follows in this case\.

IfK<32K<32, then for everylleitherdl<32d\_\{l\}<32orcapl<32​c0​L​ε2\\operatorname\{cap\}\_\{l\}<32c\_\{0\}L\\varepsilon^\{2\}, so

μl≤32​max⁡\{1,c0​L\}​min⁡\{capl,ε2\}\\mu\_\{l\}\\leq 32\\max\\\{1,c\_\{0\}L\\\}\\min\\\{\\operatorname\{cap\}\_\{l\},\\varepsilon^\{2\}\\\}for everyllandM≤C′​\(L\)​maxl⁡min⁡\{capl,ε2\}M\\leq C^\{\\prime\}\(L\)\\max\_\{l\}\\min\\\{\\operatorname\{cap\}\_\{l\},\\varepsilon^\{2\}\\\}withC′​\(L\)=32​L​max⁡\{1,c0​L\}C^\{\\prime\}\(L\)=32L\\max\\\{1,c\_\{0\}L\\\}; a two\-point argument withw=0w=0againstw=δ​ejw=\\delta e\_\{j\}forj∈Vl∗j\\in V\_\{l^\{\*\}\}at the maximising level, withσ\(l∗\)2​δ2=min⁡\{capl∗,c0​ε2\}\\sigma\_\{\(l^\{\*\}\)\}^\{2\}\\delta^\{2\}=\\min\\\{\\operatorname\{cap\}\_\{l^\{\*\}\},c\_\{0\}\\varepsilon^\{2\}\\\}, is feasible, hasKL≤c0/2\\mathrm\{KL\}\\leq c\_\{0\}/2over themmsamples, and by Le Cam’s two\-point bound gives

ℛm≥c5​min⁡\{capl∗,c0​ε2\}≥c5​c0​min⁡\{capl∗,ε2\}≥\(c5​c0/C′​\(L\)\)​M,\\mathcal\{R\}\_\{m\}\\geq c\_\{5\}\\min\\\{\\operatorname\{cap\}\_\{l^\{\*\}\},c\_\{0\}\\varepsilon^\{2\}\\\}\\geq c\_\{5\}c\_\{0\}\\min\\\{\\operatorname\{cap\}\_\{l^\{\*\}\},\\varepsilon^\{2\}\\\}\\geq\\bigl\(c\_\{5\}c\_\{0\}/C^\{\\prime\}\(L\)\\bigr\)M,withc5c\_\{5\}an absolute constant\.

\(ii\) LetVS=⨁l∈SVlV\_\{S\}=\\bigoplus\_\{l\\in S\}V\_\{l\}withDS=dimVS=∑l∈SdlD\_\{S\}=\\dim V\_\{S\}=\\sum\_\{l\\in S\}d\_\{l\}, and letf^\\hat\{f\}be ordinary least squares on the coordinates ofVSV\_\{S\}, extended by zero on the remaining levels\. SinceΣ\\Sigmais diagonal, the omitted covariates are independent of the retained ones, so conditional on the retained design the omitted signal⟨wSc∗,xSc⟩\\langle w^\{\*\}\_\{S^\{c\}\},x\_\{S^\{c\}\}\\rangleis centred noise of variance∑l∉Sσ\(l\)2​‖wl∗‖22\\sum\_\{l\\notin S\}\\sigma\_\{\(l\)\}^\{2\}\\\|w^\{\*\}\_\{l\}\\\|\_\{2\}^\{2\}, independent across observations, and the regression ofyyonxSx\_\{S\}has noise varianceσξ2\+∑l∉Sσ\(l\)2​‖wl∗‖22\\sigma\_\{\\xi\}^\{2\}\+\\sum\_\{l\\notin S\}\\sigma\_\{\(l\)\}^\{2\}\\\|w^\{\*\}\_\{l\}\\\|\_\{2\}^\{2\}\. The excess risk decomposes into the approximation term∑l∉Sσ\(l\)2​‖wl∗‖22\\sum\_\{l\\notin S\}\\sigma\_\{\(l\)\}^\{2\}\\\|w^\{\*\}\_\{l\}\\\|\_\{2\}^\{2\}and the estimation term, which for Gaussian design andm≥2​DS\+4m\\geq 2D\_\{S\}\+4is at most2​\(σξ2\+∑l∉Sσ\(l\)2​‖wl∗‖22\)​DS/m2\\bigl\(\\sigma\_\{\\xi\}^\{2\}\+\\sum\_\{l\\notin S\}\\sigma\_\{\(l\)\}^\{2\}\\\|w^\{\*\}\_\{l\}\\\|\_\{2\}^\{2\}\\bigr\)D\_\{S\}/m\. The budget constraint givesσ\(l\)2​‖wl∗‖22≤capl\\sigma\_\{\(l\)\}^\{2\}\\\|w^\{\*\}\_\{l\}\\\|\_\{2\}^\{2\}\\leq\\operatorname\{cap\}\_\{l\}uniformly over the class, andDS/m≤1/2D\_\{S\}/m\\leq 1/2, so

ℛm≤∑l∉Scapl\+2​DS​ε2\+∑l∉Scapl≤3​∑l=0L−1min⁡\{capl,dl​ε2\},\\mathcal\{R\}\_\{m\}\\ \\leq\\ \\sum\_\{l\\notin S\}\\operatorname\{cap\}\_\{l\}\\ \+\\ 2\\,D\_\{S\}\\,\\varepsilon^\{2\}\\ \+\\ \\sum\_\{l\\notin S\}\\operatorname\{cap\}\_\{l\}\\ \\leq\\ 3\\sum\_\{l=0\}^\{L\-1\}\\min\\bigl\\\{\\operatorname\{cap\}\_\{l\},\\ d\_\{l\}\\varepsilon^\{2\}\\bigr\\\},usingcapl=μl\\operatorname\{cap\}\_\{l\}=\\mu\_\{l\}forl∉Sl\\notin Sanddl​ε2=μld\_\{l\}\\varepsilon^\{2\}=\\mu\_\{l\}forl∈Sl\\in S\. ∎

###### Remark 13\.

The lower bound of[Lem\.3](https://arxiv.org/html/2607.22757#Thmlem3)holds at every sample size; the upper bound is established form≥2​∑l∈Sdl\+4m\\geq 2\\sum\_\{l\\in S\}d\_\{l\}\+4, the regime in which every level worth estimating is estimable\. The bandm≍∑l∈Sdlm\\asymp\\sum\_\{l\\in S\}d\_\{l\}, in which a level crosses its estimability threshold and its worth\-estimating threshold at comparable sample sizes, is not covered by the argument above, and the window on which the separation of[Thm\.4\.1](https://arxiv.org/html/2607.22757#S4.Thmthm1)is established must be located outside it;[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4)carries this out\.

Throughout the remainder of this subsection the profiles are geometric and the allocation uniform:al=νL−1−l/Zaa\_\{l\}=\\nu^\{L\-1\-l\}/Z\_\{a\}andsl=μl/Zss\_\{l\}=\\mu^\{l\}/Z\_\{s\}with0<μ<ν<10<\\mu<\\nu<1andZa=∑k=0L−1νkZ\_\{a\}=\\sum\_\{k=0\}^\{L\-1\}\\nu^\{k\},Zs=∑k=0L−1μkZ\_\{s\}=\\sum\_\{k=0\}^\{L\-1\}\\mu^\{k\};dl=d/Ld\_\{l\}=d/Lfor everyll; and the calibrations‖w∗‖2=1\\\|w^\{\*\}\\\|\_\{2\}=1,tr⁡Σ=1\\operatorname\{tr\}\\Sigma=1, so thatσ\(l\)2=\(L/d\)​sl\\sigma\_\{\(l\)\}^\{2\}=\(L/d\)\\,s\_\{l\}\. The grade vector is level\-constant,g=\(g0,…,gL−1\)g=\(g\_\{0\},\\dots,g\_\{L\-1\}\), and is required to satisfy

1≤g0≤g1≤⋯≤gL−1,gl\+1gl≤\(ν​μ\)−1/4\(0≤l≤L−2\),1\\leq g\_\{0\}\\leq g\_\{1\}\\leq\\cdots\\leq g\_\{L\-1\},\\qquad\\frac\{g\_\{l\+1\}\}\{g\_\{l\}\}\\leq\(\\nu\\mu\)^\{\-1/4\}\\quad\(0\\leq l\\leq L\-2\),a condition met by the uniform gradesg=𝟏g=\\mathbf\{1\}, by the fourth\-root grades of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\) evaluated on the geometric profiles, and by their clipped formgC⋆g^\{\\star\}\_\{C\}of[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10), since clipping only decreases consecutive ratios\. Under these conventions the caps of[Lem\.3](https://arxiv.org/html/2607.22757#Thmlem3)are

caplunif=Ld​sl,caplgr=Ld​sl​gl2​∑k=0L−1ak​gk−2,ε2=σξ2m\.\\operatorname\{cap\}\_\{l\}^\{\\mathrm\{unif\}\}=\\frac\{L\}\{d\}\\,s\_\{l\},\\qquad\\operatorname\{cap\}\_\{l\}^\{\\mathrm\{gr\}\}=\\frac\{L\}\{d\}\\,s\_\{l\}\\,g\_\{l\}^\{2\}\\sum\_\{k=0\}^\{L\-1\}a\_\{k\}g\_\{k\}^\{\-2\},\\qquad\\varepsilon^\{2\}=\\frac\{\\sigma\_\{\\xi\}^\{2\}\}\{m\}\.
###### Lemma 4\.

Setr=μ/ν∈\(0,1\)r=\\sqrt\{\\mu/\\nu\}\\in\(0,1\),

msat=d0​σξ2cap0unif=d2​σξ2L2​s0,W=\{m∈ℕ:4≤m≤12​msat\}\.m\_\{\\mathrm\{sat\}\}=\\frac\{d\_\{0\}\\,\\sigma\_\{\\xi\}^\{2\}\}\{\\operatorname\{cap\}\_\{0\}^\{\\mathrm\{unif\}\}\}=\\frac\{d^\{2\}\\,\\sigma\_\{\\xi\}^\{2\}\}\{L^\{2\}\\,s\_\{0\}\},\\qquad W=\\bigl\\\{m\\in\\mathbb\{N\}:4\\leq m\\leq\\tfrac\{1\}\{2\}\\,m\_\{\\mathrm\{sat\}\}\\bigr\\\}\.Then there is an absolute constantc4∈\(0,1\)c\_\{4\}\\in\(0,1\)such that:

1. \(i\)\(Non\-emptiness and width\.\)W≠∅W\\neq\\emptysetwheneverd≥3​L/σξd\\geq 3L/\\sigma\_\{\\xi\}, and\|W\|=Θ​\(d2​σξ2/L2\)\|W\|=\\Theta\\bigl\(d^\{2\}\\sigma\_\{\\xi\}^\{2\}/L^\{2\}\\bigr\)asd/L→∞d/L\\to\\inftyat fixed\(μ,ν,σξ\)\(\\mu,\\nu,\\sigma\_\{\\xi\}\); in the joint scalingd,m→∞d,m\\to\\inftythe window widens, as[Rem\.12](https://arxiv.org/html/2607.22757#Thmrem12)requires of the regime\.
2. \(ii\)\(Full saturation\.\) For everym∈Wm\\in W, every levelll, and both classes,capl≤12​dl​ε2\\operatorname\{cap\}\_\{l\}\\leq\\tfrac\{1\}\{2\}\\,d\_\{l\}\\,\\varepsilon^\{2\}; in particular the setSSof[Lem\.3](https://arxiv.org/html/2607.22757#Thmlem3)is empty for both classes and the regime restriction of[Rem\.13](https://arxiv.org/html/2607.22757#Thmrem13)is vacuous onWW\.
3. \(iii\)\(Risk pinning\.\) For each class and everym∈Wm\\in W, c4​maxl⁡capl≤ℛm​\(ℱ​\(B,D\)\)≤maxl⁡capl,c\_\{4\}\\,\\max\_\{l\}\\operatorname\{cap\}\_\{l\}\\;\\leq\\;\\mathcal\{R\}\_\{m\}\\bigl\(\\mathcal\{F\}\(B,D\)\\bigr\)\\;\\leq\\;\\max\_\{l\}\\operatorname\{cap\}\_\{l\},the upper bound attained by the zero estimator, whose worst\-case risk over the ellipsoid\{‖D​w‖2≤B\}\\\{\\\|Dw\\\|\_\{2\}\\leq B\\\}is exactlymaxl⁡capl\\max\_\{l\}\\operatorname\{cap\}\_\{l\}\.
4. \(iv\)\(Ratio\.\) Under the grade condition both cap sequences are geometric with ratio at mostrr, both maxima sit atl=0l=0, and uniformly form∈Wm\\in W, c4​\(1−r\)​Λ​\(g\)≤ℛm​\(ℱgr\)ℛm​\(ℱunif\)≤1c4​\(1−μ\)​Λ​\(g\),c\_\{4\}\\,\(1\-r\)\\;\\Lambda\(g\)\\;\\leq\\;\\frac\{\\mathcal\{R\}\_\{m\}\\bigl\(\\mathcal\{F\}^\{\\mathrm\{gr\}\}\\bigr\)\}\{\\mathcal\{R\}\_\{m\}\\bigl\(\\mathcal\{F\}^\{\\mathrm\{unif\}\}\\bigr\)\}\\;\\leq\\;\\frac\{1\}\{c\_\{4\}\\,\(1\-\\mu\)\}\\;\\Lambda\(g\),withΛ​\(g\)=\(∑lal​gl−2\)​\(∑lsl​gl2\)\\Lambda\(g\)=\\bigl\(\\sum\_\{l\}a\_\{l\}g\_\{l\}^\{\-2\}\\bigr\)\\bigl\(\\sum\_\{l\}s\_\{l\}g\_\{l\}^\{2\}\\bigr\)the gain of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\. The constants depend only on\(μ,ν\)\(\\mu,\\nu\)and are uniform inLL,dd, andm∈Wm\\in W\. At the fourth\-root grades the ratio isΘμ,ν​\(BC​\(a,s\)2\)=Θμ,ν​\(νL\)\\Theta\_\{\\mu,\\nu\}\\bigl\(\\mathrm\{BC\}\(a,s\)^\{2\}\\bigr\)=\\Theta\_\{\\mu,\\nu\}\(\\nu^\{L\}\), and atgC⋆g^\{\\star\}\_\{C\}it isΘμ,ν​\(ΛC⋆\)\\Theta\_\{\\mu,\\nu\}\(\\Lambda^\{\\star\}\_\{C\}\)\.
5. \(v\)\(Parametric floor\.\) Form≥max⁡\{2​d\+4,d0​σξ2/minl⁡capl\}m\\geq\\max\\bigl\\\{2d\+4,\\;d\_\{0\}\\sigma\_\{\\xi\}^\{2\}/\\min\_\{l\}\\operatorname\{cap\}\_\{l\}\\bigr\\\}the setSSis full for both classes, both risks areΘL​\(d​ε2\)\\Theta\_\{L\}\(d\\,\\varepsilon^\{2\}\)by[Lem\.3](https://arxiv.org/html/2607.22757#Thmlem3), and the ratio isΘL​\(1\)\\Theta\_\{L\}\(1\): the graded gain concentrates on the windowWW, where the budget is active\. On the intermediate band\(12​msat,d0​σξ2/minl⁡capl\)\\bigl\(\\tfrac\{1\}\{2\}m\_\{\\mathrm\{sat\}\},\\,d\_\{0\}\\sigma\_\{\\xi\}^\{2\}/\\min\_\{l\}\\operatorname\{cap\}\_\{l\}\\bigr\)no two\-sided comparison againstΛ​\(g\)\\Lambda\(g\)is asserted\.

###### Proof\.

Sinceggis nondecreasing,g02​∑kak​gk−2=∑kak​\(g0/gk\)2≤∑kak=1g\_\{0\}^\{2\}\\sum\_\{k\}a\_\{k\}g\_\{k\}^\{\-2\}=\\sum\_\{k\}a\_\{k\}\(g\_\{0\}/g\_\{k\}\)^\{2\}\\leq\\sum\_\{k\}a\_\{k\}=1, so

maxl⁡caplgr=cap0gr=Ld​s0​g02​∑kak​gk−2≤Ld​s0=cap0unif,\\max\_\{l\}\\operatorname\{cap\}\_\{l\}^\{\\mathrm\{gr\}\}\\;=\\;\\operatorname\{cap\}\_\{0\}^\{\\mathrm\{gr\}\}\\;=\\;\\frac\{L\}\{d\}\\,s\_\{0\}\\,g\_\{0\}^\{2\}\\sum\_\{k\}a\_\{k\}g\_\{k\}^\{\-2\}\\;\\leq\\;\\frac\{L\}\{d\}\\,s\_\{0\}\\;=\\;\\operatorname\{cap\}\_\{0\}^\{\\mathrm\{unif\}\},where the first equality holds becausesl\+1​gl\+12/\(sl​gl2\)=μ​\(gl\+1/gl\)2≤μ​\(ν​μ\)−1/2=r<1s\_\{l\+1\}g\_\{l\+1\}^\{2\}/\(s\_\{l\}g\_\{l\}^\{2\}\)=\\mu\\,\(g\_\{l\+1\}/g\_\{l\}\)^\{2\}\\leq\\mu\\,\(\\nu\\mu\)^\{\-1/2\}=r<1: both cap sequences are geometric with ratio at mostrr\(ratioμ≤r\\mu\\leq rin the uniform case\) and are maximised atl=0l=0\.

\(ii\) Form≤12​msatm\\leq\\tfrac\{1\}\{2\}m\_\{\\mathrm\{sat\}\}we havecap0unif≤12​d0​ε2\\operatorname\{cap\}\_\{0\}^\{\\mathrm\{unif\}\}\\leq\\tfrac\{1\}\{2\}d\_\{0\}\\varepsilon^\{2\}by the definition ofmsatm\_\{\\mathrm\{sat\}\}, and every other cap of either class is at mostcap0unif\\operatorname\{cap\}\_\{0\}^\{\\mathrm\{unif\}\}by the display above, whiledl=d0d\_\{l\}=d\_\{0\}for everyll\.

\(i\)W≠∅W\\neq\\emptysetrequiresmsat≥8m\_\{\\mathrm\{sat\}\}\\geq 8, that isd2≥8​L2​s0/σξ2d^\{2\}\\geq 8L^\{2\}s\_\{0\}/\\sigma\_\{\\xi\}^\{2\}, ands0=\(1−μ\)/\(1−μL\)≤1s\_\{0\}=\(1\-\\mu\)/\(1\-\\mu^\{L\}\)\\leq 1makesd≥3​L/σξd\\geq 3L/\\sigma\_\{\\xi\}sufficient\. The width claim is immediate from the formula formsatm\_\{\\mathrm\{sat\}\}\.

\(iii\) Upper bound: the zero estimator has excess risk‖Σ1/2​w‖22=∑jσj2​wj2\\\|\\Sigma^\{1/2\}w\\\|\_\{2\}^\{2\}=\\sum\_\{j\}\\sigma\_\{j\}^\{2\}w\_\{j\}^\{2\}, whose supremum over\{‖D​w‖2≤B\}\\\{\\\|Dw\\\|\_\{2\}\\leq B\\\}ismaxj⁡σj2​B2/Dj2=maxl⁡capl\\max\_\{j\}\\sigma\_\{j\}^\{2\}B^\{2\}/D\_\{j\}^\{2\}=\\max\_\{l\}\\operatorname\{cap\}\_\{l\}, attained on the maximising coordinate\. Lower bound: run the packing of the proof of[Lem\.3](https://arxiv.org/html/2607.22757#Thmlem3)\(i\) on the single levell=0l=0, withk0=min⁡\{d0,⌊cap0/\(c0​ε2\)⌋\}k\_\{0\}=\\min\\bigl\\\{d\_\{0\},\\ \\lfloor\\operatorname\{cap\}\_\{0\}/\(c\_\{0\}\\varepsilon^\{2\}\)\\rfloor\\bigr\\\}andσ\(0\)2​δ02=c0​ε2\\sigma\_\{\(0\)\}^\{2\}\\delta\_\{0\}^\{2\}=c\_\{0\}\\varepsilon^\{2\}; feasibility uses the full budget at one level,k0​D02​δ02=c0​ε2​k0​B2/cap0≤B2k\_\{0\}D\_\{0\}^\{2\}\\delta\_\{0\}^\{2\}=c\_\{0\}\\varepsilon^\{2\}k\_\{0\}B^\{2\}/\\operatorname\{cap\}\_\{0\}\\leq B^\{2\}\. Ifk0≥32k\_\{0\}\\geq 32, then eitherk0=d0k\_\{0\}=d\_\{0\}, in which casek0​ε2=d0​ε2≥2​cap0k\_\{0\}\\varepsilon^\{2\}=d\_\{0\}\\varepsilon^\{2\}\\geq 2\\operatorname\{cap\}\_\{0\}by \(ii\), ork0=⌊cap0/\(c0​ε2\)⌋≥32k\_\{0\}=\\lfloor\\operatorname\{cap\}\_\{0\}/\(c\_\{0\}\\varepsilon^\{2\}\)\\rfloor\\geq 32, in which casecap0≥32​c0​ε2\\operatorname\{cap\}\_\{0\}\\geq 32\\,c\_\{0\}\\varepsilon^\{2\}andk0≥cap0/\(2​c0​ε2\)k\_\{0\}\\geq\\operatorname\{cap\}\_\{0\}/\(2c\_\{0\}\\varepsilon^\{2\}\), sok0​ε2≥cap0/\(2​c0\)≥2​cap0k\_\{0\}\\varepsilon^\{2\}\\geq\\operatorname\{cap\}\_\{0\}/\(2c\_\{0\}\)\\geq 2\\operatorname\{cap\}\_\{0\}; in either case Fano givesℛm≥\(c0/16\)​k0​ε2≥\(c0/8\)​cap0\\mathcal\{R\}\_\{m\}\\geq\(c\_\{0\}/16\)\\,k\_\{0\}\\varepsilon^\{2\}\\geq\(c\_\{0\}/8\)\\,\\operatorname\{cap\}\_\{0\}\. Ifk0<32k\_\{0\}<32, then eithercap0<32​c0​ε2\\operatorname\{cap\}\_\{0\}<32\\,c\_\{0\}\\varepsilon^\{2\}ord0<32d\_\{0\}<32, and in the latter casecap0≤12​d0​ε2<16​ε2\\operatorname\{cap\}\_\{0\}\\leq\\tfrac\{1\}\{2\}d\_\{0\}\\varepsilon^\{2\}<16\\,\\varepsilon^\{2\}by \(ii\); in both casesmin⁡\{cap0,c0​ε2\}≥\(c0/32\)​cap0\\min\\\{\\operatorname\{cap\}\_\{0\},c\_\{0\}\\varepsilon^\{2\}\\\}\\geq\(c\_\{0\}/32\)\\,\\operatorname\{cap\}\_\{0\}, and the two\-point argument of[Lem\.3](https://arxiv.org/html/2607.22757#Thmlem3)\(i\) at level0with amplitudeσ\(0\)2​δ2=min⁡\{cap0,c0​ε2\}\\sigma\_\{\(0\)\}^\{2\}\\delta^\{2\}=\\min\\\{\\operatorname\{cap\}\_\{0\},c\_\{0\}\\varepsilon^\{2\}\\\}givesℛm≥c5​min⁡\{cap0,c0​ε2\}≥\(c5​c0/32\)​cap0\\mathcal\{R\}\_\{m\}\\geq c\_\{5\}\\min\\\{\\operatorname\{cap\}\_\{0\},c\_\{0\}\\varepsilon^\{2\}\\\}\\geq\(c\_\{5\}c\_\{0\}/32\)\\,\\operatorname\{cap\}\_\{0\}\. Takec4c\_\{4\}the smaller of the two constants\.

\(iv\) Dividing the two\-sided bounds of \(iii\),

c4​cap0grcap0unif≤ℛm​\(ℱgr\)ℛm​\(ℱunif\)≤1c4​cap0grcap0unif,cap0grcap0unif=g02​∑kak​gk−2\.c\_\{4\}\\,\\frac\{\\operatorname\{cap\}\_\{0\}^\{\\mathrm\{gr\}\}\}\{\\operatorname\{cap\}\_\{0\}^\{\\mathrm\{unif\}\}\}\\;\\leq\\;\\frac\{\\mathcal\{R\}\_\{m\}\(\\mathcal\{F\}^\{\\mathrm\{gr\}\}\)\}\{\\mathcal\{R\}\_\{m\}\(\\mathcal\{F\}^\{\\mathrm\{unif\}\}\)\}\\;\\leq\\;\\frac\{1\}\{c\_\{4\}\}\\,\\frac\{\\operatorname\{cap\}\_\{0\}^\{\\mathrm\{gr\}\}\}\{\\operatorname\{cap\}\_\{0\}^\{\\mathrm\{unif\}\}\},\\qquad\\frac\{\\operatorname\{cap\}\_\{0\}^\{\\mathrm\{gr\}\}\}\{\\operatorname\{cap\}\_\{0\}^\{\\mathrm\{unif\}\}\}=g\_\{0\}^\{2\}\\sum\_\{k\}a\_\{k\}g\_\{k\}^\{\-2\}\.It remains to compareg02​∑kak​gk−2g\_\{0\}^\{2\}\\sum\_\{k\}a\_\{k\}g\_\{k\}^\{\-2\}withΛ​\(g\)\\Lambda\(g\)\. Since∑lsl=1\\sum\_\{l\}s\_\{l\}=1,

g02​∑kak​gk−2=Λ​\(g\)⋅g02∑lsl​gl2,s0​g02≤∑lsl​gl2≤s0​g021−r,\\begin\{split\}g\_\{0\}^\{2\}\\sum\_\{k\}a\_\{k\}g\_\{k\}^\{\-2\}\\;&=\\;\\Lambda\(g\)\\cdot\\frac\{g\_\{0\}^\{2\}\}\{\\sum\_\{l\}s\_\{l\}g\_\{l\}^\{2\}\},\\\\ s\_\{0\}\\,g\_\{0\}^\{2\}\\;\\leq\\;\\sum\_\{l\}s\_\{l\}g\_\{l\}^\{2\}\\;\\leq\\;\\frac\{s\_\{0\}\\,g\_\{0\}^\{2\}\}\{1\-r\},\\end\{split\}the second line because the summandssl​gl2s\_\{l\}g\_\{l\}^\{2\}are geometric with ratio at mostrr\. Henceg02/∑lsl​gl2g\_\{0\}^\{2\}/\\sum\_\{l\}s\_\{l\}g\_\{l\}^\{2\}lies in\[1−r,1/s0\]⊆\[1−r,1/\(1−μ\)\]\[\\,1\-r,\\;1/s\_\{0\}\\,\]\\subseteq\[\\,1\-r,\\;1/\(1\-\\mu\)\\,\], which gives the display\. For the fourth\-root grades,[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)givesΛ​\(g⋆\)=BC​\(a,s\)2=Θ​\(νL\)\\Lambda\(g^\{\\star\}\)=\\mathrm\{BC\}\(a,s\)^\{2\}=\\Theta\(\\nu^\{L\}\), andΛ​\(gC⋆\)=ΛC⋆\\Lambda\(g^\{\\star\}\_\{C\}\)=\\Lambda^\{\\star\}\_\{C\}by[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\.

\(v\) For suchmmevery level of both classes satisfiesdl​ε2≤capld\_\{l\}\\varepsilon^\{2\}\\leq\\operatorname\{cap\}\_\{l\}, soSSis full, the envelope of[Lem\.3](https://arxiv.org/html/2607.22757#Thmlem3)evaluates toΘL​\(∑ldl​ε2\)=ΘL​\(d​ε2\)\\Theta\_\{L\}\(\\sum\_\{l\}d\_\{l\}\\varepsilon^\{2\}\)=\\Theta\_\{L\}\(d\\,\\varepsilon^\{2\}\)for both classes, and the ratio is bounded above and below byC​\(L\)/c​\(L\)C\(L\)/c\(L\)and its inverse: past the saturation point the budget is inactive for both classes and the comparison localises toWW\. The band assertion is a scope statement, not a theorem, and records where the separation is to be read off\. ∎

###### Theorem 4\.1\.

Assume the setting of this subsection: Gaussian level\-homogeneous design with squared loss, geometric profilesal=νL−1−l/Zaa\_\{l\}=\\nu^\{L\-1\-l\}/Z\_\{a\}andsl=μl/Zss\_\{l\}=\\mu^\{l\}/Z\_\{s\}with0<μ<ν<10<\\mu<\\nu<1, uniform allocationdl=d/Ld\_\{l\}=d/L, the calibrations‖w∗‖2=1\\\|w^\{\*\}\\\|\_\{2\}=1andtr⁡Σ=1\\operatorname\{tr\}\\Sigma=1, and a level\-constant grade vectorggsatisfying the monotonicity and ratio condition stated before[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4)\. Supposed≥3​L/σξd\\geq 3L/\\sigma\_\{\\xi\}, so that the windowWWof[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4)is nonempty\. Then there exist constants0<cμ,ν≤Cμ,ν0<c\_\{\\mu,\\nu\}\\leq C\_\{\\mu,\\nu\}depending only on\(μ,ν\)\(\\mu,\\nu\), uniform inLL,dd, andmm, such that for everym∈Wm\\in W,

cμ,ν​Λ​\(g\)≤ℛm​\(ℱgr\)ℛm​\(ℱunif\)≤Cμ,ν​Λ​\(g\),c\_\{\\mu,\\nu\}\\,\\Lambda\(g\)\\;\\leq\\;\\frac\{\\mathcal\{R\}\_\{m\}\\big\(\\mathcal\{F\}^\{\\mathrm\{gr\}\}\\big\)\}\{\\mathcal\{R\}\_\{m\}\\big\(\\mathcal\{F\}^\{\\mathrm\{unif\}\}\\big\)\}\\;\\leq\\;C\_\{\\mu,\\nu\}\\,\\Lambda\(g\),withΛ​\(g\)\\Lambda\(g\)the gain of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\. In particular:

1. \(i\)at the fourth\-root gradesg⋆g^\{\\star\}of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\) the ratio isΘμ,ν​\(BC​\(a,s\)2\)=Θμ,ν​\(νL\)\\Theta\_\{\\mu,\\nu\}\\big\(\\mathrm\{BC\}\(a,s\)^\{2\}\\big\)=\\Theta\_\{\\mu,\\nu\}\(\\nu^\{L\}\);
2. \(ii\)at the clipped gradesgC⋆g^\{\\star\}\_\{C\}of[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)the ratio isΘμ,ν​\(ΛC⋆\)\\Theta\_\{\\mu,\\nu\}\\big\(\\Lambda^\{\\star\}\_\{C\}\\big\), withΛC⋆<1\\Lambda^\{\\star\}\_\{C\}<1whenevera≠sa\\neq s\.

The gain[Eq\.14](https://arxiv.org/html/2607.22757#S4.E14)is therefore attained onWWup to constants depending only on\(μ,ν\)\(\\mu,\\nu\), over all estimators, and is a separation between the graded prior and its absence rather than a comparison of the upper bounds of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\.

###### Proof\.

The two\-sided bound is[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4)\(iv\), withcμ,ν=c4​\(1−r\)c\_\{\\mu,\\nu\}=c\_\{4\}\(1\-r\)andCμ,ν=\(c4​\(1−μ\)\)−1C\_\{\\mu,\\nu\}=\\big\(c\_\{4\}\(1\-\\mu\)\\big\)^\{\-1\}, wherec4c\_\{4\}is the constant of[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4)\(iii\) andr=μ/νr=\\sqrt\{\\mu/\\nu\}; both grade vectors named in \(i\) and \(ii\) satisfy the grade condition, as recorded before[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4), so the bound applies to them\.

The value

Λ​\(g⋆\)=BC​\(a,s\)2=Θ​\(νL\)\\Lambda\(g^\{\\star\}\)=\\mathrm\{BC\}\(a,s\)^\{2\}=\\Theta\(\\nu^\{L\}\)is[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3), andΛ​\(gC⋆\)=ΛC⋆\\Lambda\(g^\{\\star\}\_\{C\}\)=\\Lambda^\{\\star\}\_\{C\}is[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\. For the strict inequality in \(ii\),a≠sa\\neq sgives𝒬\+≠∅\\mathcal\{Q\}\_\{\+\}\\neq\\emptysetby[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(iii\); by[Rem\.9](https://arxiv.org/html/2607.22757#Thmrem9)a level\-constant grade increasing inllsatisfies the criterion of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(iv\), soε​v∈𝒬\+\\varepsilon v\\in\\mathcal\{Q\}\_\{\+\}for all smallε\>0\\varepsilon\>0along such a directionvv, and for smallε\\varepsilonthe pointε​v\\varepsilon vlies in the box𝒬C\\mathcal\{Q\}\_\{C\}as well; hence𝒬C\\mathcal\{Q\}\_\{C\}meets𝒬\+\\mathcal\{Q\}\_\{\+\}and the minimum over it is strictly below11\. Nonemptiness ofWWatd≥3​L/σξd\\geq 3L/\\sigma\_\{\\xi\}is[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4)\(i\)\. ∎

[Thm\.4\.1](https://arxiv.org/html/2607.22757#S4.Thmthm1)establishes the separation of[Conj\.2](https://arxiv.org/html/2607.22757#Thmcon2)in the regime the program occupies\. Level\-stratified geometric profiles are the hypothesis of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)and the setting of every planned configuration, andg⋆g^\{\\star\}andgC⋆g^\{\\star\}\_\{C\}are the grades those configurations carry; the loss is squared, the curved case in which minimax risk is classically posed\. The statement is a risk ratio at each sample size of the window, which is the form[Rem\.12](https://arxiv.org/html/2607.22757#Thmrem12)shows to be well posed: past saturation both risks sit on the parametric floor and the ratio isΘL​\(1\)\\Theta\_\{L\}\(1\)by[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4)\(v\), so the theorem localises the gain toWWrather than diminishing it, andWWis the token\-limited regime the framework is built for\. What remains open is the general\-profile, Lipschitz\-loss case, and[Conj\.2](https://arxiv.org/html/2607.22757#Thmcon2)states it\.

### 4\.4\.Graded Positional Encoding

Letl​\(j\)∈\{0,…,L−1\}l\(j\)\\in\\\{0,\\dots,L\-1\\\}denote the level of global dimensionjj, andi​\(j\)i\(j\)its local index withinVl​\(j\)V\_\{l\(j\)\}\. The graded positional encoding scales the standard sinusoidal encoding[Eq\.5](https://arxiv.org/html/2607.22757#S2.E5)by the grade of the dimension it occupies:

\(16\)PE′​\(t,j\)=λql​\(j\),i​\(j\)​\{sin⁡\(t100002​⌊i​\(j\)/2⌋/dl​\(j\)\),i​\(j\)​even,cos⁡\(t100002​⌊i​\(j\)/2⌋/dl​\(j\)\),i​\(j\)​odd\.\\mathrm\{PE\}^\{\\prime\}\(t,j\)=\\lambda^\{q\_\{l\(j\),\\,i\(j\)\}\}\\begin\{cases\}\\sin\\\!\\left\(\\dfrac\{t\}\{10000^\{\\,2\\lfloor i\(j\)/2\\rfloor/d\_\{l\(j\)\}\}\}\\right\),&i\(j\)\\text\{ even\},\\\\\[8\.61108pt\] \\cos\\\!\\left\(\\dfrac\{t\}\{10000^\{\\,2\\lfloor i\(j\)/2\\rfloor/d\_\{l\(j\)\}\}\}\\right\),&i\(j\)\\text\{ odd\}\.\\end\{cases\}
The wavelength is computed within the level, so that eachVlV\_\{l\}carries a complete positional basis at its own resolution rather than a fragment of a global one\. The input to the first layer isX0=E′\+PE′∈ℝn×dX\_\{0\}=E^\{\\prime\}\+\\mathrm\{PE\}^\{\\prime\}\\in\\mathbb\{R\}^\{n\\times d\}\.

### 4\.5\.The Graded Forward Pass

The stack appliesKKpre\-norm blocks in the ordering of[Sec\.2\.2](https://arxiv.org/html/2607.22757#S2.SS2), with EG\-MHSA in place of MHSA:

\(17\)Xℓ′=Xℓ−1\+EG​\-​MHSAℓ​\(LN​\(Xℓ−1\)\),Xℓ=Xℓ′\+FFNℓ​\(LN​\(Xℓ′\)\),\\begin\{split\}X\_\{\\ell\}^\{\\prime\}&=X\_\{\\ell\-1\}\+\\mathrm\{EG\\text\{\-\}MHSA\}\_\{\\ell\}\\big\(\\mathrm\{LN\}\(X\_\{\\ell\-1\}\)\\big\),\\\\ X\_\{\\ell\}&=X\_\{\\ell\}^\{\\prime\}\+\\mathrm\{FFN\}\_\{\\ell\}\\big\(\\mathrm\{LN\}\(X\_\{\\ell\}^\{\\prime\}\)\\big\),\\end\{split\}forℓ=1,…,K\\ell=1,\\dots,K, with layer normalisation as in\[ba2016layer\]\.

The feed\-forward block admits a level\-decomposed variant,

FFNℓ​\(X\)=∑l=0L−1GeGLUl​\(X​Pl\)​Pl⊤,\\mathrm\{FFN\}\_\{\\ell\}\(X\)=\\sum\_\{l=0\}^\{L\-1\}\\mathrm\{GeGLU\}\_\{l\}\\\!\\left\(XP\_\{l\}\\right\)P\_\{l\}^\{\\top\},using the gated linear unit of\[shazeer2020glu\]\. We record what this costs\. The decomposed block is block\-diagonal with respect to⨁lVl\\bigoplus\_\{l\}V\_\{l\}and therefore performs*no*cross\-level mixing; all interaction between levels must then occur in attention\. This is a substantive architectural restriction and not merely a reparameterization, and it is in tension with the informal reading of the filtrationV≤l=⨁l′≤lVl′V\_\{\\leq l\}=\\bigoplus\_\{l^\{\\prime\}\\leq l\}V\_\{l^\{\\prime\}\}as a channel by which lower levels feed higher ones\. A filtration is a nested family of subspaces; on its own it enforces no propagation\. Directed level\-to\-level transport requires block\-*triangular*mapsϕl′←l:Vl→Vl′\\phi\_\{l^\{\\prime\}\\leftarrow l\}\\colon V\_\{l\}\\to V\_\{l^\{\\prime\}\}rather than block\-diagonal ones; the morphic structure of\[sh\-111\]supplies them, and MLGE does not\.

At positionttthe final hidden statezt∈ℝdz\_\{t\}\\in\\mathbb\{R\}^\{d\}is mapped to vocabulary logits byWvoc⊤​zt∈ℝ\|𝒱\|W\_\{\\mathrm\{voc\}\}^\{\\top\}z\_\{t\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\}, withWvoc∈ℝd×\|𝒱\|W\_\{\\mathrm\{voc\}\}\\in\\mathbb\{R\}^\{d\\times\|\\mathcal\{V\}\|\}, and

\(18\)p​\(yt∣y<t\)=softmax​\(Wvoc⊤​zt\+κ​qout\),qout∈ℚ≥0\|𝒱\|,κ\>0,p\(y\_\{t\}\\mid y\_\{<t\}\)=\\mathrm\{softmax\}\\\!\\left\(W\_\{\\mathrm\{voc\}\}^\{\\top\}z\_\{t\}\+\\kappa\\,q\_\{\\mathrm\{out\}\}\\right\),\\qquad q\_\{\\mathrm\{out\}\}\\in\\mathbb\{Q\}\_\{\\geq 0\}^\{\|\\mathcal\{V\}\|\},\\ \\kappa\>0,which multiplies the unnormalised probability of tokenvvbyeκ​qout,ve^\{\\kappa q\_\{\\mathrm\{out\},v\}\}and is therefore monotone in the grade, uniformly inztz\_\{t\}\.

## 5\.Training Graded LLMs

Training a GLLM retains the autoregressive next\-token objective of classical LLMs\[radford2019language\]and modifies the weight each token carries in it\. The standard objective is the unweighted negative log\-likelihood of[Eq\.3](https://arxiv.org/html/2607.22757#S2.E3)\. The graded objective attaches to positiontta weightλqt\\lambda^\{q\_\{t\}\}recording the structural salience of that position, reweighting the summands of the decomposition[Eq\.4](https://arxiv.org/html/2607.22757#S2.E4)\. This section defines the objective, establishes what it converges to, and states precisely which of the framework’s claims it can and cannot support\.

We state the conclusion at the outset, because the section’s structure follows from it\. Grading is a*statistical*device, and its whole force is spent on the resource that is scarce\. It restricts the effective hypothesis class in a direction aligned with hierarchical targets, and the reduction in the data required to reach a given generalisation level is a consequence of that restriction, mediated by[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)and located exactly by[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\. It buys nothing in step count: exponential rescaling worsens the conditioning of the objective, and we claim no acceleration\. This distinction is central to the claim\. Extra optimisation steps are cheap while extra high\-quality tokens are not, so a device that spends conditioning to buy sample efficiency spends a cheap resource on an expensive one;[Rem\.16](https://arxiv.org/html/2607.22757#Thmrem16)prices the two sides with the same constant, the step penalty at the clipCCand the token purchase atC2C^\{2\}\.[Sec\.8](https://arxiv.org/html/2607.22757#S8)measures the two separately for exactly that reason\.

### 5\.1\.The Graded Objective and What It Minimises

The base objective weights the per\-token log\-loss exponentially by a grade:

\(19\)ℒgrade​\(y,p^;q\)=−∑t=1nλqt​log⁡p^​\(yt∣y<t\),q=\(q1,…,qn\)∈ℚ≥0n,\\mathcal\{L\}\_\{\\mathrm\{grade\}\}\(y,\\hat\{p\};q\)=\-\\sum\_\{t=1\}^\{n\}\\lambda^\{q\_\{t\}\}\\log\\hat\{p\}\(y\_\{t\}\\mid y\_\{<t\}\),\\qquad q=\(q\_\{1\},\\dots,q\_\{n\}\)\\in\\mathbb\{Q\}\_\{\\geq 0\}^\{n\},withp^\\hat\{p\}the model’s predicted conditional from[Eq\.18](https://arxiv.org/html/2607.22757#S4.E18)andλ\>1\\lambda\>1the grading base fixed in[Sec\.3](https://arxiv.org/html/2607.22757#S3)\.

Everything depends on whatqtq\_\{t\}is allowed to depend on, and the following proposition draws the line\.

###### Proposition 9\.

Letp⋆\(⋅∣y<t\)p\_\{\\star\}\(\\cdot\\mid y\_\{<t\}\)denote the true conditional and letp^\\hat\{p\}range over all ofΔ\|𝒱\|−1\\Delta^\{\|\\mathcal\{V\}\|\-1\}\.

1. \(1\)Ifqt=q​\(y<t\)q\_\{t\}=q\(y\_\{<t\}\)depends only on the context, then the population minimizer of[Eq\.19](https://arxiv.org/html/2607.22757#S5.E19)isp^⋆\(⋅∣y<t\)=p⋆\(⋅∣y<t\)\\hat\{p\}\_\{\\star\}\(\\cdot\\mid y\_\{<t\}\)=p\_\{\\star\}\(\\cdot\\mid y\_\{<t\}\)\. The objective is a reweighting of contexts and is consistent\.
2. \(2\)Ifqt=q​\(yt\)q\_\{t\}=q\(y\_\{t\}\)depends only on the realised target, then the population minimizer is the exponentially tilted conditional p^⋆​\(v∣y<t\)=λq​\(v\)​p⋆​\(v∣y<t\)∑u∈𝒱λq​\(u\)​p⋆​\(u∣y<t\)≠p⋆​\(v∣y<t\),\\hat\{p\}\_\{\\star\}\(v\\mid y\_\{<t\}\)=\\frac\{\\lambda^\{q\(v\)\}\\,p\_\{\\star\}\(v\\mid y\_\{<t\}\)\}\{\\sum\_\{u\\in\\mathcal\{V\}\}\\lambda^\{q\(u\)\}\\,p\_\{\\star\}\(u\\mid y\_\{<t\}\)\}\\;\\neq\\;p\_\{\\star\}\(v\\mid y\_\{<t\}\),and the objective is inconsistent forp⋆p\_\{\\star\}; the tilt is the same at every context\.
3. \(3\)Ifqt=q​\(yt,y<t\)q\_\{t\}=q\(y\_\{t\},y\_\{<t\}\)depends on the realised target*and*its context — a constituency parse depth, for instance — then the population minimizer is the context\-dependent tilted conditional p^⋆​\(v∣y<t\)=λq​\(v,y<t\)​p⋆​\(v∣y<t\)∑u∈𝒱λq​\(u,y<t\)​p⋆​\(u∣y<t\)≠p⋆​\(v∣y<t\),\\hat\{p\}\_\{\\star\}\(v\\mid y\_\{<t\}\)=\\frac\{\\lambda^\{q\(v,\\,y\_\{<t\}\)\}\\,p\_\{\\star\}\(v\\mid y\_\{<t\}\)\}\{\\sum\_\{u\\in\\mathcal\{V\}\}\\lambda^\{q\(u,\\,y\_\{<t\}\)\}\\,p\_\{\\star\}\(u\\mid y\_\{<t\}\)\}\\;\\neq\\;p\_\{\\star\}\(v\\mid y\_\{<t\}\),and the objective is inconsistent forp⋆p\_\{\\star\}; the tilt varies with the context\.

###### Proof\.

For fixed contexty<ty\_\{<t\}, writewv=λqtw\_\{v\}=\\lambda^\{q\_\{t\}\}for the weight attached to the outcomevvand minimise−∑vwv​p⋆​\(v\)​log⁡p^​\(v\)\-\\sum\_\{v\}w\_\{v\}\\,p\_\{\\star\}\(v\)\\log\\hat\{p\}\(v\)over the simplex\. The Lagrangian stationarity condition−wv​p⋆​\(v\)/p^​\(v\)\+μ=0\-w\_\{v\}p\_\{\\star\}\(v\)/\\hat\{p\}\(v\)\+\\mu=0givesp^​\(v\)=wv​p⋆​\(v\)/μ\\hat\{p\}\(v\)=w\_\{v\}p\_\{\\star\}\(v\)/\\mu, andμ\\muis fixed by normalisation, yieldingp^​\(v\)=wv​p⋆​\(v\)/∑uwu​p⋆​\(u\)\\hat\{p\}\(v\)=w\_\{v\}p\_\{\\star\}\(v\)/\\sum\_\{u\}w\_\{u\}p\_\{\\star\}\(u\)\. In case \(1\)wv≡λq​\(y<t\)w\_\{v\}\\equiv\\lambda^\{q\(y\_\{<t\}\)\}is constant invvand cancels\. In case \(2\)wv=λq​\(v\)w\_\{v\}=\\lambda^\{q\(v\)\}does not, and is the same function ofvvat every context\. In case \(3\)wv=λq​\(v,y<t\)w\_\{v\}=\\lambda^\{q\(v,\\,y\_\{<t\}\)\}does not cancel either, and the resulting tilt is a different function ofvvat each context\. ∎

###### Remark 14\.

Case \(2\) of[Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)is a constraint that the architecture can match\. Write the graded softmax[Eq\.18](https://arxiv.org/html/2607.22757#S4.E18)asp^​\(v∣y<t\)∝eκ​qout,v​s​\(v∣y<t\)\\hat\{p\}\(v\\mid y\_\{<t\}\)\\propto e^\{\\kappa q\_\{\\mathrm\{out\},v\}\}\\,s\(v\\mid y\_\{<t\}\), wheressis the conditional realised by the ungraded head\. Settingκ=ln⁡λ\\kappa=\\ln\\lambdaandqout,v=q​\(v\)q\_\{\\mathrm\{out\},v\}=q\(v\)makes the architectural tilt of[Eq\.18](https://arxiv.org/html/2607.22757#S4.E18)coincide with the objective tilt of[Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\(2\), so that the population minimizer satisfiess=p⋆s=p\_\{\\star\}: the underlying model is calibrated to the corpus, and the grading is carried entirely by the output head\. Under this matching, and only under it, the graded objective and the graded architecture are compatible with the standard language\-modelling objective\.

Case \(3\) admits no such matching\. The tilt of[Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\(3\) is a different function ofvvat each context, whereasqoutq\_\{\\mathrm\{out\}\}in[Eq\.18](https://arxiv.org/html/2607.22757#S4.E18)is a single fixed vector, applied identically at every context; no choice ofκ\\kappaandqoutq\_\{\\mathrm\{out\}\}can therefore absorb a context\-dependent tilt\. A grade that consults the target and its context jointly, a parse depth being the canonical example, leaves the trained model uncalibrated, with the miscalibration at each context equal to the Kullback–Leibler divergence between the corpus conditional and its tilt at that context\.

[Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\(3\) rules out the parse\-depth initialisationqt=depth⁡\(yt∣y<t\)q\_\{t\}=\\operatorname\{depth\}\(y\_\{t\}\\mid y\_\{<t\}\)outright: parse depth is a function of the target and its context jointly, so it falls under case \(3\), where by[Rem\.14](https://arxiv.org/html/2607.22757#Thmrem14)no matching is available\. We therefore takeqt=qout,ytq\_\{t\}=q\_\{\\mathrm\{out\},y\_\{t\}\}withqoutq\_\{\\mathrm\{out\}\}a per\-type grade estimated offline, which is a function of the token alone, falls under case \(2\), and admits the cancellation\. Part\-of\-speech classes supply a natural default: high grades for open\-class types, low for closed\-class function words\. A parse\-depth grading is retained in the program only as an intentionally uncalibrated ablation; by[Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\(3\) its perplexity gap measures the tilt, not the inductive bias\.

### 5\.2\.Regularisation of the Grades

The trainable parameters are

θ=\{We,Wvoc,\{WQh,WKh,WVh\}h=1H,WO,\{Pl\}l=0L−1,\{qh\},\{ql\},qout\}\.\\theta=\\\{W\_\{e\},W\_\{\\mathrm\{voc\}\},\\\{W\_\{Q\_\{h\}\},W\_\{K\_\{h\}\},W\_\{V\_\{h\}\}\\\}\_\{h=1\}^\{H\},W\_\{O\},\\\{P\_\{l\}\\\}\_\{l=0\}^\{L\-1\},\\\{q\_\{h\}\\\},\\\{q\_\{l\}\\\},q\_\{\\mathrm\{out\}\}\\\}\.Collect all grades into𝐪\\mathbf\{q\}\. The full objective is

\(20\)ℒtotal​\(θ\)=𝔼y∼𝒟​\[w​\(y\)​ℒgrade​\(y,p^;q\)\]\+νmag​‖𝐪‖22\+νorth​Ωorth​\(P\)−νdiv​∑1≤h<h′≤H‖qh−qh′‖22,\\begin\{split\}\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\)&=\\mathbb\{E\}\_\{y\\sim\\mathcal\{D\}\}\\\!\\left\[w\(y\)\\,\\mathcal\{L\}\_\{\\mathrm\{grade\}\}\(y,\\hat\{p\};q\)\\right\]\+\\nu\_\{\\mathrm\{mag\}\}\\\|\\mathbf\{q\}\\\|\_\{2\}^\{2\}\+\\nu\_\{\\mathrm\{orth\}\}\\,\\Omega\_\{\\mathrm\{orth\}\}\(P\)\\\\ &\\quad\-\\nu\_\{\\mathrm\{div\}\}\\sum\_\{1\\leq h<h^\{\\prime\}\\leq H\}\\big\\\|q\_\{h\}\-q\_\{h^\{\\prime\}\}\\big\\\|\_\{2\}^\{2\},\\end\{split\}withνmag,νorth,νdiv\>0\\nu\_\{\\mathrm\{mag\}\},\\nu\_\{\\mathrm\{orth\}\},\\nu\_\{\\mathrm\{div\}\}\>0, andΩorth\\Omega\_\{\\mathrm\{orth\}\}the orthogonality penalty of[Sec\.4](https://arxiv.org/html/2607.22757#S4)\.

Three points on[Eq\.20](https://arxiv.org/html/2607.22757#S5.E20)deserve emphasis\. First, the diversity term enters with a*negative*sign and is therefore repulsive: it rewards heads whose grade tuples differ\. This is required for consistency with the framework, since the graded bilinear forms of[Sec\.3\.1](https://arxiv.org/html/2607.22757#S3.SS1)are head\-specific only when theqhq\_\{h\}differ, and[Sec\.8](https://arxiv.org/html/2607.22757#S8)predicts head specialisation as an observable\. An attractive penalty∑h‖qh−q¯‖22\\sum\_\{h\}\\\|q\_\{h\}\-\\bar\{q\}\\\|\_\{2\}^\{2\}has its minimum atqh≡q¯q\_\{h\}\\equiv\\bar\{q\}, at which point every head carries the identical grading and the construction degenerates to a global rescaling; such a term would penalise precisely the phenomenon the framework predicts\.

Second, the repulsion must be bounded, and the magnitude termνmag​‖𝐪‖22\\nu\_\{\\mathrm\{mag\}\}\\\|\\mathbf\{q\}\\\|\_\{2\}^\{2\}supplies the cap\. Writing𝐪=\(q1,…,qH\)\\mathbf\{q\}=\(q\_\{1\},\\dots,q\_\{H\}\)with‖𝐪‖22=∑h‖qh‖22\\\|\\mathbf\{q\}\\\|\_\{2\}^\{2\}=\\sum\_\{h\}\\\|q\_\{h\}\\\|\_\{2\}^\{2\}, the polarisation identity

∑1≤h<h′≤H‖qh−qh′‖22=H​∑h=1H‖qh‖22−‖∑h=1Hqh‖22\\sum\_\{1\\leq h<h^\{\\prime\}\\leq H\}\\big\\\|q\_\{h\}\-q\_\{h^\{\\prime\}\}\\big\\\|\_\{2\}^\{2\}=H\\sum\_\{h=1\}^\{H\}\\\|q\_\{h\}\\\|\_\{2\}^\{2\}\-\\Big\\\|\\sum\_\{h=1\}^\{H\}q\_\{h\}\\Big\\\|\_\{2\}^\{2\}shows that the two quadratic terms of[Eq\.20](https://arxiv.org/html/2607.22757#S5.E20)contribute together

νmag​‖𝐪‖22−νdiv​∑h<h′‖qh−qh′‖22=\(νmag−νdiv​H\)​‖𝐪‖22\+νdiv​‖∑hqh‖22,\\nu\_\{\\mathrm\{mag\}\}\\\|\\mathbf\{q\}\\\|\_\{2\}^\{2\}\-\\nu\_\{\\mathrm\{div\}\}\\sum\_\{h<h^\{\\prime\}\}\\\|q\_\{h\}\-q\_\{h^\{\\prime\}\}\\\|\_\{2\}^\{2\}=\\big\(\\nu\_\{\\mathrm\{mag\}\}\-\\nu\_\{\\mathrm\{div\}\}H\\big\)\\\|\\mathbf\{q\}\\\|\_\{2\}^\{2\}\+\\nu\_\{\\mathrm\{div\}\}\\Big\\\|\\sum\_\{h\}q\_\{h\}\\Big\\\|\_\{2\}^\{2\},a quadratic form with eigenvalue2​νmag2\\nu\_\{\\mathrm\{mag\}\}on the head\-constant directions and2​\(νmag−νdiv​H\)2\(\\nu\_\{\\mathrm\{mag\}\}\-\\nu\_\{\\mathrm\{div\}\}H\)on their orthogonal complement\. We therefore impose

\(21\)νmag\>νdiv​H,\\nu\_\{\\mathrm\{mag\}\}\>\\nu\_\{\\mathrm\{div\}\}\\,H,which is exactly the condition under which the pair is coercive, and hence[Eq\.20](https://arxiv.org/html/2607.22757#S5.E20)bounded below in𝐪\\mathbf\{q\}, and simultaneously the condition under which it is convex\. The bound is tight: takingH=2H=2andq1=−q2=vq\_\{1\}=\-q\_\{2\}=vgives2​\(νmag−2​νdiv\)​‖v‖222\(\\nu\_\{\\mathrm\{mag\}\}\-2\\nu\_\{\\mathrm\{div\}\}\)\\\|v\\\|\_\{2\}^\{2\}, which is unbounded below as‖v‖2→∞\\\|v\\\|\_\{2\}\\to\\inftywheneverνmag<νdiv​H\\nu\_\{\\mathrm\{mag\}\}<\\nu\_\{\\mathrm\{div\}\}H\.

Third, data filtering, if desired, must act*inside*the expectation, through the per\-sequence weightw​\(y\)∈\[0,1\]w\(y\)\\in\[0,1\]; an additive penalty depending only on the data has zero gradient and no effect on training\.

###### Remark 15\.

[Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\(1\) applies tow​\(y\)w\(y\): reweighting whole sequences leaves the per\-context minimizer unchanged, sowwmay be set from a parse\-validity check without disturbing consistency\.

The grades entering[Eq\.20](https://arxiv.org/html/2607.22757#S5.E20)are refined end\-to\-end, but they are not found there: they are selected offline from estimated profiles, by the convex program of[Cor\.4](https://arxiv.org/html/2607.22757#Thmcor4)and its clipped form[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10), both developed in[Sec\.6](https://arxiv.org/html/2607.22757#S6)\.

### 5\.3\.Optimisation

Optimisation uses AdamW\[loshchilov2017decoupled\], with raw moments

mk=β1​mk−1\+\(1−β1\)​gk,vk=β2​vk−1\+\(1−β2\)​gk⊙gk,gk:=∇θℒtotal​\(θk\),\\begin\{split\}m\_\{k\}&=\\beta\_\{1\}m\_\{k\-1\}\+\(1\-\\beta\_\{1\}\)\\,g\_\{k\},\\\\ v\_\{k\}&=\\beta\_\{2\}v\_\{k\-1\}\+\(1\-\\beta\_\{2\}\)\\,g\_\{k\}\\odot g\_\{k\},\\qquad g\_\{k\}:=\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{k\}\),\\end\{split\}bias correctionsm^k=mk/\(1−β1k\)\\hat\{m\}\_\{k\}=m\_\{k\}/\(1\-\\beta\_\{1\}^\{k\}\)andv^k=vk/\(1−β2k\)\\hat\{v\}\_\{k\}=v\_\{k\}/\(1\-\\beta\_\{2\}^\{k\}\), and update

θk\+1=θk−ηk​m^kv^k\+ϵ−ηk​ω​θk,\\theta\_\{k\+1\}=\\theta\_\{k\}\-\\eta\_\{k\}\\frac\{\\hat\{m\}\_\{k\}\}\{\\sqrt\{\\hat\{v\}\_\{k\}\}\+\\epsilon\}\-\\eta\_\{k\}\\,\\omega\\,\\theta\_\{k\},takingβ1=0\.9\\beta\_\{1\}=0\.9,β2=0\.95\\beta\_\{2\}=0\.95,ω=0\.1\\omega=0\.1,ϵ=10−8\\epsilon=10^\{\-8\}\. The learning rate follows the cosine schedule of\[loshchilov2017sgdr\],

ηk=ηmin\+12​\(ηmax−ηmin\)​\(1\+cos⁡\(π​k/T\)\),\\eta\_\{k\}=\\eta\_\{\\min\}\+\\tfrac\{1\}\{2\}\\left\(\\eta\_\{\\max\}\-\\eta\_\{\\min\}\\right\)\\left\(1\+\\cos\(\\pi k/T\)\\right\),withηmax=6×10−4\\eta\_\{\\max\}=6\\times 10^\{\-4\}andηmin=10−5\\eta\_\{\\min\}=10^\{\-5\}\.

Gradient clipping must be stated carefully, since the point of clipping is to remove the grade dependence rather than to accommodate it\. We clip the global gradient norm to a constant independent ofλ\\lambdaand ofqmaxq\_\{\\max\}: a threshold set atλqmax\\lambda^\{q\_\{\\max\}\}would rescale with the very amplification it is meant to control and would impose no constraint at all\. Together with the magnitude penalty of[Eq\.20](https://arxiv.org/html/2607.22757#S5.E20)under[Eq\.21](https://arxiv.org/html/2607.22757#S5.E21)and the grade clipping of[Sec\.8\.1\.1](https://arxiv.org/html/2607.22757#S8.SS1.SSS1), this keeps the effective step size bounded uniformly in the grades\.

Distributed training uses standard sharded data parallelism with optimiser\-state partitioning\[rajbhandari2020zero\]and pipeline parallelism across devices\[huang2019gpipe\]\.

###### Lemma 5\.

Suppose each per\-token lossℓt​\(θ\)=−log⁡p^​\(yt∣y<t\)\\ell\_\{t\}\(\\theta\)=\-\\log\\hat\{p\}\(y\_\{t\}\\mid y\_\{<t\}\)isςt\\varsigma\_\{t\}\-smooth on a regionΘ\\Theta, and writeςunif:=∑t=1nςt\\varsigma\_\{\\mathrm\{unif\}\}:=\\sum\_\{t=1\}^\{n\}\\varsigma\_\{t\}for the resulting smoothness constant of the unweighted sum\. Then onΘ\\Thetathe graded loss[Eq\.19](https://arxiv.org/html/2607.22757#S5.E19)isςgrade\\varsigma\_\{\\mathrm\{grade\}\}\-smooth with

ςgrade≤λqmax​ςunif,qmax:=maxt⁡qt,\\varsigma\_\{\\mathrm\{grade\}\}\\;\\leq\\;\\lambda^\{q\_\{\\max\}\}\\,\\varsigma\_\{\\mathrm\{unif\}\},\\qquad q\_\{\\max\}:=\\max\_\{t\}q\_\{t\},and the regularisation terms of[Eq\.20](https://arxiv.org/html/2607.22757#S5.E20)contribute an additive constantcregc\_\{\\mathrm\{reg\}\}depending onνmag,νorth,νdiv,H\\nu\_\{\\mathrm\{mag\}\},\\nu\_\{\\mathrm\{orth\}\},\\nu\_\{\\mathrm\{div\}\},Hand the diameter ofΘ\\Theta, but not onλ\\lambdaor𝐪\\mathbf\{q\}\.

###### Proof\.

Forθ,θ′∈Θ\\theta,\\theta^\{\\prime\}\\in\\Theta,

‖∇ℒgrade​\(θ\)−∇ℒgrade​\(θ′\)‖2≤∑t=1nλqt​‖∇ℓt​\(θ\)−∇ℓt​\(θ′\)‖2≤\(∑t=1nλqt​ςt\)​‖θ−θ′‖2≤λqmax​ςunif​‖θ−θ′‖2,\\begin\{split\}\\big\\\|\\nabla\\mathcal\{L\}\_\{\\mathrm\{grade\}\}\(\\theta\)\-\\nabla\\mathcal\{L\}\_\{\\mathrm\{grade\}\}\(\\theta^\{\\prime\}\)\\big\\\|\_\{2\}&\\leq\\sum\_\{t=1\}^\{n\}\\lambda^\{q\_\{t\}\}\\big\\\|\\nabla\\ell\_\{t\}\(\\theta\)\-\\nabla\\ell\_\{t\}\(\\theta^\{\\prime\}\)\\big\\\|\_\{2\}\\\\ &\\leq\\Big\(\\sum\_\{t=1\}^\{n\}\\lambda^\{q\_\{t\}\}\\varsigma\_\{t\}\\Big\)\\\|\\theta\-\\theta^\{\\prime\}\\\|\_\{2\}\\leq\\lambda^\{q\_\{\\max\}\}\\varsigma\_\{\\mathrm\{unif\}\}\\\|\\theta\-\\theta^\{\\prime\}\\\|\_\{2\},\\end\{split\}using the triangle inequality, the per\-token hypothesis, andλqt≤λqmax\\lambda^\{q\_\{t\}\}\\leq\\lambda^\{q\_\{\\max\}\}\. The magnitude and diversity penalties are quadratics in𝐪\\mathbf\{q\}whose Hessians have operator norm at most2​νmag2\\nu\_\{\\mathrm\{mag\}\}and2​νdiv​H2\\nu\_\{\\mathrm\{div\}\}Hrespectively, by the eigenvalue computation of[Sec\.5\.2](https://arxiv.org/html/2607.22757#S5.SS2), andΩorth\\Omega\_\{\\mathrm\{orth\}\}is a polynomial in the entries ofPPwith gradient Lipschitz on the bounded regionΘ\\Theta; none of these constants involvesλ\\lambdaor𝐪\\mathbf\{q\}\. ∎

###### Theorem 5\.1\.

Supposeℒtotal\\mathcal\{L\}\_\{\\mathrm\{total\}\}isς\\varsigma\-smooth on the region visited, stochastic gradients are unbiased with variance at mostσ2\\sigma^\{2\}at batch sizebb, and the step size satisfiesη≤1/ς\\eta\\leq 1/\\varsigma\. Then afterTTsteps of stochastic gradient descent,

mink≤T⁡𝔼​‖∇θℒtotal​\(θk\)‖22≤2​\(ℒtotal​\(θ0\)−infℒtotal\)η​T\+ς​η​σ2b,\\min\_\{k\\leq T\}\\mathbb\{E\}\\big\\\|\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{k\}\)\\big\\\|\_\{2\}^\{2\}\\;\\leq\\;\\frac\{2\\big\(\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{0\}\)\-\\inf\\mathcal\{L\}\_\{\\mathrm\{total\}\}\\big\)\}\{\\eta T\}\+\\frac\{\\varsigma\\,\\eta\\,\\sigma^\{2\}\}\{b\},and by[Lem\.5](https://arxiv.org/html/2607.22757#Thmlem5)the smoothness constant obeysς≤λqmax​ςunif\+creg\\varsigma\\leq\\lambda^\{q\_\{\\max\}\}\\,\\varsigma\_\{\\mathrm\{unif\}\}\+c\_\{\\mathrm\{reg\}\}, withςunif\\varsigma\_\{\\mathrm\{unif\}\}the corresponding constant for the ungraded objective\.

###### Proof\.

Byς\\varsigma\-smoothness, for consecutive iteratesθk\+1=θk−η​g^k\\theta\_\{k\+1\}=\\theta\_\{k\}\-\\eta\\,\\hat\{g\}\_\{k\}withg^k\\hat\{g\}\_\{k\}the stochastic gradient,

ℒtotal​\(θk\+1\)≤ℒtotal​\(θk\)−η​⟨∇ℒtotal​\(θk\),g^k⟩\+ς​η22​‖g^k‖22\.\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{k\+1\}\)\\leq\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{k\}\)\-\\eta\\langle\\nabla\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{k\}\),\\hat\{g\}\_\{k\}\\rangle\+\\frac\{\\varsigma\\eta^\{2\}\}\{2\}\\\|\\hat\{g\}\_\{k\}\\\|\_\{2\}^\{2\}\.Taking expectations conditional onθk\\theta\_\{k\}, using unbiasedness𝔼​\[g^k\]=∇ℒtotal​\(θk\)\\mathbb\{E\}\[\\hat\{g\}\_\{k\}\]=\\nabla\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{k\}\)and the variance bound𝔼​‖g^k‖22≤‖∇ℒtotal​\(θk\)‖22\+σ2/b\\mathbb\{E\}\\\|\\hat\{g\}\_\{k\}\\\|\_\{2\}^\{2\}\\leq\\\|\\nabla\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{k\}\)\\\|\_\{2\}^\{2\}\+\\sigma^\{2\}/b,

𝔼​\[ℒtotal​\(θk\+1\)\]≤𝔼​\[ℒtotal​\(θk\)\]−η​\(1−ς​η2\)​𝔼​‖∇ℒtotal​\(θk\)‖22\+ς​η2​σ22​b\.\\mathbb\{E\}\\big\[\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{k\+1\}\)\\big\]\\leq\\mathbb\{E\}\\big\[\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{k\}\)\\big\]\-\\eta\\Big\(1\-\\frac\{\\varsigma\\eta\}\{2\}\\Big\)\\mathbb\{E\}\\\|\\nabla\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{k\}\)\\\|\_\{2\}^\{2\}\+\\frac\{\\varsigma\\eta^\{2\}\\sigma^\{2\}\}\{2b\}\.Sinceη≤1/ς\\eta\\leq 1/\\varsigmagives1−ς​η/2≥1/21\-\\varsigma\\eta/2\\geq 1/2, rearranging and summing overk=0,…,T−1k=0,\\dots,T\-1telescopes the loss terms, and dividing byη​T/2\\eta T/2bounds the minimum by the average:

mink≤T⁡𝔼​‖∇ℒtotal​\(θk\)‖22≤1T​∑k=0T−1𝔼​‖∇ℒtotal​\(θk\)‖22≤2​\(ℒtotal​\(θ0\)−infℒtotal\)η​T\+ς​η​σ2b\.\\min\_\{k\\leq T\}\\mathbb\{E\}\\\|\\nabla\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{k\}\)\\\|\_\{2\}^\{2\}\\leq\\frac\{1\}\{T\}\\sum\_\{k=0\}^\{T\-1\}\\mathbb\{E\}\\\|\\nabla\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{k\}\)\\\|\_\{2\}^\{2\}\\leq\\frac\{2\\big\(\\mathcal\{L\}\_\{\\mathrm\{total\}\}\(\\theta\_\{0\}\)\-\\inf\\mathcal\{L\}\_\{\\mathrm\{total\}\}\\big\)\}\{\\eta T\}\+\\frac\{\\varsigma\\eta\\sigma^\{2\}\}\{b\}\.The smoothness comparison is[Lem\.5](https://arxiv.org/html/2607.22757#Thmlem5)\. ∎

###### Remark 16\.

The factorλqmax≥1\\lambda^\{q\_\{\\max\}\}\\geq 1enters[Thm\.5\.1](https://arxiv.org/html/2607.22757#S5.Thmthm1)as a price: grading degrades the smoothness constant and therefore the stationarity bound, and no reading of this theorem yields a convergence speedup\. The price and the purchase are controlled by the same constant\. The grade clipping of[Sec\.8\.1\.1](https://arxiv.org/html/2607.22757#S8.SS1.SSS1)enforcesλqmax≤C\\lambda^\{q\_\{\\max\}\}\\leq CwithC=2C=2at the planned configurations, so[Thm\.5\.1](https://arxiv.org/html/2607.22757#S5.Thmthm1)degrades by at most a factor ofCC, uniformly inLLand in the profiles; the token purchase at the same clip is at mostC2C^\{2\}by[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10), and the exponential regime of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)is reached as the clip opens \([Rem\.18](https://arxiv.org/html/2607.22757#Thmrem18)\)\. A price linear in the clip against a purchase quadratic in it is the trade the framework makes, favourable at every setting, and it is the reason the two are reported separately in[Sec\.8](https://arxiv.org/html/2607.22757#S8)rather than folded into a single figure\.

Three restrictions on scope\. The bound is over stationarity and not suboptimality, sinceℒtotal\\mathcal\{L\}\_\{\\mathrm\{total\}\}is nonconvex inθ\\thetaand no global guarantee is available\. The theorem is stated for stochastic gradient descent, while[Sec\.5\.3](https://arxiv.org/html/2607.22757#S5.SS3)specifies AdamW; Adam\-family methods admit no unconditional convergence guarantee even on convex problems, as shown by Reddi, Kale, and Kumar, so the statement is a discipline on the objective rather than on the optimiser actually deployed\. And[Lem\.5](https://arxiv.org/html/2607.22757#Thmlem5)bounds the smoothness through per\-token constants, soςunif\\varsigma\_\{\\mathrm\{unif\}\}is the sum of per\-token smoothness constants rather than the \(possibly smaller\) constant of the summed objective; the comparison is honest but not tight\. Consequently, any prediction of reduced training tokens to a fixed generalisation target must be routed through[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)and[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8), and must not be attributed to this theorem\.

### 5\.4\.Pretraining and Fine\-Tuning

Pretraining is planned on The Pile\[gao2020pile\], approximately 800 GB of diverse text, at sequence lengthsn∈\[1024,8192\]n\\in\[1024,8192\]\. Per\-type gradesqoutq\_\{\\mathrm\{out\}\}are estimated once, offline, by the clipped selection problem of[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)applied to part\-of\-speech statistics over the corpus, certified against[Rem\.17](https://arxiv.org/html/2607.22757#Thmrem17), and refined end\-to\-end thereafter under the constraint of[Rem\.14](https://arxiv.org/html/2607.22757#Thmrem14)\.

Fine\-tuning uses direct preference optimisation\[rafailov2023direct\]\. Writingxxfor the prompt,ywy\_\{w\}andyly\_\{l\}for the preferred and rejected responses,πθ\\pi\_\{\\theta\}for the policy andπref\\pi\_\{\\mathrm\{ref\}\}for the reference,

\(22\)ℒDPO=−𝔼\(x,yw,yl\)∼𝒟pref​\[log⁡σ​\(βDPO​log⁡πθ​\(yw∣x\)πref​\(yw∣x\)−βDPO​log⁡πθ​\(yl∣x\)πref​\(yl∣x\)\)\],\\mathcal\{L\}\_\{\\mathrm\{DPO\}\}=\-\\,\\mathbb\{E\}\_\{\(x,y\_\{w\},y\_\{l\}\)\\sim\\mathcal\{D\}\_\{\\mathrm\{pref\}\}\}\\left\[\\log\\sigma\\\!\\left\(\\beta\_\{\\mathrm\{DPO\}\}\\log\\frac\{\\pi\_\{\\theta\}\(y\_\{w\}\\mid x\)\}\{\\pi\_\{\\mathrm\{ref\}\}\(y\_\{w\}\\mid x\)\}\-\\beta\_\{\\mathrm\{DPO\}\}\\log\\frac\{\\pi\_\{\\theta\}\(y\_\{l\}\\mid x\)\}\{\\pi\_\{\\mathrm\{ref\}\}\(y\_\{l\}\\mid x\)\}\\right\)\\right\],withβDPO\>0\\beta\_\{\\mathrm\{DPO\}\}\>0the inverse\-temperature of the implicit reward and𝒟pref\\mathcal\{D\}\_\{\\mathrm\{pref\}\}the preference dataset\. A graded variant weights the token\-level contributions tolog⁡πθ​\(y∣x\)\\log\\pi\_\{\\theta\}\(y\\mid x\)by their grades, concentrating preference signal on structurally salient positions rather than distributing it uniformly across a response\. The grades used must be theqoutq\_\{\\mathrm\{out\}\}of[Rem\.14](https://arxiv.org/html/2607.22757#Thmrem14)and no others: a weighting that depends on the realised token and its context reintroduces the tilt of[Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\(3\) into the implicit reward, where no cancellation is available to absorb it\. Subject to that constraint the variant is a construction whose behaviour the program does not yet test\.

[Sec\.6](https://arxiv.org/html/2607.22757#S6)assembles the selection results of this paper into the general procedure,[Sec\.7](https://arxiv.org/html/2607.22757#S7)instantiates that procedure for language, and[Sec\.8](https://arxiv.org/html/2607.22757#S8)states the experimental program that tests the resulting predictions\.

## 6\.Grade Selection

The results of[Sections3](https://arxiv.org/html/2607.22757#S3),[4](https://arxiv.org/html/2607.22757#S4)and[5](https://arxiv.org/html/2607.22757#S5)assign to every grade vector a price and a value: a conditioning penalty bounded by[Lem\.5](https://arxiv.org/html/2607.22757#Thmlem5), and a sample\-complexity gain computed by[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)and located by[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\. This section assembles those results into the framework’s main application\. Grades are neither a hyperparameter to sweep nor a taxonomy to posit; they are the solution of an optimisation problem whose inputs are measurable before training\. The procedure below computes them, certifies them, and specifies the experiment that tests the computation\.

The distinction that organises the section is between domains that carry canonical grades and domains that do not\. Weighted projective coordinates and graded rings arrive with their grades attached: the weights of the Igusa invariants are\(2,4,6,10\)\(2,4,6,10\)by theorem, not by choice, and in such settings the machinery of this paper adds a consistency check rather than a decision\. Language carries no canonical grading; which coordinates of an embedding space deserve amplification is exactly the question no theorem of linguistics answers, and it is the question[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\) answers in closed form\. The contribution of the framework is educated grade selection in domains where the grades are not given\.

### 6\.1\.The Selection Procedure

The procedure has six steps, and each names the result that supports it\.

1. \(1\)*Fix the basis\.*The profiles of[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2)are defined relative to the graded basis of[Eq\.13](https://arxiv.org/html/2607.22757#S4.E13)\. In language this is the embedding basis carrying the level projections\{Pl\}\\\{P\_\{l\}\\\}; in a domain with canonical structure it is the given one\.
2. \(2\)*Estimate the data profile\.*τ^j\\hat\{\\tau\}\_\{j\}is the per\-coordinate variance over the corpus, normalised to the simplex, as in[Sec\.7\.2](https://arxiv.org/html/2607.22757#S7.SS2)\. This is a corpus statistic and requires no model\.
3. \(3\)*Estimate the target profile\.*α^j\\hat\{\\alpha\}\_\{j\}is the normalised squared weight of a linear probe trained on the annotated target, reported together with the probe’s inductive bias, as in[Sec\.7\.2](https://arxiv.org/html/2607.22757#S7.SS2)\.
4. \(4\)*Test divergence\.*ComputeBC​\(α^,τ^\)\\mathrm\{BC\}\(\\hat\{\\alpha\},\\hat\{\\tau\}\)\. By[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iv\) a value near11means the profiles agree and grading buys nothing in this domain at this target; the procedure terminates here, at the cost of two estimates\. This step is the domain filter, and it is where[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)is tested rather than assumed\.
5. \(5\)*Certify candidates\.*A proposed grade vectorqq, whether taxonomic, statistical, or computed, is admissible to first order exactly when⟨q,α^−τ^⟩\>0\\langle q,\\hat\{\\alpha\}\-\\hat\{\\tau\}\\rangle\>0, by[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(iv\)\. The inner product is one line of arithmetic and is reported before training\.
6. \(6\)*Compute the optimum\.*The grades minimising the bound are the solution of the convex program of[Cor\.4](https://arxiv.org/html/2607.22757#Thmcor4)below, projected to the clipped region by[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\. The output isqC⋆q^\{\\star\}\_\{C\}, the educated grading at clipCC; by[Lem\.2](https://arxiv.org/html/2607.22757#Thmlem2)the gain it attains degrades only to second order in the estimation error of steps \(2\) and \(3\)\.

Steps \(2\)–\(4\) cost no pretraining, step \(5\) costs an inner product, and step \(6\) is a geometric program inddvariables\. The entire selection is performed, published, and falsifiable before the first training step, which is the property the staging of[Sec\.8](https://arxiv.org/html/2607.22757#S8)is built around\.

### 6\.2\.The Offline Selection Problem

The grades entering[Eq\.20](https://arxiv.org/html/2607.22757#S5.E20)are refined end\-to\-end, but they are not found there\. They are selected offline from estimated profiles, and that problem is well posed in a way the end\-to\-end problem is not\.

###### Corollary 4\.

Letα^,τ^∈Δd−1\\hat\{\\alpha\},\\hat\{\\tau\}\\in\\Delta^\{d\-1\}be strictly positive estimates of the profiles of[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2), and consider the offline selection problem

minq∈ℝ≥0d⁡\{log⁡Λ​\(q\)\+νmag​‖q‖22\},Λ​\(q\)=\(∑j=1dα^j​λ−2​qj\)​\(∑j=1dτ^j​λ2​qj\)\.\\min\_\{q\\in\\mathbb\{R\}\_\{\\geq 0\}^\{d\}\}\\ \\Big\\\{\\log\\Lambda\(q\)\+\\nu\_\{\\mathrm\{mag\}\}\\\|q\\\|\_\{2\}^\{2\}\\Big\\\},\\qquad\\Lambda\(q\)=\\Big\(\\sum\_\{j=1\}^\{d\}\\hat\{\\alpha\}\_\{j\}\\,\\lambda^\{\-2q\_\{j\}\}\\Big\)\\Big\(\\sum\_\{j=1\}^\{d\}\\hat\{\\tau\}\_\{j\}\\,\\lambda^\{2q\_\{j\}\}\\Big\)\.Then the objective is strictly convex on the convex feasible setℝ≥0d\\mathbb\{R\}\_\{\\geq 0\}^\{d\}, so the minimizer is unique and every stationary point is the global minimum\. In the variablestj=λ2​qjt\_\{j\}=\\lambda^\{2q\_\{j\}\}the functionΛ\\Lambdais a posynomial and the problem is a geometric program\. Asνmag→0\\nu\_\{\\mathrm\{mag\}\}\\to 0the value tends tolog⁡BC​\(α^,τ^\)2\\log\\mathrm\{BC\}\(\\hat\{\\alpha\},\\hat\{\\tau\}\)^\{2\}; forνmag\>0\\nu\_\{\\mathrm\{mag\}\}\>0the penalty breaks the gauge invariance of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(i\) and selects the minimum\-norm representative of the optimal orbit\. The version of this problem on the clipped region the architecture admits is solved in[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10), with the same convex structure and a saturated form of the same optimum\.

###### Proof\.

Convexity ofq↦log⁡Λ​\(q\)q\\mapsto\\log\\Lambda\(q\)is[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(ii\), andνmag​‖q‖22\\nu\_\{\\mathrm\{mag\}\}\\\|q\\\|\_\{2\}^\{2\}is strictly convex, so the sum is strictly convex;ℝ≥0d\\mathbb\{R\}\_\{\\geq 0\}^\{d\}is convex, whence uniqueness and the coincidence of stationary and global minima\. For the posynomial claim,∑jα^j​tj−1\\sum\_\{j\}\\hat\{\\alpha\}\_\{j\}t\_\{j\}^\{\-1\}and∑jτ^j​tj\\sum\_\{j\}\\hat\{\\tau\}\_\{j\}t\_\{j\}are posynomials int∈ℝ\>0dt\\in\\mathbb\{R\}\_\{\>0\}^\{d\}and a product of posynomials is a posynomial\. The limiting value is[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(vi\), and the representative inℝ≥0d\\mathbb\{R\}\_\{\\geq 0\}^\{d\}withminj⁡qj=0\\min\_\{j\}q\_\{j\}=0supplied by[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(i\) has finite norm, so the penalty is finite along the orbit and selects its minimum\-norm point\. ∎

###### Remark 17\.

[Cor\.4](https://arxiv.org/html/2607.22757#Thmcor4)has a consequence available before any pretraining run begins\. By[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(iv\), a proposed initialisationqinitq^\{\\mathrm\{init\}\}, say the part\-of\-speech grades of[Sec\.8\.1\.2](https://arxiv.org/html/2607.22757#S8.SS1.SSS2), lies in the admissible cone𝒬\+\\mathcal\{Q\}\_\{\+\}, to first order, exactly when

⟨qinit,α^−τ^⟩\>0,\\big\\langle q^\{\\mathrm\{init\}\},\\hat\{\\alpha\}\-\\hat\{\\tau\}\\big\\rangle\>0,a single inner product between the proposed grades and the estimated profiles\. Neither quantity requires a trained model:τ^\\hat\{\\tau\}is a corpus statistic andα^\\hat\{\\alpha\}a probe weight, both computed in[Sec\.7\.2](https://arxiv.org/html/2607.22757#S7.SS2)\. A GLLM may therefore be certified in advance to begin inside the region where its sample\-complexity bound is the smaller one, at the cost of one dot product, and[Cor\.4](https://arxiv.org/html/2607.22757#Thmcor4)then locates the optimum within that region by convex programming rather than by search\. The magnitude of the gain depends on the accuracy ofα^\\hat\{\\alpha\}andτ^\\hat\{\\tau\}, controlled by[Lem\.2](https://arxiv.org/html/2607.22757#Thmlem2); the sign does not, and only the sign is needed for certification\. At the clipped configurations the relevant optimum is theqC⋆q^\{\\star\}\_\{C\}of[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10), and the attainable gain is bounded byC2C^\{2\}accordingly\.

The end\-to\-end refinement is a different matter\.[Eq\.20](https://arxiv.org/html/2607.22757#S5.E20)is nonconvex inθ\\theta, and no counterpart of[Cor\.4](https://arxiv.org/html/2607.22757#Thmcor4)is available for it\. The role of the offline problem is to supply a certified starting point, not to replace training\.

### 6\.3\.Selection Under Clipping

The architecture is trained under a grade clipλqmax≤C\\lambda^\{q\_\{\\max\}\}\\leq C, withC=2C=2at the configurations of[Sec\.8\.1\.1](https://arxiv.org/html/2607.22757#S8.SS1.SSS1), and the selection problem must be solved on the region the clip admits\. The clipped problem retains the structure of the unclipped one, and the clip prices the gain exactly\.

###### Proposition 10\.

FixC=λqmax≥1C=\\lambda^\{q\_\{\\max\}\}\\geq 1and let𝒬C=\{q∈ℝd:0≤qj≤qmax\}\\mathcal\{Q\}\_\{C\}=\\\{q\\in\\mathbb\{R\}^\{d\}:0\\leq q\_\{j\}\\leq q\_\{\\max\}\\\}\. WriteΛC⋆=minq∈𝒬C⁡Λ​\(q\)\\Lambda^\{\\star\}\_\{C\}=\\min\_\{q\\in\\mathcal\{Q\}\_\{C\}\}\\Lambda\(q\)andrj=14​log⁡λ​log⁡\(αj/τj\)r\_\{j\}=\\tfrac\{1\}\{4\\log\\lambda\}\\log\(\\alpha\_\{j\}/\\tau\_\{j\}\)\.

1. \(i\)The minimum is attained, and the problem is convex:log⁡Λ\\log\\Lambdais convex on the compact convex set𝒬C\\mathcal\{Q\}\_\{C\}\.
2. \(ii\)Every minimizer has the saturated form qj⋆=min⁡\{qmax,max⁡\{0,rj\+c\}\}q\_\{j\}^\{\\star\}\\;=\\;\\min\\big\\\{q\_\{\\max\},\\,\\max\\\{0,\\;r\_\{j\}\+c\\\}\\big\\\}for some constantc∈ℝc\\in\\mathbb\{R\}: the fourth\-root law of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\) on the interior coordinates, saturation at the box on the rest\.
3. \(iii\)The gain is bounded by the clip: ΛC⋆≥max⁡\{BC​\(α,τ\)2,C−2\},\\Lambda^\{\\star\}\_\{C\}\\;\\geq\\;\\max\\big\\\{\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\},\\;C^\{\-2\}\\big\\\},and the boundC−2C^\{\-2\}is approached as the profiles concentrate on separate coordinates\. Reaching the regimeΛ⋆=Θ​\(νL\)\\Lambda^\{\\star\}=\\Theta\(\\nu^\{L\}\)of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)therefore requiresC≥ν−L/2C\\geq\\nu^\{\-L/2\}: the exponential\-in\-LLgain is the limit of a growing clip, at the conditioning price[Lem\.5](https://arxiv.org/html/2607.22757#Thmlem5)attaches to it\.

###### Proof\.

\(i\)𝒬C\\mathcal\{Q\}\_\{C\}is a product of intervals, hence compact and convex, andlog⁡Λ\\log\\Lambdais convex by[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(ii\) and continuous\.

\(ii\) In the variablesuj=2​qj​log⁡λu\_\{j\}=2q\_\{j\}\\log\\lambdaof[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8), the Karush–Kuhn–Tucker conditions for the box read∂jΦ​\(u\)=τj\(u\)−αj\(u\)=μj−−μj\+\\partial\_\{j\}\\Phi\(u\)=\\tau^\{\(u\)\}\_\{j\}\-\\alpha^\{\(u\)\}\_\{j\}=\\mu\_\{j\}^\{\-\}\-\\mu\_\{j\}^\{\+\}withμj±≥0\\mu\_\{j\}^\{\\pm\}\\geq 0supported on the active constraints\. On an interior coordinate both multipliers vanish andαj​e−uj/A​\(u\)=τj​euj/T​\(u\)\\alpha\_\{j\}e^\{\-u\_\{j\}\}/A\(u\)=\\tau\_\{j\}e^\{u\_\{j\}\}/T\(u\), givinguj=12​log⁡\(αj/τj\)\+12​log⁡\(T​\(u\)/A​\(u\)\)u\_\{j\}=\\tfrac\{1\}\{2\}\\log\(\\alpha\_\{j\}/\\tau\_\{j\}\)\+\\tfrac\{1\}\{2\}\\log\\\!\\big\(T\(u\)/A\(u\)\\big\), which isqj=rj\+cq\_\{j\}=r\_\{j\}\+cwithc=14​log⁡λ​log⁡\(T​\(u⋆\)/A​\(u⋆\)\)c=\\tfrac\{1\}\{4\\log\\lambda\}\\log\\\!\\big\(T\(u^\{\\star\}\)/A\(u^\{\\star\}\)\\big\)common to all interior coordinates\. On a coordinate active at the lower bound the condition∂jΦ≥0\\partial\_\{j\}\\Phi\\geq 0holds exactly when the interior law would giverj\+c≤0r\_\{j\}\+c\\leq 0, and symmetrically at the upper bound, which is the saturated form\. SinceΛ\\Lambdais invariant along𝟏\\mathbf\{1\}by[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(i\), the minimizer set is a segment in that direction intersected with𝒬C\\mathcal\{Q\}\_\{C\}, and every point of it has the stated form under the corresponding shift ofcc\.

\(iii\) The boundBC​\(α,τ\)2\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\}is the unconstrained minimum of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\)\. For the second bound, every feasiblegj=λqjg\_\{j\}=\\lambda^\{q\_\{j\}\}lies in\[1,C\]\[1,C\], so

Λ​\(q\)=\(∑j=1dαj​gj−2\)​\(∑j=1dτj​gj2\)≥C−2​\(∑jαj\)⋅1⋅\(∑jτj\)=C−2\.\\Lambda\(q\)=\\Big\(\\sum\_\{j=1\}^\{d\}\\alpha\_\{j\}g\_\{j\}^\{\-2\}\\Big\)\\Big\(\\sum\_\{j=1\}^\{d\}\\tau\_\{j\}g\_\{j\}^\{2\}\\Big\)\\;\\geq\\;C^\{\-2\}\\Big\(\\sum\_\{j\}\\alpha\_\{j\}\\Big\)\\cdot 1\\cdot\\Big\(\\sum\_\{j\}\\tau\_\{j\}\\Big\)=C^\{\-2\}\.For tightness taked=2d=2,α→\(1,0\)\\alpha\\to\(1,0\),τ→\(0,1\)\\tau\\to\(0,1\), andq=\(qmax,0\)q=\(q\_\{\\max\},0\): the first factor tends toC−2C^\{\-2\}and the second to11\. For the final claim,ΛC⋆≤νL\\Lambda^\{\\star\}\_\{C\}\\leq\\nu^\{L\}forcesC−2≤νLC^\{\-2\}\\leq\\nu^\{L\}\. ∎

###### Remark 18\.

[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\(iii\) and[Lem\.5](https://arxiv.org/html/2607.22757#Thmlem5)are controlled by the same constant, and together they give the framework’s cost–benefit statement in final form: at clipCCthe conditioning penalty is at mostCCand the sample\-complexity gain is at mostC2C^\{2\}\. The price is linear in the clip and the purchase quadratic, so the trade is favourable at every setting, and the planned configurationC=2C=2buys a gain of at most44at a penalty of at most22\. The exponential regime of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)is not available at fixed clip; it is reached as the clip grows, with both sides of the trade priced at every point along the way\. The experiments of[Sec\.8](https://arxiv.org/html/2607.22757#S8)are accordingly designed to detect a bounded effect atC=2C=2and to sweepCCas an explicit arm\.

### 6\.4\.Level Reduction

Under the stratification of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)the selection problem collapses fromddvariables toLL, and this reduction is what makes step \(6\) implementable at scale\.

###### Proposition 11\.

Suppose the profiles are uniform within levels in the sense of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3):αj=al/dl\\alpha\_\{j\}=a\_\{l\}/d\_\{l\}andτj=sl/dl\\tau\_\{j\}=s\_\{l\}/d\_\{l\}forj∈Vlj\\in V\_\{l\}, witha,s∈ΔL−1a,s\\in\\Delta^\{L\-1\}strictly positive\. Then every minimizer ofΛ\\Lambdaoverℝd\\mathbb\{R\}^\{d\}, and every minimizer over the clipped box𝒬C\\mathcal\{Q\}\_\{C\}of[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10), is constant on each level\. Consequently the selection problem is the box\-constrained convex program inLLvariables

minv∈\[0,qmax\]L⁡ΛL​\(v\),ΛL​\(v\)=\(∑l=0L−1al​λ−2​vl\)​\(∑l=0L−1sl​λ2​vl\),\\min\_\{v\\in\[0,q\_\{\\max\}\]^\{L\}\}\\ \\Lambda\_\{L\}\(v\),\\qquad\\Lambda\_\{L\}\(v\)=\\left\(\\sum\_\{l=0\}^\{L\-1\}a\_\{l\}\\lambda^\{\-2v\_\{l\}\}\\right\)\\left\(\\sum\_\{l=0\}^\{L\-1\}s\_\{l\}\\lambda^\{2v\_\{l\}\}\\right\),whose solutionv⋆v^\{\\star\}determinesq⋆q^\{\\star\}byqj⋆=vl⋆q^\{\\star\}\_\{j\}=v^\{\\star\}\_\{l\}forj∈Vlj\\in V\_\{l\}\. AtL=4L=4the program has four variables, whateverddmay be\.

###### Proof\.

Work in the variablesuj=2​qj​log⁡λu\_\{j\}=2q\_\{j\}\\log\\lambdaand writeΦ​\(u\)=log⁡A​\(u\)\+log⁡T​\(u\)\\Phi\(u\)=\\log A\(u\)\+\\log T\(u\)as in the proof of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8), withA​\(u\)=∑jαj​e−ujA\(u\)=\\sum\_\{j\}\\alpha\_\{j\}e^\{\-u\_\{j\}\}andT​\(u\)=∑jτj​eujT\(u\)=\\sum\_\{j\}\\tau\_\{j\}e^\{u\_\{j\}\}\. LetΣ\\Sigmabe the group of coordinate permutationsσ\\sigmawithσ​\(Vl\)=Vl\\sigma\(V\_\{l\}\)=V\_\{l\}for everyll\. Under the stratification hypothesisα\\alphaandτ\\tauare constant on each level, henceΣ\\Sigma\-invariant, soΦ​\(σ⋅u\)=Φ​\(u\)\\Phi\(\\sigma\\cdot u\)=\\Phi\(u\)for everyσ∈Σ\\sigma\\in\\Sigma; and both feasible sets,ℝd\\mathbb\{R\}^\{d\}and the box, areΣ\\Sigma\-invariant and convex\.

Letℳ\\mathcal\{M\}denote the set of minimizers, nonempty by[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\(i\) in the clipped case and by[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\) in the unclipped one\. SinceΦ\\Phiis convex and the feasible set is convex,ℳ\\mathcal\{M\}is convex\. Ifu⋆∈ℳu^\{\\star\}\\in\\mathcal\{M\}thenσ⋅u⋆∈ℳ\\sigma\\cdot u^\{\\star\}\\in\\mathcal\{M\}for everyσ∈Σ\\sigma\\in\\Sigma, so the averageu¯=\|Σ\|−1​∑σ∈Σσ⋅u⋆\\bar\{u\}=\|\\Sigma\|^\{\-1\}\\sum\_\{\\sigma\\in\\Sigma\}\\sigma\\cdot u^\{\\star\}lies inℳ\\mathcal\{M\}, andu¯\\bar\{u\}is constant on each level because it is aΣ\\Sigma\-average\.

It remains to show that every minimizer is of this form\. Letu1,u2∈ℳu^\{1\},u^\{2\}\\in\\mathcal\{M\}\. The segment\[u1,u2\]\[u^\{1\},u^\{2\}\]lies in the feasible set by convexity and inℳ\\mathcal\{M\}by convexity ofΦ\\Phi, on whichΦ\\Phiis therefore constant; a convex function constant along a segment has vanishing second directional derivative along it, so\(u2−u1\)⊤​∇2Φ​\(u\)​\(u2−u1\)=0\(u^\{2\}\-u^\{1\}\)^\{\\top\}\\nabla^\{2\}\\Phi\(u\)\(u^\{2\}\-u^\{1\}\)=0at everyuuof the segment\. By the proof of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8),∇2Φ​\(u\)=C​\(α\(u\)\)\+C​\(τ\(u\)\)\\nabla^\{2\}\\Phi\(u\)=\\mathrm\{C\}\(\\alpha^\{\(u\)\}\)\+\\mathrm\{C\}\(\\tau^\{\(u\)\}\)withv⊤​C​\(p\)​v=Varp​\(v\)v^\{\\top\}\\mathrm\{C\}\(p\)v=\\mathrm\{Var\}\_\{p\}\(v\), and both tilted vectorsα\(u\),τ\(u\)\\alpha^\{\(u\)\},\\tau^\{\(u\)\}are strictly positive becauseα\\alphaandτ\\tauare\. HenceVarα\(u\)​\(u2−u1\)=0\\mathrm\{Var\}\_\{\\alpha^\{\(u\)\}\}\(u^\{2\}\-u^\{1\}\)=0, which forcesu2−u1∈ℝ​𝟏u^\{2\}\-u^\{1\}\\in\\mathbb\{R\}\\mathbf\{1\}\. Every minimizer is thereforeu¯\+c​𝟏\\bar\{u\}\+c\\mathbf\{1\}for somec∈ℝc\\in\\mathbb\{R\}, and is constant on each level\.

For the reduced form, evaluateΛ\\Lambdaon a level\-constantqq:∑j∈Vl\(al/dl\)​λ−2​vl=al​λ−2​vl\\sum\_\{j\\in V\_\{l\}\}\(a\_\{l\}/d\_\{l\}\)\\lambda^\{\-2v\_\{l\}\}=a\_\{l\}\\lambda^\{\-2v\_\{l\}\}and likewise forτ\\tau, givingΛL\\Lambda\_\{L\}\. Convexity ofv↦log⁡ΛL​\(v\)v\\mapsto\\log\\Lambda\_\{L\}\(v\)is[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(ii\) restricted to the level\-constant subspace, and\[0,qmax\]L\[0,q\_\{\\max\}\]^\{L\}is compact and convex\. ∎

### 6\.5\.The Selection Ladder

The experiment that tests educated selection is not a two\-arm comparison but a ladder\. Every arm is a GLLM of identical architecture, data, and optimisation budget, distinguished only by what its grades consulted; the standard transformer is the bottom rung, the arm whose grades consulted nothing, and by the gauge invariance of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(i\) its admissibility inner product is0identically, with[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(iii\) reading that zero as the boundary position of the uniform grades\.

Table 2\.The selection ladder\. All arms share architecture, data, and budget; only the provenance of the grades varies\. The third column is computed offline and published before training\.The prediction, conditional on[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)and the scope of[Rem\.11](https://arxiv.org/html/2607.22757#Thmrem11), is an ordering: the token\-efficiency measurement of[Sec\.8\.2\.2](https://arxiv.org/html/2607.22757#S8.SS2.SSS2)ranks the arms as the third column of[Tab\.2](https://arxiv.org/html/2607.22757#S6.T2)ranks them\. The ordering is fixed by offline measurement before any arm is trained, which makes the prediction pre\-registered by construction and sharper than any pairwise comparison: a graded model beating an ungraded one admits many explanations, while six arms ordering by a number computed in advance admits few\.

The outcomes partition cleanly\. If the ordering holds, the first\-order reading of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)survives the passage from the single\-layer analysis to the full stack, and the selection procedure works as claimed\. If the educated arm falls below the part\-of\-speech arm, the quadratic degradation of[Lem\.2](https://arxiv.org/html/2607.22757#Thmlem2)has exceeded the refinement the estimates purchased, on estimates whose error the bootstrap of[Sec\.7\.2](https://arxiv.org/html/2607.22757#S7.SS2)reports in advance\. If all arms tie, either the profiles fail to diverge, which is caught at step \(4\) before training at negligible cost, or the single\-layer analysis fails at depth, which is the first limitation of[Rem\.11](https://arxiv.org/html/2607.22757#Thmrem11)\. Each outcome points to a specific result\.

### 6\.6\.Scope of the Procedure

The procedure applies wherever its first three steps are executable: a basis, second moments, and a target admitting a probe\. Domains sort into three classes against that criterion\.

Domains with canonical grades, such as weighted projective spaces, graded rings, and the weighted moduli of algebraic curves, carry the answer to step \(6\) already, and there the procedure inverts into a consistency test: the estimatedq⋆q^\{\\star\}should correlate with the canonical weights, and a failure of that correlation would indicate a target that does not respect the graded structure\. The machine\-learning treatment of the genus\-two moduli space is the natural site for this check, since the weights\(2,4,6,10\)\(2,4,6,10\)and the invariant\-theoretic targets are both exact and the profiles are computable without estimation error\.

Domains without canonical grades but with measurable profiles are where the procedure is the contribution, and language is the motivating case: the levels are real, their coordinates are not given, and[Sec\.7](https://arxiv.org/html/2607.22757#S7)instantiates the procedure for it, with[Sec\.8](https://arxiv.org/html/2607.22757#S8)running the ladder\.

Domains where step \(3\) has no probe target, such as end\-to\-end control where no annotated intermediate definesα\\alpha, lie outside the procedure’s present scope, and extending it there requires a surrogate for the target profile before anything else\.

### 6\.7\.The Admissible Cone in Geometric Invariant Theory

The structure assembled in[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8), the gauge invariance along𝟏\\mathbf\{1\}, the convexity oflog⁡Λ\\log\\Lambda, the boundary position of the uniform grades, and the first\-order criterion[Eq\.15](https://arxiv.org/html/2607.22757#S4.E15), is an instance of a classical one\. Under the exponential change of coordinates the gain becomes a Kempf–Ness functional for an action of the grading torus, the admissible set becomes the negativity region of a homogeneous hypersurface, the optimal grades of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\) become the coincidence point of two moment maps, and[Eq\.15](https://arxiv.org/html/2607.22757#S4.E15)becomes the descent criterion of geometric invariant theory at the uniform point\[mumford\-GIT,kempf\-ness\]\. This subsection records the dictionary; no result elsewhere in the paper depends on it, and its purpose is to place the selection theory of this section in the same geometry as the canonically graded domains of[Sec\.6\.6](https://arxiv.org/html/2607.22757#S6.SS6), where the stability theory is classical and the grades arrive by theorem\.

Let𝕋=\(ℂ×\)d\\mathbb\{T\}=\(\\mathbb\{C\}^\{\\times\}\)^\{d\}act diagonally onℂd\\mathbb\{C\}^\{d\}byt⋅z=\(tj​zj\)t\\cdot z=\(t\_\{j\}z\_\{j\}\), with maximal compact\(S1\)d\(S^\{1\}\)^\{d\}and Lie\-algebra coordinatesu∈ℝdu\\in\\mathbb\{R\}^\{d\}acting through

exp⁡\(u/2\)⋅z=\(euj/2​zj\)\.\\exp\(u/2\)\\cdot z=\(e^\{u\_\{j\}/2\}z\_\{j\}\)\.For a strictly positivep∈Δd−1p\\in\\Delta^\{d\-1\}set

zp=\(p11/2,…,pd1/2\)∈ℂdz\_\{p\}=\(p\_\{1\}^\{1/2\},\\dots,p\_\{d\}^\{1/2\}\)\\in\\mathbb\{C\}^\{d\}and define the Kempf–Ness functional ofzpz\_\{p\},

ψp​\(u\)=log⁡‖exp⁡\(u/2\)⋅zp‖2=log​∑j=1dpj​euj,∇ψp​\(u\)=μ​\(exp⁡\(u/2\)⋅zp\),\\begin\{split\}\\psi\_\{p\}\(u\)&=\\log\\big\\\|\\exp\(u/2\)\\cdot z\_\{p\}\\big\\\|^\{2\}=\\log\\sum\_\{j=1\}^\{d\}p\_\{j\}\\,e^\{u\_\{j\}\},\\\\ \\nabla\\psi\_\{p\}\(u\)&=\\mu\\big\(\\exp\(u/2\)\\cdot z\_\{p\}\\big\),\\end\{split\}whereμ​\(z\)=\(\|zj\|2/‖z‖2\)j∈Δd−1\\mu\(z\)=\\big\(\|z\_\{j\}\|^\{2\}/\\\|z\\\|^\{2\}\\big\)\_\{j\}\\in\\Delta^\{d\-1\}is the moment map of the compact torus, with image in the simplex\.

###### Lemma 6\.

In the variablesuj=2​qj​log⁡λu\_\{j\}=2q\_\{j\}\\log\\lambdaof[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8), the log\-gain is the Kempf–Ness functional of the pair\(zα,zτ\)∈ℂd×ℂd\(z\_\{\\alpha\},z\_\{\\tau\}\)\\in\\mathbb\{C\}^\{d\}\\times\\mathbb\{C\}^\{d\}under the anti\-diagonal actiont⋅\(x,y\)=\(t−1​x,t​y\)t\\cdot\(x,y\)=\(t^\{\-1\}x,\\,t\\,y\):

Φ​\(u\)=log⁡Λ=ψα​\(−u\)\+ψτ​\(u\),∇Φ​\(u\)=μ​\(eu/2​zτ\)−μ​\(e−u/2​zα\)=τ\(u\)−α\(u\),\\begin\{split\}\\Phi\(u\)=\\log\\Lambda&=\\psi\_\{\\alpha\}\(\-u\)\+\\psi\_\{\\tau\}\(u\),\\\\ \\nabla\\Phi\(u\)&=\\mu\\big\(e^\{u/2\}z\_\{\\tau\}\\big\)\-\\mu\\big\(e^\{\-u/2\}z\_\{\\alpha\}\\big\)=\\tau^\{\(u\)\}\-\\alpha^\{\(u\)\},\\end\{split\}so the tilted profiles of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)are the moment maps of the two factors, and the Hessian∇u2Φ=C​\(α\(u\)\)\+C​\(τ\(u\)\)\\nabla^\{2\}\_\{u\}\\Phi=\\mathrm\{C\}\(\\alpha^\{\(u\)\}\)\+\\mathrm\{C\}\(\\tau^\{\(u\)\}\)is the sum of their moment\-map derivatives\.

###### Proof\.

By definitionψα​\(−u\)=log​∑jαj​e−uj=log⁡A​\(u\)\\psi\_\{\\alpha\}\(\-u\)=\\log\\sum\_\{j\}\\alpha\_\{j\}e^\{\-u\_\{j\}\}=\\log A\(u\)andψτ​\(u\)=log⁡T​\(u\)\\psi\_\{\\tau\}\(u\)=\\log T\(u\), so the sum islog⁡\(A​T\)=log⁡Λ\\log\(AT\)=\\log\\Lambda\. Differentiating,∂jψα​\(−u\)=−αj​e−uj/A​\(u\)=−αj\(u\)\\partial\_\{j\}\\psi\_\{\\alpha\}\(\-u\)=\-\\alpha\_\{j\}e^\{\-u\_\{j\}\}/A\(u\)=\-\\alpha^\{\(u\)\}\_\{j\}and∂jψτ​\(u\)=τj\(u\)\\partial\_\{j\}\\psi\_\{\\tau\}\(u\)=\\tau^\{\(u\)\}\_\{j\}; moreoverμ​\(e−u/2​zα\)j=αj​e−uj/∑kαk​e−uk=αj\(u\)\\mu\(e^\{\-u/2\}z\_\{\\alpha\}\)\_\{j\}=\\alpha\_\{j\}e^\{\-u\_\{j\}\}/\\sum\_\{k\}\\alpha\_\{k\}e^\{\-u\_\{k\}\}=\\alpha^\{\(u\)\}\_\{j\}, and likewise forτ\\tau\. The Hessian identity is the computation in the proof of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\. ∎

###### Proposition 12\.

Letα,τ\\alpha,\\taube the profiles of[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2)and settj=λ2​qjt\_\{j\}=\\lambda^\{2q\_\{j\}\}\.

1. \(i\)*\(The boundary hypersurface\.\)*In the coordinatest∈ℝ\>0dt\\in\\mathbb\{R\}^\{d\}\_\{\>0\}the admissible set is𝒬\+=\{F<0\}∩ℝ\>0d\\mathcal\{Q\}\_\{\+\}=\\\{F<0\\\}\\cap\\mathbb\{R\}^\{d\}\_\{\>0\}, where F​\(t\)=\(∑j=1dαj​∏k≠jtk\)​\(∑l=1dτl​tl\)−∏k=1dtkF\(t\)=\\Big\(\\sum\_\{j=1\}^\{d\}\\alpha\_\{j\}\\prod\_\{k\\neq j\}t\_\{k\}\\Big\)\\Big\(\\sum\_\{l=1\}^\{d\}\\tau\_\{l\}\\,t\_\{l\}\\Big\)\-\\prod\_\{k=1\}^\{d\}t\_\{k\}is homogeneous of degreedd\. The homogeneity is the gauge invariance of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(i\): the translationq↦q\+c​𝟏q\\mapsto q\+c\\mathbf\{1\}is the scalingt↦λ2​c​tt\\mapsto\\lambda^\{2c\}t, so𝒬\+\\mathcal\{Q\}\_\{\+\}is a cone in the exact sense, invariant underℝ\>0\\mathbb\{R\}\_\{\>0\}\-scaling, and∂𝒬\+\\partial\\mathcal\{Q\}\_\{\+\}descends to the positive real locus of the projective hypersurfaceV​\(F\)⊂ℙd−1V\(F\)\\subset\\mathbb\{P\}^\{d\-1\}\. The uniform point\[1:⋯:1\]\[1:\\cdots:1\]lies onV​\(F\)V\(F\), with∇F​\(𝟏\)=τ−α\\nabla F\(\\mathbf\{1\}\)=\\tau\-\\alpha, and is a smooth point of it wheneverα≠τ\\alpha\\neq\\tau\.
2. \(ii\)*\(Coercivity and existence\.\)*For a one\-parameter subgroupv∈ℝdv\\in\\mathbb\{R\}^\{d\}of𝕋\\mathbb\{T\}, the recession slope of the Kempf–Ness functional alongvvis ρ​\(v\):=lims→∞Φ​\(s​v\)s=maxj⁡vj−minj⁡vj,\\rho\(v\):=\\lim\_\{s\\to\\infty\}\\frac\{\\Phi\(sv\)\}\{s\}=\\max\_\{j\}v\_\{j\}\-\\min\_\{j\}v\_\{j\},strictly positive for everyv∉ℝ​𝟏v\\notin\\mathbb\{R\}\\mathbf\{1\}because both profiles are strictly positive by[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2)\. The functionalΦ\\Phiis therefore coercive onℝd/ℝ​𝟏\\mathbb\{R\}^\{d\}/\\mathbb\{R\}\\mathbf\{1\}, hence proper modulo the gauge direction, and the pair\(zα,zτ\)\(z\_\{\\alpha\},z\_\{\\tau\}\)is polystable for the quotient torus𝕋/ℂ×​𝟏\\mathbb\{T\}/\\mathbb\{C\}^\{\\times\}\\mathbf\{1\}; the Kempf–Ness theorem then yields existence of a minimizer and uniqueness modulo gauge, recovering[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(vi\)\. The recession slopeρ\\rhois the coercivity certificate and is symmetric underv↦−vv\\mapsto\-v; the sign\-sensitive Hilbert–Mumford\-type criterion that decides admissibility is the first\-order pairing of \(iv\), notρ\\rho\.
3. \(iii\)*\(The optimum as a moment\-map coincidence\.\)*The minimizer uj⋆=12​log⁡\(αj/τj\)\+cu^\{\\star\}\_\{j\}=\\tfrac\{1\}\{2\}\\log\(\\alpha\_\{j\}/\\tau\_\{j\}\)\+cis characterised by the coincidence of the two moment maps, μ​\(e−u⋆/2​zα\)=μ​\(eu⋆/2​zτ\)=β,βj=αj​τjBC​\(α,τ\),\\mu\\big\(e^\{\-u^\{\\star\}/2\}z\_\{\\alpha\}\\big\)=\\mu\\big\(e^\{u^\{\\star\}/2\}z\_\{\\tau\}\\big\)=\\beta,\\qquad\\beta\_\{j\}=\\frac\{\\sqrt\{\\alpha\_\{j\}\\tau\_\{j\}\}\}\{\\mathrm\{BC\}\(\\alpha,\\tau\)\},with attained valueΦ​\(u⋆\)=log⁡BC​\(α,τ\)2\\Phi\(u^\{\\star\}\)=\\log\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\}: the fourth\-root law of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\) is the balancing of the two moment maps, and the profileβ\\betais their common value\.
4. \(iv\)*\(The first\-order criterion as the descent criterion\.\)*Atu=𝟎u=\\mathbf\{0\}the derivative ofΦ\\Phialongvvis⟨v,τ−α⟩\\langle v,\\tau\-\\alpha\\rangle, so[Eq\.15](https://arxiv.org/html/2607.22757#S4.E15)is precisely the condition thatvvbe a Kempf–Ness descent direction at the uniform point\. This pairing is sign\-sensitive, reversing underv↦−vv\\mapsto\-v, and is the Hilbert–Mumford\-type object of the dictionary: the character it pairs against is the moment\-map defectτ−α\\tau\-\\alpha, which is the normal toV​\(F\)V\(F\)at the uniform point by \(i\), and the full gauge orbitℝ​𝟏\\mathbb\{R\}\\mathbf\{1\}of the uniform grades lies in\{Λ=1\}\\\{\\Lambda=1\\\}\.

###### Proof\.

\(i\) WithA​\(t\)=∑jαj​tj−1A\(t\)=\\sum\_\{j\}\\alpha\_\{j\}t\_\{j\}^\{\-1\}andT​\(t\)=∑lτl​tlT\(t\)=\\sum\_\{l\}\\tau\_\{l\}t\_\{l\}, clearing denominators gives

Λ−1=F​\(t\)/∏ktk\\Lambda\-1=F\(t\)/\\prod\_\{k\}t\_\{k\}onℝ\>0d\\mathbb\{R\}^\{d\}\_\{\>0\}, soΛ<1\\Lambda<1isF<0F<0there\. The first factor ofFFis homogeneous of degreed−1d\-1, the second of degree11, and the monomial of degreedd\. Scalingt↦s​tt\\mapsto stwiths=λ2​cs=\\lambda^\{2c\}corresponds toq↦q\+c​𝟏q\\mapsto q\+c\\mathbf\{1\}, which is[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(i\)\. Att=𝟏t=\\mathbf\{1\},

F​\(𝟏\)=Λ​\(𝟏\)−1=0,F\(\\mathbf\{1\}\)=\\Lambda\(\\mathbf\{1\}\)\-1=0,and

∇F=\(∇tΛ\)​∏ktk\+\(Λ−1\)​∇t​∏ktk\\nabla F=\(\\nabla\_\{t\}\\Lambda\)\\prod\_\{k\}t\_\{k\}\+\(\\Lambda\-1\)\\,\\nabla\_\{t\}\\prod\_\{k\}t\_\{k\}evaluates att=𝟏t=\\mathbf\{1\}to∇tΛ​\(𝟏\)\\nabla\_\{t\}\\Lambda\(\\mathbf\{1\}\), whosejj\-th entry is

−αj​T​\(𝟏\)\+A​\(𝟏\)​τj=τj−αj;\-\\alpha\_\{j\}T\(\\mathbf\{1\}\)\+A\(\\mathbf\{1\}\)\\tau\_\{j\}=\\tau\_\{j\}\-\\alpha\_\{j\};this is nonzero exactly whenα≠τ\\alpha\\neq\\tau, which is smoothness ofV​\(F\)V\(F\)at the uniform point\.

\(ii\) Ass→∞s\\to\\infty,

ψτ​\(s​v\)=s​maxj⁡vj\+O​\(1\)\\psi\_\{\\tau\}\(sv\)=s\\max\_\{j\}v\_\{j\}\+O\(1\)andψα​\(−s​v\)=−s​minj⁡vj\+O​\(1\)\\psi\_\{\\alpha\}\(\-sv\)=\-s\\min\_\{j\}v\_\{j\}\+O\(1\), since each sum is dominated by its extremal exponent and all coefficients are strictly positive; the slope isρ​\(v\)=maxj⁡vj−minj⁡vj\\rho\(v\)=\\max\_\{j\}v\_\{j\}\-\\min\_\{j\}v\_\{j\}, vanishing exactly onℝ​𝟏\\mathbb\{R\}\\mathbf\{1\}\. A convex function with strictly positive recession slope in every direction transverse to𝟏\\mathbf\{1\}is coercive, hence proper, onℝd/ℝ​𝟏\\mathbb\{R\}^\{d\}/\\mathbb\{R\}\\mathbf\{1\}, so a minimizer exists; uniqueness moduloℝ​𝟏\\mathbb\{R\}\\mathbf\{1\}follows as in the proof of[Prop\.11](https://arxiv.org/html/2607.22757#Thmprop11), since vanishing of the Hessian form along a segment of minimizers forces the segment intoℝ​𝟏\\mathbb\{R\}\\mathbf\{1\}\. Symmetry ofρ\\rhounderv↦−vv\\mapsto\-vis immediate from the displayed formula, which distinguishes it from a Hilbert–Mumford weight\.

\(iii\) Atu⋆u^\{\\star\},αj​e−uj⋆=αj​τj​e−c\\alpha\_\{j\}e^\{\-u^\{\\star\}\_\{j\}\}=\\sqrt\{\\alpha\_\{j\}\\tau\_\{j\}\}\\,e^\{\-c\}, soA​\(u⋆\)=e−c​BC​\(α,τ\)A\(u^\{\\star\}\)=e^\{\-c\}\\,\\mathrm\{BC\}\(\\alpha,\\tau\)and

αj\(u⋆\)=αj​τj​e−ce−c​BC​\(α,τ\)=αj​τjBC​\(α,τ\);\\alpha^\{\(u^\{\\star\}\)\}\_\{j\}=\\frac\{\\sqrt\{\\alpha\_\{j\}\\tau\_\{j\}\}\\,e^\{\-c\}\}\{e^\{\-c\}\\,\\mathrm\{BC\}\(\\alpha,\\tau\)\}=\\frac\{\\sqrt\{\\alpha\_\{j\}\\tau\_\{j\}\}\}\{\\mathrm\{BC\}\(\\alpha,\\tau\)\};symmetricallyτj​euj⋆=αj​τj​ec\\tau\_\{j\}e^\{u^\{\\star\}\_\{j\}\}=\\sqrt\{\\alpha\_\{j\}\\tau\_\{j\}\}\\,e^\{c\},T​\(u⋆\)=ec​BC​\(α,τ\)T\(u^\{\\star\}\)=e^\{c\}\\,\\mathrm\{BC\}\(\\alpha,\\tau\), andτj\(u⋆\)=αj​τj/BC​\(α,τ\)\\tau^\{\(u^\{\\star\}\)\}\_\{j\}=\\sqrt\{\\alpha\_\{j\}\\tau\_\{j\}\}/\\mathrm\{BC\}\(\\alpha,\\tau\)\. The moment maps coincide atβ\\betaand∇Φ​\(u⋆\)=0\\nabla\\Phi\(u^\{\\star\}\)=0by[Lem\.6](https://arxiv.org/html/2607.22757#Thmlem6)\. The value is

ψα​\(−u⋆\)\+ψτ​\(u⋆\)=log⁡A​\(u⋆\)\+log⁡T​\(u⋆\)=\(log⁡BC−c\)\+\(log⁡BC\+c\)=log⁡BC​\(α,τ\)2\.\\psi\_\{\\alpha\}\(\-u^\{\\star\}\)\+\\psi\_\{\\tau\}\(u^\{\\star\}\)=\\log A\(u^\{\\star\}\)\+\\log T\(u^\{\\star\}\)=\\big\(\\log\\mathrm\{BC\}\-c\\big\)\+\\big\(\\log\\mathrm\{BC\}\+c\\big\)=\\log\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\}\.
\(iv\) The derivative is

⟨∇Φ​\(𝟎\),v⟩=⟨τ−α,v⟩\\langle\\nabla\\Phi\(\\mathbf\{0\}\),v\\rangle=\\langle\\tau\-\\alpha,v\\rangleby[Lem\.6](https://arxiv.org/html/2607.22757#Thmlem6), andΛ​\(c​𝟏\)=1\\Lambda\(c\\mathbf\{1\}\)=1for everyccis[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(i\); the normal direction is∇F​\(𝟏\)=τ−α\\nabla F\(\\mathbf\{1\}\)=\\tau\-\\alphafrom \(i\)\. The pairing⟨τ−α,v⟩\\langle\\tau\-\\alpha,v\\ranglereverses sign underv↦−vv\\mapsto\-v, unlike the recession slopeρ\\rhoof \(ii\)\. ∎

###### Remark 19\.

Two statements coexist in[Prop\.12](https://arxiv.org/html/2607.22757#Thmprop12)and operate at different points of the dictionary\.

The first is the solvability of the selection problem, certified by \(ii\): the recession slopeρ​\(v\)\\rho\(v\)is strictly positive along every one\-parameter subgroup transverse to the gauge direction, because both profiles have full support, so the Kempf–Ness functional is coercive modulo gauge\. This is what makes the minimisation of[Cor\.4](https://arxiv.org/html/2607.22757#Thmcor4)well posed: the minimisation has a minimizer, unique modulo gauge, rather than an infimum escaping to infinity along a direction of vanishing slope\. A coordinate on whichα\\alphaorτ\\tauvanished would supply such a direction, and[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2)deletes those coordinates for this reason: full support of the profiles is the analytic form of stability, and the deletion convention of[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2)is its enforcement\. The sign\-sensitive criterion that classifies directions, as opposed to the symmetric slope that certifies coercivity, is the first\-order pairing of \(iv\)\.

The second concerns the*uniform grades*, and it is a statement about a point of the orbit rather than about the orbit’s type: the gauge orbitℝ​𝟏\\mathbb\{R\}\\mathbf\{1\}lies inside the level set\{Λ=1\}\\\{\\Lambda=1\\\}, which by \(i\) is the boundary hypersurfaceV​\(F\)V\(F\), while the moment\-map defect∇Φ​\(𝟎\)=τ−α\\nabla\\Phi\(\\mathbf\{0\}\)=\\tau\-\\alphais nonzero whenever the profiles differ\. The uniform point is thus positioned the way a strictly semistable point is positioned in a GIT quotient, on the boundary of the favourable locus with its entire orbit trapped there, and with the destabilising data, here the profile difference, naming the direction of escape\. The analogy is positional and not literal, since the pair itself is stable; what the position encodes is that the uniform grades are the unique gauge orbit on which the Kempf–Ness functional is critical for*no*pair of profiles and equal to its reference value for*every*pair, which is[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(iii\) read through the dictionary\.

###### Remark 20\.

The admissible cone is not a nullcone, and the comparison should be drawn at exactly its strength\. A nullcone is Zariski\-closed, cut out by the ideal of positive\-degree invariants, generally non\-convex, and stratified;𝒬\+\\mathcal\{Q\}\_\{\+\}is open, semialgebraic, defined by the single inequalityF<0F<0, and log\-convex in the grade variables, and the convexity is the property the selection of[Cor\.4](https://arxiv.org/html/2607.22757#Thmcor4)runs on\. What the dictionary identifies is the complementary structure: the gauge orbit of the uniform grades sits inside the boundary hypersurface in the way the orbit of a strictly semistable point sits inside the boundary of the stable locus, as[Rem\.19](https://arxiv.org/html/2607.22757#Thmrem19)makes precise, and[Eq\.15](https://arxiv.org/html/2607.22757#S4.E15)is a Hilbert–Mumford\-shaped pairing that reads admissibility of a one\-parameter subgroup off a fixed character of the torus\. The Hilbert–Mumford\-type object of the dictionary is the sign\-sensitive first\-order pairing of[Prop\.12](https://arxiv.org/html/2607.22757#Thmprop12)\(iv\); the recession slope of[Prop\.12](https://arxiv.org/html/2607.22757#Thmprop12)\(ii\) is not such an object but its coercivity certificate, symmetric under reversal of the one\-parameter subgroup\. The clipped problem of[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)is, in this language, the Kempf–Ness minimisation restricted to a box in the Lie algebra, and the saturated form of[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\(ii\) is the facial structure of that box under the moment\-map flow\.

###### Remark 21\.

The graded program originates in the invariant theory of binary forms, and the stability theory above closes that circle at its source\. For degree\-ddbinary forms underSL2\\mathrm\{SL\}\_\{2\}, the Hilbert–Mumford criterion reduces, after conjugating the one\-parameter subgroup into the maximal torus, to the weight computationμ​\(f,λ\)=max⁡\{2​i−d:ai≠0\}\\mu\(f,\\lambda\)=\\max\\\{2i\-d:a\_\{i\}\\neq 0\\\}, a sign\-sensitive pairing of the kind that appears in[Prop\.12](https://arxiv.org/html/2607.22757#Thmprop12)\(iv\); stability is equivalent to every root having multiplicity belowd/2d/2, and the invariant ring is graded with canonical weights, the weights\(2,4,6,10\)\(2,4,6,10\)above being the sextic instance\. The equivalence class of a form is its weighted moduli point in the weighted projective space carrying those weights as grades, and the entire stability theory is read off that graded point: a form is unstable exactly when its moduli point vanishes, and, over a number field, semistable over a residue field exactly when the prime does not divide the point’s coordinates, with semistability quantified by the weighted height of the point\[eshaska\]\.

In both settings a graded coordinate system converts stability into an arithmetic condition on a single point: there, divisibility and weighted height of the moduli point decide residual semistability; here, positivity of the recession slope of[Prop\.12](https://arxiv.org/html/2607.22757#Thmprop12)\(ii\), guaranteed by full support of the profiles, gives solvability of the selection problem\. The direction of inference is complementary: the binary\-form setting fixes the grades, given by theorem as the weights of the invariants, and studies the stability of points; this section fixes the point, the pair of profiles, and selects the grades\. The consistency test of[Sec\.6\.6](https://arxiv.org/html/2607.22757#S6.SS6)sits at the intersection: in a canonically graded domain the estimated optimum of[Cor\.4](https://arxiv.org/html/2607.22757#Thmcor4)is predicted to recover the weights the invariant theory prescribes\.

###### Remark 22\.

The domains of[Sec\.6\.6](https://arxiv.org/html/2607.22757#S6.SS6)that carry canonical grades, weighted projective spaces and graded rings among them, are precisely those in which the torus action and its linearisation are given by theorem and the moment\-map data need not be estimated\. The present construction equips a domain without canonical grades with the same objects from measurement: the two profiles of[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2)determine the pair\(zα,zτ\)\(z\_\{\\alpha\},z\_\{\\tau\}\), the estimated moment maps determine the descent direction at the uniform point, and the selection of[Cor\.4](https://arxiv.org/html/2607.22757#Thmcor4)is the Kempf–Ness minimisation for that pair\. The consistency test of[Sec\.6\.6](https://arxiv.org/html/2607.22757#S6.SS6)for the canonically graded case is, in this language, the statement that the estimated moment\-map coincidence point should recover the canonical weights\.

## 7\.Case Study: Hierarchical Grading for Natural Language

[Sec\.6](https://arxiv.org/html/2607.22757#S6)states the selection procedure in the generality the framework supports: fix a basis, estimate two profiles, test their divergence, certify a candidate, compute the optimum\. This section carries it out for language\. Each subsection below is one step of[Sec\.6\.1](https://arxiv.org/html/2607.22757#S6.SS1), taken to the point at which it can be implemented and reproduced, and the section as a whole is the input the program of[Sec\.8](https://arxiv.org/html/2607.22757#S8)uses: it fixes the graded basis the configurations use, the grades every arm of[Tab\.2](https://arxiv.org/html/2607.22757#S6.T2)carries, and the effect size the runs must detect\.

Two things distinguish this stage from the runs it precedes\. It is offline in the strict sense: a treebank, a matrix of second moments, and a convex program in four variables\. And it is falsifiable on its own terms, since step \(4\) can terminate the program before a single parameter is initialised\. The estimates themselves are not reported here; what is reported is the construction, and the consequences that follow from it in closed form\.

### 7\.1\.The Level Taxonomy and the Graded Basis

Step \(1\) of[Sec\.6\.1](https://arxiv.org/html/2607.22757#S6.SS1)asks for the basis in which the profiles of[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2)are read\. Language supplies no canonical one, which is exactly the situation the framework addresses; what it does supply is a taxonomy, and the basis is built to carry it\.

We takeL=4L=4, matching the configurations of[Sec\.8\.1\.1](https://arxiv.org/html/2607.22757#S8.SS1.SSS1):V0V\_\{0\}for subword and morphological structure,V1V\_\{1\}for syntactic constituency and dependency,V2V\_\{2\}for sentential semantics,V3V\_\{3\}for discourse and pragmatics\. Each level is given a feature mapφl\\varphi\_\{l\}sending a token in context to a vector of annotations drawn from a corpus that carries them\. Forl=0l=0these are the segmentation identity, affix class, part\-of\-speech tag, and orthographic case\. Forl=1l=1they are the label of the incoming dependency arc, the signed distance to the head, the labels of the constituent ancestors, and the depth of the token in the parse\. Forl=2l=2they are the predicate–argument role, the polarity of the governing negation, and the scope index of the governing quantifier\. Forl=3l=3they are coreference\-chain membership, the discourse\-relation label attaching the containing clause to its neighbour, and the position of the sentence within the document\. The Penn Treebank and the Universal Dependencies corpora supplyl∈\{0,1\}l\\in\\\{0,1\\\}; a proposition\-bank annotation suppliesl=2l=2; a coreference\-annotated corpus and a discourse treebank supplyl=3l=3\.

The projections\{Pl\}\\\{P\_\{l\}\\\}are then fixed by principal component analysis: for each level, the annotated featureφl\\varphi\_\{l\}is regressed onto a reference representation of widthdd, the residual covariance of the fitted directions is formed, andPl∈ℝd×dlP\_\{l\}\\in\\mathbb\{R\}^\{d\\times d\_\{l\}\}is taken to span its leadingdld\_\{l\}eigenvectors\. Gram–Schmidt across levels enforces mutual orthogonality of the ranges at initialisation, which is the hypothesisη=0\\eta=0of[Prop\.6](https://arxiv.org/html/2607.22757#Thmprop6); thereafter the columns are learnable and the orthogonality penalty of[Sec\.4](https://arxiv.org/html/2607.22757#S4)controls the drift, with[Prop\.6](https://arxiv.org/html/2607.22757#Thmprop6)bounding the residual\.

###### Remark 23\.

The construction above requires a representation of widthddbefore the model that will carry it exists, and the circularity is real rather than apparent: at initialisationWeW\_\{e\}is isotropic by construction, so the profiles read in its coordinates are uniform and carry no information\. Two routes resolve it, and the program uses both, at different steps\.

For steps \(2\)–\(5\), the divergence test and the certification, the reference is a fixed public encoder of matching width, or static type embeddings projected toℝd\\mathbb\{R\}^\{d\}\. This costs no pretraining, which is what makes the falsification test of[Sec\.7\.3](https://arxiv.org/html/2607.22757#S7.SS3)available before the first GPU\-hour\. The profiles so obtained are the reference’s, and transporting them to the GLLM’s own geometry is an assumption, not a theorem\.

For step \(6\), the educated gradingqC⋆q^\{\\star\}\_\{C\}, the reference is the embedding matrix of the uniform arm itself\. This is exact, since the profiles are then read in precisely the space in which the comparison of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)is made, and it is not an added cost: the uniform arm is the control of[Tab\.2](https://arxiv.org/html/2607.22757#S6.T2)and must be trained regardless\.

What makes the cheap route sufficient for its purpose is[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(iv\)\. Certification turns on the*sign*of an inner product, not on the magnitudes ofα^\\hat\{\\alpha\}andτ^\\hat\{\\tau\}, and a coarse reference that gets the sign right certifies as well as an exact one\.

### 7\.2\.Estimating the Profiles

Steps \(2\) and \(3\) produce the two probability vectors on which everything downstream depends\. Both are computed in the basis of[Sec\.7\.1](https://arxiv.org/html/2607.22757#S7.SS1)and both are estimated on held\-out data\.

The data profile is a corpus statistic and requires no model\. Writingεt∈ℝd\\varepsilon\_\{t\}\\in\\mathbb\{R\}^\{d\}for the reference representation of the token at positionttandε¯\\bar\{\\varepsilon\}for its mean over the corpus, the estimator is the normalised per\-coordinate second moment in the graded basis,

τ^j=σ^j2∑k=1dσ^k2,σ^j2=1N​∑t=1N⟨ej,εt−ε¯⟩2,\\hat\{\\tau\}\_\{j\}=\\frac\{\\hat\{\\sigma\}\_\{j\}^\{2\}\}\{\\sum\_\{k=1\}^\{d\}\\hat\{\\sigma\}\_\{k\}^\{2\}\},\\qquad\\hat\{\\sigma\}\_\{j\}^\{2\}=\\frac\{1\}\{N\}\\sum\_\{t=1\}^\{N\}\\big\\langle e\_\{j\},\\varepsilon\_\{t\}\-\\bar\{\\varepsilon\}\\big\\rangle^\{2\},with\{ej\}\\\{e\_\{j\}\\\}the columns of\[P0​∣⋯∣​PL−1\]\[P\_\{0\}\\mid\\cdots\\mid P\_\{L\-1\}\]\. Tokens are drawn from the pretraining corpus\[gao2020pile\]so thatτ^\\hat\{\\tau\}is the variance profile of the data the runs will actually see, andNNis set by the width of the resulting confidence interval rather than by convention\.

The target profile is a probe weight\. For a hierarchical targetyy, such as acceptability, dependency\-arc label, or nesting depth, a linear probew^\\hat\{w\}is fitted on the annotated treebank in the same basis, and

α^j=w^j2‖w^‖22\.\\hat\{\\alpha\}\_\{j\}=\\frac\{\\hat\{w\}\_\{j\}^\{2\}\}\{\\\|\\hat\{w\}\\\|\_\{2\}^\{2\}\}\.The probe is fitted with ridge regularisation and the penalty is selected by cross\-validation on a split disjoint from the one on whichα^\\hat\{\\alpha\}is reported, so that the shrinkage does not itself manufacture the concentration that[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)asserts\. Three properties of the estimate must be reported with it\. It identifiesα\\alphaonly up to the probe’s inductive bias, and a probe that cannot express the target reports the profile of what it can express instead\. Coordinates on whichw^\\hat\{w\}vanishes are deleted rather than carried, as[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2)requires\. And the ridge penalty biasesα^\\hat\{\\alpha\}towards uniformity, so the estimate is conservative for the framework’s purposes: it understates divergence rather than overstating it\.

The quantity that must survive estimation error is the sign of⟨q,α^−τ^⟩\\langle q,\\hat\{\\alpha\}\-\\hat\{\\tau\}\\rangle, not the value of either profile, and it is reported with a bootstrap over the treebank sentences: resample, refit, recompute, and report the fraction of resamples on which the sign is preserved\. This is the only uncertainty quantification the certification of[Sec\.7\.4](https://arxiv.org/html/2607.22757#S7.SS4)requires\. The magnitude ofBC​\(α^,τ^\)\\mathrm\{BC\}\(\\hat\{\\alpha\},\\hat\{\\tau\}\), which the effect\-size statement does require, carries its own interval from the same bootstrap\.

### 7\.3\.The Divergence Test

Step \(4\) is the domain filter, and it is where[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)is tested rather than assumed\. The program reports

BC​\(α^,τ^\)=∑j=1dα^j​τ^j\\mathrm\{BC\}\(\\hat\{\\alpha\},\\hat\{\\tau\}\)=\\sum\_\{j=1\}^\{d\}\\sqrt\{\\hat\{\\alpha\}\_\{j\}\\,\\hat\{\\tau\}\_\{j\}\}per task, together with the level\-aggregated profiles\(a^,s^\)\(\\hat\{a\},\\hat\{s\}\)of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)obtained by summingα^\\hat\{\\alpha\}andτ^\\hat\{\\tau\}within levels, and a goodness\-of\-fit check of the geometric hypothesisal∝νL−1−la\_\{l\}\\propto\\nu^\{L\-1\-l\},sl∝μls\_\{l\}\\propto\\mu^\{l\}on which every exponential\-in\-LLstatement of this work rests\.

By[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iv\) a value near11says the profiles agree, the gain of grading is11, and the procedure terminates for that task at the cost of two estimates\. By[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)the hierarchical targets of[Sec\.8\.2\.3](https://arxiv.org/html/2607.22757#S8.SS2.SSS3)are the tasks at which it should not be near11, and the lexically dominated contrast tasks are the tasks at which it should be: the framework predicts a*spread*across tasks, and a uniform value across tasks would be as informative against it as a uniform value near11\. This is the cheapest disconfirmation available anywhere in the program, and the staging of[Sec\.8](https://arxiv.org/html/2607.22757#S8)exists so that it is taken first\.

### 7\.4\.Certifying the Candidate Gradings

Step \(5\) costs one inner product per candidate\. By[Rem\.17](https://arxiv.org/html/2607.22757#Thmrem17), a proposed gradingqinitq^\{\\mathrm\{init\}\}enters the admissible cone𝒬\+\\mathcal\{Q\}\_\{\+\}of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8), to first order, exactly when⟨qinit,α^−τ^⟩\>0\\langle q^\{\\mathrm\{init\}\},\\hat\{\\alpha\}\-\\hat\{\\tau\}\\rangle\>0, and the program reports the quantity for every arm of[Tab\.2](https://arxiv.org/html/2607.22757#S6.T2)before pretraining begins\. A negative value falsifies that candidate and not the framework, and the corrected grades are then supplied by[Sec\.7\.5](https://arxiv.org/html/2607.22757#S7.SS5)by convex programming rather than by search\.

Two arms are settled in advance of the measurement, and in opposite directions\.

The uniform arm scores⟨𝟎,α^−τ^⟩=0\\langle\\mathbf\{0\},\\hat\{\\alpha\}\-\\hat\{\\tau\}\\rangle=0identically, whatever the profiles turn out to be, and by the gauge invariance of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(i\) so does every uniform gradec​𝟏c\\mathbf\{1\}\. This is not an artefact of the comparison but the content of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(iii\): the standard transformer sits on∂𝒬\+\\partial\\mathcal\{Q\}\_\{\+\}with gradient2​\(log⁡λ\)​\(τ−α\)≠02\(\\log\\lambda\)\(\\tau\-\\alpha\)\\neq 0, so a strictly descending direction exists at its location and the profiles determine it\. The isotropic architecture forgoes the gain of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)not because the profiles are unfavourable to it but because its grade vector does not read them\.

###### Remark 24\.

The taxonomic arm is admissible*a priori*, before any estimate exists\. By[Rem\.9](https://arxiv.org/html/2607.22757#Thmrem9), under the level\-stratified profiles of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)a level\-constant gradingvj=vlv\_\{j\}=v\_\{l\}forj∈Vlj\\in V\_\{l\}satisfies the criterion[Eq\.15](https://arxiv.org/html/2607.22757#S4.E15)exactly when∑l=0L−1vl​\(al−sl\)\>0\\sum\_\{l=0\}^\{L\-1\}v\_\{l\}\(a\_\{l\}\-s\_\{l\}\)\>0, and under the geometric hypothesis this holds for*every*vvaffine and increasing inll, with no estimate ofν\\nu, ofμ\\mu, or of the profiles at all\.

The linguistic ordering is therefore sufficient on its own: assigning higher grades to deeper levels, which is what the taxonomy of[Sec\.7\.1](https://arxiv.org/html/2607.22757#S7.SS1)does by construction, enters𝒬\+\\mathcal\{Q\}\_\{\+\}\. What the estimates of[Sec\.7\.2](https://arxiv.org/html/2607.22757#S7.SS2)buy is not admissibility but the location of the optimum within it, and the circularity of[Rem\.23](https://arxiv.org/html/2607.22757#Thmrem23)therefore cannot reach the framework’s positive claim: it reaches only the sharpness of the grading, not its direction\.

### 7\.5\.Computing the Educated Grading

Step \(6\) is a convex program, and for language it is a small one\. The unclipped problem of[Cor\.4](https://arxiv.org/html/2607.22757#Thmcor4)and the clipped problem of[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)are posed inddvariables,10241024atGLLM\-Mediumand40964096atGLLM\-7B, but under the stratification the taxonomy imposes they collapse by[Prop\.11](https://arxiv.org/html/2607.22757#Thmprop11)to theLL\-variable programΛL\\Lambda\_\{L\}, withL=4L=4at the planned configurations\.

[Prop\.11](https://arxiv.org/html/2607.22757#Thmprop11)is what makes the selection implementable rather than merely well posed\. The program it names is solved by projected gradient descent with coordinatewise clipping as the projection, using the gradient supplied by the proof of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)in level form,

∂∂vl​log⁡ΛL​\(v\)=2​log⁡λ​\(sl\(v\)−al\(v\)\),al\(v\)=al​λ−2​vl∑kak​λ−2​vk,sl\(v\)=sl​λ2​vl∑ksk​λ2​vk,\\begin\{split\}\\frac\{\\partial\}\{\\partial v\_\{l\}\}\\log\\Lambda\_\{L\}\(v\)&=2\\log\\lambda\\,\\big\(s^\{\(v\)\}\_\{l\}\-a^\{\(v\)\}\_\{l\}\\big\),\\\\ a^\{\(v\)\}\_\{l\}=\\frac\{a\_\{l\}\\lambda^\{\-2v\_\{l\}\}\}\{\\sum\_\{k\}a\_\{k\}\\lambda^\{\-2v\_\{k\}\}\},\\qquad s^\{\(v\)\}\_\{l\}&=\\frac\{s\_\{l\}\\lambda^\{2v\_\{l\}\}\}\{\\sum\_\{k\}s\_\{k\}\\lambda^\{2v\_\{k\}\}\},\\end\{split\}which is the difference of two tilted level profiles and vanishes at the interior optimum exactly when they coincide\. Four variables, a closed\-form gradient, a convex objective with no spurious stationary points by[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(ii\), and a projection that is a clip: the educated grading of[Tab\.2](https://arxiv.org/html/2607.22757#S6.T2)is the output of a computation that runs in milliseconds once\(a^,s^\)\(\\hat\{a\},\\hat\{s\}\)are in hand\. By[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\(ii\) the solution has the saturated form, the fourth\-root law of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iii\) on the interior levels and the box on the rest, and the solver is verified against that form\.

### 7\.6\.The Predicted Effect Size

The output of the procedure is a number the runs of[Sec\.8](https://arxiv.org/html/2607.22757#S8)must detect, and the theory fixes it in closed form once the level profiles are named\. Under the geometric hypothesis of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)the profiles are governed by two parameters,ν\\nufor the concentration of the target towards the deepest level andμ\\mufor the decay of the data away from the shallowest, and step \(4\) of[Sec\.6\.1](https://arxiv.org/html/2607.22757#S6.SS1)estimates them\.[Tab\.3](https://arxiv.org/html/2607.22757#S7.T3)records what[Prop\.11](https://arxiv.org/html/2607.22757#Thmprop11)and[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)deliver across the range those estimates may fall in, at the plannedL=4L=4andC=2C=2\.

Table 3\.The gain atL=4L=4and clipC=2C=2, as a function of the free parameters\(ν,μ\)\(\\nu,\\mu\)of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)\.*No corpus enters this table\.*It is arithmetic:Λ⋆\\Lambda^\{\\star\}is the closed form of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3), andΛC⋆\\Lambda^\{\\star\}\_\{C\}is the four\-variable program of[Prop\.11](https://arxiv.org/html/2607.22757#Thmprop11)solved on the box, withgC⋆=λqC⋆g^\{\\star\}\_\{C\}=\\lambda^\{q^\{\\star\}\_\{C\}\}the resulting per\-level multiplier\. The pair\(ν,μ\)\(\\nu,\\mu\)is what step \(4\) of[Sec\.6\.1](https://arxiv.org/html/2607.22757#S6.SS1)estimates; the table says what each possible estimate would imply\.###### Remark 25\.

Three observations follow from the table\.

The predicted effect is bounded and it is large enough to see\. Across the range, the token gain runs from1\.4×1\.4\\timesto2\.4×2\.4\\times, which is the quantity[Sec\.8\.2\.2](https://arxiv.org/html/2607.22757#S8.SS2.SSS2)measures and the magnitude the runs must be powered for\. It is not the exponential regime of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3), and[Rem\.18](https://arxiv.org/html/2607.22757#Thmrem18)says why: atC=2C=2the exponential is unreachable by construction, and what[Tab\.3](https://arxiv.org/html/2607.22757#S7.T3)quantifies is the first point of that trajectory\.

The regime of the table, geometric level\-stratified profiles at level\-constant clipped grades, is moreover exactly the hypothesis set of[Thm\.4\.1](https://arxiv.org/html/2607.22757#S4.Thmthm1), so each tabulated gain is attained up to constants depending only on\(ν,μ\)\(\\nu,\\mu\)as a ratio of minimax risks over all estimators, throughout the window of[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4): the effect size the runs are powered for is backed by the two\-sided separation, not only by the upper bounds of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\.

The clip is the binding constraint, and it binds hardest exactly where the profiles diverge most\. At\(0\.8,0\.4\)\(0\.8,0\.4\)the clipped gain is within about1%1\\%of the unclipped one; at\(0\.5,0\.2\)\(0\.5,0\.2\)the clip costs a third of it, and the top two levels saturate atg=Cg=C\. The pattern is[Rem\.18](https://arxiv.org/html/2607.22757#Thmrem18)made quantitative: the more the corpus rewards grading, the more the clip withholds, and theCC\-sweep of[Sec\.8\.1\.1](https://arxiv.org/html/2607.22757#S8.SS1.SSS1)is the arm that recovers it\.

The optimal grading is stable in a way the profiles are not\. Across the whole range the multiplier on the syntactic level sits between1\.271\.27and1\.321\.32, and the discourse level saturates atCCthroughout\. The ordering ofgC⋆g^\{\\star\}\_\{C\}, increasing inll, is invariant across every entry, which is[Rem\.24](https://arxiv.org/html/2607.22757#Thmrem24)recovered numerically: the estimates move the magnitudes and leave the direction alone\.

The boundΛC⋆≥max⁡\{BC​\(α,τ\)2,C−2\}\\Lambda^\{\\star\}\_\{C\}\\geq\\max\\\{\\mathrm\{BC\}\(\\alpha,\\tau\)^\{2\},C^\{\-2\}\\\}of[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\(iii\) is a lower bound and is not tight at these parameters; at\(0\.6,0\.3\)\(0\.6,0\.3\)it gives0\.4590\.459against the true0\.5310\.531\. The effect size the program is powered for is therefore the solution of the reduced program and not the bound\. This is the operative use of[Prop\.11](https://arxiv.org/html/2607.22757#Thmprop11): the bound is what the theory guarantees, and the four\-variable program is what the experiment must detect\.

### 7\.7\.Cost of the Offline Stage

The accounting closes the case study\. Steps \(2\) and \(3\) are one pass over a corpus for a covariance and one ridge regression on a treebank of order10610^\{6\}tokens\. Step \(4\) is a sum ofddsquare roots\. Step \(5\) is one inner product per arm of[Tab\.2](https://arxiv.org/html/2607.22757#S6.T2), six in total\. Step \(6\) is the four\-variable convex program of[Prop\.11](https://arxiv.org/html/2607.22757#Thmprop11)\. Against the100100B tokens and150150K steps of[Sec\.8\.1\.5](https://arxiv.org/html/2607.22757#S8.SS1.SSS5), the whole of it is free, and it is free in the sense that matters: it does not consume the resource the framework is spending its conditioning to buy\.

That asymmetry is the case study’s main point\. The procedure of[Sec\.6](https://arxiv.org/html/2607.22757#S6)publishes, before the first GPU\-hour, a per\-task affinity that can falsify[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1), a certificate for each candidate grading, an ordering of the arms of[Tab\.2](https://arxiv.org/html/2607.22757#S6.T2), and the effect size of[Tab\.3](https://arxiv.org/html/2607.22757#S7.T3)that the runs must then detect\. Whatever the runs return, the prediction they test was fixed in advance by a computation anyone can repeat, and that is a property the ungraded baseline cannot have, not because its profiles are unfavourable, but because the gauge invariance of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(i\) puts its admissibility inner product at0identically\.

## 8\.The Experimental Programme

[Sections3](https://arxiv.org/html/2607.22757#S3),[4](https://arxiv.org/html/2607.22757#S4)and[5](https://arxiv.org/html/2607.22757#S5)developed the Graded Large Language Model: graded attention preserving the asymptotic cost of standard attention \([Prop\.4](https://arxiv.org/html/2607.22757#Thmprop4)\) and compiling to a standard transformer at deployment \([Cor\.1](https://arxiv.org/html/2607.22757#Thmcor1)\), multi\-level graded embeddings with an exact sample\-complexity accounting \([Propositions7](https://arxiv.org/html/2607.22757#Thmprop7)and[3](https://arxiv.org/html/2607.22757#Thmcor3)\) and a closed\-form characterisation of the grades that purchase it \([Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\), and a graded training objective whose consistency conditions are characterised in[Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\. This section specifies the training program that will test the framework, states the predictions the theory supports, and marks explicitly the hypotheses on which each depends\. No experiments have been conducted at the time of writing, and this section reports none\.

The program is staged so that the cheapest test comes first\. Its first stage is a measurement rather than an experiment: both profiles of[Def\.2](https://arxiv.org/html/2607.22757#Thmdefn2)can be estimated offline, so the Bhattacharyya affinityBC​\(α,τ\)\\mathrm\{BC\}\(\\alpha,\\tau\)that[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)identifies with the entire gain of grading is reported per task, and every candidate grading is certified against[Eq\.15](https://arxiv.org/html/2607.22757#S4.E15)at the cost of one inner product, before any model is pretrained\. That stage is the selection procedure of[Sec\.6](https://arxiv.org/html/2607.22757#S6), carried out for language in[Sec\.7](https://arxiv.org/html/2607.22757#S7): the arms of the ladder are certified, the educated gradingqC⋆q^\{\\star\}\_\{C\}is computed by[Propositions10](https://arxiv.org/html/2607.22757#Thmprop10)and[11](https://arxiv.org/html/2607.22757#Thmprop11), and the effect size the runs below must detect is fixed in advance\. The present section takes those quantities as given: it fixes the configurations, pretrains at 345M and 7B, and evaluates\. A framework whose governing hypothesis is measurable before its first GPU\-hour should be tested in that order\.

We state the status of each prediction explicitly\. Several predictions, in particular any reduction in the data required to reach a given generalisation target, follow from[Propositions7](https://arxiv.org/html/2607.22757#Thmprop7)and[8](https://arxiv.org/html/2607.22757#Thmprop8)only under hypotheses that are empirical, not mathematical: that the energy profiles of language diverge \([Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)\), and that the scope limitations of[Rem\.11](https://arxiv.org/html/2607.22757#Thmrem11)do not overturn the single\-layer analysis at the full stack\. Those predictions are marked*conditional*below and name the hypotheses they depend on\.

### 8\.1\.Planned Experimental Setup

#### 8\.1\.1\.Model configurations

Two scales are planned\.GLLM\-Medium\(∼\\sim345M parameters\) mirrors GPT\-2\-medium at 24 layers,d=1024d=1024,H=16H=16, with EG\-MHSA substituting standard MHSA and MLGE substituting standard embeddings atL=4L=4levels and uniform allocationdl=256d\_\{l\}=256\.GLLM\-7B\(∼\\sim7B parameters\) mirrors LLaMA\-7B at 32 layers,d=4096d=4096,H=32H=32, withL=4L=4anddl=1024d\_\{l\}=1024\. Both use the baseλ=e1/dk\\lambda=e^\{1/d\_\{k\}\}fixed in[Sec\.3\.1](https://arxiv.org/html/2607.22757#S3.SS1), with grade clipping enforcingλqmax≤C\\lambda^\{q\_\{\\max\}\}\\leq CatC=2C=2\. The clip is the framework’s single cost–benefit parameter \([Rem\.18](https://arxiv.org/html/2607.22757#Thmrem18)\): it caps the optimisation penalty of[Thm\.5\.1](https://arxiv.org/html/2607.22757#S5.Thmthm1)at a factor ofCC, uniformly inLLand in the profiles, and caps the token gain atC2C^\{2\}by[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\. AtGLLM\-Mediumthe clip is swept overC∈\{1\.5,2,4\}C\\in\\\{1\.5,2,4\\\}as an explicit arm, tracing the price–purchase curve of[Rem\.18](https://arxiv.org/html/2607.22757#Thmrem18)rather than reporting a single point on it\.

#### 8\.1\.2\.Grade initialisation

Grades are initialised from part\-of\-speech statistics computed offline over the pretraining corpus, assigning high grades to open\-class types and low grades to closed\-class function words, and are certified against[Eq\.15](https://arxiv.org/html/2607.22757#S4.E15)as in[Sec\.7\.4](https://arxiv.org/html/2607.22757#S7.SS4)\. Per\-type initialisation is not a matter of convenience:[Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\(3\) establishes that a grade depending on the realised target and its context jointly, a constituency parse depth for instance, renders the training objective inconsistent for the corpus conditional, and by[Rem\.14](https://arxiv.org/html/2607.22757#Thmrem14)the resulting context\-dependent tilt admits no matching by the fixed output grades\. Per\-type grades fall under[Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\(2\), the case in which the matching is available\. A parse\-depth initialisation appears only in the ablation arm described in[Sec\.5\.1](https://arxiv.org/html/2607.22757#S5.SS1)\.

#### 8\.1\.3\.Parameter overhead

The overhead comprises the projection matrices\{Pl\}\\\{P\_\{l\}\\\}at∑ld​dl=d2\\sum\_\{l\}d\\,d\_\{l\}=d^\{2\}parameters, the head grades\{qh\}\\\{q\_\{h\}\\\}atK​H​dkKHd\_\{k\}, the level grades\{ql\}\\\{q\_\{l\}\\\}atdd, and the output gradesqoutq\_\{\\mathrm\{out\}\}at\|𝒱\|\|\\mathcal\{V\}\|, so the total isO​\(d2\+K​H​dk\+\|𝒱\|\)O\(d^\{2\}\+KHd\_\{k\}\+\|\\mathcal\{V\}\|\)and is dominated byd2d^\{2\}\. AtGLLM\-Mediumthis is1\.05​M\+24\.6​K\+1​K\+50\.3​K≈1\.13​M1\.05\\text\{M\}\+24\.6\\text\{K\}\+1\\text\{K\}\+50\.3\\text\{K\}\\approx 1\.13\\text\{M\}, or0\.33%0\.33\\%of345​M345\\text\{M\}; atGLLM\-7Bit is16\.8​M\+131​K\+4​K\+32​K≈17\.0​M16\.8\\text\{M\}\+131\\text\{K\}\+4\\text\{K\}\+32\\text\{K\}\\approx 17\.0\\text\{M\}, or0\.24%0\.24\\%of7​B7\\text\{B\}\. The overhead is below one percent at both scales and negligible against the12​d212d^\{2\}per block of[Prop\.1](https://arxiv.org/html/2607.22757#Thmprop1), but this follows fromd2≪Nd^\{2\}\\ll Nat these configurations rather than from any sublinearity indd, and must be recomputed rather than assumed at other scales\. By[Corollaries1](https://arxiv.org/html/2607.22757#Thmcor1)and[3](https://arxiv.org/html/2607.22757#Thmrem3)the figure is a training cost only, and it is an upper bound on the lifetime cost of the prior\.

#### 8\.1\.4\.Baselines and the ladder

The arms are those of the selection ladder,[Tab\.2](https://arxiv.org/html/2607.22757#S6.T2): uniform \(the standard transformer\), random\-grade, taxonomic, part\-of\-speech, educated \(qC⋆q^\{\\star\}\_\{C\}\), and oracle, all of identical architecture, data, compute, and optimisation budget, distinguished only by what their grades consulted\. The uniform arm is realised as identically sizedGPT\-2\-MediumandLLaMA\-7Bmodels with standard MHSA and embeddings\. The random\-grade arm is the control that isolates the prior from the parameterization: by[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(iv\) a grade direction drawn independently ofα−τ\\alpha\-\\tauhas⟨v,α−τ⟩\\langle v,\\alpha\-\\tau\\rangleof either sign, so it is predicted to straddle the uniform arm rather than to improve on it, and any gain the informed arms show over it is attributable to the prior and not to the extra parameters of[Sec\.8\.1\.3](https://arxiv.org/html/2607.22757#S8.SS1.SSS3)\. Two further baselines sit outside the ladder: a probe\-augmented uniform model, adding syntactic probing classifiers over frozen representations to indicate what post\-hoc structural annotation recovers without architectural grading, and a linear\-grading variant evaluated alongside the exponential to separate their contributions\.

#### 8\.1\.5\.Training

GLLM\-Mediumis pretrained on a 100B\-token subset of The Pile\[gao2020pile\]for 150K steps at batch size 512\.GLLM\-7Bis pretrained on 300B tokens\. Both use AdamW\[loshchilov2017decoupled\]withηmax=6×10−4\\eta\_\{\\max\}=6\\times 10^\{\-4\}and the cosine schedule of\[loshchilov2017sgdr\], under the objective[Eq\.20](https://arxiv.org/html/2607.22757#S5.E20)with the boundedness condition[Eq\.21](https://arxiv.org/html/2607.22757#S5.E21)of[Sec\.5\.2](https://arxiv.org/html/2607.22757#S5.SS2)\. Distributed training uses sharded data parallelism with optimiser\-state partitioning\[rajbhandari2020zero\]and pipeline parallelism\[huang2019gpipe\]\.

#### 8\.1\.6\.Context length

Training sequence lengths must be set by the evaluation the configuration is intended to support\. Discourse\-level grading atl=3l=3is the setting in which MLGE is expected to matter most, and it cannot be assessed at a context of 1024 or 2048 tokens; nor does the graded sinusoidal encoding[Eq\.16](https://arxiv.org/html/2607.22757#S4.E16)extrapolate beyond training length\. Long\-context evaluation therefore requires training at the target context, with the compute that implies, and a rotary or otherwise extrapolable graded positional encoding\. Absent that, the long\-context benchmarks below are out of scope for these configurations and the corresponding predictions are untestable\.

### 8\.2\.Planned Benchmarks

#### 8\.2\.1\.Intrinsic metrics

Held\-out perplexity[Eq\.2](https://arxiv.org/html/2607.22757#S2.E2)on Penn Treebank and WikiText\-103 gives short\-range coverage; PG\-19 gives long\-document coverage at contexts the configurations support\. Perplexity is the*unweighted*loss, and the graded model does not optimise it\. The configurations of[Sec\.8\.1\.2](https://arxiv.org/html/2607.22757#S8.SS1.SSS2)enforce the matched\-tilt condition of[Rem\.14](https://arxiv.org/html/2607.22757#Thmrem14), under which the graded and unweighted objectives share a population minimizer and the perplexity comparison is therefore made on equal terms\. The parse\-depth arm of the component ablation of[Sec\.8\.2\.5](https://arxiv.org/html/2607.22757#S8.SS2.SSS5)is the single arm in which that matching is unavailable, by[Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\(3\), and there any perplexity gap is attributed to the grade\-tilt, the Kullback–Leibler divergence between the corpus conditional and its context\-dependent tilt, and not to the inductive bias\. Every perplexity comparison states which condition holds\.

#### 8\.2\.2\.Sample efficiency

The quantity[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)speaks to is tokens, not steps, and the program measures it directly\. For each task of[Sec\.8\.2\.3](https://arxiv.org/html/2607.22757#S8.SS2.SSS3)we record the held\-out error of every arm as a function of pretraining tokens consumed, and report the ratio of token counts at which each first attains a fixed target error\.[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(ii\) predicts that ratio to beΛ​\(g\)\\Lambda\(g\), estimated in advance by[Sec\.7\.2](https://arxiv.org/html/2607.22757#S7.SS2)and bounded below byC−2C^\{\-2\}at the clipped configurations \([Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\); the scope of[Rem\.11](https://arxiv.org/html/2607.22757#Thmrem11)means the prediction is of the ordering and the sign, not of the constant\. This is the measurement on which the framework’s central claim stands, and it is reported separately from steps\-to\-loss for the reason given in[Rem\.16](https://arxiv.org/html/2607.22757#Thmrem16)\.

#### 8\.2\.3\.Hierarchical tasks

CoLA \(linguistic acceptability\) and MultiRC \(multi\-sentence reading comprehension\) are the primary targets, rewarding syntactic sensitivity and compositional inference respectively\. BoolQ, CB, and WiC provide contrast at lower structural demand, distinguishing gains specific to hierarchy from gains distributed across tasks\.

#### 8\.2\.4\.Long\-context tasks

Subject to the context\-length condition of[Sec\.8\.1\.6](https://arxiv.org/html/2607.22757#S8.SS1.SSS6), SCROLLS\[shaham2022scrolls\], NarrativeQA, and QUALITY test discourse\-level grading\.

#### 8\.2\.5\.Ablation protocol

Two ablations run orthogonally\. The first isolates each graded component: EG\-MHSA alone; MLGE alone; graded loss alone; and their combinations\. It also carries the parse\-depth arm, the intentionally uncalibrated grading of[Sec\.8\.1\.2](https://arxiv.org/html/2607.22757#S8.SS1.SSS2), retained to measure the tilt cost that[Prop\.9](https://arxiv.org/html/2607.22757#Thmprop9)\(3\) predicts\. The second is the selection ladder of[Tab\.2](https://arxiv.org/html/2607.22757#S6.T2), varying only the provenance of the grades across the six arms of[Sec\.8\.1\.4](https://arxiv.org/html/2607.22757#S8.SS1.SSS4): uniform, random, taxonomic level\-constant, POS\-based, the educatedqC⋆q^\{\\star\}\_\{C\}of[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)computed from the estimated profiles, and oracle\. Each arm reports its admissibility inner product⟨qinit,α^−τ^⟩\\langle q^\{\\mathrm\{init\}\},\\hat\{\\alpha\}\-\\hat\{\\tau\}\\ranglefrom[Sec\.7\.4](https://arxiv.org/html/2607.22757#S7.SS4)alongside its result, so that the ladder tests[Eq\.15](https://arxiv.org/html/2607.22757#S4.E15)directly: the arms are predicted to order by that inner product, an arm with a negative value is predicted to underperform the uniform baseline, and the interpretation of each possible outcome is set out in[Sec\.6\.5](https://arxiv.org/html/2607.22757#S6.SS5)\. For the random arm the inner product is of either sign on any given draw, so its predicted position is fixed only once its draw is measured; the aggregate prediction is the straddle of[Sec\.8\.1\.4](https://arxiv.org/html/2607.22757#S8.SS1.SSS4)\. The level count is swept overL∈\{1,2,4,8,16\}L\\in\\\{1,2,4,8,16\\\}; the choiceL=4L=4reflects a linguistic taxonomy, and the sweep is exploratory rather than confirmatory\.

#### 8\.2\.6\.Interpretability

Let𝒜gold\\mathcal\{A\}\_\{\\mathrm\{gold\}\}be the set of gold dependency arcs in a held\-out treebank andAh,i​jA\_\{h,ij\}the attention weight from tokeniito tokenjjin headhh\. Define the structural attention mass

\(23\)SAM​\(h\)=1\|𝒜gold\|​∑\(i,j\)∈𝒜goldAh,i​j,\\mathrm\{SAM\}\(h\)=\\frac\{1\}\{\|\\mathcal\{A\}\_\{\\mathrm\{gold\}\}\|\}\\sum\_\{\(i,j\)\\in\\mathcal\{A\}\_\{\\mathrm\{gold\}\}\}A\_\{h,ij\},the mean attention weight placed on gold arcs\. This is a recall\-like quantity, not a precision, and it is not normalised against the attention placed off gold arcs; it should be reported alongside the total attention mass per head\. Grade\-stratified accuracyAcc​\(l∗\)\\mathrm\{Acc\}\(l^\{\*\}\)partitions benchmark instances by maximum dependency nesting depthl∗l^\{\*\}\.

### 8\.3\.Predictions

Every prediction below names the result it follows from\.

#### 8\.3\.1\.Zero inference overhead

\(established\)\. By[Corollaries1](https://arxiv.org/html/2607.22757#Thmcor1)and[3](https://arxiv.org/html/2607.22757#Thmrem3), the grading transformations are absorbed into the projection and embedding weights by an invertible change of parameters\. A trained GLLM therefore compiles to a standard transformer of identical architecture and identical inference cost, and the0\.33%0\.33\\%and0\.24%0\.24\\%training overheads of[Sec\.8\.1\.3](https://arxiv.org/html/2607.22757#S8.SS1.SSS3)vanish entirely at deployment\. This follows from the construction and is not subject to experimental uncertainty\. By[Rem\.2](https://arxiv.org/html/2607.22757#Thmrem2)it distinguishes grading from every variant of[Tab\.1](https://arxiv.org/html/2607.22757#S2.T1), each of which carries its inductive bias in the computation and pays for it at every forward pass\.

#### 8\.3\.2\.The educated grading is certifiably admissible

\(established given the profile estimates\)\. By[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(iv\) and[Rem\.17](https://arxiv.org/html/2607.22757#Thmrem17), a candidate grading enters the cone𝒬\+\\mathcal\{Q\}\_\{\+\}to first order exactly when⟨qinit,α^−τ^⟩\>0\\langle q^\{\\mathrm\{init\}\},\\hat\{\\alpha\}\-\\hat\{\\tau\}\\rangle\>0, and we predict a strictly positive value for the POS and educated arms on CoLA and MultiRC\. Only the sign must survive the estimation error of[Sec\.7\.2](https://arxiv.org/html/2607.22757#S7.SS2); the magnitudes need not\. The ungraded baseline scores0on this quantity identically, by the gauge invariance of[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(i\), and cannot score otherwise;[Prop\.8](https://arxiv.org/html/2607.22757#Thmprop8)\(iii\) is the interpretation of that zero, as the boundary position of the uniform grades\. The prediction is reported before pretraining and is falsified by a non\-positive value\.

#### 8\.3\.3\.Selection\-ladder ordering

\(conditional on[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)and the scope of[Rem\.11](https://arxiv.org/html/2607.22757#Thmrem11)\)\. The arms of[Tab\.2](https://arxiv.org/html/2607.22757#S6.T2)are predicted to rank, in the token\-efficiency measurement of[Sec\.8\.2\.2](https://arxiv.org/html/2607.22757#S8.SS2.SSS2), in the order of their admissibility inner products, computed offline in[Sec\.7](https://arxiv.org/html/2607.22757#S7)and published before training\. This is the central prediction of the program and the sharpest, because an ordering fixed in advance is far harder to satisfy by accident than any single pairwise comparison\. The partition of outcomes and what each would establish is given in[Sec\.6\.5](https://arxiv.org/html/2607.22757#S6.SS5)\.

#### 8\.3\.4\.Hierarchical task gains

\(conditional on[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)and the scope of[Rem\.11](https://arxiv.org/html/2607.22757#Thmrem11)\)\. Gains are predicted to concentrate where the estimatedBC​\(α,τ\)\\mathrm\{BC\}\(\\alpha,\\tau\)is smallest, by[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)on CoLA and MultiRC, and to diminish on lexically dominated tasks, where the profiles approach agreement and[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)\(iv\) forcesΛ⋆→1\\Lambda^\{\\star\}\\to 1\. That contrast is what makes the gain attributable to hierarchy rather than to the parameterization, and the measurements of[Sec\.7](https://arxiv.org/html/2607.22757#S7)turn it from a qualitative expectation into a quantitative one: the predicted ordering of per\-task gains is the ordering of the measured affinities, fixed in advance of training, with each gain bounded byC2C^\{2\}at the clipped configurations \([Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10)\)\. The measurement is that of[Sec\.8\.2\.2](https://arxiv.org/html/2607.22757#S8.SS2.SSS2), in tokens to a fixed error\. The gain lives in the norm\-controlled regime, which is the regime the runs occupy:[Eq\.20](https://arxiv.org/html/2607.22757#S5.E20)is optimised under the decoupled weight decay of[Sec\.5\.3](https://arxiv.org/html/2607.22757#S5.SS3), so the comparison of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7), with each class evaluated at the smallest budget at which it represents the target, is the one the experiment realises\.

#### 8\.3\.5\.Depth\-stratified robustness

\(conditional on[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1)and the scope of[Rem\.11](https://arxiv.org/html/2607.22757#Thmrem11)\)\. Given[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)and profiles that diverge in the sense of[Assumption1](https://arxiv.org/html/2607.22757#Thmassum1), the graded model’s accuracy is predicted to decay more slowly in nesting depthl∗l^\{\*\}than the ungraded baseline’s, at equal data\. The prediction is qualitative: no result in this manuscript supports a rate forAcc​\(l∗\)\\mathrm\{Acc\}\(l^\{\*\}\), and the sample\-complexity ratio of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)governs the data required to reach a target error, not the shape of the accuracy\-versus\-depth curve\. The two are distinct quantities and should not be conflated\.

#### 8\.3\.6\.Head specialisation

\(conditional on[Eq\.20](https://arxiv.org/html/2607.22757#S5.E20)\)\. The repulsive diversity term of[Eq\.20](https://arxiv.org/html/2607.22757#S5.E20)rewards distinct grade tuples, so heads are predicted to acquire distinguishable gradings and, if the linguistic prior is informative, to align differentially with dependency categories, observable asSAM​\(h\)\\mathrm\{SAM\}\(h\)varying systematically withqhq\_\{h\}\. This is a prediction of the*regulariser*, not of the algebraic structure of EG\-MHSA, which by[Rem\.1](https://arxiv.org/html/2607.22757#Thmrem1)is silent on the question\. Under an attractive penalty the prediction reverses\.

#### 8\.3\.7\.Resource separation

\(established\)\.[Theorems5\.1](https://arxiv.org/html/2607.22757#S5.Thmthm1)and[5](https://arxiv.org/html/2607.22757#Thmlem5)giveς≤λqmax​ςunif\+creg\\varsigma\\leq\\lambda^\{q\_\{\\max\}\}\\varsigma\_\{\\mathrm\{unif\}\}\+c\_\{\\mathrm\{reg\}\}withλqmax≥1\\lambda^\{q\_\{\\max\}\}\\geq 1: grading degrades the smoothness constant\. We therefore predict that graded training does not reduce, and may increase, the optimisation steps to a fixed training loss at matched hyperparameters\. The two resources are priced by the same constant and in the framework’s favour: the step\-count penalty is capped atCCby the grade clipping of[Sec\.8\.1\.1](https://arxiv.org/html/2607.22757#S8.SS1.SSS1), uniformly inLL, while the token saving is capped atC2C^\{2\}by[Prop\.10](https://arxiv.org/html/2607.22757#Thmprop10), a price linear in the clip against a purchase quadratic in it \([Rem\.18](https://arxiv.org/html/2607.22757#Thmrem18)\), with the exponential regime of[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)reached as the clip opens\. TheCC\-sweep of[Sec\.8\.1\.1](https://arxiv.org/html/2607.22757#S8.SS1.SSS1)traces this curve directly\. Since steps are the cheap resource and high\-quality tokens the expensive one, the trade favours the graded model at every setting of the clip\. Any reduction in*tokens to a fixed generalisation target*arises from[Propositions7](https://arxiv.org/html/2607.22757#Thmprop7)and[8](https://arxiv.org/html/2607.22757#Thmprop8)and is visible as a gap in held\-out rather than training loss\. These are different measurements and the program reports them separately, in[Sec\.8\.2\.2](https://arxiv.org/html/2607.22757#S8.SS2.SSS2)and[Sec\.8\.2\.1](https://arxiv.org/html/2607.22757#S8.SS2.SSS1)respectively\.

### 8\.4\.Remaining Theoretical Work

The framework’s central quantity is settled in the regime the program occupies\.[Thm\.4\.1](https://arxiv.org/html/2607.22757#S4.Thmthm1)establishes the separation of[Conj\.2](https://arxiv.org/html/2607.22757#Thmcon2)for level\-stratified geometric profiles at squared loss: a minimax lower bound over the uniform target class, with the estimator unrestricted, met by the matching upper bound of[Lem\.3](https://arxiv.org/html/2607.22757#Thmlem3)throughout the window of[Lem\.4](https://arxiv.org/html/2607.22757#Thmlem4), so that on that window the ratioΛ​\(g\)\\Lambda\(g\)of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)is a genuine separation between the graded prior and its absence rather than a comparison of bounds\. What remains is the general case:[Conj\.2](https://arxiv.org/html/2607.22757#Thmcon2)for arbitrary profiles and Lipschitz losses, whose proof requires extending the packing argument of[Lem\.3](https://arxiv.org/html/2607.22757#Thmlem3)beyond level\-homogeneous designs\. It is independent of the experiments above and could be completed before any of them\.

Three further directions follow from limitations recorded above\. The first limitation of[Rem\.11](https://arxiv.org/html/2607.22757#Thmrem11)calls for the layerwise extension of[Prop\.7](https://arxiv.org/html/2607.22757#Thmprop7)through the graded stack\.[Sec\.4\.5](https://arxiv.org/html/2607.22757#S4.SS5)records that the level\-decomposed feed\-forward block performs no cross\-level mixing, and that directed transport between levels requires block\-triangular maps rather than block\-diagonal ones; the morphic structure of\[sh\-111\]supplies them\. And[Sec\.8\.1\.6](https://arxiv.org/html/2607.22757#S8.SS1.SSS6)makes an extrapolable graded positional encoding a precondition for testing discourse\-level grading at all, which is the setting in which[Cor\.3](https://arxiv.org/html/2607.22757#Thmcor3)predicts the largest effect at open clip\.

## References