Black-Box Inference of LLM Architectural Properties with Restrictive API Access

# Black-Box Inference of LLM Architectural Properties with Restrictive API Access
Source: [https://arxiv.org/html/2607.01313](https://arxiv.org/html/2607.01313)
Christopher Ellis Carnegie Mellon University Pittsburgh, PA 15213 crellis@andrew\.cmu\.edu &Shreyas Chaudhari∗ Carnegie Mellon University Pittsburgh, PA 15213 shreyasc@andrew\.cmu\.edu Mei\-Yu Wang∗ Pittsburgh Supercomputing Center Pittsburgh, PA 15213 mwang7@psc\.edu &Leighton Barnes Carnegie Mellon University Pittsburgh, PA 15213 leightonb@cmu\.edu &Giulia Fanti Carnegie Mellon University Pittsburgh, PA 15213 gfanti@andrew\.cmu\.edu &José M\. F\. Moura Carnegie Mellon University Pittsburgh, PA 15213 moura@andrew\.cmu\.edu

###### Abstract

In practice, most commercial LLM providers do not publicly release details of underlying LLM architectures\. However, prior work has shown that given limited API access to an LLM \(namely, top\-kklogits and/or a logit bias function\), one can recover certain architectural details of an LLM, such as the hidden dimension of the feed\-forward network\. Perhaps in response to these results, most commercial LLM providers have restricted their APIs to expose only the single logit for each decoded token, and they no longer give users the ability to bias logits\. We show that even under current restrictive APIs, several architectural parameters are still recoverable\. We present NightVision, an attack that uses restrictive black\-box API access to estimate the hidden dimension, 0pt, and parameter count of an LLM\. Algorithmically, NightVision relies on a novel*common set*prompting technique in which multiple prompts expose log probabilities for the same set of output tokens; a spectral analysis of these results is used to infer hidden dimension\. NightVision additionally uses end\-to\-end time to first token \(TTFT\) measurements and the estimated hidden dimension to estimate 0ptand parameter count\. We empirically evaluate NightVision on 32 open\-source LLMs, recovering hidden dimension to within23%23\\%average relative error across all models \(9%9\\%on MoE models\), and 0ptand parameter count to within53%53\\%for models exceeding three billion parameters\. We run extensive ablations to demonstrate how these accuracies scale with token budget and model properties\. Overall, our results suggest that current LLM APIs are not sufficiently restricted to fully obfuscate the architectural details of their underlying models\.

## 1Introduction

Can attackers infer architectural properties of a large language model, e\.g\., its hidden dimension, depth, or total parameter count, from API access alone? This question is of great interest to both LLM developers and servers, who may seek to keep their hyperparameters and associated model intellectual property secret\(Tramèret al\.,[2016](https://arxiv.org/html/2607.01313#bib.bib34)\)\. It is also relevant to security researchers and digital forensics experts, who may need to characterize unknown LLM APIs encountered in the wild\. The feasibility of black\-box architectural inference thus has direct implications for both API design\(Hartenstein,[2025](https://arxiv.org/html/2607.01313#bib.bib29); Liuet al\.,[2026](https://arxiv.org/html/2607.01313#bib.bib30)\)and model auditing\(Amirizanianiet al\.,[2024](https://arxiv.org/html/2607.01313#bib.bib31); Mökanderet al\.,[2024](https://arxiv.org/html/2607.01313#bib.bib32); Rastogiet al\.,[2023](https://arxiv.org/html/2607.01313#bib.bib33)\)\.

Recent black\-box attacks\(Zhaoet al\.,[2025](https://arxiv.org/html/2607.01313#bib.bib37)\)have shown that hidden dimension and final\-layer weights \(up to orthogonal transformation\) can be recovered via spectral analysis of output logits\(Carliniet al\.,[2024](https://arxiv.org/html/2607.01313#bib.bib14); Finlaysonet al\.,[2024](https://arxiv.org/html/2607.01313#bib.bib13)\)\. These attacks critically depend on either*full log probabilities*or a*logit\-bias*parameter that allows the attacker to reconstruct full logit vectors from top\-kkprobabilities\. In response, as modern language model APIs continue to evolve\(Chauvinet al\.,[2025](https://arxiv.org/html/2607.01313#bib.bib38)\), providers have started to restrict access to both top\-kklogits and logit bias\(Carliniet al\.,[2024](https://arxiv.org/html/2607.01313#bib.bib14)\), rendering these attacks inapplicable on such APIs\. A natural question is whether architectural inference remains feasible under a more restrictive API that models those of current frontier models\. We answer affirmatively\. Assuming only the log probability of a*single*decoded token at each step \(the minimal probabilistic information consistent with modern LLM APIs\) together with end\-to\-end response timing, which cannot easily be hidden, we recover not only the hidden dimension but also depth and total parameter count, within a margin of error\.

Contributions\.Our contributions are threefold and summarized below\. Overall, our results demonstrate that meaningful inference of sensitive architectural properties remains possible even under restrictive API access, and that closing this channel will require defending against timing side channels in addition to logit access\.

*\(1\) Algorithmic:*We introduce NightVision, a two\-pronged inference algorithm that uses black\-box API access to jointly recover the hidden dimension, 0pt, and parameter count of transformer\-based LLM architectures\.

The first prong of NightVision introduces*common\-set prompting*, a novel method that recovers the hidden dimension under single\-logit API access, requiring neither logit bias nor top\-kkaccess\. Second, we develop a*timing\-based recovery*procedure for depth and parameter count that exploits the distinctive scaling of prefill\-dominated transformer inference with depth, sequence length, and hidden dimension; this stage uses our hidden\-dimension estimate as a building block\.

*\(2\) Theoretical:*We analytically bound in Section[4\.1\.2](https://arxiv.org/html/2607.01313#S4.SS1.SSS2)the number of API calls needed to successfully recover the requisite “common set” used in NightVision\. Via a concentration argument on the size of the intersection of randomly\-sampled token sets across multiple prompts, we derive a sample\-complexity result that quantifies how many samples per prompt are required to form a common set of a target size, or equivalently, the largest hidden dimension recoverable with high probability under a fixed sample budget\.

*\(3\) Experimental:*We evaluate NightVision on 32 open\-source LLMs, showing that it recovers the hidden dimension within23%23\\%mean relative error \(exact recovery in 4 of 32 cases, and within10%10\\%in 12 of 32\)\. Conditioned on an accurate hidden dimension, NightVision recovers depth and total parameter count within∼53%\{\\sim\}53\\%mean relative error on≥3\\geq 3B\-parameter models\.

The accuracy of our estimates depends in part on the*budget*of tokens given to NightVision\. We also provide a fine\-grained characterization of how estimation accuracy scales with budget and architecture properties\.

## 2Technical Preliminaries

##### Architectural Parameters of LLMs\.

Most LLMs can be modeled as stacks ofLLtransformer decoder layers\(Vaswaniet al\.,[2017](https://arxiv.org/html/2607.01313#bib.bib39)\)\. In these architectures, each transformer block operates on hidden representations of a fixed dimensiondd\. We refer toLLas the*depth*,ddas the*hidden dimension*, andPPas the*parameter count*totaled over the entire architecture; together they determine inference cost, memory footprint, and capability, and are commonly held proprietary by LLM providers\. The hidden dimensionddadmits a particularly clean recovery target: the next\-token logits are produced as𝐳=𝐖𝐡\\mathbf\{z\}=\\mathbf\{W\}\\mathbf\{h\}, where𝐡∈ℝd\\mathbf\{h\}\\in\\mathbb\{R\}^\{d\}is the last\-layer hidden state and𝐖∈ℝ\|𝒱\|×d\\mathbf\{W\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\\times d\}is the output embedding matrix over a vocabulary𝒱\\mathcal\{V\}\. Since\|𝒱\|≫d\|\\mathcal\{V\}\|\\gg din practice, the matrix of logits collected across many prompts has rank at mostdd\. This*softmax bottleneck*is the structural fact exploited by prior attacks\(Carliniet al\.,[2024](https://arxiv.org/html/2607.01313#bib.bib14); Finlaysonet al\.,[2024](https://arxiv.org/html/2607.01313#bib.bib13)\)and by our spectral method in[Section˜4\.1\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS1.Px3)\.

##### Threat Model\.

Consider a*target model*ϕ\\phithat consists of stacked transformer blocks, each with the same hidden dimensionddand 0ptLL, as is common in modern LLM architectures\. Given API access toϕ\\phi, our goal is to estimatedd,LL, and parameter countPP\. Given an input sequenceS=\(x1,…,xt−1\)S=\(x\_\{1\},\\dots,x\_\{t\-1\}\), an LLM produces a next\-token logit vector𝐳S∈ℝ\|𝒱\|\\mathbf\{z\}\_\{S\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\}with entrieszS\(i\)z\_\{S\}^\{\(i\)\}\. Many APIs expose a temperature parameterτ\>0\\tau\>0that rescales the logits prior to the softmax, yielding a probability vector𝐩S​\(τ\)=softmax​\(𝐳S/τ\)∈Δ\|𝒱\|−1\\mathbf\{p\}\_\{S\}\(\\tau\)=\\mathrm\{softmax\}\(\\mathbf\{z\}\_\{S\}/\\tau\)\\in\\Delta^\{\|\\mathcal\{V\}\|\-1\}with entriespS\(i\)​\(τ\)=exp⁡\(zS\(i\)/τ\)/∑jexp⁡\(zS\(j\)/τ\)=ℙ​\(xt=vi∣x1,…,xt−1\)p\_\{S\}^\{\(i\)\}\(\\tau\)\\\!=\\\!\\exp\(z\_\{S\}^\{\(i\)\}/\\tau\)/\\sum\_\{j\}\\exp\(z\_\{S\}^\{\(j\)\}/\\tau\)\\\!=\\\!\\mathbb\{P\}\(x\_\{t\}=v\_\{i\}\\mid x\_\{1\},\\dots,x\_\{t\-1\}\)and a corresponding log\-probability vector𝐲S​\(τ\)\\mathbf\{y\}\_\{S\}\(\\tau\)with entriesyS\(i\)​\(τ\)=log⁡pS\(i\)​\(τ\)y\_\{S\}^\{\(i\)\}\(\\tau\)\\\!=\\\!\\log p\_\{S\}^\{\(i\)\}\(\\tau\); largerτ\\tauflattens the distribution toward uniform, andτ=1\\tau=1recovers the unscaled distribution𝐩S≜𝐩S​\(1\)\\mathbf\{p\}\_\{S\}\\triangleq\\mathbf\{p\}\_\{S\}\(1\)\. We useviv\_\{i\}fori∈\[\|𝒱\|\]i\\in\[\|\\mathcal\{V\}\|\]to denote theiith token label\. We operate in a restricted black\-box setting in which none of𝐳S\\mathbf\{z\}\_\{S\},𝐳S​\(τ\)\\mathbf\{z\}\_\{S\}\(\\tau\),𝐩S\\mathbf\{p\}\_\{S\}, or𝐩S​\(τ\)\\mathbf\{p\}\_\{S\}\(\\tau\)is observed\. Once the model decodes a next tokenvi0v\_\{i\_\{0\}\}by sampling under𝐩S​\(τ\)\\mathbf\{p\}\_\{S\}\(\\tau\), only the realized log\-probabilityyS\(i0\)​\(τ\)y\_\{S\}^\{\(i\_\{0\}\)\}\(\\tau\)is revealed\. When prompts are indexed asSqS\_\{q\}\(as in our common\-set construction in[Section˜4\.1](https://arxiv.org/html/2607.01313#S4.SS1)\), we abbreviatezq\(i\)≜zSq\(i\)z\_\{q\}^\{\(i\)\}\\triangleq z\_\{S\_\{q\}\}^\{\(i\)\},pq\(i\)≜pSq\(i\)p\_\{q\}^\{\(i\)\}\\triangleq p\_\{S\_\{q\}\}^\{\(i\)\}, andyq\(i\)≜ySq\(i\)y\_\{q\}^\{\(i\)\}\\triangleq y\_\{S\_\{q\}\}^\{\(i\)\}\. Hence, our threat models are strictly weaker \(give less information to the attacker\) than those ofCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)andFinlaysonet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib13)\), which additionally grant top\-kkentries of𝐩S​\(τ\)\\mathbf\{p\}\_\{S\}\(\\tau\)or logit\-bias control prior to the final softmax\. Our common\-set prompting method \([Section˜4\.1](https://arxiv.org/html/2607.01313#S4.SS1)\) benefits from flatter distributions, so we useτ=2\\tau=2\(the cap on most current APIs\)

## 3Related Work

Carliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)andFinlaysonet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib13)\)concurrently showed that an LLM’s hidden dimension can be recovered from API access by exploiting the softmax bottleneck: the matrix of output logits across many prompts has rank equal to the hidden dimension, which can be readily determined via singular value decomposition\.Carliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)further recover the final output layer up to orthogonal transformations\. Crucially, both methods depend on a logit\-bias parameter to reconstruct full logit vectors from the API’s top\-kklog\-probability outputs \(withk≥1k\\geq 1\)\(Morriset al\.,[2023](https://arxiv.org/html/2607.01313#bib.bib36)\), and both are rendered ineffective once this functionality is removed\. We provide a detailed algorithmic comparison between our method and that ofCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)in[Section˜4\.1](https://arxiv.org/html/2607.01313#S4.SS1)\.

Table 1:Prior work on black\-box recovery of architectural properties from production LLM APIs\. The left block shows the access the attack*requires*; the right block shows what it*recovers*\. All prior attacks that recover hidden dimension require a logit\-bias parameter \(used to*reconstruct*full logit vectors from a single decoded log\-probability, which is why “Full logits” is not checked\), a feature that has been removed or restricted by major providers in response\.Required accessRecoveredWorkTextonlyFulllogitsLogitbiasSinglelogprobInferencetimingHiddenDepth\#ParamsCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)✓✓✓Finlaysonet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib13)\)✓✓✓Li\([2026](https://arxiv.org/html/2607.01313#bib.bib27)\)✓✓NightVision \(basic common\-set\)✓✓NightVision \(with timing\)✓✓✓✓✓More recently,Li \([2026](https://arxiv.org/html/2607.01313#bib.bib27)\)take an alternative approach, in which rather than probing the model’s internals, they attempt to probe what it has learned\. Their method,*Incompressible Knowledge Probes*\(IKPs\), tests a model on obscure factual questions whose answers cannot be inferred from general reasoning and must instead be memorized in the model’s weights\. Since storing more facts requires more parameters, a model’s accuracy on these probes serves as a proxy for its size, once calibrated against open\-weight models of known size\. Table[1](https://arxiv.org/html/2607.01313#S3.T1)positions our work against these methods by threat model, namely, the API access each attack requires, and which architectural properties it recovers\. Our method requires the strictly weakest probabilistic API access of any approach that recovers the hidden dimension, and it is the only method that jointly recovers depth, hidden dimension, and parameter count\. Due to space constraints, we discuss additional related work in Appendix[A](https://arxiv.org/html/2607.01313#A1)\.

## 4NightVision: Algorithm and Analysis

![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/nightvision.png)Figure 1:NightVision overviewWe first give an overview of NightVision, as illustrated in Figure[1](https://arxiv.org/html/2607.01313#S4.F1), followed by a detailed explanation of its components\. NightVision proceeds in two phases\. The first phase, which we call the*common\-set prompting attack*, is used to recover the hidden dimensiondd\([Section˜4\.1](https://arxiv.org/html/2607.01313#S4.SS1)\)\. The second phase, which we refer to as the*timing attack*recovers the parameter count and 0pt, given an estimate for hidden dimensiond^\\hat\{d\}obtained in phase 1 \([Section˜4\.2](https://arxiv.org/html/2607.01313#S4.SS2)\)\. Given a total budget ofBBtokens for the attack \(we find thatB\>2×1011B\>2\\times 10^\{11\}tokens are required in practice for LLMs with billions of parameters\), we allocate the first1010million tokens to the timing attack, and the rest to the common\-set prompting attack\.

### 4\.1Phase 1: NightVision Common\-Set Prompting Attack

We first review the attack ofCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)under top\-1 logit access and explain why it is infeasible under modern APIs\. We then present our common\-set prompting attack \([Section˜4\.1\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS1)\) and analyze its sample complexity \([Section˜4\.1\.2](https://arxiv.org/html/2607.01313#S4.SS1.SSS2)\)\.

Carliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\): Top\-11binary logit\-bias attackNightVision Phase 1: Common\-set prompting attack1:2:Top\-11logprob APIwith binary biasb∈\{−1,0\}b\\in\\\{\-1,0\\\}3:Prompts𝒟=\{Sq\}q=1n\\mathcal\{D\}=\\\{S\_\{q\}\\\}\_\{q=1\}^\{n\}withn\>dn\>d1:2:Single sampled logprob API\(no logit bias\)3:High\-entropyprompts𝒟=\{Sq\}q=1D\\mathcal\{D\}=\\\{S\_\{q\}\\\}\_\{q=1\}^\{D\}, number of samplesNN, temperatureτ\\tauStage 1: Collect Outputs3:forq=1,…,nq=1,\\dots,ndo4:Query unbiased; record top tokenrrand its logprobytopy\_\{\\mathrm\{top\}\}5:foreacht∈𝒱∖\{r\}t\\in\\mathcal\{V\}\\setminus\\\{r\\\}do6:Query biased; record logprobytop′y^\{\\prime\}\_\{\\mathrm\{top\}\}7:Recover prob\.:pq\(t\)=exp⁡\(ytop−ytop′\)−11/e−1p\_\{q\}^\{\(t\)\}=\\dfrac\{\\exp\(y\_\{\\mathrm\{top\}\}\-y^\{\\prime\}\_\{\\mathrm\{top\}\}\)\-1\}\{1/e\-1\}8:Convert to logit:zq\(t\)=log⁡pq\(t\)−ytopz\_\{q\}^\{\(t\)\}=\\log p\_\{q\}^\{\(t\)\}\-y\_\{\\mathrm\{top\}\}9:endfor10:endfor3:forq=1,…,Dq=1,\\dots,Ddo4:fork=1,…,Nk=1,\\dots,Ndo5:Sample at temperatureτ\\tau; record\(vik,yq\(i\)\)\\left\(v\_\{i\_\{k\}\},\\,y\_\{q\}^\{\(i\)\}\\right\)6:endfor7:𝒯q←\{unique tokens observed\}\\mathcal\{T\}\_\{q\}\\leftarrow\\\{\\text\{unique tokens observed\}\\\}8:endforStage 2: Form Matrix9:Build𝐐∈ℝn×\|𝒱\|\\mathbf\{Q\}\\in\\mathbb\{R\}^\{n\\times\|\\mathcal\{V\}\|\}with𝐐q,t=zq\(t\)\\mathbf\{Q\}\_\{q,t\}=z\_\{q\}^\{\(t\)\}9:Common set𝒞\\mathcal\{C\}and common prompts𝒟′\\mathcal\{D\}^\{\\prime\}formed via greedy pruning of sparseD×\|𝒱\|D\{\\times\}\|\\mathcal\{V\}\|matrix \([Section˜4\.1\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS1.Px2)\)10:Build𝐘∈ℝD′×\|𝒞\|\\mathbf\{Y\}\\in\\mathbb\{R\}^\{D^\{\\prime\}\\times\|\\mathcal\{C\}\|\}withYq,i=yq\(vi\)Y\_\{q,i\}=y\_\{q\}^\{\(v\_\{i\}\)\}forvi∈𝒞v\_\{i\}\\in\\mathcal\{C\}Stage 3: Estimateddvia spectrum11:Compute SVD of𝐐\{\\mathbf\{Q\}\}; extractσ1≥σ2≥⋯\\sigma\_\{1\}\\geq\\sigma\_\{2\}\\geq\\cdots12:returnd^\\hat\{d\}at the largest gap in\{σi\}\\\{\\sigma\_\{i\}\\\}11:Eigendecompose𝐆=𝐘𝐘⊤\\mathbf\{G\}=\\mathbf\{Y\}\\mathbf\{Y\}^\{\\top\}; extractλ1≥λ2≥⋯\\lambda\_\{1\}\\geq\\lambda\_\{2\}\\geq\\cdots12:returnd^\\hat\{d\}via saddle\-then\-peak inflection onai=arctan⁡\(λi\)a\_\{i\}=\\arctan\(\\lambda\_\{i\}\)\(saddle atai≤arctan⁡\(π\)a\_\{i\}\\\!\\leq\\\!\\arctan\(\\pi\), peak atai≤arctan⁡\(π/2\)a\_\{i\}\\\!\\leq\\\!\\arctan\(\\pi/2\);[Section˜4\.1\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS1.Px3)\)Figure 2:Comparison of our common\-set prompting attack to Top\-1 binary logit\-bias attack ofCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)\.##### The attack ofCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)

The attack ofCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)follows a three\-stage procedure, illustrated in Figure[2](https://arxiv.org/html/2607.01313#S4.F2)\(left\): \(1\) collect outputs from prompts, \(2\) assemble a logit matrix, and \(3\) estimated^\\hat\{d\}from its singular values\. In stage \(1\), they query the target LLMϕ\\phiwith a set𝒟\\mathcal\{D\}ofn\>dn\>ddistinct prompts\. For each promptqq, they record the top tokenrrand its log\-probabilityytopy\_\{\\mathrm\{top\}\}\. Then, for each non\-top tokentt, they submit a query with a logit bias of−1\-1ontt, which suppressesttand yields a new top log\-probabilityytop′y^\{\\prime\}\_\{\\mathrm\{top\}\}forrr\. The shiftytop−ytop′y\_\{\\mathrm\{top\}\}\-y^\{\\prime\}\_\{\\mathrm\{top\}\}depends on the original probability oftt, and inverting it in closed form givespq\(t\)=\(exp⁡\(ytop−ytop′\)−1\)/\(1/e−1\)p\_\{q\}^\{\(t\)\}=\(\\exp\(y\_\{\\mathrm\{top\}\}\-y^\{\\prime\}\_\{\\mathrm\{top\}\}\)\-1\)/\(1/e\-1\), the probability ofttunder promptqq\. In stage \(2\), these probabilities are converted to logits and collected into ann×\|𝒱\|n\\times\|\\mathcal\{V\}\|matrix𝐐\\mathbf\{Q\}, with one row per prompt and one column per token in𝒱\\mathcal\{V\}\. In stage \(3\), the singular values of𝐐\\mathbf\{Q\}recover the hidden dimension: the rank shows up as a pronounced drop in the ordered singular values \([Section˜B\.1\.3](https://arxiv.org/html/2607.01313#A2.SS1.SSS3)\), at an index that typically matches the truedd\. This attack is infeasible in our threat model, where logit bias is unavailable: without it, individual token probabilities cannot be recovered and𝐐\\mathbf\{Q\}cannot be constructed\.

#### 4\.1\.1Common\-Set Prompting

Our common\-set prompting attack follows the same three high\-level stages asCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\): \(1\) output collection, \(2\) matrix formation, and \(3\) spectral estimation\. Each stage is instantiated differently, as shown in[Figure˜2](https://arxiv.org/html/2607.01313#S4.F2)\.

##### \(1\) Output Collection: Common\-Set Prompting

Our goal is to construct a matrix𝐘\\mathbf\{Y\}analogous to𝐐\\mathbf\{Q\}fromCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\), in which each row corresponds to a prompt from a set𝒟′\\mathcal\{D\}^\{\\prime\}, each column corresponds to a token from a set𝒞⊆𝒱\\mathcal\{C\}\\subseteq\\mathcal\{V\}, and entries are log\-probabilities of tokensc∈𝒞c\\in\\mathcal\{C\}under promptsS∈𝒟′S\\in\\mathcal\{D\}^\{\\prime\}\. Without logit bias, a token’s log\-probability is only visible when the model decodes it\. We pick𝒟′\\mathcal\{D\}^\{\\prime\}and𝒞\\mathcal\{C\}jointly so that, under repeated sampling ofϕ\\phi,*every*token in𝒞\\mathcal\{C\}appears at least once for*every*prompt in𝒟′\\mathcal\{D\}^\{\\prime\}\. Spectral recovery requires\|𝒞\|\>d\|\\mathcal\{C\}\|\>d, so we seek high\-entropy input prompts likely to produce a large vocabulary of output tokens\.

Specifically, we start with a set𝒟=\{Sq\}q=1D\\mathcal\{D\}=\\\{S\_\{q\}\\\}\_\{q=1\}^\{D\}ofDDprompts chosen to induce high\-entropy distributions over the next token \(details below\)\. For each promptSq∈𝒟S\_\{q\}\\in\\mathcal\{D\}, we drawNNindependent samples fromϕ​\(Sq\)\\phi\(S\_\{q\}\)at temperatureτ\\tau\(concrete values forDD,NN, andτ\\tauare listed in[Section˜B\.2](https://arxiv.org/html/2607.01313#A2.SS2.SSS0.Px1)\)\. Each callk∈\[N\]k\\in\[N\]returns a single decoded tokenvikv\_\{i\_\{k\}\}sampled from the temperature\-scaled distribution𝐩​\(τ\)\\mathbf\{p\}\(\\tau\), together with its log\-probabilityyq\(ik\)y\_\{q\}^\{\(i\_\{k\}\)\}under that distribution\. No other entries of the output distribution are revealed\.

Let𝒯q⊆𝒱\\mathcal\{T\}\_\{q\}\\subseteq\\mathcal\{V\}denote the set of unique tokens decoded under promptSqS\_\{q\}across itsNNsamples,𝒯q≜∪k∈\[N\]\{vik\}\\mathcal\{T\}\_\{q\}\\triangleq\\cup\_\{k\\in\[N\]\}\\\{v\_\{i\_\{k\}\}\\\}\. We define the*common set*as𝒞≜∩q∈\[D\]𝒯q\\mathcal\{C\}\\;\\triangleq\\;\\cap\_\{q\\in\[D\]\}\\mathcal\{T\}\_\{q\}, i\.e\., the set of tokens decoded at least once under every prompt\.

The number of samplesNNrequired to form a common set of a given size depends on the next\-token distributions\.

Increasing temperatureτ\\taualone is typically insufficient to drive\|𝒞\|\|\\mathcal\{C\}\|abovedd, so prompt selection becomes the dominant lever for vocabulary coverage \([Section˜B\.1\.1](https://arxiv.org/html/2607.01313#A2.SS1.SSS1)\)\. We find that out\-of\-distribution token sequences \(inputs the model has likely not seen during training\) reliably produce diffuse predictions\.

Our automated prompt search \([Section˜B\.1\.1](https://arxiv.org/html/2607.01313#A2.SS1.SSS1)\) draws candidates from character groups including Latin, CJK, emoji, Arabic, Hebrew, Braille, and mathematical symbols\.

##### \(2\) Sampling and Common Token Matrix Construction

We populate a sparse log\-probability matrix𝐘full^∈ℝD×\|𝒱\|\\hat\{\\mathbf\{Y\}\_\{\\mathrm\{full\}\}\}\\in\\mathbb\{R\}^\{D\\times\|\\mathcal\{V\}\|\}with\[𝐘full^\]q,i=yq\(i\)\[\\hat\{\\mathbf\{Y\}\_\{\\mathrm\{full\}\}\}\]\_\{q,i\}=y\_\{q\}^\{\(i\)\}forvi∈𝒯qv\_\{i\}\\in\\mathcal\{T\}\_\{q\}andNaN\\mathrm\{NaN\}elsewhere\. Different prompts cover different parts of the vocabulary, so𝐘full^\\hat\{\\mathbf\{Y\}\_\{\\mathrm\{full\}\}\}contains many missing entries and cannot be fed to the spectral step directly\. We apply greedy NaN\-pruning \([Section˜B\.1\.2](https://arxiv.org/html/2607.01313#A2.SS1.SSS2)\) to extract a dense submatrix: the surviving columns form the*common set*𝒞\\mathcal\{C\}, and the surviving rows define a subset of prompts𝒟′⊆𝒟\\mathcal\{D\}^\{\\prime\}\\subseteq\\mathcal\{D\}retained for spectral analysis\. LetD′=\|𝒟′\|D^\{\\prime\}=\|\\mathcal\{D\}^\{\\prime\}\|denote the post\-pruning prompt count; the resulting dense submatrix𝐘∈ℝD′×\|𝒞\|\\mathbf\{Y\}\\in\\mathbb\{R\}^\{D^\{\\prime\}\\times\|\\mathcal\{C\}\|\}feeds the spectral step\.

##### \(3\) Spectral Hidden\-Dimension Estimation

We recoverddfrom𝐘\\mathbf\{Y\}via spectral analysis, followingCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)\. The singular values of𝐘\\mathbf\{Y\}are noisy, so we work instead with the Gram matrix𝐆=𝐘𝐘⊤\\mathbf\{G\}=\\mathbf\{Y\}\\mathbf\{Y\}^\{\\top\}\(or𝐘⊤​𝐘\\mathbf\{Y\}^\{\\top\}\\mathbf\{Y\}whenD′\>\|𝒞\|D^\{\\prime\}\>\|\\mathcal\{C\}\|\) to amplify the signal\. Its eigendecomposition𝐆=𝐔​𝚲​𝐔⊤\\mathbf\{G\}=\\mathbf\{U\}\\,\\boldsymbol\{\\Lambda\}\\,\\mathbf\{U\}^\{\\top\}yields ordered eigenvaluesλ1≥λ2≥⋯≥λmin⁡\(D′,\|𝒞\|\)≥0\\lambda\_\{1\}\\geq\\lambda\_\{2\}\\geq\\cdots\\geq\\lambda\_\{\\min\(D^\{\\prime\},\|\\mathcal\{C\}\|\)\}\\geq 0\. The leading eigenvalues span several orders of magnitude while the tail is comparatively flat, so the elbow rule ofCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)does not isolate a clean inflection\. We use a custom change\-point detector \([Section˜B\.1\.3](https://arxiv.org/html/2607.01313#A2.SS1.SSS3)\): a transform of the spectrum exposes the head\-to\-tail transition, two geometric landmarks bracketd^\\hat\{d\}from above and below, and the point of maximum curvature within that bracket is taken asd^\\hat\{d\}\. The detector exposes a few hyperparameters \(slope half\-window, smoothing window, prominence fraction\); the values used throughout the evaluation are listed in[Section˜B\.2](https://arxiv.org/html/2607.01313#A2.SS2.SSS0.Px1)\. Finally, we snapd^\\hat\{d\}to a grid spacingg=128g=128, since hidden dimensions of deployed models are almost always multiples of powers of two\.

#### 4\.1\.2Theoretical Analysis: Sample Complexity of the Common\-Set Attack

We bound the number of samples per prompt sufficient to construct a common\-set matrix large enough for spectral recovery of the hidden dimension\. The spectral method \([Section˜4\.1\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS1.Px3)\) operates on a matrix𝐘∈ℝD′×\|𝒞\|\\mathbf\{Y\}\\in\\mathbb\{R\}^\{D^\{\\prime\}\\times\|\\mathcal\{C\}\|\}whose rank reflects the hidden dimensiondd, so recovery requires bothD′≥dD^\{\\prime\}\\geq dand\|𝒞\|≥d\|\\mathcal\{C\}\|\\geq d\. The first condition is controlled by the attacker via the sizeD≥D′D\\geq D^\{\\prime\}of the prompt collection; the second is a probabilistic event determined by how many samples are drawn per prompt\. We hence study the sample sizeNNrequired to ensure\|𝒞\|≥d\|\\mathcal\{C\}\|\\geq dwith high probability\.

###### Theorem 1\.

\[Sample complexity for spectral recovery\] Let𝒱\\mathcal\{V\}be a finite vocabulary of size\|𝒱\|\|\\mathcal\{V\}\|, and let𝒯1,…,𝒯D\\mathcal\{T\}\_\{1\},\\dots,\\mathcal\{T\}\_\{D\}beDDindependent random subsets of𝒱\\mathcal\{V\}, each formed by drawingNNtokens uniformly at random with replacement from𝒱\\mathcal\{V\}\. Let𝒞≜𝒯1∩⋯∩𝒯D\\mathcal\{C\}\\triangleq\\mathcal\{T\}\_\{1\}\\cap\\cdots\\cap\\mathcal\{T\}\_\{D\}denote their common intersection, andd≤Dd\\leq Dbe a target threshold\. If

N≥ln⁡\[1−\(d\+2​d​ln⁡\(1/δ\)\+2​ln⁡\(1/δ\)\|𝒱\|\)1/D\]ln⁡\(1−1/\|𝒱\|\)N\\geq\\frac\{\\ln\\\!\\left\[1\-\\left\(\\frac\{d\+\\sqrt\{2d\\ln\(1/\\delta\)\}\+2\\ln\(1/\\delta\)\}\{\|\\mathcal\{V\}\|\}\\right\)^\{1/D\}\\right\]\}\{\\ln\(1\-1/\|\\mathcal\{V\}\|\)\}\(1\)then\|𝒞\|≥d\|\\mathcal\{C\}\|\\geq dwith probability at least1−δ1\-\\delta\.

\(Proof in[Appendix˜F](https://arxiv.org/html/2607.01313#A6)\) Expanding \([1](https://arxiv.org/html/2607.01313#S4.E1)\) \(also in[Appendix˜F](https://arxiv.org/html/2607.01313#A6)\) shows that the total sufficient query budget roughly scales asD​N=O​\(\|𝒱\|​D​ln⁡D\)DN=O\(\|\\mathcal\{V\}\|D\\ln D\)\.[Theorem˜1](https://arxiv.org/html/2607.01313#Thmtheorem1)assumes uniformity, an idealization of the high\-entropy prompts produced by our search \([Section˜4\.1](https://arxiv.org/html/2607.01313#S4.SS1)\)\.[Corollary˜1](https://arxiv.org/html/2607.01313#Thmcorollary1)in[Appendix˜F](https://arxiv.org/html/2607.01313#A6)generalizes the bound to non\-uniform distributions over a sub\-vocabulary𝒱′⊆𝒱\\mathcal\{V\}^\{\\prime\}\\subseteq\\mathcal\{V\}\. The same form holds with\|𝒱\|\|\\mathcal\{V\}\|and1/\|𝒱\|1/\|\\mathcal\{V\}\|in the numerator and denominator respectively replaced by\|𝒱′\|\|\\mathcal\{V\}^\{\\prime\}\|andttin the appropriate places\. We empirically validate the predicted scaling against measured common\-set sizes in[Section˜5\.2](https://arxiv.org/html/2607.01313#S5.SS2)\.

### 4\.2Phase 2: NightVision Timing Attack

In Phase 2, NightVision uses end\-to\-end inference time, from API call to token return, as a complementary side\-channel signal to infer 0ptand parameter count\.

The timing attack \(full pseudocode in[Figure˜11](https://arxiv.org/html/2607.01313#A2.F11)\) has three sub\-phases: \(1\) timing collection, \(2\) 0ptestimation, and \(3\) parameter count inference\. Our attack assumes access to a collection ofKKmodelsℱ=\{ϕi\}i=1K\\mathcal\{F\}=\\\{\\phi\_\{i\}\\\}\_\{i=1\}^\{K\}, which could be obtained from open\-source models\. These are used to learn a timing scaling relation, which is applied to our target modelϕ\\phi\.

##### \(1\) Timing Collection

Transformer\-based LLM inference comprises two stages:prefill, in which the model processes the input prompt to build hidden states and populate the per\-layer key\-value \(KV\) cache, anddecode, in which tokens are generated autoregressively, reusing cached keys and values to avoid recomputing the full context\. The prefill stage tends to be more predictable computationally due to optimizations once the KV cache is filled, such as Grouped\-Query Attention \(GQA\)\(Ainslieet al\.,[2023](https://arxiv.org/html/2607.01313#bib.bib4)\)\. Hence, we design prompts with long input sequences and minimal generation length so that end\-to\-end latency is dominated by prefill\.

To make prefill dominate, we create a set ofN′N^\{\\prime\}diverse input sequence lengthsℒ=\{ℓj\}j=1N′\\mathcal\{L\}=\\\{\\ell\_\{j\}\\\}\_\{j=1\}^\{N^\{\\prime\}\}\(distinct from the prompt set𝒟\\mathcal\{D\}in Section[4\.1\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS1.Px3)\)\. For each input lengthℓj\\ell\_\{j\}\(j∈\[N′\]j\\in\[N^\{\\prime\}\]\) and each training modelϕi\\phi\_\{i\}, we collect an end\-to\-end timing signalTi,jT\_\{i,j\}\.

##### \(2\) DepthLLEstimation

To estimateLL, we model the prefill\-dominated runtimeTi,jT\_\{i,j\}for modelϕi\\phi\_\{i\}on input lengthℓj\\ell\_\{j\}as:

Ti,j≈β​Li​ℓj2​di\+α,T\_\{i,j\}\\approx\\beta\\,L\_\{i\}\\,\\ell\_\{j\}^\{2\}\\,d\_\{i\}\+\\alpha,\(2\)whereLiL\_\{i\}is the layer count,did\_\{i\}the true hidden dimension, andβ\\beta,α\\alphaare empirical fitting parameters\. TheLi​ℓj2​diL\_\{i\}\\,\\ell\_\{j\}^\{2\}\\,d\_\{i\}term reflects the per\-layerO​\(ℓ2​d\)O\(\\ell^\{2\}d\)attention cost\(Vaswaniet al\.,[2017](https://arxiv.org/html/2607.01313#bib.bib39); Daoet al\.,[2022](https://arxiv.org/html/2607.01313#bib.bib5)\), which dominates theO​\(ℓ​d2\)O\(\\ell\\,d^\{2\}\)projection and FFN terms in the long\-context regime\(Kaplanet al\.,[2020](https://arxiv.org/html/2607.01313#bib.bib1)\)\. We show that this is a good model for timing measurements empirically in[Section˜5\.3](https://arxiv.org/html/2607.01313#S5.SS3)\. The coefficientβ\\betareflects a combination of system\-level factors \(kernel efficiency, e\.g\., FlashAttention\(Daoet al\.,[2022](https://arxiv.org/html/2607.01313#bib.bib5)\); hardware bandwidth; numerical precision; parallelism\) and architectural choices \(multi\-head versus grouped\-query attention \(GQA\)\(Ainslieet al\.,[2023](https://arxiv.org/html/2607.01313#bib.bib4)\); two\-matrix GELU versus three\-matrix SwiGLU FFN\)\. Despite these sources of variation, we find that a single globalβ\\betaprovides a good approximation for models exceeding approximately 3B parameters that employ LLaMA\-style transformer blocks, the predominant architectural choice in modern large language models\.

Our basic estimator forLLfor a target modelϕ\\philearns a globalβ\\betaandα\\alphafrom our training setℱ\\mathcal\{F\}via linear regression, using the variety of prompt lengths in the prompting set\. Next, we plugα\\alpha,β\\beta, and the hidden dimension estimated^\\hat\{d\}from the common\-set token attack into \([2](https://arxiv.org/html/2607.01313#S4.E2)\) to solve forLL\. We intentionally retain a single linear term in the fitting relation and ignore system\-level variations stemming from diverse inference engine configurations and accelerator types, which would otherwise confound multi\-term fits\. We discuss the effect of model architecture \(including MoE models\) in[Section˜B\.3\.2](https://arxiv.org/html/2607.01313#A2.SS3.SSS2)and extend this algorithm to a setting where the training set is run on different hardware than the target model in[Section˜B\.3\.3](https://arxiv.org/html/2607.01313#A2.SS3.SSS3)\.

##### \(3\) Total Parameter CountPPEstimation

Given our estimated hidden dimensiond^\\hat\{d\}, estimated layer countL^\\hat\{L\}, and an upper bound on vocabulary sizeV^\\hat\{V\}, we estimate the total parameter count assuming a LLaMA\-style decoder\-only transformer with GQA, SwiGLU feed\-forward layers, and untied input/output embeddings\. Although this architecture is quite different from many LLMs, we find that this model is sufficiently close in practice to predict parameter counts to reasonable accuracy \(at least within an order of magnitude\)\.

P^=2​V^​d^\+L^​\(2\+2​nkvnheads\+3​dffnd^\)​d^2\+\(2​L^\+1\)​d^,\\hat\{P\}=2\\hat\{V\}\\hat\{d\}\+\\hat\{L\}\\left\(2\+2\\,\\frac\{n\_\{\\mathrm\{kv\}\}\}\{n\_\{\\mathrm\{heads\}\}\}\+3\\,\\frac\{d\_\{\\mathrm\{ffn\}\}\}\{\\hat\{d\}\}\\right\)\\hat\{d\}^\{\\,2\}\+\(2\\hat\{L\}\+1\)\\hat\{d\},\(3\)wherenkv/nheadsn\_\{\\mathrm\{kv\}\}/n\_\{\\mathrm\{heads\}\}is the key\-value head ratio anddffn/d^d\_\{\\mathrm\{ffn\}\}/\\hat\{d\}is the feed\-forward expansion ratio\. The first term accounts for the input embedding and output projection matrices \(untied, as in LLaMA\)\. The second term captures the per\-layer parameter budget: query and output projections contribute2​d^22\\hat\{d\}^\{2\}, key\-value projections contribute2​\(nkv/nheads\)​d^22\(n\_\{\\mathrm\{kv\}\}/n\_\{\\mathrm\{heads\}\}\)\\hat\{d\}^\{2\}, and the three SwiGLU weight matrices \(WgateW\_\{\\mathrm\{gate\}\},WupW\_\{\\mathrm\{up\}\},WdownW\_\{\\mathrm\{down\}\}\) contribute3​\(dffn/d^\)​d^23\(d\_\{\\mathrm\{ffn\}\}/\\hat\{d\}\)\\hat\{d\}^\{2\}\. The third term accounts for RMSNorm parameters: two per layer \(pre\-attention and pre\-FFN\) plus one final norm, each withd^\\hat\{d\}learnable scale parameters\.111In practice this term is negligible, e\.g\., ford^=4096\\hat\{d\}=4096andL^=32\\hat\{L\}=32it constitutes∼0\.3\{\\sim\}0\.3M out of∼8\{\\sim\}8B total parameters\. Architecture\-specific ratios are retrieved from a per\-model registry when available; otherwise, we default to values representative of LLaMA\-3 8B \(nkv/nheads=1/4n\_\{\\mathrm\{kv\}\}/n\_\{\\mathrm\{heads\}\}=1/4,dffn/d^=3\.5d\_\{\\mathrm\{ffn\}\}/\\hat\{d\}=3\.5\)\.

## 5Results

[Section˜5\.1](https://arxiv.org/html/2607.01313#S5.SS1)establishes overall efficacy;[Section˜5\.2](https://arxiv.org/html/2607.01313#S5.SS2)isolates the factors governing hidden\-dimension estimation via common\-set prompting;[Section˜5\.3](https://arxiv.org/html/2607.01313#S5.SS3)examines parameter\-count and depth estimation\.

##### Models

We evaluate NightVision across 32 LLMs \(full list in[Appendix˜C](https://arxiv.org/html/2607.01313#A3)\)\. Our suite spans two architectural classes,*dense*and*mixture\-of\-experts \(MoE\)*transformers, drawn from the Qwen, IBM Granite, Llama, OLMo, and HuggingFace model families\. Together they cover 135M–30B parameters, hidden dimensionsd∈\[576,4096\]d\\in\[576,4096\], and depthsL∈\[16,48\]L\\in\[16,48\]\.

##### Test bench

Log\-probability matrices and timing measurements are collected with HuggingFacetransformersand spot\-checked against[vLLM](https://github.com/vllm-project/vllm)\(v0\.16\.0\)\. For the timing measurements used in the scaling\-relation analysis, we serve models via vLLM and measure end\-to\-end inference latency through its API\. The NightVision hyperparameters used throughout this section \(number of prompts, samples per prompt, temperature, change\-point detector windows, grid spacing, and precision\) are fixed across all models and listed in[Section˜B\.2](https://arxiv.org/html/2607.01313#A2.SS2.SSS0.Px1); further details on our timing measurements and experimental setup are in[Appendix˜D](https://arxiv.org/html/2607.01313#A4)\.

##### Metrics

We report cost in two units, token consumption and number of LLM calls,

Token Consumption=\(System Prompt Length\+Input Tokens\+Output Tokens\)⋅\(\#​Samples\)\\text\{Token Consumption\}=\(\\text\{System Prompt Length\}\+\\text\{Input Tokens\}\+\\text\{Output Tokens\}\)\\cdot\(\\\#\\,\\text\{Samples\}\)that conservatively assumes no system\-prompt caching\. Plots showing error vs\. token budget use the average system\-prompt length across the 39 models,ℓ¯sys≈31\\bar\{\\ell\}\_\{\\text\{sys\}\}\\approx 31tokens\. We also report*relative error*: for parameterθ\\thetawith estimateθ^\\hat\{\\theta\}, we defineR​\(θ^;θ\)=\|θ−θ^\|/θR\(\\hat\{\\theta\};\\theta\)=\|\\theta\-\\hat\{\\theta\}\|/\\theta\.

### 5\.1Overall Results

We measure the accuracy of NightVision in estimating hidden dimension, parameter count, and 0ptas a function of total budget, measured in tokens\. Figure[3](https://arxiv.org/html/2607.01313#S5.F3)shows this tradeoff for all dense transformers in our dataset with parameter counts of at least 2B parameters; results are averaged across models with one run per model\. For these curves, we use the estimated^\\hat\{d\}of hidden dimension as an input to the estimate of 0ptand parameter count, which causes error to propagate toL^\\hat\{L\}andP^\\hat\{P\}\. On average,NightVision estimates hidden dimension, 0pt, and parameter count on average to within∼40%\\sim 40\\%,25%25\\%, and65%65\\%relative error, respectively, using a total budget of∼\\sim800B tokens for model sizes ranging from 4B dense to 30B MoE\. While there is significant variability across models \(shown as lighter curves in[Figure˜3](https://arxiv.org/html/2607.01313#S5.F3)\), we find that most of the models generally improve in relative error as budget increases, as expected\.

![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/err_vs_H_pred_budget_avg_gt2b.png)Figure 3:Relative error vs\. token budget\. Errors fordd,LL, andPPare shown as a function of total token budget, with bold curves denoting averages across target models and faint curves denoting individual models\. Larger budgets generally reduce error, withLLimproving fastest andPPremaining harder to estimate\. Results include≥\\geq3B\-parameter dense models and MoE models\.##### NightVision works better on large models\.

Figure[13](https://arxiv.org/html/2607.01313#A2.F13)shows how relative error for estimating 0pt, parameter count, and hidden dimension, respectively depends on model size for a collection of model architectures\. We see that while the common\-set prompting attack works well at all model sizes, 0ptand parameter count are more sensitive to model size\. In particular, on larger models \(≥3\\geq 3B parameters\), NightVision achieves low relative error \(often below 30%\) for 0ptand parameter count estimation\. However, for models below 3B parameters, the estimation error is often over 100% for both parameters\. This is likely because in smaller models, the dominant computational cost is no longer prefill, because the asymptotics have not yet kicked in\. For larger models, which are more common in production, TTFT is dominated by prefill costs, so our model is more accurate\.

### 5\.2Hidden\-Dimension Recovery via the Common\-Set Prompting Attack

We evaluate the spectral hidden\-dimension estimator on a held\-out validation set of 32 models\. Figure[4](https://arxiv.org/html/2607.01313#S5.F4)summarizes the per\-model rounded relative error, averaged across runs, as a function of total prompt\-token budget; the right\-most point of each curve corresponds to the maximum sample budget ofN=106N\{=\}10^\{6\}per prompt \(∼800\{\\sim\}800B prompt tokens\)\. Each line averages models within a group; eachxx\-axis tick is the mean tokens consumed over a batch of 50k samples\. We separate models by size \(small≤2\\leq\{\}2B parameters,mid≈2\{\\approx\}2–44B, andlarge≥7\\geq\{\}7B\) and architecture \(dense vs\. MoE\)\.

![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/hd_relative_err_fig.png)Figure 4:Mean relative error of the spectral hidden\-dimension estimated^\\hat\{d\}on the validation set vs\. total prompt\-token budget\. Each curve averages over thennvalidation models in the corresponding size and architecture group\.[Figure˜4](https://arxiv.org/html/2607.01313#S5.F4)highlights three points: \(1\) Average relative error drops monotonically with budget for all model classes\. \(2\) Smaller models converge faster on average\. This is expected, as small models tend to have smaller values ofdd, which in turn requires a smaller common set for recovery, and a lower token budget\. \(3\) MoE models tend to converge noticeably faster than dense ones\. We hypothesize that this is because the expert routing structure in MoE models sends each token through a small subset of experts, so the effective next\-token distribution depends on fewer parameters and this helps saturate the eigenspace quicker than dense models\.

Overall, the most challenging instances for the methodology incur errors between 49\-64% on dense architectures from the Llama, Granite, and Qwen families\. One possible explanation is that our token budget for these models was not high enough to saturate the eigenspectrum, in which case the estimator can only lower\-bound the true rank and additional sampling is required to detect the true elbow\.[Figure˜15](https://arxiv.org/html/2607.01313#A5.F15)illustrates this for Llama\-3\.1\-8B \(d∗=4,096d^\{\*\}\{=\}4\{,\}096\): at our maximum budget, the saddle and peak landmarks that bracket the change\-point detector’s area of interest both fall well belowd∗d^\{\*\}, so the spectrum has not saturated enough for the detector to reliably latch onto the true rank\.

##### NightVision incurs≈12−190×\{\\approx\}12\-190\\timesthe cost ofCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)

The baseline ofCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)assumes access to top\-kklogits and a logit bias function\. Figure[5](https://arxiv.org/html/2607.01313#S5.F5)shows what that extra access buys\. For each of the 23 configurations that reach relative error≤30%\\leq 30\\%, we usedD=20,000D=20,000prompts and grew the per\-prompt sample budget until NightVision’sd^\\hat\{d\}reached that threshold, and we plot the cost ratio of our attack at that point divided by the cost of the top\-11logit\-bias attack inCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)\. Their cost isd​\|𝒱\|​\(ℓ¯sys\+2\)d\\,\|\\mathcal\{V\}\|\(\\bar\{\\ell\}\_\{\\text\{sys\}\}\{\+\}2\)tokens, counting one input plus one output token per logit\-bias query on top of the average system promptℓ¯sys\\bar\{\\ell\}\_\{\\text\{sys\}\}; NightVision’s cost is the realized prompt\-token total\.

![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/carlini_vs_us_1.png)Figure 5:Cost overhead ratio as a function of hidden dimension\. Cost overhead ratio = \(Our cost\)/\(Cost ofCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\)\)\. We report the cost to reach relative error≤30%\\leq 30\\%\. Marker color encodes the number of samples used to reach that error at each point\.Overall, the cost ratio varies with hidden dimensionddbut NightVision spends about 1 to 2\.3 orders of magnitude \(median∼1\.7\{\\sim\}1\.7\) more tokens than the logit\-bias attack to reach the same accuracy on hidden dimension\. The overhead is substantial; these numbers partially reflect the inherent cost of working under the weaker single\-decoded\-logprob threat model, which still applies on production APIs that have removed top\-kkand logit\-bias access\. However, they may also reflect inefficiencies in NightVision\. For example, there may be more efficient prompting methods, or more efficient attacks altogether, that we have yet to identify\.

The overhead shrinks to∼33×\{\\sim\}33\\timeson OLMo\-2\-7B and OLMo\-3\-7B atd=4,096d\{=\}4\{,\}096\. This matches the analysis: our per\-prompt sample requirement scales sub\-linearly indd, while Carlini’s query count scales asd​\|𝒱\|d\\,\|\\mathcal\{V\}\|\.

##### Theory predicts the empirical sample complexity\.

![Refer to caption](https://arxiv.org/html/2607.01313v1/x1.png)Figure 6:Required samples per promptNNto reach a common token set of sizedd, forD∈\{5000,10000,20000\}D\\in\\\{5000,10000,20000\\\}prompts\. Token distributions are from the smallest HuggingFace SmolLM model \(135135M parameters; vocabularyV=49,152V=49\{,\}152, temperatureT=1T=1\)\. The two theory lines, expectedNN\(blue\) and the95%95\\%upper bound \(orange,[Corollary˜1](https://arxiv.org/html/2607.01313#Thmcorollary1)\), analyze the strict requirement that a token appear in*all*DDprompts\. The empirical95%95\\%NN\(green\) is Common Set Prompting, which greedily prunes prompts\.Figure[6](https://arxiv.org/html/2607.01313#S5.F6)reports the samples per promptNNneeded to obtain a common token set of sizedd, as a function ofdd, forD∈\{5000,10000,20000\}D\\in\\\{5000,10000,20000\\\}prompts on output distributions from the smallest SmolLM model\. We drawNNtokens from each prompt and call a token present in a prompt if it is drawn at least once\. The quantity plotted is alwaysNNfor a target ofddshared tokens\. The curves differ in which tokens count as shared\. The theory analyzes the*strict*common set, i\.e\., the tokens present in allDDprompts\. Common Set Prompting instead produces a*pruned*common set as it greedily discards the prompts that are hardest to satisfy and keeps the tokens shared across the remaining subset, so a token need not appear in every prompt to be counted\. For the strict common set we report two theoretical quantities: the expectedNN, obtained by inverting the expected intersection sizeμ​\(N\)=∑i∏q=1D\(1−\(1−pq\(i\)\)N\)\\mu\(N\)=\\sum\_\{i\}\\prod\_\{q=1\}^\{D\}\(1\-\(1\-p^\{\(i\)\}\_\{q\}\)^\{N\}\)atμ​\(N\)=d\\mu\(N\)=d, and the95%95\\%upper bound from Corollary[1](https://arxiv.org/html/2607.01313#Thmcorollary1), the smallestNNthat guarantees size at leastddwith probability1−δ=0\.951\-\\delta=0\.95\. For the pruned common set we report the empirical95%95\\%NNas the per\-run mean plus1\.6451\.645standard deviations over4040runs of the algorithm\. Since pruning requires agreement across only a subset rather than allDDprompts, the empirical cost lies below the theoretical upper bound and within a small constant factor of it\. The theory, though it analyzes the stricter strict\-intersection problem, reliably predicts the sample complexity of Common Set Prompting\.

### 5\.3Depth and Total Parameter Count Estimation via the Timing Attack

We first validate the assumptions made in[Section˜4\.2](https://arxiv.org/html/2607.01313#S4.SS2), particularly the scaling relation in \([2](https://arxiv.org/html/2607.01313#S4.E2)\)\. To this end,[Figure˜8](https://arxiv.org/html/2607.01313#S5.F8)shows the end\-to\-end TTFT as a function of our desired scaling, which we model as being linear inL​ℓ2​dL\\ell^\{2\}d\. Notice first that for most LLMs,a linear timing model is reasonable, if not perfect, with the same slopeβ\\betaand offsetα\\alphaacross models\. However, the Qwen 3\.5 family \(shown in red, pink, and purple\) requires a different linear fit for prediction\. This suggests that timing attacks could benefit from an initial phase to estimate the model family, possibly via model fingerprinting techniques\. We leave this question to future work\.

![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/err_vs_LH_pred_budget_avg_timing.png)Figure 7:Mean relative error of the number of layers and total model parameter count estimates on the validation set vs\. total prompt\-token budget for timing method\. Here we assume perfect knowledge of hidden dimension\. Each curve averages over thennvalidation models in the corresponding size and architecture group\.
\\begin\{overpic\}\[width=433\.62pt\]\{figs/scaling\_fit\_gf\.png\} \\put\(50\.0,0\.0\)\{\\makebox\[0\.0pt\]\{\{\\small$L\\ell^\{2\}d$\}\}\} \\end\{overpic\}Figure 8:Measured runtime versusL​ℓ2​dL\\,\\ell^\{2\}\\,drelation for several models considered in this work\. The Qwen3\.5 family \(points in red, pink, dark purple, and light purple\) shows a different slope than others, which fall into LLaMA\-style with SwiGLU type of blocks\.
Figure[8](https://arxiv.org/html/2607.01313#S5.F8)\(and Figure[13](https://arxiv.org/html/2607.01313#A2.F13)in[Section˜E\.2](https://arxiv.org/html/2607.01313#A5.SS2)\) presents the relative error for the layer count and total parameter count\. Layer count estimates are obtained by combining the spectral hidden\-dimension estimate with the timing scaling relation of Eq\. \([2](https://arxiv.org/html/2607.01313#S4.E2)\)\. For total parameter estimation, we use Eq\. \([3](https://arxiv.org/html/2607.01313#S4.E3)\) and further assume that the vocabulary size is known exactly; this is a modeling simplification whose effect is small, typically contributing less than2%2\\%to the overall estimate\. While the average relative error for layer count and total parameters is approximately50%50\\%for models≥\\geq3B parameter counts, several individual models yield substantially more accurate estimates:Qwen2\.5\-7B\-Instruct\(∼16%\{\\sim\}16\\%forLL,∼14%\{\\sim\}14\\%forNN\),OLMo\-3\-7B\-Instruct\(∼20%\{\\sim\}20\\%,∼18%\{\\sim\}18\\%\), andQwen3\-8B\(∼35%\{\\sim\}35\\%,∼30%\{\\sim\}30\\%\)\. The notable exceptions aregranite\-3\.0\-3B\-A800Mandgranite\-3\.0\-8B, whose limited context windows prevent processing sufficiently long inputs to reach the prefill\-dominated regime required by our scaling relation\.

##### Performance does not improve significantly with budget\.

Figure[16](https://arxiv.org/html/2607.01313#A5.F16)shows that for larger models \(≥\\geq3B parameters\), the accuracy of our method does not significantly improve with token budget\. Recall that our method requires conducting linear regression on timing measurements for input prompts of different lengths; the slope is used to estimate hidden dimension and parameter count\. It appears that we can accurately estimate the slope with relatively few samples at each sequence length\. Hence, our estimation is not very sensitive to token budget for large models, at budgets that are orders of magnitude smaller than the requirements for the common\-set prompting attack\. This explains why we set the token budget to a flat 10M tokens in Figure[1](https://arxiv.org/html/2607.01313#S4.F1)\.

## 6Conclusion, Limitations, and Future Work

We present NightVision, an attack that recovers certain hyperparameters of transformer\-based architectures based only on black\-box API access to a target model\. Unlike prior attacks, NightVision does not make use of top\-kklogits or a logit\-bias function\. We show that even with the very limited information available in modern APIs, we can reconstruct the hidden dimension, 0pt, and parameter count of many LLMs, up to an accuracy threshold\.

This work has several limitations and directions for future research\. First, NightVision can incur nontrivial estimation error at high cost: reducing relative error requires token budgets around101110^\{11\}to101210^\{12\}per target \(median∼4×1011\{\\sim\}4\\times 10^\{11\}\), which is1212to190190times more expensive thanCarliniet al\.\([2024](https://arxiv.org/html/2607.01313#bib.bib14)\), reflecting the cost of using the weaker single\-decoded\-logprob threat model\. Second, our evaluation focuses on standard dense and MoE transformers from a few model families, and we have not yet studied how spectral signals behave in other architectures, such as state\-space or hybrid models\. Finally, the timing\-based estimates of 0ptand parameter count depend on a prefill\-dominated regime and a calibrated serving stack\. These estimates are less reliable for small models where prefill is not the main bottleneck, and they have not been tested on MoE deployments or in varied serving environments\.

## Acknowledgments

This effort was sponsored in whole or in part by the United States Government \(USG\)\. The U\.S\. Government is authorized to reproduce and distribute original submissions for publication for Governmental purposes notwithstanding any copyright notation thereon\. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the United States Government \(USG\)\. The authors also thank Srinivasa Pranav for helpful discussions\.

## References

- J\. Ainslie, J\. Lee\-Thorp, M\. de Jong, Y\. Zemlyanskiy, F\. Lebron, and S\. Sanghai \(2023\)GQA: training generalized multi\-query transformer models from multi\-head checkpoints\.InThe 2023 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://openreview.net/forum?id=hmOwOZWzYE)Cited by:[§4\.2](https://arxiv.org/html/2607.01313#S4.SS2.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2607.01313#S4.SS2.SSS0.Px2.p1.13)\.
- S\. Alhazbi, A\. Hussain, G\. Oligeri, and P\. Papadimitratos \(2025\)LLMs have rhythm: fingerprinting large language models using inter\-token times and network traffic analysis\.IEEE Open Journal of the Communications Society\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p1.1)\.
- M\. Amirizaniani, E\. Martin, T\. Roosta, A\. Chadha, and C\. Shah \(2024\)AuditLLM: a tool for auditing large language models using multiprobe approach\.InProceedings of the 33rd ACM International Conference on Information and Knowledge Management,pp\. 5174–5179\.Cited by:[§1](https://arxiv.org/html/2607.01313#S1.p1.1)\.
- N\. Carlini, D\. Paleka, K\. D\. Dvijotham, T\. Steinke, J\. Hayase, A\. F\. Cooper, K\. Lee, M\. Jagielski, M\. Nasr, A\. Conmy,et al\.\(2024\)Stealing part of a production language model\.arXiv preprint arXiv:2403\.06634\.Cited by:[Figure 10](https://arxiv.org/html/2607.01313#A2.F10),[Figure 10](https://arxiv.org/html/2607.01313#A2.F10.1.1.1),[§B\.1\.3](https://arxiv.org/html/2607.01313#A2.SS1.SSS3.p2.14),[§B\.1\.3](https://arxiv.org/html/2607.01313#A2.SS1.SSS3.p4.3),[§1](https://arxiv.org/html/2607.01313#S1.p2.2),[§2](https://arxiv.org/html/2607.01313#S2.SS0.SSS0.Px1.p1.12),[§2](https://arxiv.org/html/2607.01313#S2.SS0.SSS0.Px2.p1.35),[Table 1](https://arxiv.org/html/2607.01313#S3.T1.12.3.1),[§3](https://arxiv.org/html/2607.01313#S3.p1.2),[Figure 2](https://arxiv.org/html/2607.01313#S4.F2),[Figure 2](https://arxiv.org/html/2607.01313#S4.F2.pic1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1),[§4\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS0.Px1),[§4\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS0.Px1.p1.24),[§4\.1\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS1.Px1.p1.12),[§4\.1\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS1.Px3.p1.12),[§4\.1\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS1.p1.1),[§4\.1](https://arxiv.org/html/2607.01313#S4.SS1.p1.1),[Figure 5](https://arxiv.org/html/2607.01313#S5.F5),[§5\.2](https://arxiv.org/html/2607.01313#S5.SS2.SSS0.Px1),[§5\.2](https://arxiv.org/html/2607.01313#S5.SS2.SSS0.Px1.p1.7),[§6](https://arxiv.org/html/2607.01313#S6.p2.5)\.
- T\. Chauvin, E\. L\. Merrer, F\. Taïani, and G\. Tredan \(2025\)Log probability tracking of llm apis\.arXiv preprint arXiv:2512\.03816\.Cited by:[§1](https://arxiv.org/html/2607.01313#S1.p2.2)\.
- T\. Dao, D\. Y\. Fu, S\. Ermon, A\. Rudra, and C\. Re \(2022\)FlashAttention: fast and memory\-efficient exact attention with IO\-awareness\.InAdvances in Neural Information Processing Systems,A\. H\. Oh, A\. Agarwal, D\. Belgrave, and K\. Cho \(Eds\.\),External Links:[Link](https://openreview.net/forum?id=H4DqfPSibmx)Cited by:[§4\.2](https://arxiv.org/html/2607.01313#S4.SS2.SSS0.Px2.p1.13)\.
- D\. P\. Dubhashi and D\. Ranjan \(1996\)Balls and bins: a study in negative dependence\.BRICS Report Series3\(25\)\.Cited by:[Appendix F](https://arxiv.org/html/2607.01313#A6.2.p2.1),[Appendix F](https://arxiv.org/html/2607.01313#A6.3.p3.15),[Appendix F](https://arxiv.org/html/2607.01313#A6.8.p3.3)\.
- V\. Duddu, D\. Samanta, D\. V\. Rao, and V\. E\. Balas \(2018\)Stealing neural networks via timing side channels\.arXiv preprint arXiv:1812\.11720\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p2.1)\.
- M\. Finlayson, X\. Ren, and S\. Swayamdipta \(2024\)Logits of API\-protected LLMs leak proprietary information\.InConference on Language Modeling \(COLM\),External Links:[Link](https://openreview.net/forum?id=oRcYFm8vyB)Cited by:[§1](https://arxiv.org/html/2607.01313#S1.p2.2),[§2](https://arxiv.org/html/2607.01313#S2.SS0.SSS0.Px1.p1.12),[§2](https://arxiv.org/html/2607.01313#S2.SS0.SSS0.Px2.p1.35),[Table 1](https://arxiv.org/html/2607.01313#S3.T1.12.4.1),[§3](https://arxiv.org/html/2607.01313#S3.p1.2)\.
- M\. Gubri, D\. Ulmer, H\. Lee, S\. Yun, and S\. J\. Oh \(2024\)Trap: targeted random adversarial prompt honeypot for black\-box identification\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 11496–11517\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p1.1)\.
- S\. Hartenstein \(2025\)Bridging the security gap: an empirical analysis of llm\-api integration vulnerabilities and mitigation strategies\.InProceedings of the 2025 14th International Conference on Software and Computer Applications,pp\. 90–95\.Cited by:[§1](https://arxiv.org/html/2607.01313#S1.p1.1)\.
- X\. Hu, L\. Liang, S\. Li, L\. Deng, P\. Zuo, Y\. Ji, X\. Xie, Y\. Ding, C\. Liu, T\. Sherwood,et al\.\(2020\)Deepsniffer: a DNN model extraction framework based on learning architectural hints\.InProceedings of the Twenty\-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems,pp\. 385–399\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p2.1)\.
- K\. Joag\-Dev and F\. Proschan \(1983\)Negative association of random variables with applications\.The Annals of Statistics11\(1\),pp\. 286–295\.External Links:ISSN 00905364, 21688966,[Link](http://www.jstor.org/stable/2240482)Cited by:[Appendix F](https://arxiv.org/html/2607.01313#A6.5.p5.12)\.
- J\. Kaplan, S\. McCandlish, T\. Henighan, T\. B\. Brown, B\. Chess, R\. Child, S\. Gray, A\. Radford, J\. Wu, and D\. Amodei \(2020\)Scaling laws for neural language models\.External Links:2001\.08361,[Link](https://arxiv.org/abs/2001.08361)Cited by:[§4\.2](https://arxiv.org/html/2607.01313#S4.SS2.SSS0.Px2.p1.13)\.
- B\. Li \(2026\)Incompressible knowledge probes: estimating black\-box LLM parameter counts via factual capacity\.arXiv preprint arXiv:2604\.24827\.Cited by:[Table 1](https://arxiv.org/html/2607.01313#S3.T1.12.5.1),[§3](https://arxiv.org/html/2607.01313#S3.p2.1)\.
- J\. Li, J\. Xu, S\. Huang, Y\. Chen, W\. Li, J\. Liu, Y\. Lian, J\. Pan, L\. Ding, H\. Zhou,et al\.\(2024\)Large language model inference acceleration: a comprehensive hardware perspective\.arXiv preprint arXiv:2410\.04466\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p2.1)\.
- Z\. Liu, L\. Hu, Y\. Tang, T\. Zhou, F\. Liu, and Z\. Cai \(2026\)Exploring and exploiting security vulnerabilities in self\-hosted llm services\.InProceedings of the ACM Web Conference 2026,pp\. 2535–2546\.Cited by:[§1](https://arxiv.org/html/2607.01313#S1.p1.1)\.
- J\. Mökander, J\. Schuett, H\. R\. Kirk, and L\. Floridi \(2024\)Auditing large language models: a three\-layered approach\.AI and Ethics4\(4\),pp\. 1085–1115\.Cited by:[§1](https://arxiv.org/html/2607.01313#S1.p1.1)\.
- J\. X\. Morris, W\. Zhao, J\. T\. Chiu, V\. Shmatikov, and A\. M\. Rush \(2023\)Language model inversion\.arXiv preprint arXiv:2311\.13647\.Cited by:[§3](https://arxiv.org/html/2607.01313#S3.p1.2)\.
- J\. Pan and G\. Li \(2025\)A survey of LLM inference systems\.arXiv preprint arXiv:2506\.21901\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p2.1)\.
- D\. Pasquini, E\. M\. Kornaropoulos, and G\. Ateniese \(2025\)\{\\\{llmmap\}\\\}: Fingerprinting for large language models\.In34th USENIX Security Symposium \(USENIX Security 25\),pp\. 299–318\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p1.1)\.
- C\. Rastogi, M\. Tulio Ribeiro, N\. King, H\. Nori, and S\. Amershi \(2023\)Supporting human\-ai collaboration in auditing llms with llms\.InProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society,pp\. 913–926\.Cited by:[§1](https://arxiv.org/html/2607.01313#S1.p1.1)\.
- S\. Shao, Y\. Li, Y\. He, H\. Yao, W\. Yang, D\. Tao, and Z\. Qin \(2025a\)Sok: large language model copyright auditing via fingerprinting\.arXiv preprint arXiv:2508\.19843\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p1.1)\.
- S\. Shao, Y\. Li, H\. Yao, Y\. Chen, Y\. Yang, and Z\. Qin \(2025b\)Reading between the lines: towards reliable black\-box LLM fingerprinting via zeroth\-order gradient estimation\.arXiv preprint arXiv:2510\.06605\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p1.1)\.
- N\. Shazeer \(2020\)GLU variants improve transformer\.External Links:2002\.05202,[Link](https://arxiv.org/abs/2002.05202)Cited by:[§B\.3\.2](https://arxiv.org/html/2607.01313#A2.SS3.SSS2.p2.1)\.
- F\. Tramèr, F\. Zhang, A\. Juels, M\. K\. Reiter, and T\. Ristenpart \(2016\)Stealing machine learning models via prediction\{\\\{apis\}\\\}\.In25th USENIX security symposium \(USENIX Security 16\),pp\. 601–618\.Cited by:[§1](https://arxiv.org/html/2607.01313#S1.p1.1)\.
- S\. Ubaru, Y\. Saad, and A\. Seghouane \(2016\)Fast estimation of approximate matrix ranks using spectral densities\.arXiv preprint arXiv:1608\.05754\.Cited by:[§B\.1\.3](https://arxiv.org/html/2607.01313#A2.SS1.SSS3.p3.6)\.
- A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin \(2017\)Attention is all you need\.Advances in Neural Information Processing Systems30\.Cited by:[§2](https://arxiv.org/html/2607.01313#S2.SS0.SSS0.Px1.p1.12),[§4\.2](https://arxiv.org/html/2607.01313#S4.SS2.SSS0.Px2.p1.13)\.
- A\. Verma, S\. Krishna, S\. Gehrmann, M\. Seshadri, A\. Pradhan, T\. Ault, L\. Barrett, D\. Rabinowitz, J\. Doucette, and N\. Phan \(2024\)Operationalizing a threat model for red\-teaming large language models \(LLMs\)\.arXiv preprint arXiv:2407\.14937\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p1.1)\.
- U\. von Luxburg \(2007\)A tutorial on spectral clustering\.Statistics and Computing17\(4\),pp\. 395–416\.Cited by:[§B\.1\.3](https://arxiv.org/html/2607.01313#A2.SS1.SSS3.p3.6)\.
- J\. Xu, F\. Wang, M\. Ma, P\. W\. Koh, C\. Xiao, and M\. Chen \(2024\)Instructional fingerprinting of large language models\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 3277–3306\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p1.1)\.
- S\. Yang, J\. Kautz, and A\. Hatamizadeh \(2025\)Gated delta networks: improving mamba2 with delta rule\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=r8H7xhYPwz)Cited by:[§B\.3\.2](https://arxiv.org/html/2607.01313#A2.SS3.SSS2.p1.2)\.
- B\. Zeng, L\. Wang, Y\. Hu, Y\. Xu, C\. Zhou, X\. Wang, Y\. Yu, and Z\. Lin \(2024\)Huref: human\-readable fingerprint for large language models\.Advances in Neural Information Processing Systems37,pp\. 126332–126362\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p1.1)\.
- J\. Zhang, D\. Liu, C\. Qian, L\. Zhang, Y\. Liu, Y\. Qiao, and J\. Shao \(2024\)Reef: representation encoding fingerprints for large language models\.arXiv preprint arXiv:2410\.14273\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p1.1)\.
- K\. Zhao, L\. Li, K\. Ding, N\. Z\. Gong, Y\. Zhao, and Y\. Dong \(2025\)A survey on model extraction attacks and defenses for large language models\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\. 2,pp\. 6227–6236\.Cited by:[§1](https://arxiv.org/html/2607.01313#S1.p2.2)\.
- X\. Zheng, H\. Han, S\. Shi, Q\. Fang, Z\. Du, X\. Hu, and Q\. Guo \(2024\)Inputsnatch: stealing input in LLM services via timing side\-channel attacks\.arXiv preprint arXiv:2411\.18191\.Cited by:[Appendix A](https://arxiv.org/html/2607.01313#A1.p2.1)\.
- M\. Zhu and A\. Ghodsi \(2006\)Automatic dimensionality selection from the scree plot via the use of profile likelihood\.Computational Statistics & Data Analysis51\(2\),pp\. 918–930\.Cited by:[§B\.1\.3](https://arxiv.org/html/2607.01313#A2.SS1.SSS3.p3.6)\.

## Appendix

## Appendix AAdditional Related Work

In addition to the related work from[Section˜3](https://arxiv.org/html/2607.01313#S3), a separate and growing body of work on LLM*fingerprinting*\[Alhazbiet al\.,[2025](https://arxiv.org/html/2607.01313#bib.bib28), Pasquiniet al\.,[2025](https://arxiv.org/html/2607.01313#bib.bib20), Zhanget al\.,[2024](https://arxiv.org/html/2607.01313#bib.bib21), Zenget al\.,[2024](https://arxiv.org/html/2607.01313#bib.bib22), Gubriet al\.,[2024](https://arxiv.org/html/2607.01313#bib.bib23), Shaoet al\.,[2025b](https://arxiv.org/html/2607.01313#bib.bib18), Xuet al\.,[2024](https://arxiv.org/html/2607.01313#bib.bib9)\]aims to identify which model from a known set is being served, rather than to recover architectural parameters of a previously\-unseen model; closely related is the application of such fingerprints to copyright auditing\[Shaoet al\.,[2025a](https://arxiv.org/html/2607.01313#bib.bib19)\]\. These methods rely on per\-target calibration data and are therefore not directly comparable to ours\. The threat\-model landscape for black\-box LLM attacks more broadly is taxonomized inVermaet al\.\[[2024](https://arxiv.org/html/2607.01313#bib.bib24)\]\.

Methodologically, our use of inference timing as an auxiliary signal for architecture inference draws on a longer tradition outside the LLM setting\.Dudduet al\.\[[2018](https://arxiv.org/html/2607.01313#bib.bib6)\]combine end\-to\-end runtime with architecture search and distillation to recover substitute DNN models, andHuet al\.\[[2020](https://arxiv.org/html/2607.01313#bib.bib7)\]use learning\-based methods on similar signals\. Surveys of modern LLM inference systems\[Pan and Li,[2025](https://arxiv.org/html/2607.01313#bib.bib25), Liet al\.,[2024](https://arxiv.org/html/2607.01313#bib.bib26)\]provide the hardware and software context that makes timing a useful signal in our setting\. While recent timing\-based attacks on LLM APIs have demonstrated input recovery from cache effects\[Zhenget al\.,[2024](https://arxiv.org/html/2607.01313#bib.bib8)\]rather than architecture extraction, they reflect the same broader trend of treating runtime as exploitable auxiliary information\.

## Appendix BAlgorithmic Details

### B\.1Common\-Set Attack Details

#### B\.1\.1Finding High\-Entropy Prompts

Our common\-set prompting method seeks prompts whose output distributions are as flat as possible\. As discussed in[Section˜2](https://arxiv.org/html/2607.01313#S2), increasing the temperature parameterτ\\taupushes the distribution over tokens toward uniform\. Empirically, we find that temperature alone is not enough\. When the unscaled distribution is highly peaked, even largeτ\\tauleaves substantial probability mass on the top tokens, as illustrated in[Figure˜9](https://arxiv.org/html/2607.01313#A2.F9)\. For a prompt inducing high entropy over the tokens \(top row\), the unscaled top\-100 tokens hold68%68\\%of the probability mass; temperature scaling spreads this further, dropping the top\-100 mass to just14%14\\%and yielding a near\-uniform distribution\. For a low\-entropy prompt \(bottom row\), the unscaled top\-100 tokens hold95%95\\%of the mass, and even after scaling they retain34%34\\%, which is still far from uniform\. Prompt selection is hence the dominant lever for vocabulary coverage\. We find that out\-of\-distribution token sequences, i\.e\., inputs the model has likely not seen during training, reliably produce diffuse predictions, since the model lacks a strong prior over what token should come next\. The next subsection describes how we identify such prompts\.

![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/temp_entropy_plot.png)Figure 9:Next\-token distributions for a high\-entropy prompt \(top\) and a low\-entropy prompt \(bottom\), shown without \(left\) and with \(right\) temperature scaling\. High\-entropy prompts yield substantially flatter distributions, enabling broader vocabulary coverage per sample\.##### Automated High\-Entropy Prompt Search

High\-entropy prompts can be specific to each model, so it is desirable for us to have an automated way of finding enough of them for the common\-set method given a new black\-box model\. In our experiments, our main method of finding these prompts is to sample random prompts from various Unicode character groups\. Concretely, we consider character groups like Latin, CJK, emoji, Arabic, Hebrew, Braille, mathematical symbols, etc\., and form candidate prompts by randomly sampling characters from these groups\.

Our search runs for a fixed number of iterations and combines exploration with greedy search over a best candidate prompt\. The first few rounds draw fresh batches of random prompts across all character groups and record the prompt whose argmax token has the lowest probability seen so far\. From the second round onwards, half of each batch consists of single\-character variants of the current best prompt, so promising candidates are exploited while new groups can still surface; this exploration phase ends early once any prompt drives the argmax probability below a small threshold\. The remaining iterations switch to pure refinement: every candidate is a one\-character mutation of the current best, distributed evenly across character groups, and the best prompt is updated greedily after each batch\.

All candidates are deduplicated against previously\-evaluated prompts to ensure each string is evaluated only once\.

After the final iteration, we sort every evaluated prompt by argmax probability and keep theDDlowest\-probability prompts as the prompt set𝒟\\mathcal\{D\}passed to the common\-set construction\. In practice, a largerDDis preferable: more prompts yield broader vocabulary coverage in the common set\.

#### B\.1\.2Greedy NaN Pruning

We prune𝐘full^\\hat\{\\mathbf\{Y\}\_\{\\mathrm\{full\}\}\}to a large NaN\-free submatrix using a greedy heuristic: At each step, find the surviving row or column with the highest fraction of NaNs and drop it, then update the per\-row and per\-column NaN counts\. The loop terminates when no NaNs remain\. The surviving rows index theD′≤DD^\{\\prime\}\\leq Dprompts kept for the spectral method, the surviving columns form the common set𝒞\\mathcal\{C\}, and the submatrix itself is the common\-set log\-probability matrix𝐘∈ℝD′×\|𝒞\|\\mathbf\{Y\}\\in\\mathbb\{R\}^\{D^\{\\prime\}\\times\|\\mathcal\{C\}\|\}\.

#### B\.1\.3Inflection\-Point Detection

Under the standard transformer output head, the logit for tokenviv\_\{i\}given promptSqS\_\{q\}iszq\(i\)=𝐡q⊤​𝐰iz\_\{q\}^\{\(i\)\}=\\mathbf\{h\}\_\{q\}^\{\\top\}\\mathbf\{w\}\_\{i\}with𝐡q,𝐰i∈ℝd\\mathbf\{h\}\_\{q\},\\mathbf\{w\}\_\{i\}\\in\\mathbb\{R\}^\{d\}, so the logit matrix𝐙\\mathbf\{Z\}has rank at mostdd\. The log\-probability matrix𝐘\\mathbf\{Y\}\(with entriesyq\(i\)y\_\{q\}^\{\(i\)\}\) differs from𝐙\\mathbf\{Z\}only by a row\-wise log\-partition termlog​∑jexp⁡\(zq\(j\)\)\\log\\sum\_\{j\}\\exp\(z\_\{q\}^\{\(j\)\}\), which is identical across all columns within a given row; temperature scaling preserves this structure\. The rank\-ddsignal in𝐙\\mathbf\{Z\}is therefore recoverable from𝐘\\mathbf\{Y\}up to a row\-wise additive constant\.

Deployed models run inference in reduced\-precision formats such asbfloat16\(8\-bit exponent, 7\-bit mantissa,≈3\\approx 3decimal digits of precision\)\. The logitszq\(i\)z\_\{q\}^\{\(i\)\}are therefore quantized, and the returned log\-probabilities carry rounding noise\. AsCarliniet al\.\[[2024](https://arxiv.org/html/2607.01313#bib.bib14)\]observe, this means𝐘\\mathbf\{Y\}is essentially always full rank and we must recover its*practical numerical rank*\. We model this as𝐘=𝐘⋆\+𝜼\\mathbf\{Y\}=\\mathbf\{Y\}^\{\\star\}\+\\boldsymbol\{\\eta\}, with a low\-rank*head*𝐘⋆\\mathbf\{Y\}^\{\\star\}of rank at mostd\+1d\{\+\}1holding the signal from𝐙\\mathbf\{Z\}, and a full\-rank floating\-point*noise tail*𝜼\\boldsymbol\{\\eta\}, and recoveringddamounts to separating the two\. To do this, we take the Gram matrix𝐆=𝐘𝐘⊤\\mathbf\{G\}=\\mathbf\{Y\}\\mathbf\{Y\}^\{\\top\}whenD′≤\|𝒞\|D^\{\\prime\}\\leq\|\\mathcal\{C\}\|, and𝐆=𝐘⊤​𝐘\\mathbf\{G\}=\\mathbf\{Y\}^\{\\top\}\\mathbf\{Y\}otherwise\. Its eigendecomposition𝐆=𝐔​𝚲​𝐔⊤\\mathbf\{G\}=\\mathbf\{U\}\\,\\boldsymbol\{\\Lambda\}\\,\\mathbf\{U\}^\{\\top\}produces ordered eigenvaluesλ1≥λ2≥⋯≥λmin⁡\(D′,\|𝒞\|\)≥0\\lambda\_\{1\}\\geq\\lambda\_\{2\}\\geq\\cdots\\geq\\lambda\_\{\\min\(D^\{\\prime\},\|\\mathcal\{C\}\|\)\}\\geq 0\.

Formally, this is a numerical rank estimation problem\[Ubaruet al\.,[2016](https://arxiv.org/html/2607.01313#bib.bib15)\]: we fix a thresholdε≥0\\varepsilon\\geq 0and report theε\\varepsilon\-rank\|\{i:λi\>ε\}\|\|\\\{i:\\lambda\_\{i\}\>\\varepsilon\\\}\|, the index that divides the signal eigenvalues of𝐘⋆\\mathbf\{Y\}^\{\\star\}from the noise tail of𝜼\\boldsymbol\{\\eta\}\. The standard recipes for choosingε\\varepsilonassume structure our spectrum lacks: a dominant eigengap or a low\-density valley in the spectral density separating signal from noise\[von Luxburg,[2007](https://arxiv.org/html/2607.01313#bib.bib17), Ubaruet al\.,[2016](https://arxiv.org/html/2607.01313#bib.bib15)\], or signal and noise eigenvalues that each concentrate around a level\[Zhu and Ghodsi,[2006](https://arxiv.org/html/2607.01313#bib.bib16)\]\.

Our goal is to identify the inflection point in this sequence, as illustrated in[Figure˜10](https://arxiv.org/html/2607.01313#A2.F10)\. The contrast with prior work motivates the design of our detector\. The logit\-bias attack ofCarliniet al\.\[[2024](https://arxiv.org/html/2607.01313#bib.bib14)\]reconstructs nearly all entries of the dense logit matrix, so the spectrum exhibits a sharp drop of several orders of magnitude precisely at indexdd\([Figure˜10](https://arxiv.org/html/2607.01313#A2.F10)a\), and a simple elbow rule suffices\. In our restricted\-API setting we observe only sampled log\-probabilities over the common set𝒞\\mathcal\{C\}, and the resulting spectrum decays smoothly: the head merges into the noise tail through a gradual transition rather than a visible drop\-off \([Figure˜10](https://arxiv.org/html/2607.01313#A2.F10)b\), soddmust be recovered from the curvature of the transition rather than from a discontinuity in magnitude\.

![Refer to caption](https://arxiv.org/html/2607.01313v1/x2.png)

\(a\) Logit\-bias attack ofCarliniet al\.\[[2024](https://arxiv.org/html/2607.01313#bib.bib14)\]\.

![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/eignevalues_2olmo_3_7b_1000000.png)

\(b\) Our common\-set spectrum \(OLMo\-3 7B\)\.

![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/aoi_plots.png)

\(c\) Arctan\-transformed spectrum\.\(d\) Windowed slope∂iai\\partial\_\{i\}a\_\{i\}with detected saddle, peak, and AOI\.

Figure 10:Spectra used to recover the hidden dimensiondd\. \(a\) Under the logit\-bias attack ofCarliniet al\.\[[2024](https://arxiv.org/html/2607.01313#bib.bib14)\], the singular\-value spectrum drops sharply at indexdd\(dashed line\) once the number of queries exceedsdd, and a simple elbow rule recoversdd\. \(b\) In our restricted\-API common\-set setting \(OLMo\-3 7B, trued⋆=4096d^\{\\star\}=4096\), the eigenvalue spectrum decays smoothly across several orders of magnitude with no clean elbow\. \(c\) Applying the element\-wise transformai=arctan⁡\(λi\)a\_\{i\}=\\arctan\(\\lambda\_\{i\}\)compresses the head toward the plateau atπ/2\\pi/2and exposes the head\-to\-tail transition to slope\-based detection\. \(d\) The windowed slope∂iai\\partial\_\{i\}a\_\{i\}exhibits a saddle \(local minimum\) and a subsequent peak \(first non\-increase\) that bracket the area of interest \(AOI\) containingdd; within this AOI the algorithm returns the index of maximum negative second difference asd^\\hat\{d\}, which closely tracks the trued⋆=4096d^\{\\star\}=4096\.We first apply the element\-wise transform

ai=arctan⁡\(λi\),i=1,…,min⁡\(D′,\|𝒞\|\),\\displaystyle a\_\{i\}=\\arctan\(\\lambda\_\{i\}\),\\qquad i=1,\\dots,\\min\(D^\{\\prime\},\|\\mathcal\{C\}\|\),\(4\)which compresses the signal eigenvalues toward a plateau bounded above byπ2\\frac\{\\pi\}\{2\}and leaves the noise tail as a smooth decay to a final linear drop off at extremely smallaia\_\{i\}\([Figure˜10](https://arxiv.org/html/2607.01313#A2.F10)c\)\. This transform exposes the transition region between the head and its decay to slope\-based detection when its head is sufficiently saturated\.

Letn=min⁡\(D′,\|𝒞\|\)n=\\min\(D^\{\\prime\},\|\\mathcal\{C\}\|\)\. We compute the windowed least\-squares slope∂iai\\partial\_\{i\}a\_\{i\}over\[i−w,i\+w\]\[i\-w,i\+w\]withw=max⁡\(20,⌊n/50⌋\)w=\\max\(20,\\lfloor n/50\\rfloor\), then smooth it with a centered moving averages~i=MAws​\(∂iai\)\\tilde\{s\}\_\{i\}=\\mathrm\{MA\}\_\{w\_\{s\}\}\(\\partial\_\{i\}a\_\{i\}\)of half\-windowws=max⁡\(20,⌊n/100⌋\)w\_\{s\}=\\max\(20,\\lfloor n/100\\rfloor\)\. As shown in[Figure˜10](https://arxiv.org/html/2607.01313#A2.F10)d,s~\\tilde\{s\}exhibits a shallow local minimum near the head\-to\-tail transition followed by a recovery toward zero before the deep negative excursion of the noise floor\. We exploit this structure to localizeddbetween two landmarks ofs~\\tilde\{s\}, which together delimit the*area of interest*\(AOI\) shaded in[Figure˜10](https://arxiv.org/html/2607.01313#A2.F10)d:

- •Saddle:isadi\_\{\\mathrm\{sad\}\}is the first local minimum ofs~\\tilde\{s\}with prominence at leastp⋅\(max⁡s~−min⁡s~\)p\\cdot\(\\max\\tilde\{s\}\-\\min\\tilde\{s\}\), whereai≤arctan⁡\(π\)a\_\{i\}\\leq\\arctan\(\\pi\)\. This marks the lower edge of the AOI \([Figure˜10](https://arxiv.org/html/2607.01313#A2.F10)d, dotted line atisad=3585i\_\{\\mathrm\{sad\}\}=3585for OLMo\-3 7B\)\.
- •Peak:ipeaki\_\{\\mathrm\{peak\}\}is the first non\-increase ofs~\\tilde\{s\}afterisadi\_\{\\mathrm\{sad\}\}, withaipeak≤arctan⁡\(π/2\)a\_\{i\_\{\\mathrm\{peak\}\}\}\\leq\\arctan\(\\pi/2\)\(near the noise floor\)\. This marks the upper edge of the AOI \([Figure˜10](https://arxiv.org/html/2607.01313#A2.F10)d, dotted line atipeak=4730i\_\{\\mathrm\{peak\}\}=4730\)\.

Within this bracket we recover the geometric inflection of the head\-saturatedarctan\\arctancurve via the maximum negative second difference,

d^=arg⁡maxk∈\[isad,ipeak\]⁡\(−Δ2​ak\)\.\\displaystyle\\hat\{d\}=\\arg\\max\_\{k\\,\\in\\,\[i\_\{\\mathrm\{sad\}\},\\,i\_\{\\mathrm\{peak\}\}\]\}\\bigl\(\-\\Delta^\{2\}a\_\{k\}\\bigr\)\.\(5\)For the OLMo\-3 7B example in[Figure˜10](https://arxiv.org/html/2607.01313#A2.F10), the AOI\[3585,4730\]\[3585,4730\]tightly brackets the true hidden dimensiond⋆=4096d^\{\\star\}=4096, and the second\-difference maximum within the AOI recoversd^\\hat\{d\}to within a handful of indices\.

### B\.2Hidden Dimension Attack

Algorithm 1NightVision: common\-set construction via greedy NaN\-pruning1:Black\-box model

ffwith sampled\-token \+ log\-prob API only; high\-entropy prompt set

𝒟=\{Sq\}q=1D\\mathcal\{D\}=\\\{S\_\{q\}\\\}\_\{q=1\}^\{D\}; samples per prompt

NN; temperature

τ\\tau\.

2:Pruned log\-probability matrix

𝐘∈ℝD′×\|𝒞\|\\mathbf\{Y\}\\in\\mathbb\{R\}^\{D^\{\\prime\}\\times\|\\mathcal\{C\}\|\}over the common set

𝒞\\mathcal\{C\}\.

3:for

q=1,…,Dq=1,\\dots,Ddo

4:Draw

NNsamples from

f​\(Sq\)f\(S\_\{q\}\)at temperature

τ\\tau, recording each

\(vk,yq\(k\)\)\(v\_\{k\},y\_\{q\}^\{\(k\)\}\)\.

5:Set

𝒯q←\{unique tokens observed under​Sq\}\\mathcal\{T\}\_\{q\}\\leftarrow\\\{\\text\{unique tokens observed under \}S\_\{q\}\\\}\.

6:endfor

7:Assemble sparse

𝐘full^∈ℝD×\|𝒱\|\\hat\{\\mathbf\{Y\}\_\{\\mathrm\{full\}\}\}\\in\\mathbb\{R\}^\{D\\times\|\\mathcal\{V\}\|\}with

\[𝐘full^\]q,i=yq\(i\)\[\\hat\{\\mathbf\{Y\}\_\{\\mathrm\{full\}\}\}\]\_\{q,i\}=y\_\{q\}^\{\(i\)\}for

vi∈𝒯qv\_\{i\}\\in\\mathcal\{T\}\_\{q\},

NaN\\mathrm\{NaN\}otherwise\.

8:Greedy NaN\-prune

𝐘full^\\hat\{\\mathbf\{Y\}\_\{\\mathrm\{full\}\}\}: repeatedly drop the surviving row or column with the highest NaN fraction until no NaNs remain\. Surviving columns form the common set

𝒞\\mathcal\{C\}; surviving rows index the prompts retained for spectral analysis \([Section˜4\.1\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS1.Px2)\)\.

9:Set

𝐘∈ℝD′×\|𝒞\|\\mathbf\{Y\}\\in\\mathbb\{R\}^\{D^\{\\prime\}\\times\|\\mathcal\{C\}\|\}to the pruned submatrix\.

10:returnpruned matrix

𝐘\\mathbf\{Y\}and common set

𝒞\\mathcal\{C\}\.

Algorithm 2NightVision: hidden\-dimension estimation from the pruned spectrum1:Pruned log\-probability matrix

𝐘∈ℝD′×\|𝒞\|\\mathbf\{Y\}\\in\\mathbb\{R\}^\{D^\{\\prime\}\\times\|\\mathcal\{C\}\|\}from[Algorithm˜1](https://arxiv.org/html/2607.01313#alg1); slope half\-window

ww; smoothing window

wsw\_\{s\}; prominence fraction

pp; grid

gg\.

2:Estimated hidden dimension

d^\\hat\{d\}\.

3:Form the smaller\-side Gram matrix

𝐆←𝐘𝐘⊤\\mathbf\{G\}\\leftarrow\\mathbf\{Y\}\\mathbf\{Y\}^\{\\top\}if

D′≤\|𝒞\|D^\{\\prime\}\\leq\|\\mathcal\{C\}\|, else

𝐆←𝐘⊤​𝐘\\mathbf\{G\}\\leftarrow\\mathbf\{Y\}^\{\\top\}\\mathbf\{Y\}\.

4:Eigendecompose:

𝐆=𝐔​𝚲​𝐔⊤\\mathbf\{G\}=\\mathbf\{U\}\\,\\boldsymbol\{\\Lambda\}\\,\\mathbf\{U\}^\{\\top\}, take

λ1≥⋯≥λmin⁡\(D′,\|𝒞\|\)\\lambda\_\{1\}\\geq\\cdots\\geq\\lambda\_\{\\min\(D^\{\\prime\},\|\\mathcal\{C\}\|\)\}\.

5:Transform:

ai←arctan⁡\(λi\)a\_\{i\}\\leftarrow\\arctan\(\\lambda\_\{i\}\)\.

6:Compute the smoothed first derivative

s~i←MAws​\(∂iai\)\\tilde\{s\}\_\{i\}\\leftarrow\\mathrm\{MA\}\_\{w\_\{s\}\}\\\!\\big\(\\partial\_\{i\}a\_\{i\}\\big\), where

∂iai\\partial\_\{i\}a\_\{i\}is the least\-squares slope of

aaon

\[i−w,i\+w\]\[i\-w,\\,i\+w\]with

w=max⁡\(20,⌊n/50⌋\)w=\\max\(20,\\lfloor n/50\\rfloor\)and

MAws\\mathrm\{MA\}\_\{w\_\{s\}\}is a centered moving average with

ws=max⁡\(20,⌊n/100⌋\)w\_\{s\}=\\max\(20,\\lfloor n/100\\rfloor\)\.

7:Find saddle:

isad←i\_\{\\mathrm\{sad\}\}\\leftarrowfirst local min of

s~\\tilde\{s\}with prominence

≥p⋅\(max⁡s~−min⁡s~\)\\geq p\\cdot\(\\max\\tilde\{s\}\-\\min\\tilde\{s\}\), where

ai=arctan⁡\(λi\)≤arctan⁡\(π\)a\_\{i\}=\\arctan\(\\lambda\_\{i\}\)\\leq\\arctan\(\\pi\)\.

8:Find Peak:

ipeak←i\_\{\\mathrm\{peak\}\}\\leftarrowfirst non\-increase of

s~\\tilde\{s\}after

isadi\_\{\\mathrm\{sad\}\}, with

aipeak≤arctan⁡\(π/2\)a\_\{i\_\{\\mathrm\{peak\}\}\}\\leq\\arctan\(\\pi/2\)\(near the noise floor\)\.

9:

d^←arg⁡maxk∈\[isad,ipeak\]⁡\(−Δ2​ak\)\\hat\{d\}\\leftarrow\\arg\\max\_\{k\\,\\in\\,\[i\_\{\\mathrm\{sad\}\},\\,i\_\{\\mathrm\{peak\}\}\]\}\\big\(\-\\Delta^\{2\}a\_\{k\}\\big\), the geometric inflection of the head\-saturated

arctan\\arctancurve via second differences\.

10:return

d^rnd=g​⌈d^/g⌉\\hat\{d\}\_\{\\mathrm\{rnd\}\}=g\\,\\lceil\\hat\{d\}/g\\rceil⊳\\trianglerightceiling\-snap to the grid

##### NightVision hyperparameters

The hyperparameters in[Algorithms˜1](https://arxiv.org/html/2607.01313#alg1)and[2](https://arxiv.org/html/2607.01313#alg2)are fixed across all models in our evaluation, with values chosen on the development set \([Appendix˜C](https://arxiv.org/html/2607.01313#A3)\):

- •Number of promptsDD:D=20,000D=20\{,\}000high\-entropy prompts produced by the search procedure of[Section˜B\.1\.1](https://arxiv.org/html/2607.01313#A2.SS1.SSS1)\.
- •Samples per promptNN: swept up toN=106N=10^\{6\}in the budget curves \([Figure˜3](https://arxiv.org/html/2607.01313#S5.F3)\); per\-model results in[Appendix˜C](https://arxiv.org/html/2607.01313#A3.SS0.SSS0.Px2)use the maximum budget\.
- •Sampling temperatureτ\\tau:τ=2\.0\\tau=2\.0, applied at the API to flatten the next\-token distribution\.
- •Prompt length:55Unicode characters per prompt, sampled from the character groups described in[Section˜B\.1\.1](https://arxiv.org/html/2607.01313#A2.SS1.SSS1)\(not55tokens; tokenized length varies by model\)\.
- •Slope half\-windowww:w=max⁡\(20,⌊n/50⌋\)w=\\max\(20,\\,\\lfloor n/50\\rfloor\), wheren=min⁡\(D′,\|𝒞\|\)n=\\min\(D^\{\\prime\},\\,\|\\mathcal\{C\}\|\)is the spectrum length after pruning\.
- •Smoothing windowwsw\_\{s\}:ws=max⁡\(20,⌊n/100⌋\)w\_\{s\}=\\max\(20,\\,\\lfloor n/100\\rfloor\)for the centered moving average applied to the slope\.
- •Prominence fractionpp:p=0\.54p=0\.54, used to gate saddle detection ons~\\tilde\{s\}\.
- •Grid spacinggg:g=128g=128, the ceiling\-snap applied tod^\\hat\{d\}\(chosen to match the alignment of hidden dimensions in mainstream transformer configurations\)\.
- •Model precision:bfloat16for all log\-probability collection, matching the precision used for serving\.

### B\.3Timing Attack Details

#### B\.3\.1Overall Algorithm

The end\-to\-end algorithm for our timing attack can be found in[Figure˜11](https://arxiv.org/html/2607.01313#A2.F11)\.

NightVision Phase 2: Timing\-based architecture inference1:2:Reference modelsℱ=\{ϕi\}i=1K\\mathcal\{F\}=\\\{\\phi\_\{i\}\\\}\_\{i=1\}^\{K\}with knowndid\_\{i\},LiL\_\{i\}3:Target modelϕg\\phi\_\{g\}with unknownLgL\_\{g\};spectral estimated^g\\hat\{d\}\_\{g\}\([Section˜4\.1\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS1.Px3)\)4:Sequence lengthsℒ=\{ℓj\}j=1N′\\mathcal\{L\}=\\\{\\ell\_\{j\}\\\}\_\{j=1\}^\{N^\{\\prime\}\}5:Platform flag:SameorCross6:Inferred layer countL^g\\hat\{L\}\_\{g\}and total parametersP^g\\hat\{P\}\_\{g\}Stage 1: Collect prefill\-dominated runtimes1:foreach modelϕi∈ℱ∪\{ϕg\}\\phi\_\{i\}\\in\\mathcal\{F\}\\cup\\\{\\phi\_\{g\}\\\}do2:forj=1,…,N′j=1,\\dots,N^\{\\prime\}do3:Queryϕi\\phi\_\{i\}with input lengthℓj\\ell\_\{j\}\(long context, minimal generation\)4:Record runtimeTi,jT\_\{i,j\}5:endfor6:endforStage 2: InferL^g\\hat\{L\}\_\{g\}7:Fit L\-knownoverℱgref\\mathcal\{F\}^\{\\mathrm\{ref\}\}\_\{g\}:Ti,jref≈βgref​Li​ℓj2​di\+αrefT\_\{i,j\}^\{\\mathrm\{ref\}\}\\approx\\beta^\{\\mathrm\{ref\}\}\_\{g\}\\,L\_\{i\}\\,\\ell\_\{j\}^\{2\}\\,d\_\{i\}\+\\alpha^\{\\mathrm\{ref\}\}⊳\\trianglerightEq\. \([2](https://arxiv.org/html/2607.01313#S4.E2)\)8:Fit L\-unknownoverℱgref\\mathcal\{F\}^\{\\mathrm\{ref\}\}\_\{g\}:Ti,jref≈γgref,d​ℓj2​di\+ζrefT\_\{i,j\}^\{\\mathrm\{ref\}\}\\approx\\gamma^\{\\mathrm\{ref\},d\}\_\{g\}\\,\\ell\_\{j\}^\{2\}\\,d\_\{i\}\+\\zeta^\{\\mathrm\{ref\}\}⊳\\trianglerightEq\. \([6](https://arxiv.org/html/2607.01313#A2.E6)\)9:Fit L\-unknownoverℱgtgt\\mathcal\{F\}^\{\\mathrm\{tgt\}\}\_\{g\}:Ti,jtgt≈γgtgt,d^​ℓj2​d^i\+ζ′⁣tgtT\_\{i,j\}^\{\\mathrm\{tgt\}\}\\approx\\gamma^\{\\mathrm\{tgt\},\\hat\{d\}\}\_\{g\}\\,\\ell\_\{j\}^\{2\}\\,\\hat\{d\}\_\{i\}\+\\zeta^\{\\prime\\mathrm\{tgt\}\}⊳\\trianglerightEq\. \([7](https://arxiv.org/html/2607.01313#A2.E7)\)10:Compute transfer factor:k←γgtgt,d^/γgref,dk\\leftarrow\\gamma^\{\\mathrm\{tgt\},\\hat\{d\}\}\_\{g\}\\,/\\,\\gamma^\{\\mathrm\{ref\},d\}\_\{g\}⊳\\trianglerightEq\. \([8](https://arxiv.org/html/2607.01313#A2.E8)\)11:Fit per\-modelforfgf\_\{g\}withd^g\\hat\{d\}\_\{g\}:Tg,jtgt≈γgtgt​ℓj2​d^g\+ζgtgtT\_\{g,j\}^\{\\mathrm\{tgt\}\}\\approx\\gamma^\{\\mathrm\{tgt\}\}\_\{g\}\\,\\ell\_\{j\}^\{2\}\\,\\hat\{d\}\_\{g\}\+\\zeta^\{\\mathrm\{tgt\}\}\_\{g\}⊳\\trianglerightEq\. \([9](https://arxiv.org/html/2607.01313#A2.E9)\)12:L^g←γgtgt/\(k⋅βgref\)\\hat\{L\}\_\{g\}\\leftarrow\\gamma^\{\\mathrm\{tgt\}\}\_\{g\}\\,/\\,\(k\\cdot\\beta^\{\\mathrm\{ref\}\}\_\{g\}\)⊳\\trianglerightEq\. \([10](https://arxiv.org/html/2607.01313#A2.E10)\)Stage 3: Estimate parametersP^g\\hat\{P\}\_\{g\}15:Assume LLaMA\-styledecoder\-only transformer with GQA and SwiGLU16:P^g←2​V^​d^g\+L^g​\(2\+2​nkvnheads\+3​dffnd^g\)​d^g2\\hat\{P\}\_\{g\}\\leftarrow 2\\hat\{V\}\\hat\{d\}\_\{g\}\+\\hat\{L\}\_\{g\}\\left\(2\+2\\,\\frac\{n\_\{\\mathrm\{kv\}\}\}\{n\_\{\\mathrm\{heads\}\}\}\+3\\,\\frac\{d\_\{\\mathrm\{ffn\}\}\}\{\\hat\{d\}\_\{g\}\}\\right\)\\hat\{d\}\_\{g\}^\{\\,2\}, wherenkvnheads\{n\_\{\\mathrm\{kv\}\}\\over n\_\{\\mathrm\{heads\}\}\}= 1/4,dffnd^g\{d\_\{\\mathrm\{ffn\}\}\\over\\hat\{d\}\_\{g\}\}= 3\.5⊳\\trianglerightEq\. \([3](https://arxiv.org/html/2607.01313#S4.E3)\)17:returnd^g,L^g,P^g\\hat\{d\}\_\{g\},\\;\\hat\{L\}\_\{g\},\\;\\hat\{P\}\_\{g\}Figure 11:Timing\-based architecture inference pipeline\. The procedure fits prefill\-dominated scaling relations to infer layer countL^g\\hat\{L\}\_\{g\}and total parametersP^g\\hat\{P\}\_\{g\}from end\-to\-end runtime measurements, with optional cross\-platform calibration\.
#### B\.3\.2Effect of model architecture on accuracy

In practice, models largely obey the same scaling relation within each of three architectural categories:\(A\)GPT\-style with GELU and a two\-matrix FFN,\(B\)LLaMA\-style with SwiGLU and a three\-matrix gated FFN, and\(C\)Variants, such as Qwen3\.5, for which it contains a hybrid combining LLaMA\-style transformer blocks with DeltaNet linear\-attention layers\[Yanget al\.,[2025](https://arxiv.org/html/2607.01313#bib.bib2)\]\. Most modern LLMs fall into category \(B\), which dominates our evaluation\. Qwen3\.5 requires separate treatment \(e\.g\., see Fig[8](https://arxiv.org/html/2607.01313#S5.F8)\) because its DeltaNet layers replace quadraticO​\(ℓ2\)O\(\\ell^\{2\}\)attention with a linear\-complexity recurrence\. Consequently, only a subset of its layers exhibit theℓ2\\ell^\{2\}\-dependent cost assumed by Eq\. \([2](https://arxiv.org/html/2607.01313#S4.E2)\), reducing the effective layer count below the total architectural depth and necessitating category\-specific coefficients\.

As shown in Fig[12](https://arxiv.org/html/2607.01313#A2.F12), category \(A\) exhibits noticeably poorer adherence to the scaling relation than category \(B\)\. We attribute this to the structural difference in their FFN blocks: GPT\-style models use a two\-matrix FFN with GELU, whereas SwiGLU\-based models use a three\-matrix gated FFN\[Shazeer,[2020](https://arxiv.org/html/2607.01313#bib.bib3)\], resulting in 50% more general matrix–matrix multiplications \(GEMMs\) per layer\. This higher GEMM fraction means compute\-bound operations dominate a larger share of prefill wall\-clock time in category \(B\), making the quadratic scaling ansatz in Eq\. \([2](https://arxiv.org/html/2607.01313#S4.E2)\) a tighter fit\. In category \(A\), the smaller GEMM share allows non\-scaling overhead to perturb the relation more substantially\.

![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/scaling_law.png)Figure 12:Measured runtime versus predicted runtime with scaling relation as function of number of layers, hidden dimension, and input sequence length\.
#### B\.3\.3Inferring DepthLLunder Hardware Mismatch

In practice, the target model is rarely benchmarked on the same platform and configuration used to establish the reference scaling relation\. We therefore introduce a cross\-platform calibration procedure that transfers the reference relation to a new target platform via a single global scale factor\. The same framework also applies to same\-platform settings, accounting for residual system\-level variations such as inference configuration and runtime conditions\.

Concretely, we establish fitted global scaling relations Eq\. \([2](https://arxiv.org/html/2607.01313#S4.E2)\) on a reference platform using a set of reference modelsℱref=\{ϕiref\}i=1K\\mathcal\{F\}^\{\\mathrm\{ref\}\}=\\\{\\phi\_\{i\}^\{\\mathrm\{ref\}\}\\\}\_\{i=1\}^\{K\}with knowndrd\_\{r\}andLrL\_\{r\}:

Tr,j≈βref​Lr​ℓj2​dr\+αref≈γref​ℓj2​dr\+ζref,T\_\{r,j\}\\approx\\beta^\{\\mathrm\{ref\}\}\\,L\_\{r\}\\,\\ell\_\{j\}^\{2\}\\,d\_\{r\}\+\\alpha^\{\\mathrm\{ref\}\}\\approx\\gamma^\{\\mathrm\{ref\}\}\\,\\ell\_\{j\}^\{2\}\\,d\_\{r\}\+\\zeta^\{\\mathrm\{ref\}\},\(6\)
![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/val_metrics_bars_L_param_timing.png)Figure 13:Relative error for the estimate of the number of layers \(top panel\), total number of parameters \(middle panel\), and hidden dimension \(bottom panel\) for various models\. Here we assume the reference and target platform are the same \(both with NVIDIA H200 GPU\)\. Dark bars show the raw estimate; light bars show the estimate after rounding to the nearest integer\. Some of the largest errors are driven by small models with<<3B parameters\. For the layer\-count and total\-parameter estimates, the large errors forgranite\-3\.0\-3b\-a800mandgranite\-3\.0\-8barise from their small context window of 4096 tokens, which is insufficient to effectively probe the prefill stage\.![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/cross_gpu_comparison.png)Figure 14:Cross\-platform relative error for the estimated layer count \(top\) and total parameter count \(bottom\) across models, where the platform transfer is between NVIDIA H200 and A100 GPUs\. Dark bars denote H200 as the reference platform and A100 as the target; light bars denote the reverse configuration\.On the target platform, we assume a set of models availableℱtgt=\{ϕitgt\}i=1K′\\mathcal\{F\}^\{\\mathrm\{tgt\}\}=\\\{\\phi^\{\\mathrm\{tgt\}\}\_\{i\}\\\}\_\{i=1\}^\{K^\{\\prime\}\}\.

Letfgtgtf^\{\\mathrm\{tgt\}\}\_\{g\}denote the target model, we fit a global relation using spectral estimatesd^\\hat\{d\}\(Section[4\.1\.1](https://arxiv.org/html/2607.01313#S4.SS1.SSS1.Px3)\):

Ti,jtgt≈γtgt​ℓj2​d^i\+ζtgt\.T\_\{i,j\}^\{\\mathrm\{tgt\}\}\\approx\\gamma^\{\\mathrm\{tgt\}\}\\,\\ell\_\{j\}^\{2\}\\,\\hat\{d\}\_\{i\}\+\\zeta^\{\\mathrm\{tgt\}\}\.\(7\)Assumingd^≈d\\hat\{d\}\\approx d, we compute the platform transfer factor

k=γtgt/γref,k=\\gamma^\{\\mathrm\{tgt\}\}/\\gamma^\{\\mathrm\{ref\}\}\\;,\(8\)which captures the aggregate throughput ratio between the two platforms\. Applyingkkto the layer\-aware reference relation yields the cross\-platform approximation

Ti,jtgt≈k​βref​Li​ℓj2​di\+αref,T\_\{i,j\}^\{\\mathrm\{tgt\}\}\\approx k\\,\\beta^\{\\mathrm\{ref\}\}\\,L\_\{i\}\\,\\ell\_\{j\}^\{2\}\\,d\_\{i\}\+\\alpha^\{\\mathrm\{ref\}\},\(9\)from which the inferred layer count is recovered as:

L^g=γgtgtk⋅βref=γgtgt⋅γrefγtgt⋅βref,\\hat\{L\}\_\{g\}=\\frac\{\\gamma\_\{g\}^\{\\mathrm\{tgt\}\}\}\{k\\cdot\\beta^\{\\mathrm\{ref\}\}\}=\{\\gamma\_\{g\}^\{\\mathrm\{tgt\}\}\\cdot\\gamma^\{\\mathrm\{ref\}\}\\over\\gamma^\{\\mathrm\{tgt\}\}\\cdot\\beta^\{\\mathrm\{ref\}\}\},\(10\)whereγgtgt\\gamma\_\{g\}^\{\\mathrm\{tgt\}\}is the held\-out target model fitting parameter, obtained by solving \([7](https://arxiv.org/html/2607.01313#A2.E7)\) forγtgt\\gamma^\{\\mathrm\{tgt\}\}\.

This formulation assumes that the relative scaling behavior across layers is preserved across platforms, i\.e\., that platform differences manifest primarily as a multiplicative constant rather than altering the functional form of the relation\.

Figure[14](https://arxiv.org/html/2607.01313#A2.F14)presents cross\-platform estimation results between NVIDIA H200 and A100 GPUs, showing the inferred layer count \(top panel\) and total parameter count \(bottom panel\)\. Compared with the same\-platform setting of Figure[13](https://arxiv.org/html/2607.01313#A2.F13)\(reference and target models both on NVIDIA H200\), the cross\-platform results exhibit a similar range of relative errors and comparable trends across models, with an average relative error of approximately50%50\\%and the same model\-size dependency observed in the same\-platform case \(larger models tend to yield lower error\)\. These results demonstrate that our cross\-platform calibration procedure \(Section[B\.3\.3](https://arxiv.org/html/2607.01313#A2.SS3.SSS3)\) effectively compensates for hardware\-specific throughput differences without degrading inference accuracy\.

## Appendix CModel List

Table[C](https://arxiv.org/html/2607.01313#A3.SS0.SSS0.Px2)summarizes the architectural details of all language models evaluated in this work\. We report the configurations published on the Hugging Face Hub\. All values are extracted from each model’sconfig\.json\.

The 39 models span eight families: Alibaba’s Qwen 2\.5, Qwen 3, and Qwen 3\.5 series; IBM’s Granite 3\.0 through 3\.3 and the Granite 4\.0 dense models; Meta’s Llama 3\.1 and 3\.2 instruction models; AI2’s OLMo 2, OLMoE, and OLMo 3; and Hugging Face’s SmolLM2 and SmolLM3\. Together they cover more than two orders of magnitude of total parameters \(135M to 30B\) and span various architectures in current use: standard dense transformers \(Qwen 2\.5/3, Llama, OLMo 2, SmolLM, Granite 3\.x and 4\.0 dense\), sparse mixture\-of\-experts \(Qwen 3 30B\-A3B, Granite 3\.x MoE, OLMoE\), dense transformers with interleaved sliding\-window and full softmax\-attention layers \(OLMo 3\), and dense transformers with interleaved Gated DeltaNet linear\-attention and full softmax\-attention layers \(Qwen 3\.5\)\.

The table reports the number of decoder layersLL, the hidden dimensiondd, and the feed\-forward intermediate dimensiondffnd\_\{\\mathrm\{ffn\}\}, which corresponds to the size of a single expert’s MLP for mixture\-of\-experts \(MoE\) models and to the dense MLP otherwise\. Attention head counts are given ashq/hk​vh\_\{q\}/h\_\{kv\}, where equal values indicate standard multi\-head attention andhk​v<hqh\_\{kv\}<h\_\{q\}indicates grouped\-query attention\. The vocabulary size is denoted by\|𝒱\|\|\\mathcal\{V\}\|and the “Tied” column indicates whether the input embedding matrix is shared with the output projection\. The “Type” column reports the type of attention if full softmax\-attention is not used in all layers \(full softmax\-attention interleaved with sliding\-window attention iSWA\) or interleaved with Gated DeltaNet \(iGDN\) linear\-attention \(GDN\)\), and the type of feed\-forward network \(Dense or MoE\)\. For MoE models we additionally list the total and active expert counts per layer,Etot/EactE\_\{\\mathrm\{tot\}\}/E\_\{\\mathrm\{act\}\}, whereEactE\_\{\\mathrm\{act\}\}is the top\-kkrouting parameter\. For MoE models we further report active parameters per token, computed analytically from the configuration as\|𝒱\|⋅d\+L⋅\(Pattn\+Eact⋅Pexpert\+Prouter\)\|\\mathcal\{V\}\|\\cdot d\+L\\cdot\(P\_\{\\mathrm\{attn\}\}\+E\_\{\\mathrm\{act\}\}\\cdot P\_\{\\mathrm\{expert\}\}\+P\_\{\\mathrm\{router\}\}\), wherePattnP\_\{\\mathrm\{attn\}\}is the per\-layer attention parameter count,Pexpert=3​d⋅dffnP\_\{\\mathrm\{expert\}\}=3d\\cdot d\_\{\\mathrm\{ffn\}\}is the per\-expert SwiGLU MLP parameter count, andProuter=d⋅EtotP\_\{\\mathrm\{router\}\}=d\\cdot E\_\{\\mathrm\{tot\}\}is the per\-layer router parameter count\.

##### Development models\.

The following seven model configurations were used during method development \(hyperparameter selection, algorithmic choices, and prompt design\) and are held out from the validation set reported in[Section˜5](https://arxiv.org/html/2607.01313#S5):

- •HuggingFaceTB/SmolLM2\-360M\-Instruct
- •HuggingFaceTB/SmolLM2\-1\.7B\-Instruct
- •HuggingFaceTB/SmolLM3\-3B
- •Qwen/Qwen3\.5\-9B
- •Qwen/Qwen3\-0\.6B
- •ibm\-granite/granite\-3\.1\-1b\-a400m\-instruct
- •ibm\-granite/granite\-3\.1\-2b\-instruct

Together with the 32 validation models, these account for the 39 model configurations referenced throughout the paper\.

##### Reasoning mode\.

For Qwen models that support a toggleable reasoning \(“thinking”\) mode \(e\.g\., the Qwen 3 and Qwen 3\.5 series\), we disable reasoning mode for all experiments by passingenable\_thinking=Falseto the chat template\. This ensures that output tokens are sampled directly from the model’s next\-token distribution rather than from a post\-reasoning distribution, keeping the log\-probability and timing measurements consistent with the non\-reasoning models in our suite\.

Table 2:Architectural details of the language models evaluated in experiments\.ModelLLdddffnd\_\{\\mathrm\{ffn\}\}hq/hk​vh\_\{q\}/h\_\{kv\}\|𝒱\|\|\\mathcal\{V\}\|TiedTypeEtot/EactE\_\{\\mathrm\{tot\}\}/E\_\{\\mathrm\{act\}\}Params \(B\)\[\]\[\]Qwen 2\.5 \(Qwen/Qwen2\.5\-\*\-Instruct\)\[\]\[\] 0\.5B248964,86414/2151,936✓dense–0\.49\[\]\[\] 1\.5B281,5368,96012/2151,936✓dense–1\.54\[\]\[\] 3B362,04811,00816/2151,936✓dense–3\.09\[\]\[\] 7B283,58418,94428/4152,064–dense–7\.62\[\]\[\]Qwen 3 \(Qwen/Qwen3\-\*\)\[\]\[\] 0\.6B281,0243,07216/8151,936✓dense–0\.60\[\]\[\] 1\.7B282,0486,14416/8151,936✓dense–1\.72\[\]\[\] 4B362,5609,72832/8151,936✓dense–4\.02\[\]\[\] 8B364,09612,28832/8151,936–dense–8\.19\[\]\[\] 30B\-A3B482,04876832/4151,936–MoE128/830\.5 / 3\.4\[\]\[\] 30B\-A3B\-Inst\.482,04876832/4151,936–MoE128/830\.5 / 3\.4\[\]\[\]Qwen 3\.5 \(Qwen/Qwen3\.5\-\*;\[\]\[\] 0\.8B241,0243,5848/2248,320✓dense \+ GDN–0\.87\[\]\[\] 2B242,0486,1448/2248,320✓dense \+ GDN–2\.27\[\]\[\] 4B322,5609,21616/4248,320✓dense \+ GDN–4\.66\[\]\[\] 9B324,09612,28816/4248,320–dense \+ GDN–9\.65\[\]\[\]IBM Granite 3\.0–3\.3 \(ibm\-granite/granite\-3\.x\-\*\-instruct\)\[\]\[\] 3\.0\-1B\-A400M241,02451216/849,155✓MoE32/81\.33 / 0\.43\[\]\[\] 3\.0\-3B\-A800M321,53651224/849,155✓MoE40/83\.37 / 0\.88\[\]\[\] 3\.0\-2B402,0488,19232/849,155✓dense–2\.63\[\]\[\] 3\.0\-8B404,09612,80032/849,155✓dense–8\.17\[\]\[\] 3\.1\-1B\-A400M241,02451216/849,155✓MoE32/81\.33 / 0\.43\[\]\[\] 3\.1\-3B\-A800M321,53651224/849,155✓MoE40/83\.30 / 0\.88\[\]\[\] 3\.1\-2B402,0488,19232/849,155✓dense–2\.53\[\]\[\] 3\.1\-8B404,09612,80032/849,155✓dense–8\.17\[\]\[\] 3\.2\-2B402,0488,19232/849,155✓dense–2\.53\[\]\[\] 3\.2\-8B404,09612,80032/849,155✓dense–8\.17\[\]\[\] 3\.3\-2B402,0488,19232/849,159✓dense–2\.53\[\]\[\] 3\.3\-8B404,09612,80032/849,159✓dense–8\.17\[\]\[\]IBM Granite 4\.0 \(ibm\-granite/granite\-4\.0\-\*;\[\]\[\] 350M281,0242,04816/4100,352✓hybrid–0\.35\[\]\[\] Micro402,5608,19240/8100,352✓hybrid–3\.40\[\]\[\]Meta Llama 3\.x \(meta\-llama/Llama\-3\.x\-\*\-Instruct\)\[\]\[\] 3\.2\-1B162,0488,19232/8128,256✓dense–1\.24\[\]\[\] 3\.2\-3B283,0728,19224/8128,256✓dense–3\.21\[\]\[\] 3\.1\-8B324,09614,33632/8128,256–dense–8\.03\[\]\[\]AI2 OLMo \(allenai/OLMo\*\-\*\-Instruct\)\[\]\[\] OLMo\-2\-1B \(0425\)162,0488,19216/16100,352–dense–1\.48\[\]\[\] OLMo\-2\-7B \(1124\)324,09611,00832/32100,352–dense–7\.30\[\]\[\] OLMoE\-1B\-7B \(0924\)162,0481,02416/1650,304–MoE64/86\.92 / 1\.28\[\]\[\] OLMo\-3\-7B324,09611,00832/32100,278–dense \+ SWA–7\.30\[\]\[\]HuggingFace SmolLM \(HuggingFaceTB/SmolLM\*\-\*\)\[\]\[\] SmolLM2\-135M305761,5369/349,152✓dense–0\.13\[\]\[\] SmolLM2\-360M329602,56015/549,152✓dense–0\.36\[\]\[\] SmolLM2\-1\.7B242,0488,19232/3249,152✓dense–1\.71\[\]\[\] SmolLM3\-3B362,04811,00816/4128,256✓dense–3\.08
## Appendix DExperimental Setup Details

##### Hardware

Each model runs on a single NVIDIA H100 \(RHEL 8 / glibc 2\.28; CUDA 12\.6; PyTorch with CUDA 12\.6; Python 3\.12\)\. Timing measurements are additionally repeated on H200 and A100 GPUs to test cross\-platform calibration\.

Timing protocol\.For each model, we build synthetic prompts ofℓ\\elltokens by repeating a fixed passage\. After three warmup requests, we issue100100batch\-size\-one chat requests generating one output token, measuring client\-side latency viatime\.time\(\)and subtracting anℓ=1\\ell\{=\}1baseline to isolate the prefill cost\. At eachℓ\\ellwe report the10%10\\%\-trimmed mean overN=100N\{=\}100trials, weighting regressions byσT=std/Nkept\\sigma\_\{T\}=\\mathrm\{std\}/\\sqrt\{N\_\{\\text\{kept\}\}\}\. We sweepn=24n=24lengths spanning the projection\-dominated \(O​\(L​ℓ​d2\)O\(L\\ell d^\{2\}\)\) to attention\-dominated \(O​\(L​ℓ2​d\)O\(L\\ell^\{2\}d\)\) transition, with crossoverℓ⋆=⌊\(d\+dkv\+32​dffn\)​r⌋\\ell^\{\\star\}=\\lfloor\(d\+d\_\{\\mathrm\{kv\}\}\+\\tfrac\{3\}\{2\}d\_\{\\mathrm\{ffn\}\}\)r\\rfloor, wheredkv=nkv​\(d/nheads\)d\_\{\\mathrm\{kv\}\}=n\_\{\\mathrm\{kv\}\}\(d/n\_\{\\mathrm\{heads\}\}\)andr≈0\.6r\\approx 0\.6is the empirical attention\-to\-MLP throughput ratio; points span\[0\.5​ℓ⋆,min⁡\(3​ℓ⋆,ℓcap\)\]\[0\.5\\ell^\{\\star\},\\,\\min\(3\\ell^\{\\star\},\\ell\_\{\\mathrm\{cap\}\}\)\]withℓcap=min⁡\(max⁡\(2​ℓ⋆,4​d\),pmax−200\)\\ell\_\{\\mathrm\{cap\}\}=\\min\(\\max\(2\\ell^\{\\star\},4d\),p\_\{\\max\}\-200\), and we discard 12 points to restrict the fit to the attention\-dominated regime\.

##### Sequence\-length sweep

We sweep input lengthsℒ=\{ℓj\}j=1N′\\mathcal\{L\}=\\\{\\ell\_\{j\}\\\}\_\{j=1\}^\{N^\{\\prime\}\}chosen to span the transition between the projection\-dominated \(O​\(L​ℓ​d2\)O\(L\\,\\ell\\,d^\{2\}\)\) and attention\-dominated \(O​\(L​ℓ2​d\)O\(L\\,\\ell^\{2\}\\,d\)\) regimes\. We estimate the crossover as

ℓ⋆=⌊\(d\+dkv\+32​dffn\)⋅r⌋,\\ell^\{\\star\}=\\left\\lfloor\\left\(d\+d\_\{\\mathrm\{kv\}\}\+\\tfrac\{3\}\{2\}\\,d\_\{\\mathrm\{ffn\}\}\\right\)\\cdot r\\right\\rfloor,\(11\)wheredkv=nkv⋅\(d/nheads\)d\_\{\\mathrm\{kv\}\}=n\_\{\\mathrm\{kv\}\}\\cdot\(d/n\_\{\\mathrm\{heads\}\}\)withnheadsn\_\{\\mathrm\{heads\}\}attention heads andnkvn\_\{\\mathrm\{kv\}\}KV heads,dffnd\_\{\\mathrm\{ffn\}\}is the feed\-forward intermediate dimension, andr≈0\.6r\\approx 0\.6reflects the empirical attention\-to\-MLP throughput ratio\. We sweepn=24n=24uniformly spaced points fromℓlo=0\.5​ℓ⋆\\ell\_\{\\mathrm\{lo\}\}=0\.5\\,\\ell^\{\\star\}toℓhi=min⁡\(3​ℓ⋆,ℓcap\)\\ell\_\{\\mathrm\{hi\}\}=\\min\(3\\,\\ell^\{\\star\},\\,\\ell\_\{\\mathrm\{cap\}\}\), whereℓcap=min⁡\(max⁡\(2​ℓ⋆,4​d\),pmax−200\)\\ell\_\{\\mathrm\{cap\}\}=\\min\(\\max\(2\\,\\ell^\{\\star\},\\,4d\),\\;p\_\{\\max\}\-200\)keeps sequences within the positional encoding limitpmaxp\_\{\\max\}, and additionally includeℓ⋆\\ell^\{\\star\}when it falls inside the sweep\. When fitting the scaling relation, we discard 12 points to restrict the fit to the attention\-dominated regime, excluding sequences too short for prefill cost to dominate as well as outliers\.

## Appendix EAdditional Results

### E\.1Spectral Shape and Eigenspectrum Saturation

![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/llama_3.1_8b_shape_plot.png)Figure 15:Windowed sloped​arctan⁡\(λ\)/d​id\\arctan\(\\lambda\)/diof the ordered eigenvalues for Llama\-3\.1\-8B at the maximum sample budget \(N=106N\{=\}10^\{6\}per prompt\)\. The true hidden dimensiond∗=4,096d^\{\*\}\{=\}4\{,\}096is marked with a vertical line\. The change\-point detector \([Section˜B\.1\.3](https://arxiv.org/html/2607.01313#A2.SS1.SSS3)\) localizes the elbow inside the area of interest \(AOI\) bracketed by the saddle \(1655\) and peak \(3077\)\. Both bracket landmarks fall well below the trued∗d^\{\*\}, so the estimated^\\hat\{d\}underestimates the rank: at this budget the eigenspectrum has not yet saturated, and the elbow associated with the true rank is not yet pronounced enough for the detector to recover it\. Larger sample budgets are needed to push the AOI up tod∗d^\{\*\}\.
### E\.2Timing

![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/L_param_error_budget.png)Figure 16:Relative error in estimating the number of layers and total number of parameters as a function of API query budget \(measured in number of tokens\)\. Some of the largest errors are driven by small models with≤\\leq4B parameters\.![Refer to caption](https://arxiv.org/html/2607.01313v1/figs/err_vs_H_pred_budget_avg_cost.png)Figure 17:Estimation relative error versus number of API calls \(left panel\) and estimated cost in USD \(right panel\)\. Relative errors for d, L, and N are shown as a function of total token budget, with bold curves denoting averages across target models and faint curves denoting individual models\. Larger budgets generally reduce error, with L improving fastest and N remaining harder to estimate\. Results include≥\\geq3B\-parameter dense models and MoE models\.
## Appendix FProofs

Here we prove[Theorem˜1](https://arxiv.org/html/2607.01313#Thmtheorem1)in[Section˜4\.1\.2](https://arxiv.org/html/2607.01313#S4.SS1.SSS2)and provide a generalization to the non\-uniform case\. Throughout, we let𝒱\\mathcal\{V\}be a finite vocabulary of size\|𝒱\|\|\\mathcal\{V\}\|, with tokensv1,…,v\|𝒱\|v\_\{1\},\\dots,v\_\{\|\\mathcal\{V\}\|\}\. We drawDDrandom subsets𝒯1,𝒯2,…,𝒯D⊆𝒱\\mathcal\{T\}\_\{1\},\\mathcal\{T\}\_\{2\},\\dots,\\mathcal\{T\}\_\{D\}\\subseteq\\mathcal\{V\}as follows: for eachq∈\[D\]q\\in\[D\]we sampleNNtokens from𝒱\\mathcal\{V\}independently and with replacement, and let𝒯q\\mathcal\{T\}\_\{q\}be the set of distinct tokens obtained\. Theqq\-th subset is in general drawn according to its own probability distribution over𝒱\\mathcal\{V\},

𝐩q=\(pq\(1\),pq\(2\),…,pq\(\|𝒱\|\)\),∑i=1\|𝒱\|pq\(i\)=1\.\\mathbf\{p\}\_\{q\}=\\left\(p^\{\(1\)\}\_\{q\},p^\{\(2\)\}\_\{q\},\\dots,p^\{\(\|\\mathcal\{V\}\|\)\}\_\{q\}\\right\),\\qquad\\sum\_\{i=1\}^\{\|\\mathcal\{V\}\|\}p^\{\(i\)\}\_\{q\}=1\.\(12\)The subscriptqqindexes the subset \(which of theDDdistributions is in force\) while the parenthesized superscript indexes the token; hencepq\(i\)p^\{\(i\)\}\_\{q\}is the probability of drawing tokenviv\_\{i\}when forming𝒯q\\mathcal\{T\}\_\{q\}\. The uniform case of[Theorem˜1](https://arxiv.org/html/2607.01313#Thmtheorem1)is recovered by taking every distribution to be uniform over𝒱\\mathcal\{V\}, i\.e\.pq\(i\)=1/\|𝒱\|p^\{\(i\)\}\_\{q\}=1/\|\\mathcal\{V\}\|for every tokenviv\_\{i\},i∈\[\|𝒱\|\]i\\in\[\|\\mathcal\{V\}\|\], and every subsetq∈\[D\]q\\in\[D\]\. We drop the subscriptqqwhenever the distribution does not depend onqq, writing𝐩=\(p\(1\),…,p\(\|𝒱\|\)\)\\mathbf\{p\}=\(p^\{\(1\)\},\\dots,p^\{\(\|\\mathcal\{V\}\|\)\}\)\. Finally, we denote by𝒞≜⋂q∈\[D\]𝒯q\\mathcal\{C\}\\triangleq\\bigcap\_\{q\\in\[D\]\}\\mathcal\{T\}\_\{q\}the common intersection of theDDsubsets, i\.e\., the tokens that appear in every subset\.

###### Lemma 1\(Negative association of intersection indicators\)\.

LetXi≜𝟏​\(vi∈𝒞\)∈\{0,1\}X\_\{i\}\\triangleq\\mathbf\{1\}\(v\_\{i\}\\in\\mathcal\{C\}\)\\in\\\{0,1\\\}be the indicator random variable for the event that tokenviv\_\{i\}is in the common token set𝒞\\mathcal\{C\}\. ThenX1,…,XVX\_\{1\},\\dots,X\_\{V\}are negatively associated\.

###### Proof\.

A collection of random variables\(X1,…,Xm\)\(X\_\{1\},\\dots,X\_\{m\}\)is*negatively associated*\(NA\) if for every pair of disjoint index setsI,J⊆\[m\]I,J\\subseteq\[m\]and every pair of functionsf,gf,gthat are both non\-decreasing \(or both non\-increasing\),

𝔼​\[f​\(Xi,i∈I\)​g​\(Xj,j∈J\)\]≤𝔼​\[f​\(Xi,i∈I\)\]​𝔼​\[g​\(Xj,j∈J\)\]\.\\mathbb\{E\}\\left\[f\(X\_\{i\},i\\in I\)\\,g\(X\_\{j\},j\\in J\)\\right\]\\;\\leq\\;\\mathbb\{E\}\\left\[f\(X\_\{i\},i\\in I\)\\right\]\\,\\mathbb\{E\}\\left\[g\(X\_\{j\},j\\in J\)\\right\]\.
NA captures the idea that the variables compete: when one subset of them takes on larger values, any disjoint subset tends to take on smaller values, so they cannot all be large simultaneously\. NA is stronger than negative correlation\. Correlation tracks how variables move in pairs, whereas NA says that any group of them competes with any disjoint group\. So NA variables are always negatively correlated, but variables can be negatively correlated in pairs without exhibiting this fuller competition\. We refer the reader toDubhashi and Ranjan\[[1996](https://arxiv.org/html/2607.01313#bib.bib12)\]for additional background\.

In our setting, we performNNdraws in each of theDDsubsets, thusN​DNDdraws in total\. For eachq∈\[D\]q\\in\[D\]andk∈\[N\]k\\in\[N\], let𝐰q,k∈\{0,1\}\|𝒱\|\\mathbf\{w\}\_\{q,k\}\\in\\\{0,1\\\}^\{\|\\mathcal\{V\}\|\}be the one\-hot encoding of thekk\-th token drawn in subsetqq\. Its coordinatewq,k\(i\)w\_\{q,k\}^\{\(i\)\}indicates whether tokenviv\_\{i\}was the one drawn on thekk\-th draw of subsetqq\. By construction∑i=1\|𝒱\|wq,k\(i\)=1\\sum\_\{i=1\}^\{\|\\mathcal\{V\}\|\}w\_\{q,k\}^\{\(i\)\}=1with eachwq,k\(i\)∈\{0,1\}w\_\{q,k\}^\{\(i\)\}\\in\\\{0,1\\\}, so by Lemma 8 ofDubhashi and Ranjan\[[1996](https://arxiv.org/html/2607.01313#bib.bib12)\]the entries of𝐰q,k\\mathbf\{w\}\_\{q,k\}are NA\.

Next, we show that NA of𝐰q,k\\mathbf\{w\}\_\{q,k\}implies NA of the indicators\(X1,…,X\|𝒱\|\)\(X\_\{1\},\\dots,X\_\{\|\\mathcal\{V\}\|\}\), whereXi≜𝟏​\(vi∈𝒞\)X\_\{i\}\\triangleq\\mathbf\{1\}\(v\_\{i\}\\in\\mathcal\{C\}\)records whether tokenviv\_\{i\}is drawn at least once in every one of theDDsubsets\. The indicators can be expressed in terms of the one\-hot encodings as

Xi\\displaystyle X\_\{i\}=∏q=1Dmaxk∈\[N\]⁡wq,k\(i\),\\displaystyle=\\prod\_\{q=1\}^\{D\}\\max\_\{k\\in\[N\]\}w\_\{q,k\}^\{\(i\)\},\(13\)sincemaxk∈\[N\]⁡wq,k\(i\)=1\\max\_\{k\\in\[N\]\}w\_\{q,k\}^\{\(i\)\}=1exactly whenvi∈𝒯qv\_\{i\}\\in\\mathcal\{T\}\_\{q\}, and the product overqqis11exactly whenvi∈𝒯qv\_\{i\}\\in\\mathcal\{T\}\_\{q\}for allqq, i\.e\.vi∈𝒞v\_\{i\}\\in\\mathcal\{C\}\.

Each𝐰q,k\\mathbf\{w\}\_\{q,k\}is a deterministic function of a single draw, and theN​DNDdraws are mutually independent, so the vectors\{𝐰q,k\}\\\{\\mathbf\{w\}\_\{q,k\}\\\}are mutually independent\. Combining the within\-draw NA of each𝐰q,k\\mathbf\{w\}\_\{q,k\}with this independence, the union of independent sets of NA random variables is again NA by property 7 ofJoag\-Dev and Proschan\[[1983](https://arxiv.org/html/2607.01313#bib.bib10)\], so the full collection\{wq,k\(i\)\}i,q,k\\\{w\_\{q,k\}^\{\(i\)\}\\\}\_\{i,q,k\}is NA\. Each indicatorXiX\_\{i\}depends only on the coordinates\{wq,k\(i\):q∈\[D\],k∈\[N\]\}\\\{w\_\{q,k\}^\{\(i\)\}:q\\in\[D\],k\\in\[N\]\\\}indexed by tokenviv\_\{i\}, and these subsets are disjoint across distinctii\. Sincemax\\maxand products of\{0,1\}\\\{0,1\\\}\-valued terms are non\-decreasing, eachXiX\_\{i\}is a non\-decreasing function defined over disjoint subsets of NA random variables, and hence the indicators are NA by property 6 ofJoag\-Dev and Proschan\[[1983](https://arxiv.org/html/2607.01313#bib.bib10)\]\. ∎

See[1](https://arxiv.org/html/2607.01313#Thmtheorem1)

###### Proof\.

The intersection size can be expressed asC≜\|𝒞\|=∑i=1\|𝒱\|XiC\\triangleq\|\\mathcal\{C\}\|=\\sum\_\{i=1\}^\{\|\\mathcal\{V\}\|\}X\_\{i\}, whereXi=𝟏​\(vi∈𝒞\)X\_\{i\}=\\mathbf\{1\}\(v\_\{i\}\\in\\mathcal\{C\}\)as in[Lemma˜1](https://arxiv.org/html/2607.01313#Thmlemma1)\.

*Step 1: Expected intersection size\.*By independence across theDDsubsets,

μ≜𝔼​\[C\]=∑i=1\|𝒱\|𝔼​\[Xi\]=∑i=1\|𝒱\|∏q=1Dℙ​\(vi∈𝒯q\)\.\\displaystyle\\mu\\triangleq\\mathbb\{E\}\[C\]=\\sum\_\{i=1\}^\{\|\\mathcal\{V\}\|\}\\mathbb\{E\}\[X\_\{i\}\]=\\sum\_\{i=1\}^\{\|\\mathcal\{V\}\|\}\\prod\_\{q=1\}^\{D\}\\mathbb\{P\}\(v\_\{i\}\\in\\mathcal\{T\}\_\{q\}\)\.\(14\)Since each draw within𝒯q\\mathcal\{T\}\_\{q\}samples tokenviv\_\{i\}uniformly with probability1/\|𝒱\|1/\|\\mathcal\{V\}\|, the probability thatviv\_\{i\}never appears inNNdraws is\(1−1/\|𝒱\|\)N\(1\-1/\|\\mathcal\{V\}\|\)^\{N\}, soℙ​\(vi∈𝒯q\)=1−\(1−1/\|𝒱\|\)N\\mathbb\{P\}\(v\_\{i\}\\in\\mathcal\{T\}\_\{q\}\)=1\-\(1\-1/\|\\mathcal\{V\}\|\)^\{N\}\. Substituting gives

μ=\|𝒱\|​\(1−\(1−1/\|𝒱\|\)N\)D\.\\displaystyle\\mu=\|\\mathcal\{V\}\|\\bigl\(1\-\(1\-1/\|\\mathcal\{V\}\|\)^\{N\}\\bigr\)^\{D\}\.\(15\)
*Step 2: Concentration via a Chernoff bound for NA variables\.*By[Lemma˜1](https://arxiv.org/html/2607.01313#Thmlemma1),CCis a sum of the NA indicatorsX1,…,X\|𝒱\|∈\{0,1\}X\_\{1\},\\dots,X\_\{\|\\mathcal\{V\}\|\}\\in\\\{0,1\\\}\. The multiplicative Chernoff lower\-tail bound holds exactly as in the independent case\[Dubhashi and Ranjan,[1996](https://arxiv.org/html/2607.01313#bib.bib12)\]\. Thus for anyε∈\(0,1\)\\varepsilon\\in\(0,1\),

ℙ​\(C≤\(1−ε\)​μ\)≤exp⁡\(−ε2​μ2\)\.\\displaystyle\\mathbb\{P\}\\bigl\(C\\leq\(1\-\\varepsilon\)\\mu\\bigr\)\\leq\\exp\\\!\\left\(\-\\frac\{\\varepsilon^\{2\}\\mu\}\{2\}\\right\)\.\(16\)Setting the right\-hand side equal toδ\\deltagivesε=2​ln⁡\(1/δ\)/μ\\varepsilon=\\sqrt\{2\\ln\(1/\\delta\)/\\mu\}, so with probability at least1−δ1\-\\delta,

C≥μ−2​μ​ln⁡\(1/δ\)\.\\displaystyle C\\geq\\mu\-\\sqrt\{2\\mu\\ln\(1/\\delta\)\}\.\(17\)
*Step 3: Sufficient condition onμ\\mu\.*It therefore suffices to ensureμ−2​μ​ln⁡\(1/δ\)≥d\\mu\-\\sqrt\{2\\mu\\ln\(1/\\delta\)\}\\geq d\. Writingu=μu=\\sqrt\{\\mu\}anda=2​ln⁡\(1/δ\)a=\\sqrt\{2\\ln\(1/\\delta\)\}, this is the quadratic inequalityu2−a​u−d≥0u^\{2\}\-au\-d\\geq 0, whose positive root givesu≥12​\(a\+a2\+4​d\)u\\geq\\tfrac\{1\}\{2\}\(a\+\\sqrt\{a^\{2\}\+4d\}\)\. Squaring and usinga2\+4​d≤a\+2​d\\sqrt\{a^\{2\}\+4d\}\\leq a\+2\\sqrt\{d\},

μ≥d\+2​d​ln⁡\(1/δ\)\+2​ln⁡\(1/δ\)\.\\displaystyle\\mu\\geq d\+\\sqrt\{2d\\ln\(1/\\delta\)\}\+2\\ln\(1/\\delta\)\.\(18\)
*Step 4: Solving forNN\.*Combining \([15](https://arxiv.org/html/2607.01313#A6.E15)\) and \([18](https://arxiv.org/html/2607.01313#A6.E18)\), it suffices that

\|𝒱\|​\(1−\(1−1/\|𝒱\|\)N\)D≥d\+2​d​ln⁡\(1/δ\)\+2​ln⁡\(1/δ\)\.\\displaystyle\|\\mathcal\{V\}\|\\bigl\(1\-\(1\-1/\|\\mathcal\{V\}\|\)^\{N\}\\bigr\)^\{D\}\\geq d\+\\sqrt\{2d\\ln\(1/\\delta\)\}\+2\\ln\(1/\\delta\)\.\(19\)Solving forNN,

N≥ln⁡\[1−\(d\+2​d​ln⁡\(1/δ\)\+2​ln⁡\(1/δ\)\|𝒱\|\)1/D\]ln⁡\(1−1/\|𝒱\|\)\.\\displaystyle N\\geq\\frac\{\\ln\\\!\\left\[1\-\\left\(\\dfrac\{d\+\\sqrt\{2d\\ln\(1/\\delta\)\}\+2\\ln\(1/\\delta\)\}\{\|\\mathcal\{V\}\|\}\\right\)^\{1/D\}\\right\]\}\{\\ln\(1\-1/\|\\mathcal\{V\}\|\)\}\.\(20\)∎

##### Interpretation of the bound\.

Here we interpret sample complexity provided by the bound in[Theorem˜1](https://arxiv.org/html/2607.01313#Thmtheorem1)\. Letmmbe the required value of the required value ofμ\\mufrom \([18](https://arxiv.org/html/2607.01313#A6.E18)\), namely

m≜d\+2​d​ln⁡\(1/δ\)\+2​ln⁡\(1/δ\),m\\;\\triangleq\\;d\+\\sqrt\{2d\\ln\(1/\\delta\)\}\+2\\ln\(1/\\delta\),so that the sufficient condition isN≥N⋆N\\geq N^\{\\star\}with

N⋆≜−ln⁡\[1−\(m/\|𝒱\|\)1/D\]−ln⁡\(1−1/\|𝒱\|\)\.N^\{\\star\}\\;\\triangleq\\;\\frac\{\-\\ln\\\!\\left\[1\-\(m/\|\\mathcal\{V\}\|\)^\{1/D\}\\right\]\}\{\-\\ln\(1\-1/\|\\mathcal\{V\}\|\)\}\.We proceed bound the numerator and denominator\. The denominator satisfies−ln⁡\(1−1/\|𝒱\|\)≥1/\|𝒱\|\-\\ln\(1\-1/\|\\mathcal\{V\}\|\)\\geq 1/\|\\mathcal\{V\}\|, so its reciprocal is at most\|𝒱\|\|\\mathcal\{V\}\|\. For the numerator, let\(m/\|𝒱\|\)1/D=e−t\(m/\|\\mathcal\{V\}\|\)^\{1/D\}=e^\{\-t\}witht≜1D​ln⁡\|𝒱\|mt\\triangleq\\tfrac\{1\}\{D\}\\ln\\tfrac\{\|\\mathcal\{V\}\|\}\{m\}\. The inequality1−e−t≥t1\+t1\-e^\{\-t\}\\geq\\tfrac\{t\}\{1\+t\}gives−ln⁡\(1−e−t\)≤ln⁡\(1\+1/t\)\-\\ln\(1\-e^\{\-t\}\)\\leq\\ln\(1\+1/t\), and in the regimem≤\|𝒱\|/em\\leq\|\\mathcal\{V\}\|/ewe havet≥1/Dt\\geq 1/D, hence1/t≤D1/t\\leq D\. Combining,

N⋆≤\|𝒱\|​ln⁡\(1\+D\)=O​\(\|𝒱\|​ln⁡D\),N^\{\\star\}\\;\\leq\\;\|\\mathcal\{V\}\|\\,\\ln\(1\+D\)\\;=\\;O\(\|\\mathcal\{V\}\|\\ln D\),so it suffices to takeN=O​\(\|𝒱\|​ln⁡D\)N=O\(\|\\mathcal\{V\}\|\\ln D\)samples per subset, for a total query budgetD​N=O​\(\|𝒱\|​D​ln⁡D\)DN=O\(\|\\mathcal\{V\}\|\\,D\\ln D\)\.

[Theorem˜1](https://arxiv.org/html/2607.01313#Thmtheorem1)assumes the output distribution over tokens is exactly uniform\. In practice, the proposed common\-set prompting search method finds prompts whose resulting output distributions are approximately, but not perfectly, uniform in the high temperature\. We subsequently generalize the theorem by introducing a notion oftt\-flatness, which captures the degree to which an output distribution is spread across a sub\-vocabulary of interest\.

###### Definition 1\.

\[tt\-flat distribution\] A probability distribution𝐩=\(p\(1\),…,p\(\|𝒱\|\)\)\\mathbf\{p\}=\(p^\{\(1\)\},\\dots,p^\{\(\|\\mathcal\{V\}\|\)\}\)over a vocabulary𝒱\\mathcal\{V\}is*tt\-flat over𝒱′⊆𝒱\\mathcal\{V\}^\{\\prime\}\\subseteq\\mathcal\{V\}*ifp\(i\)≥tp^\{\(i\)\}\\geq tfor allvi∈𝒱′v\_\{i\}\\in\\mathcal\{V\}^\{\\prime\}\.

Equipped with this definition, we generalize[Theorem˜1](https://arxiv.org/html/2607.01313#Thmtheorem1)to the non\-uniform setting where each of theDDsubsets may be drawn from its own𝐩q\\mathbf\{p\}\_\{q\}, provided they share a common floortton the sub\-vocabulary𝒱′\\mathcal\{V\}^\{\\prime\}of interest\.

###### Corollary 1\.

\[Sample complexity undertt\-flat distributions\] Let𝒯1,…,𝒯D\\mathcal\{T\}\_\{1\},\\dots,\\mathcal\{T\}\_\{D\}beDDindependent random subsets of𝒱\\mathcal\{V\}, each formed by drawingNNtokens with replacement from its own distribution𝐩q\\mathbf\{p\}\_\{q\}, where every𝐩q\\mathbf\{p\}\_\{q\}istt\-flat over a common sub\-vocabulary𝒱′⊆𝒱\\mathcal\{V\}^\{\\prime\}\\subseteq\\mathcal\{V\}\. If

N≥ln⁡\[1−\(d\+2​d​ln⁡\(1/δ\)\+2​ln⁡\(1/δ\)\|𝒱′\|\)1/D\]ln⁡\(1−t\)N\\geq\\frac\{\\ln\\\!\\left\[1\-\\left\(\\frac\{d\+\\sqrt\{2d\\ln\(1/\\delta\)\}\+2\\ln\(1/\\delta\)\}\{\|\\mathcal\{V\}^\{\\prime\}\|\}\\right\)^\{1/D\}\\right\]\}\{\\ln\(1\-t\)\}\(21\)then\|𝒞\|≥d\|\\mathcal\{C\}\|\\geq dwith probability at least1−δ1\-\\delta\.

###### Proof\.

LetC′≜∑vi∈𝒱′XiC^\{\\prime\}\\triangleq\\sum\_\{v\_\{i\}\\in\\mathcal\{V\}^\{\\prime\}\}X\_\{i\}be the intersection size restricted to𝒱′\\mathcal\{V\}^\{\\prime\}, withXi=𝟏​\(vi∈𝒞\)X\_\{i\}=\\mathbf\{1\}\(v\_\{i\}\\in\\mathcal\{C\}\)as in[Lemma˜1](https://arxiv.org/html/2607.01313#Thmlemma1)\. Since𝒱′⊆𝒱\\mathcal\{V\}^\{\\prime\}\\subseteq\\mathcal\{V\},C′≤\|𝒞\|C^\{\\prime\}\\leq\|\\mathcal\{C\}\|, so it suffices to guaranteeC′≥dC^\{\\prime\}\\geq d\. The indicators\{Xi:vi∈𝒱′\}\\\{X\_\{i\}:v\_\{i\}\\in\\mathcal\{V\}^\{\\prime\}\\\}are a subcollection of the NA family of[Lemma˜1](https://arxiv.org/html/2607.01313#Thmlemma1)and any subset of an NA family is NA\. Steps 2 and 3 of the proof of[Theorem˜1](https://arxiv.org/html/2607.01313#Thmtheorem1)therefore apply toC′C^\{\\prime\}verbatim and it is sufficient thatμ′≜𝔼​\[C′\]≥d\+2​d​ln⁡\(1/δ\)\+2​ln⁡\(1/δ\)\\mu^\{\\prime\}\\triangleq\\mathbb\{E\}\[C^\{\\prime\}\]\\geq d\+\\sqrt\{2d\\ln\(1/\\delta\)\}\+2\\ln\(1/\\delta\)\. By independence across theDDsubsets andtt\-flatness,pq\(i\)≥tp\_\{q\}^\{\(i\)\}\\geq tfor everyq∈\[D\]q\\in\[D\]andvi∈𝒱′v\_\{i\}\\in\\mathcal\{V\}^\{\\prime\}, so

μ′=∑vi∈𝒱′∏q=1D\(1−\(1−pq\(i\)\)N\)≥\|𝒱′\|​\(1−\(1−t\)N\)D\.\\displaystyle\\mu^\{\\prime\}=\\sum\_\{v\_\{i\}\\in\\mathcal\{V\}^\{\\prime\}\}\\prod\_\{q=1\}^\{D\}\\bigl\(1\-\(1\-p\_\{q\}^\{\(i\)\}\)^\{N\}\\bigr\)\\;\\geq\\;\|\\mathcal\{V\}^\{\\prime\}\|\\bigl\(1\-\(1\-t\)^\{N\}\\bigr\)^\{D\}\.\(22\)Ensuring the right\-hand side is at leastd\+2​d​ln⁡\(1/δ\)\+2​ln⁡\(1/δ\)d\+\\sqrt\{2d\\ln\(1/\\delta\)\}\+2\\ln\(1/\\delta\)and solving forNN\(as in Step 4 of[Theorem˜1](https://arxiv.org/html/2607.01313#Thmtheorem1), usingln⁡\(1−t\)<0\\ln\(1\-t\)<0\) gives the stated bound\. ∎