When More Sampling Hurts: The Modal Ceiling and Correlation Ceiling of Test-Time Scaling

# 1 Introduction and roadmap
Source: [https://arxiv.org/html/2606.28661](https://arxiv.org/html/2606.28661)
When More Sampling Hurts: The Modal Ceiling and Correlation Ceiling of Test\-Time Scaling

Yong Yi Bay\*Kathleen A\. Yearick\*

PhD, University of Illinois at Urbana\-Champaign

††footnotetext:\*Equal contribution\. Correspondence:\{yongyibay, kallie\.a\.yearick\}@gmail\.com\.

Abstract

- People overthink; language models over\-sample, and the extra effort can talk both into a*worse*answer\. Reasoning systems answer a hard question by sampling it many times \(*test\-time scaling*\), and the more they draw, the more often a correct answer turns up somewhere, so*coverage*, the fraction of problems with at least one correct try, climbs and appears to be progress\. But a deployed system must return one answer, and choosing it, not knowing which try is right, is*selection*; selection is capped, and past a point extra samples only make the model surer of a confident mistake, even as every draw adds cost\. The gap between climbing coverage and stalled selection, the*identifiability gap*, is the answer a model can produce but not pick\. So the real question is not whether to sample but how far, and the answer is: not far\. For picking an answer, the vote has already settled within a few dozen draws, the*modal ceiling*; for scoring a benchmark, sooner still, the*correlation ceiling*\. Beyond that, extra draws cost compute and add nothing, and can even make the answer worse\. This paper turns the cutoff into a single number, the*effective number of samples*, that any sampling run already reveals\. The bottleneck is recognizing a right answer, not generating one\.

Keywordstest\-time scaling⋅\\cdotinference\-time compute⋅\\cdotrepeated sampling⋅\\cdotpass@k⋅\\cdotcoverage⋅\\cdotbest\-of\-n⋅\\cdotself\-consistency⋅\\cdoteffective sample size⋅\\cdotdesign effect⋅\\cdotintraclass correlation⋅\\cdotcorrelation ceiling⋅\\cdotmodal ceiling⋅\\cdotidentifiability gap

A reasoning model can be made stronger at inference time by spending more compute, and the compute goes to one of two levers: longer reasoning, letting the model think more before it answers, or more sampling, drawingnnanswers to one prompt and combining them\[[1](https://arxiv.org/html/2606.28661#bib.bib1),[2](https://arxiv.org/html/2606.28661#bib.bib2),[3](https://arxiv.org/html/2606.28661#bib.bib3),[4](https://arxiv.org/html/2606.28661#bib.bib4)\]\. This is test\-time scaling, and it drives much of the progress in reasoning systems\[[5](https://arxiv.org/html/2606.28661#bib.bib5),[6](https://arxiv.org/html/2606.28661#bib.bib6),[7](https://arxiv.org/html/2606.28661#bib.bib7),[8](https://arxiv.org/html/2606.28661#bib.bib8)\]; the sampling lever is the subject here\. Its headline curve is*coverage*, the fraction of problems on which at least one ofnnsamples is correct\[[9](https://arxiv.org/html/2606.28661#bib.bib9)\], and it climbs over several orders of magnitude ofnn\[[2](https://arxiv.org/html/2606.28661#bib.bib2)\], reading as steadily rising capability\. That reading is too generous\. A deployed system must return a single answer, and with nothing to certify which sample is right it can only*select*one, by frequency or a learned score; selection is capped, and more sampling does not lift it and can even lower it\. A correct answer thus grows ever easier to reach but no easier to return: the bottleneck in test\-time scaling is not generating a right answer, it is recognizing one\. And because each draw costs compute, the question that matters is how far to sample, which the ceilings below answer with a small, goal\-dependent budget \(Figure[1](https://arxiv.org/html/2606.28661#S1.F1)\)\.

![Refer to caption](https://arxiv.org/html/2606.28661v1/x1.png)Figure 1:The two ceilings of test\-time sampling\.\(a\)Coverage, self\-consistency, and the identifiability gap between them\.\(b\)The effective number of samplesneffn\_\{\\mathrm\{eff\}\}against the nominal countnn, with the correlation ceiling1/ρ1/\\rho\.The split\.Coverage and selection ask different questions of the same samples, and only one of them keeps improving\. Coverage asks whether*any*sample is correct; a verifier that could spot the correct one would let coverage ride past every limit toward what the model can ever reach\. Selection must commit to one answer with no such verifier, and a plurality vote converges to the model’s most common answer; on the problems where that answer is wrong, more samples only make the wrong answer win more surely, so selection saturates at the fraction of problems whose most common answer is correct, and can even decline as the budget grows\.Brown et al\. \[[2](https://arxiv.org/html/2606.28661#bib.bib2)\]report exactly this pattern without a mechanism, that “majority voting and reward models plateau beyond several hundred samples and fail to fully scale with the sample budget\.” The distance between rising coverage and stalled selection is the set of problems the model can solve but cannot pick out, the*identifiability gap*: it can generate the right answer without being able to select it \(Figure[2](https://arxiv.org/html/2606.28661#S1.F2)\)\.

![Refer to caption](https://arxiv.org/html/2606.28661v1/x2.png)Figure 2:Generated but not selected\. On a hard problem the correct answer appears among the sampled attempts, so coverage finds it; but it is not the most common answer, so the plurality vote returns a confident wrong one\.A second, separate ceiling\.Even the benchmark accuracy that sampling is meant to estimate is worth less than the sample count suggests, for a different reason\. Thennattempts to a problem are notnnindependent tries; they are a cluster drawn from one prompt, alike the way several people from one household are alike in a survey\. The classical correction is the*design effect*deff=1\+\(n−1\)​ρd\_\{\\mathrm\{eff\}\}=1\+\(n\-1\)\\rhoofKish \[[10](https://arxiv.org/html/2606.28661#bib.bib10)\], whereρ\\rhois the intraclass correlation among the attempts, so thenncorrelated draws are worth onlyn/deffn/d\_\{\\mathrm\{eff\}\}independent ones\[[11](https://arxiv.org/html/2606.28661#bib.bib11)\]\. Read for test\-time sampling, this is the*effective number of samples*

neff=n1\+\(n−1\)​ρ,neff⟶1ρas​n→∞,n\_\{\\mathrm\{eff\}\}=\\frac\{n\}\{1\+\(n\-1\)\\rho\},\\qquad n\_\{\\mathrm\{eff\}\}\\;\\longrightarrow\\;\\frac\{1\}\{\\rho\}\\quad\\text\{as \}n\\to\\infty,\(1\)which saturates at a hard*correlation ceiling*1/ρ1/\\rho: no budget makesnncorrelated attempts worth more than1/ρ1/\\rhoindependent ones \(Section[2\.1](https://arxiv.org/html/2606.28661#S2.SS1)\)\. This ceiling governs estimation, not selection; the two are kept apart throughout\. The identifiability gap is measured in Section[4\.3](https://arxiv.org/html/2606.28661#S4.SS3)\.

What is and is not new\.The design effect and effective sample size are not new here; they areKish \[[10](https://arxiv.org/html/2606.28661#bib.bib10)\]andCochran \[[11](https://arxiv.org/html/2606.28661#bib.bib11)\], and the correction of Condorcet’s theorem for correlated voters is older still\[[12](https://arxiv.org/html/2606.28661#bib.bib12),[13](https://arxiv.org/html/2606.28661#bib.bib13)\]\. Three recent works apply the same instrument to language\-model outputs, but to different objects:Kohli \[[14](https://arxiv.org/html/2606.28661#bib.bib14)\]measure the effective votes of a panel of*distinct judge models*in evaluation,Goel et al\. \[[15](https://arxiv.org/html/2606.28661#bib.bib15)\]quantify error correlation*across models*, andNitarach \[[16](https://arxiv.org/html/2606.28661#bib.bib16)\], in a competition report, track the effective sample size of majority voting across mixed models\. None treats single\-model test\-time scaling as cluster sampling, separates the estimation ceiling from a distinct selection ceiling, derives the modal\-answer ceiling and its anti\-scaling, or connects to the survey\-sampling literature\. On the coverage side,Schaeffer et al\. \[[17](https://arxiv.org/html/2606.28661#bib.bib17)\]andKazdan et al\. \[[18](https://arxiv.org/html/2606.28661#bib.bib18)\]explain the power\-law shape of coverage by a heavy\-tailed distribution of*per\-problem*difficulty, andLevi \[[19](https://arxiv.org/html/2606.28661#bib.bib19)\]reaches a power law through a memorization ansatz; all assume attempts are conditionally independent given the problem, the within\-problemρw=0\\rho\_\{w\}=0case of the model below, from which the analysis here recovers a power law of the same form\. The contribution here is the lens that separates the three quantities a nominal sample count confounds: a between\-problem difficulty spreadρb\\rho\_\{b\}that caps benchmark estimation and shapes coverage, a within\-problem dependence measured to be near zero, and the concentration of the answer distribution that caps selection at a modal\-hit rateπmode\\pi\_\{\\mathrm\{mode\}\}and makes it anti\-scale, all grounded on released logs\.

This work is a citable derivation, not a new algorithm\. Section[2](https://arxiv.org/html/2606.28661#S2)sets up test\-time sampling as cluster sampling and fences the scope of the exact claims\. Section[3](https://arxiv.org/html/2606.28661#S3)derives the effective number of samples, the correlation ceiling, and the marginal value of a sample\. Section[4](https://arxiv.org/html/2606.28661#S4)separates coverage from selection, explains the identifiability gap between them, and measures it on both an independent\-draw log\[[2](https://arxiv.org/html/2606.28661#bib.bib2)\]and a dependent\-draw log\[[20](https://arxiv.org/html/2606.28661#bib.bib20)\]\. Section[5](https://arxiv.org/html/2606.28661#S5)decomposes within\- and between\-problem correlation, gives the compute\-allocation rule, an estimator forρ\\rho, and a summary table\.

## 2 Test\-time sampling is cluster sampling

The entire argument rests on one reframing: a problem and its repeated attempts form a cluster, not a fresh draw each time\. This section makes the correspondence precise and fences exactly where the claims that follow are exact\.

Fix a promptqqand drawnnresponseso1,…,ono\_\{1\},\\dots,o\_\{n\}from one model at a fixed decoding configuration \(the same sampling settings each time\)\. A verifier \(an automatic checker that marks each answer right or wrong\) scores each, giving binary success indicators

Yi=𝟏​\{oi​is correct for​q\}∈\{0,1\},i=1,…,n\.Y\_\{i\}=\\mathbf\{1\}\\\{\\,o\_\{i\}\\text\{ is correct for \}q\\,\\\}\\in\\\{0,1\\\},\\qquad i=1,\\dots,n\.\(2\)Writes=ℙ​\[Yi=1\]s=\\mathbb\{P\}\[Y\_\{i\}=1\]for the per\-attempt success probability andK=∑i=1nYiK=\\sum\_\{i=1\}^\{n\}Y\_\{i\}for the number correct\. The three headline quantities of test\-time scaling are functions of\(Y1,…,Yn\)\(Y\_\{1\},\\dots,Y\_\{n\}\):

pass​@​n=ℙ​\[K≥1\]​⏟coverage,p^=K/n​⏟success fraction,𝟏​\{K\>n/2\}​⏟majority vote\.\\underbrace\{\\mathrm\{pass\}@n=\\mathbb\{P\}\[K\\geq 1\]\\rule\[\-5\.0pt\]\{0\.0pt\}\{0\.0pt\}\}\_\{\\text\{coverage\}\},\\qquad\\underbrace\{\\hat\{p\}=K/n\\rule\[\-5\.0pt\]\{0\.0pt\}\{0\.0pt\}\}\_\{\\text\{success fraction\}\},\\qquad\\underbrace\{\\mathbf\{1\}\\\{K\>n/2\\\}\\rule\[\-5\.0pt\]\{0\.0pt\}\{0\.0pt\}\}\_\{\\text\{majority vote\}\}\.\(3\)Best\-of\-nn\(drawingnnanswers and returning the one a learned reward model scores highest\)\[[21](https://arxiv.org/html/2606.28661#bib.bib21)\], weighted voting, and self\-consistency \(returning the answer the samples agree on most often\) are all read off the sampled distribution and so are functions of the same draws\.

The independence baseline assumesY1,…,YnY\_\{1\},\\dots,Y\_\{n\}are i\.i\.d\. \(independent and identically distributed\) Bernoulli\(s\)\(s\)draws, each a coin that lands correct with probabilityss\. Thenpass​@​n=1−\(1−s\)n\\mathrm\{pass\}@n=1\-\(1\-s\)^\{n\},Var⁡\(p^\)=s​\(1−s\)/n\\operatorname\{Var\}\(\\hat\{p\}\)=s\(1\-s\)/n, and, fors\>12s\>\\tfrac\{1\}\{2\}, the majority vote is correct with probability tending to11asn→∞n\\to\\infty\[[9](https://arxiv.org/html/2606.28661#bib.bib9)\]\. The analysis keeps the marginalssand replaces independence with exchangeability\.

Exchangeable attempts\.The attempts to one problem are exchangeable: relabeling them does not change their joint distribution, so only how many succeed matters, not which ones\. By de Finetti’s theorem\[[22](https://arxiv.org/html/2606.28661#bib.bib22)\], an infinitely exchangeable binary sequence \(one a fresh session can in principle extend to arbitrarily many attempts\) is a mixture of i\.i\.d\. sequences,

Yi∣θ​∼i\.i\.d\.​Bernoulli​\(θ\),θ∼G​on​\[0,1\],Y\_\{i\}\\mid\\theta\\;\\overset\{\\text\{i\.i\.d\.\}\}\{\\sim\}\\;\\text\{Bernoulli\}\(\\theta\),\\qquad\\theta\\sim G\\text\{ on \}\[0,1\],\(4\)for some mixing distributionGG\(the spread of these hidden success rates across problems\) with mean𝔼​\[θ\]=s\\mathbb\{E\}\[\\theta\]=s\. In words, the model behaves as if each fresh session first draws a hidden success rateθ\\thetafor the problem, then every attempt in that session is an independentθ\\theta\-weighted coin: a good session lands in a strong reasoning basin \(largeθ\\theta\), a weak one in a poor basin \(smallθ\\theta\)\. The single number that summarizes the dependence is the*intraclass correlation*ρ\\rho, how strongly two attempts on the same problem move together, and the standard measure for clustered binary data,

ρ=Corr⁡\(Yi,Yj\)=Var⁡\(θ\)s​\(1−s\)∈\[0,1\],i≠j\.\\rho\\;=\\;\\operatorname\{Corr\}\(Y\_\{i\},Y\_\{j\}\)\\;=\\;\\frac\{\\operatorname\{Var\}\(\\theta\)\}\{s\(1\-s\)\}\\;\\in\\;\[0,1\],\\qquad i\\neq j\.\(5\)The representation forcesVar⁡\(θ\)≥0\\operatorname\{Var\}\(\\theta\)\\geq 0, soρ≥0\\rho\\geq 0: exchangeable attempts are non\-negatively correlated\. Independence isρ=0\\rho=0\(a degenerateGGatss\);ρ=1\\rho=1is total collapse, where a session is all correct or all wrong together\. Equation \([5](https://arxiv.org/html/2606.28661#S2.E5)\) is the bridge from the spread of the latent rateθ\\thetato the intraclass correlation that drives the design effect\. Thisρ\\rhois a correlation of*correctness*; the concentration of the answer*strings*, which governs selection, is a separate quantity, introduced in Section[4\.2](https://arxiv.org/html/2606.28661#S4.SS2)\. Figure[3](https://arxiv.org/html/2606.28661#S2.F3)states the correspondence\.

sampling frameclusterwithin\-cluster drawscorrectionpopulationhouseholdmmmembersdeff=1\+\(m−1\)​ρd\_\{\\mathrm\{eff\}\}=1\+\(m\{\-\}1\)\\rhom↦m/deffm\\mapsto m/d\_\{\\mathrm\{eff\}\}benchmarkproblemqqnnattemptsoio\_\{i\}deff=1\+\(n−1\)​ρd\_\{\\mathrm\{eff\}\}=1\+\(n\{\-\}1\)\\rhon↦neffn\\mapsto n\_\{\\mathrm\{eff\}\}surveytest\-time

Figure 3:The survey\-to\-test\-time correspondence\.*Top:*a survey, where a household ofmmmembers maps throughdeff=1\+\(m−1\)​ρd\_\{\\mathrm\{eff\}\}=1\+\(m\-1\)\\rho\.*Bottom:*test\-time sampling, where a problem’snnattempts map throughdeff=1\+\(n−1\)​ρd\_\{\\mathrm\{eff\}\}=1\+\(n\-1\)\\rhotoneffn\_\{\\mathrm\{eff\}\}\.### 2\.1 Scope of the exact claims

A canonical claim is only as strong as the assumptions it names, and the main one here is weaker than it looks\. The load\-bearing assumption is*exchangeability*: the attempts to a problem are taken order\-free, which gives them a common success ratessand a common pairwise correlationρ\\rhoand makesdeff=1\+\(n−1\)​ρd\_\{\\mathrm\{eff\}\}=1\+\(n\-1\)\\rhoand the ceiling1/ρ1/\\rhoexact\. Little of this rests on the correlation being the*same*for every pair\. The variance identity of Proposition[1](https://arxiv.org/html/2606.28661#Thmproposition1)needs only the common ratess: for any pattern of pairwise correlations it holds withρ\\rhoread as the*mean*pairwise correlationρ¯=1n​\(n−1\)​∑i≠jCorr⁡\(Yi,Yj\)\\bar\{\\rho\}=\\tfrac\{1\}\{n\(n\-1\)\}\\sum\_\{i\\neq j\}\\operatorname\{Corr\}\(Y\_\{i\},Y\_\{j\}\), sodeff=1\+\(n−1\)​ρ¯d\_\{\\mathrm\{eff\}\}=1\+\(n\-1\)\\bar\{\\rho\}and the ceiling is1/ρ¯1/\\bar\{\\rho\}\. Real sampling departs from equicorrelation \(attempts that share a long prefix are more alike than attempts that branch early\); then1/ρ1/\\rhois simply read as1/ρ¯1/\\bar\{\\rho\}, the average the estimator of Section[5\.3](https://arxiv.org/html/2606.28661#S5.SS3)already returns\. What the ceiling truly requires is only that the correlation be positive and not vanish, that is, that sampling diversity stay bounded: the de Finetti mixture forcesVar⁡\(θ\)≥0\\operatorname\{Var\}\(\\theta\)\\geq 0, henceρ≥0\\rho\\geq 0, and infinite exchangeability, the one further assumption, carries everyn→∞n\\to\\inftylimit below\. Equicorrelation is thus a convenience that turns an average into a single number, not a crutch the ceiling stands on\.

Under these assumptions the design effect in Section[3](https://arxiv.org/html/2606.28661#S3)is*exact*for one estimand: the success fractionp^=K/n\\hat\{p\}=K/n, which depends on\(Yi\)\(Y\_\{i\}\)only through their sum and so estimates the per\-problem ratessor, pooled, the benchmark mean\. It governs how precisely a sampling budget pins down accuracy, the estimation use of test\-time sampling\. Two other quantities read the same draws but are not this mean and are treated on their own terms\. Coverage,pass​@​n=ℙ​\[K≥1\]\\mathrm\{pass\}@n=\\mathbb\{P\}\[K\\geq 1\], depends on the full mixing distributionGG, and Section[4\.1](https://arxiv.org/html/2606.28661#S4.SS1)treats it directly through Equation \([4](https://arxiv.org/html/2606.28661#S2.E4)\)\. Selection \(self\-consistency, best\-of\-nn\) returns the most\-favored answer, the mode of the categorical answer distribution rather than a sum of theYiY\_\{i\}, and Section[4\.2](https://arxiv.org/html/2606.28661#S4.SS2)gives it its own ceiling\. The three are kept apart on purpose, because their different dependence on the same draws is the point here, and it is what lets one budget buy different amounts of each\. The verifier is assumed accurate; an imperfect verifier adds a second, independent ceiling on usable accuracy, one not folded intoρ\\rhohere\.

## 3 The effective number of samples

Everything downstream, the ceiling and the budget rule, follows from a single quantity: the variance of the number correct\. This section computes it and reads off the effective number of samples\.

###### Proposition 1\(Design effect of test\-time sampling\)\.

LetY1,…,YnY\_\{1\},\\dots,Y\_\{n\}be exchangeable withℙ​\[Yi=1\]=s\\mathbb\{P\}\[Y\_\{i\}=1\]=sandCorr⁡\(Yi,Yj\)=ρ\\operatorname\{Corr\}\(Y\_\{i\},Y\_\{j\}\)=\\rhofori≠ji\\neq j\. Then the countK=∑iYiK=\\sum\_\{i\}Y\_\{i\}has

𝔼​\[K\]=n​s,Var⁡\(K\)=n​s​\(1−s\)​\[1\+\(n−1\)​ρ\],\\mathbb\{E\}\[K\]=ns,\\qquad\\operatorname\{Var\}\(K\)=ns\(1\-s\)\\,\\big\[\\,1\+\(n\-1\)\\rho\\,\\big\],\(6\)and the success fractionp^=K/n\\hat\{p\}=K/nhasVar⁡\(p^\)=s​\(1−s\)/neff\\operatorname\{Var\}\(\\hat\{p\}\)=s\(1\-s\)/n\_\{\\mathrm\{eff\}\}with the effective number of samples

neff=n1\+\(n−1\)​ρ\.n\_\{\\mathrm\{eff\}\}=\\frac\{n\}\{1\+\(n\-1\)\\rho\}\.\(7\)

###### Proof\.

Linearity gives𝔼​\[K\]=n​s\\mathbb\{E\}\[K\]=ns\. For the variance,Var⁡\(K\)=∑iVar⁡\(Yi\)\+∑i≠jCov⁡\(Yi,Yj\)\\operatorname\{Var\}\(K\)=\\sum\_\{i\}\\operatorname\{Var\}\(Y\_\{i\}\)\+\\sum\_\{i\\neq j\}\\operatorname\{Cov\}\(Y\_\{i\},Y\_\{j\}\)\. EachVar⁡\(Yi\)=s​\(1−s\)\\operatorname\{Var\}\(Y\_\{i\}\)=s\(1\-s\)and each of then​\(n−1\)n\(n\-1\)covariances equalsρ​s​\(1−s\)\\rho\\,s\(1\-s\), soVar⁡\(K\)=n​s​\(1−s\)\+n​\(n−1\)​ρ​s​\(1−s\)=n​s​\(1−s\)​\[1\+\(n−1\)​ρ\]\\operatorname\{Var\}\(K\)=ns\(1\-s\)\+n\(n\-1\)\\rho\\,s\(1\-s\)=ns\(1\-s\)\[1\+\(n\-1\)\\rho\]\. Dividing byn2n^\{2\}givesVar⁡\(p^\)=s​\(1−s\)​\[1\+\(n−1\)​ρ\]/n=s​\(1−s\)/neff\\operatorname\{Var\}\(\\hat\{p\}\)=s\(1\-s\)\[1\+\(n\-1\)\\rho\]/n=s\(1\-s\)/n\_\{\\mathrm\{eff\}\}\. ∎

The bracketdeff=1\+\(n−1\)​ρd\_\{\\mathrm\{eff\}\}=1\+\(n\-1\)\\rhois the design effect: the factor by which correlation inflates the variance of the count relative tonnindependent draws\[[10](https://arxiv.org/html/2606.28661#bib.bib10),[11](https://arxiv.org/html/2606.28661#bib.bib11)\]\. Equivalently, then×nn\\times ncovariance of the attempts is a constant diagonal plus a single rank\-one term,Σ=s​\(1−s\)​\[\(1−ρ\)​𝐈\+ρ​11⊤\]\\Sigma=s\(1\-s\)\\big\[\(1\-\\rho\)\\mathbf\{I\}\+\\rho\\,\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}\\big\], so the variance of the count collapses to the scalardeffd\_\{\\mathrm\{eff\}\}without ever forming that matrix: the same structured\-operator economy that lets large numerical systems be solved matrix\-free\[[23](https://arxiv.org/html/2606.28661#bib.bib23)\]\. The estimatorp^\\hat\{p\}behaves as if it were built fromneffn\_\{\\mathrm\{eff\}\}independent samples; Appendix[B](https://arxiv.org/html/2606.28661#A2)confirms the variance identity \([6](https://arxiv.org/html/2606.28661#S3.E6)\) against Monte Carlo simulation\.

### 3\.1 The correlation ceiling

This is the result this work is named for, and its single most important fact: the effective number of samples does not grow without bound\. Equation \([7](https://arxiv.org/html/2606.28661#S3.E7)\) has a finite limit\.

###### Corollary 1\(Correlation ceiling\)\.

Ifρ\>0\\rho\>0, thenneffn\_\{\\mathrm\{eff\}\}increases innnto the finite limit

limn→∞neff=1ρ,\\lim\_\{n\\to\\infty\}n\_\{\\mathrm\{eff\}\}=\\frac\{1\}\{\\rho\},\(8\)andneffn\_\{\\mathrm\{eff\}\}reaches half of this ceiling atn=\(1−ρ\)/ρ≈1/ρn=\(1\-\\rho\)/\\rho\\approx 1/\\rho\.

###### Proof\.

Writeneff=n/\[1−ρ\+n​ρ\]→1/ρn\_\{\\mathrm\{eff\}\}=n/\[\\,1\-\\rho\+n\\rho\\,\]\\to 1/\\rhoasn→∞n\\to\\infty\. Settingneff=1/\(2​ρ\)n\_\{\\mathrm\{eff\}\}=1/\(2\\rho\)gives2​ρ​n=1\+\(n−1\)​ρ2\\rho n=1\+\(n\-1\)\\rho, i\.e\.ρ​n=1−ρ\\rho n=1\-\\rho, son=\(1−ρ\)/ρn=\(1\-\\rho\)/\\rho\. ∎

This is the central fact\. A correlated cluster with intraclass correlationρ\\rhois worth at most1/ρ1/\\rhoindependent draws, however large the budget \(exactly under the equicorrelation of Section[2\.1](https://arxiv.org/html/2606.28661#S2.SS1), and as a limiting average otherwise\); and it gets halfway there byn≈1/ρn\\approx 1/\\rho\. Figure[4](https://arxiv.org/html/2606.28661#S3.F4)plots the ceiling\. A model withρ=0\.1\\rho=0\.1caps out near ten effective samples: the thousandth draw is almost worthless\. The ceiling is also why the marginal value of a sample collapses\.

![Refer to caption](https://arxiv.org/html/2606.28661v1/x3.png)Figure 4:The correlation ceiling\. The effective number of samplesneff=n/\[1\+\(n−1\)​ρ\]n\_\{\\mathrm\{eff\}\}=n/\[1\+\(n\-1\)\\rho\]against the number drawn, for four correlation levels, each saturating at its ceiling1/ρ1/\\rho\(dashed\); the independent idealneff=nn\_\{\\mathrm\{eff\}\}=nis the diagonal\.###### Corollary 2\(Value of thenn\-th sample\)\.

The marginal effective sample contributed by thenn\-th draw is

d​neffd​n=1−ρ\[1\+\(n−1\)​ρ\]2∼1−ρρ2​n2\(n→∞\),\\frac\{d\\,n\_\{\\mathrm\{eff\}\}\}\{dn\}=\\frac\{1\-\\rho\}\{\\big\[\\,1\+\(n\-1\)\\rho\\,\\big\]^\{2\}\}\\;\\sim\\;\\frac\{1\-\\rho\}\{\\rho^\{2\}\\,n^\{2\}\}\\quad\(n\\to\\infty\),\(9\)so the worth of an added sample decays quadratically and is negligible oncen≫1/ρn\\gg 1/\\rho\.

The marginal value starts atd​neff/d​n=1−ρd\\,n\_\{\\mathrm\{eff\}\}/dn=1\-\\rhoatn=1n=1\(the second draw already adds only\(1−ρ\)/\(1\+ρ\)\(1\-\\rho\)/\(1\+\\rho\)effective samples\) and then falls off as1/\(ρ​n\)21/\(\\rho n\)^\{2\}\. Figure[5](https://arxiv.org/html/2606.28661#S3.F5)shows the collapse and marks the break\-even pointn≈1/ρn\\approx 1/\\rhoat which the curve has already given up most of its value\. This is a budget rule, derived in Section[5\.2](https://arxiv.org/html/2606.28661#S5.SS2): spending past1/ρ1/\\rhosamples buys redundancy, not signal\.

A worked reading makes the decay concrete\. Atρ=0\.1\\rho=0\.1the first draw is worth a full independent sample; by Corollary[2](https://arxiv.org/html/2606.28661#Thmcorollary2)the tenth is worth\(1−ρ\)/\[1\+9​ρ\]2=0\.9/1\.92≈0\.25\(1\-\\rho\)/\[1\+9\\rho\]^\{2\}=0\.9/1\.9^\{2\}\\approx 0\.25, and the hundredth only0\.9/10\.92≈0\.0080\.9/10\.9^\{2\}\\approx 0\.008\. A thousand\-sample run therefore carries the estimation information of roughly its first ten draws, and the rest refine the estimate by almost nothing\. The decay is quadratic, so each tenfold increase in budget past the ceiling returns about a hundredth as much, which is why the curves of Figure[5](https://arxiv.org/html/2606.28661#S3.F5)fall to the axis so quickly\.

![Refer to caption](https://arxiv.org/html/2606.28661v1/x4.png)Figure 5:The value of thenn\-th sample,d​neff/d​nd\\,n\_\{\\mathrm\{eff\}\}/dnfrom Corollary[2](https://arxiv.org/html/2606.28661#Thmcorollary2), against the sample index\. Break\-even pointsn≈1/ρn\\approx 1/\\rhomarked dotted \(one perρ\\rho\)\.

## 4 Coverage rises, selection saturates

The most\-cited puzzle in test\-time scaling is that one curve keeps rising while another stalls, and no single mechanism connects them\. This section gives the mechanism: coverage and selection read the same samples for different things, so they obey different ceilings\. Coverage is capped only by what the model could ever produce; selection by whether its most common answer is correct\. The gap between the two is exactly the solvable problems whose correct answer the vote fails to return\.

Why selection matters at all bears stating, since coverage looks like the whole story\. Coverage can be cashed only through a verifier that certifies a correct answer, and a perfect, general verifier is the rare exception rather than the rule\. A few domains supply one: code against a test suite, a proof against a proof assistant, a numeric answer against a key\. Most do not, and the verifiers that do exist score a proxy rather than truth, a finite test suite or a learned reward model that a wrong answer can still pass\. Wherever no sound verifier is available, which is the typical deployment, a single answer can be returned only by selection, so the modal ceiling of Section[4\.2](https://arxiv.org/html/2606.28661#S4.SS2), not coverage, fixes the accuracy a system delivers\. Coverage measures what the model could reach given a verifier it will not have; selection measures what ships\.

### 4\.1 Coverage and the correlation tax

Coverage is the optimistic half of the split, the curve that keeps climbing; this subsection prices what correlation quietly subtracts from it\. Coverage asks only whether some sample is correct, with no need to know which one\. Under the mixture \([4](https://arxiv.org/html/2606.28661#S2.E4)\),

pass​@​n=1−ℙ​\[K=0\]=1−𝔼θ​\[\(1−θ\)n\]\.\\mathrm\{pass\}@n=1\-\\mathbb\{P\}\[K=0\]=1\-\\mathbb\{E\}\_\{\\theta\}\\big\[\(1\-\\theta\)^\{n\}\\big\]\.\(10\)Coverage can only rise as the budget grows, and outside a hard core of unreachable problems it rises all the way\.

###### Proposition 2\(Coverage rises without a within\-problem ceiling\)\.

Under the mixture \([4](https://arxiv.org/html/2606.28661#S2.E4)\),pass​@​n\\mathrm\{pass\}@nis non\-decreasing innn, strictly increasing wheneverGGplaces mass on\(0,1\)\(0,1\), and converges to1−π01\-\\pi\_\{0\}, whereπ0=G​\(\{0\}\)\\pi\_\{0\}=G\(\\\{0\\\}\)is the fraction of problems the model never solves\. For a fixed problem with reachabilitypq​\(cq\)=π\>0p\_\{q\}\(c\_\{q\}\)=\\pi\>0, coverage1−\(1−π\)n1\-\(1\-\\pi\)^\{n\}strictly increases to11\.

###### Proof\.

For eachθ∈\[0,1\]\\theta\\in\[0,1\]the mapn↦\(1−θ\)nn\\mapsto\(1\-\\theta\)^\{n\}is non\-increasing, strictly forθ∈\(0,1\)\\theta\\in\(0,1\)\. Taking the mixture expectation,ℙ​\[K=0\]=𝔼θ​\[\(1−θ\)n\]\\mathbb\{P\}\[K=0\]=\\mathbb\{E\}\_\{\\theta\}\[\(1\-\\theta\)^\{n\}\]is non\-increasing innn, strictly ifGGcharges\(0,1\)\(0,1\), sopass​@​n=1−ℙ​\[K=0\]\\mathrm\{pass\}@n=1\-\\mathbb\{P\}\[K=0\]is non\-decreasing, strictly so\. Since\(1−θ\)n→𝟏​\{θ=0\}\(1\-\\theta\)^\{n\}\\to\\mathbf\{1\}\\\{\\theta=0\\\}pointwise, dominated convergence givesℙ​\[K=0\]→π0\\mathbb\{P\}\[K=0\]\\to\\pi\_\{0\}, hencepass​@​n→1−π0\\mathrm\{pass\}@n\\to 1\-\\pi\_\{0\}\. The fixed\-problem case is the point massθ=π\\theta=\\pi\. ∎

Correlation always taxes coverage relative to the independent reading\.

###### Proposition 3\(Correlation tax on coverage\)\.

For exchangeable attempts with per\-attempt successss,

pass​@​n≤1−\(1−s\)n,\\mathrm\{pass\}@n\\;\\leq\\;1\-\(1\-s\)^\{n\},\(11\)with equality if and only ifρ=0\\rho=0\. The gap grows with the dispersion ofθ\\theta: a mean\-preserving spread of the difficulty distribution \(more very\-easy and very\-hard problems at the same average\) can only widen it\.

###### Proof\.

The mapθ↦\(1−θ\)n\\theta\\mapsto\(1\-\\theta\)^\{n\}is strictly convex on\[0,1\]\[0,1\]forn≥2n\\geq 2\. By Jensen’s inequality \(the average of a convex function is at least the function of the average\),𝔼θ​\[\(1−θ\)n\]≥\(1−𝔼​\[θ\]\)n=\(1−s\)n\\mathbb\{E\}\_\{\\theta\}\[\(1\-\\theta\)^\{n\}\]\\geq\(1\-\\mathbb\{E\}\[\\theta\]\)^\{n\}=\(1\-s\)^\{n\}, with equality iffθ\\thetais degenerate, i\.e\.ρ=0\\rho=0\. Negating gives Equation \([11](https://arxiv.org/html/2606.28661#S4.E11)\)\. ∎

So the textbook curve1−\(1−s\)n1\-\(1\-s\)^\{n\}is an upper bound, not a prediction: real coverage runs below it whenever samples are correlated\. The shape of the shortfall is governed by the lower tail ofGG, the problems the model rarely solves\.

###### Proposition 4\(Power\-law coverage from heterogeneity\)\.

Letθ∼Beta​\(α,β\)\\theta\\sim\\mathrm\{Beta\}\(\\alpha,\\beta\)\(a flexible family of success\-rate distributions on the unit interval, standing in for the spread of problem difficulty\), sos=α/\(α\+β\)s=\\alpha/\(\\alpha\+\\beta\)andρ=1/\(α\+β\+1\)\\rho=1/\(\\alpha\+\\beta\+1\)\. Then the miss rate is

ℙ​\[K=0\]=B​\(α,β\+n\)B​\(α,β\)∼Γ​\(α\+β\)Γ​\(β\)​n−α\(n→∞\),\\mathbb\{P\}\[K=0\]=\\frac\{B\(\\alpha,\\beta\+n\)\}\{B\(\\alpha,\\beta\)\}\\;\\sim\\;\\frac\{\\Gamma\(\\alpha\+\\beta\)\}\{\\Gamma\(\\beta\)\}\\,n^\{\-\\alpha\}\\qquad\(n\\to\\infty\),\(12\)so coverage approaches its limit as a power lawn−αn^\{\-\\alpha\}, not the exponential\(1−s\)n\(1\-s\)^\{n\}\.

###### Proof\.

Withθ∼Beta​\(α,β\)\\theta\\sim\\mathrm\{Beta\}\(\\alpha,\\beta\),𝔼​\[\(1−θ\)n\]=B​\(α,β\+n\)/B​\(α,β\)\\mathbb\{E\}\[\(1\-\\theta\)^\{n\}\]=B\(\\alpha,\\beta\+n\)/B\(\\alpha,\\beta\)is the standard beta\-binomial \(a binomial whose success rate is itself drawn from the Beta\) probability of zero successes\. Writing it asΓ​\(α\+β\)Γ​\(β\)⋅Γ​\(β\+n\)Γ​\(α\+β\+n\)\\frac\{\\Gamma\(\\alpha\+\\beta\)\}\{\\Gamma\(\\beta\)\}\\cdot\\frac\{\\Gamma\(\\beta\+n\)\}\{\\Gamma\(\\alpha\+\\beta\+n\)\}and usingΓ​\(β\+n\)/Γ​\(α\+β\+n\)∼n−α\\Gamma\(\\beta\+n\)/\\Gamma\(\\alpha\+\\beta\+n\)\\sim n^\{\-\\alpha\}\(Stirling\) gives the tail\. The intraclass correlation of a beta\-binomial isρ=1/\(α\+β\+1\)\\rho=1/\(\\alpha\+\\beta\+1\)\. ∎

Proposition[4](https://arxiv.org/html/2606.28661#Thmproposition4)is the difficulty\-heterogeneity story ofSchaeffer et al\. \[[17](https://arxiv.org/html/2606.28661#bib.bib17)\]andKazdan et al\. \[[18](https://arxiv.org/html/2606.28661#bib.bib18)\], recovered here as the coverage face of the same model for a Beta difficulty prior: a heavy lower tail \(smallα\\alpha\) yields the slowly rising, log\-linear coverage thatBrown et al\. \[[2](https://arxiv.org/html/2606.28661#bib.bib2)\]fit empirically, with the exponent set by the lower tail ofGG\. Figure[6](https://arxiv.org/html/2606.28661#S4.F6)contrasts the exponential and power\-law regimes\. The overdispersed binomial behind it \(a count with more spread than independent draws would give\) is classical\[[24](https://arxiv.org/html/2606.28661#bib.bib24)\]\. IfGGadditionally places an atomπ0\\pi\_\{0\}atθ=0\\theta=0, a hard core of attempts the model never makes correct, thenpass​@​n→1−π0<1\\mathrm\{pass\}@n\\to 1\-\\pi\_\{0\}<1and no budget crosses it; this is mode collapse in its starkest form\. The atom is a ceiling of model capability rather than of sampling: those problems lie outside the model’s reach, a generalization limit that no sampling budget repairs\[[25](https://arxiv.org/html/2606.28661#bib.bib25)\]\.

The practical force of the power law is how slowly it pays\. An exponential miss rate\(1−s\)n\(1\-s\)^\{n\}clears any target in a handful of samples, each one cutting the miss rate by a constant factor; a power\-law miss raten−αn^\{\-\\alpha\}does not\. Halving an exponential miss rate costs a fixed number of further samples; halving a power law costs a fixed*multiplicative*factor21/α2^\{1/\\alpha\}, so for a heavy lower tail \(smallα\\alpha\) each successive halving costs as much sampling as everything before it combined\. Coverage keeps rising, but with sharply diminishing speed: the slow, log\-linear climb that the released logs show over four orders of magnitude\. Figure[6](https://arxiv.org/html/2606.28661#S4.F6)plots both regimes on log–log axes, where the exponential falls off a cliff and the power law settles onto a straight line of slope−α\-\\alpha\.

![Refer to caption](https://arxiv.org/html/2606.28661v1/x5.png)Figure 6:Exponential versus power\-law coverage\. The miss rate1−pass​@​n1\-\\mathrm\{pass\}@non log–log axes: the independent exponential against the power lawn−αn^\{\-\\alpha\}of Proposition[4](https://arxiv.org/html/2606.28661#Thmproposition4)\. Heres=0\.5s=0\.5andρ∈\{0\.1,0\.3\}\\rho\\in\\\{0\.1,0\.3\\\}\.
### 4\.2 Selection and the modal answer

Selection is where the visible gains stop, and the cause is not correlation but the shape of the answer distribution\. To return an answer without a verifier, a method keeps the most common answer \(self\-consistency\[[1](https://arxiv.org/html/2606.28661#bib.bib1)\]\) or the highest\-scored one \(reward\-model best\-of\-nn\); either way it reads the whole distribution over answer strings, not just the per\-attempt correctness rate\. Fix a problemqqand let the model induce a distributionpq​\(a\)p\_\{q\}\(a\)over candidate answersaa, with correct answercqc\_\{q\}and a unique most common answer, the*mode*aq⋆=arg⁡maxa⁡pq​\(a\)a^\{\\star\}\_\{q\}=\\arg\\max\_\{a\}p\_\{q\}\(a\)\. A plurality vote amongnnattempts returns the empirical mode\.

###### Proposition 5\(Selection ceiling\)\.

Asn→∞n\\to\\inftythe plurality answer converges almost surely to the modeaq⋆a^\{\\star\}\_\{q\}, so self\-consistency accuracy on a fixed problem tends to𝟏​\{aq⋆=cq\}\\mathbf\{1\}\\\{a^\{\\star\}\_\{q\}=c\_\{q\}\\\}\. Averaged over a benchmark, self\-consistency accuracy converges to the*modal\-hit rate*

πmode=ℙq​\[aq⋆=cq\],\\pi\_\{\\mathrm\{mode\}\}\\;=\\;\\mathbb\{P\}\_\{q\}\\big\[\\,a^\{\\star\}\_\{q\}=c\_\{q\}\\,\\big\],\(13\)a hard ceiling, the*modal ceiling*, with no dependence on the sample budgetnn\.

###### Proof\.

For a fixed problem the empirical answer frequencies converge topqp\_\{q\}almost surely \(the law of large numbers applied to the multinomial counts\), andarg⁡max\\arg\\maxis continuous wherever the maximizer is unique, so the empirical mode converges toaq⋆a^\{\\star\}\_\{q\}almost surely\. Hence𝟏​\{plurality correct\}→𝟏​\{aq⋆=cq\}\\mathbf\{1\}\\\{\\text\{plurality correct\}\\\}\\to\\mathbf\{1\}\\\{a^\{\\star\}\_\{q\}=c\_\{q\}\\\}, and averaging over problems gives the limit \([13](https://arxiv.org/html/2606.28661#S4.E13)\), which is constant innn\. ∎

The ceiling is a wall, not a slowdown: once the budget reveals the mode, more samples cannot move the vote, and on the problems where the mode is wrong they move it the wrong way\.

###### Corollary 3\(Anti\-scaling of selection\)\.

On any problem whose mode is incorrect \(aq⋆≠cqa^\{\\star\}\_\{q\}\\neq c\_\{q\}\) yet whose correct answer is reachable \(pq​\(cq\)\>0p\_\{q\}\(c\_\{q\}\)\>0\), self\-consistency accuracy falls to0while coverage rises to11asn→∞n\\to\\infty: more sampling makes selection worse and coverage better on the same problem\.

###### Proof\.

By Proposition[5](https://arxiv.org/html/2606.28661#Thmproposition5)the plurality converges toaq⋆≠cqa^\{\\star\}\_\{q\}\\neq c\_\{q\}, so𝟏​\{plurality correct\}→0\\mathbf\{1\}\\\{\\text\{plurality correct\}\\\}\\to 0, whilepass​@​n=1−\(1−pq​\(cq\)\)n→1\\mathrm\{pass\}@n=1\-\(1\-p\_\{q\}\(c\_\{q\}\)\)^\{n\}\\to 1becausepq​\(cq\)\>0p\_\{q\}\(c\_\{q\}\)\>0\. ∎

Anti\-scaling is the mechanism behind the limited and sometimes non\-monotone returns that self\-consistency and reward\-model selection show in practice\[[26](https://arxiv.org/html/2606.28661#bib.bib26),[27](https://arxiv.org/html/2606.28661#bib.bib27)\]: extra samples sharpen a confident wrong answer\. How fast a problem reaches its ceiling is set by the concentration ofpqp\_\{q\}, summarized by the*effective number of answers*1/∑apq​\(a\)21/\\sum\_\{a\}p\_\{q\}\(a\)^\{2\}; the ceiling*height*πmode\\pi\_\{\\mathrm\{mode\}\}is set by whether the mode it converges to happens to be correct\. Figure[7](https://arxiv.org/html/2606.28661#S4.F7)shows both outcomes: coverage rises to one whether or not the mode is correct, while selection rises to one only when the mode is correct and otherwise falls to zero\. Self\-consistency and plurality voting meet this modal ceiling exactly\. Best\-of\-nnwith a learned reward model is bounded instead by how well that scorer ranks answers: a perfect verifier lifts it to coverage, a frequency\-like scorer collapses it to the modal ceiling, and real reward models fall between\.

![Refer to caption](https://arxiv.org/html/2606.28661v1/x6.png)Figure 7:Anti\-scaling of selection\. Two problems sampled tonnattempts each: coverage \(any attempt correct\) and self\-consistency \(the plurality answer\) againstnn\. Left: a problem whose most common answer is correct\. Right: a problem whose most common answer is wrong\.The familiar majority\-vote reading is the binary special case, and it is a strict lower bound\. Collapse the answers to two, the correct one against a single modal error, and letθ\\thetabe the latent per\-attempt correctness of Equation \([4](https://arxiv.org/html/2606.28661#S2.E4)\); then plurality is majority,K/n→θK/n\\to\\thetaalmost surely, and the vote is correct in the limit exactly whenθ\>12\\theta\>\\tfrac\{1\}\{2\}\.

###### Corollary 4\(Condorcet jury, correlated\)\.

If the difficulty distribution places positive mass on problems the model gets wrong more often than right \(ℙ​\[θ<12\]\>0\\mathbb\{P\}\[\\theta<\\tfrac\{1\}\{2\}\]\>0\), majority\-vote accuracy converges toℙ​\[θ\>12\]<1\\mathbb\{P\}\[\\theta\>\\tfrac\{1\}\{2\}\]<1, below the value11that independent jurors reach whens\>12s\>\\tfrac\{1\}\{2\}\[[12](https://arxiv.org/html/2606.28661#bib.bib12),[13](https://arxiv.org/html/2606.28661#bib.bib13)\]; the normal approximationθ≈𝒩​\(s,ρ​s​\(1−s\)\)\\theta\\approx\\mathcal\{N\}\(s,\\rho\\,s\(1\-s\)\)givesℙ​\[θ\>12\]≈Φ​\(\(s−12\)/ρ​s​\(1−s\)\)\\mathbb\{P\}\[\\theta\>\\tfrac\{1\}\{2\}\]\\approx\\Phi\\big\(\(s\-\\tfrac\{1\}\{2\}\)/\\sqrt\{\\rho\\,s\(1\-s\)\}\\big\)\. This never exceeds the plurality ceiling,ℙ​\[θ\>12\]≤πmode\\mathbb\{P\}\[\\theta\>\\tfrac\{1\}\{2\}\]\\leq\\pi\_\{\\mathrm\{mode\}\}, becauseθ\>12\\theta\>\\tfrac\{1\}\{2\}forces every wrong answer below12\\tfrac\{1\}\{2\}and so below the correct one\.

The two bounds part company on real logs, and the distance between them is the point\. On the dependent\-draw log of Section[4\.3](https://arxiv.org/html/2606.28661#S4.SS3)the majority bound readsℙ​\[θ\>12\]=0\.20\\mathbb\{P\}\[\\theta\>\\tfrac\{1\}\{2\}\]=0\.20while the plurality ceiling isπmode=0\.45\\pi\_\{\\mathrm\{mode\}\}=0\.45: more than half of the selectable accuracy comes from problems the model answers correctly less than half the time, where the correct answer is still the single most common one because the errors scatter\. Figure[8](https://arxiv.org/html/2606.28661#S4.F8)shows the binary plateau and that simulated accuracy lands on it\.

![Refer to caption](https://arxiv.org/html/2606.28661v1/x7.png)Figure 8:The binary \(majority\-vote\) special case of the selection ceiling\. Majority\-vote accuracy againstnnfor severalρ\\rho: the independent case \(ρ→0\\rho\\to 0, top\) and correlated cases plateauing atℙ​\[θ\>12\]\\mathbb\{P\}\[\\theta\>\\tfrac\{1\}\{2\}\]\(Corollary[4](https://arxiv.org/html/2606.28661#Thmcorollary4)\), with its normal approximation dashed\. Markers: simulation;s=0\.65s=0\.65\.
### 4\.3 The coverage–selection gap, measured

Theory has predicted a gap; this is where it is measured on real sampling logs\. Put the two together\. Coverage \(Proposition[3](https://arxiv.org/html/2606.28661#Thmproposition3)\) climbs toward what the model can reach,1−π01\-\\pi\_\{0\}, slowly\. Selection \(Proposition[5](https://arxiv.org/html/2606.28661#Thmproposition5)\) saturates at the modal\-hit rateπmode\\pi\_\{\\mathrm\{mode\}\}\. The two diverge, and the divergence is the identifiability gap: the problems whose correct answer is in the pool but is not its most common answer\. The split shows up in two released logs that probe the two effects separately: an independent\-draw log for the between\-problem difficulty spread, and a dependent\-draw log for the within\-problem answer collapse\.

The between\-problem term, on independent draws\.Brown et al\. \[[2](https://arxiv.org/html/2606.28661#bib.bib2)\]sampled up to10410^\{4\}solutions per problem on GSM8K\[[21](https://arxiv.org/html/2606.28661#bib.bib21)\]\(grade\-school math word problems\) and MATH\[[28](https://arxiv.org/html/2606.28661#bib.bib28)\]\(harder competition mathematics\) and recorded the correctness of each\. Their attempts are drawn independently, so the within\-problem correlation is zero by construction; what the logs expose is the*between\-problem*difficulty spreadρb\\rho\_\{b\}and its consequences: theρw=0\\rho\_\{w\}=0face of the model\. The analysis here estimates, per configuration, the difficulty meanss, the intraclass correlationρ^=Var⁡\(θ\)/\[s​\(1−s\)\]\\hat\{\\rho\}=\\operatorname\{Var\}\(\\theta\)/\[s\(1\-s\)\], coverage by the unbiased estimator ofChen et al\. \[[9](https://arxiv.org/html/2606.28661#bib.bib9)\], and self\-consistency by plurality vote of the extracted answers \(validated against the logs’ own correctness labels\)\. The self\-consistency plateau is reported only where the numeric extractor reproduces those labels; MATH answers are boxed expressions a numeric extractor cannot parse, so MATH shows coverage andρ^\\hat\{\\rho\}alone\. Table[1](https://arxiv.org/html/2606.28661#S4.T1)reports them\.

Table 1:The between\-problem difficulty correlationρ^b\\hat\{\\rho\}\_\{b\}and the coverage–selection gap, from the released logs ofBrown et al\. \[[2](https://arxiv.org/html/2606.28661#bib.bib2)\]\(10410^\{4\}samples per problem\)\. Theρ^b\\hat\{\\rho\}\_\{b\}95%95\\%CI is a problem\-level clustered bootstrap \(10410^\{4\}resamples\)\. Self\-consistency is n/a on MATH \(boxed answers a numeric extractor cannot parse\)\.Two readings stand out\. First, the difficulty correlation is large and stable,ρ^≈0\.4\\hat\{\\rho\}\\approx 0\.4–0\.60\.6across models and benchmarks, so by Equation \([7](https://arxiv.org/html/2606.28661#S3.E7)\) the10410^\{4\}samples drawn for each problem carry the benchmark\-mean information of onlyneff≈2n\_\{\\mathrm\{eff\}\}\\approx 2independent samples: for estimating benchmark accuracy, ten thousand samples of one problem are worth about two independent ones\. Second, coverage and selection diverge in the direction the split predicts\. Figure[9](https://arxiv.org/html/2606.28661#S4.F9)shows the GSM8K, Llama\-3\-8B\-Instruct curves: coverage reaches1\.001\.00while self\-consistency plateaus at0\.870\.87\. For roughly an eighth of problems the correct answer is somewhere in the pool but the vote does not return it even at the plateau: the usable\-signal gap, on real data\.

![Refer to caption](https://arxiv.org/html/2606.28661v1/x8.png)Figure 9:The coverage–selection gap on real logs\[[2](https://arxiv.org/html/2606.28661#bib.bib2)\]: GSM8K, Llama\-3\-8B\-Instruct, up to10410^\{4\}samples per problem\. Coverage rises to1\.001\.00; self\-consistency plateaus at0\.870\.87; the shaded band is the usable\-signal gap\.The reading is consistent across the repeated\-sampling literature\.Chen et al\. \[[26](https://arxiv.org/html/2606.28661#bib.bib26)\]find voting performance non\-monotone in the number of calls because a benchmark mixes easy and hard problems, the anti\-scaling of Corollary[3](https://arxiv.org/html/2606.28661#Thmcorollary3);Wang et al\. \[[27](https://arxiv.org/html/2606.28661#bib.bib27)\]show that prompt\-diversity interventions, which spread the answer distribution, are what move best\-of\-nn; andKirk et al\. \[[29](https://arxiv.org/html/2606.28661#bib.bib29)\]document that post\-training \(the fine\-tuning applied after pretraining\) sharpens the modal answer\. Each is a face of one split: selection is held at its modal ceiling, coverage is not\.

The selection ceiling, on raw answers\.The modal\-hit rateπmode\\pi\_\{\\mathrm\{mode\}\}needs the answer strings, which the correctness\-only logs above do not record; a log of raw completions supplies them\. The best\-of\-nnrelease ofBeeching et al\. \[[20](https://arxiv.org/html/2606.28661#bib.bib20)\]samples each of the500500MATH\-500 problems256256times from one model \(Llama\-3\.2\-1B\-Instruct\) at a fixed decoding configuration \(temperature0\.80\.8, top\-pp1\.01\.0\), recording every raw completion\. Across the500500problems the256256answers carry a median of only about thirteen distinct values \(an effective answer count1/∑apa21/\\sum\_\{a\}p\_\{a\}^\{2\}\), not256256: the answer distributionpqp\_\{q\}is sharply concentrated\. Figure[10](https://arxiv.org/html/2606.28661#S4.F10)measures the consequence: coverage \(any attempt correct\) climbs to0\.880\.88while self\-consistency \(the plurality answer\) plateaus atπmode=0\.45\\pi\_\{\\mathrm\{mode\}\}=0\.45by aboutn=64n=64and moves little after\. Correctness is graded with the same verifier that produced the dataset’s labels \(math\-verify\), reproducing its reported single\-sample accuracy \(≈0\.27\\approx 0\.27\) to about a point\. The binary majority bound on the same log reads onlyℙ​\[θ\>12\]=0\.20\\mathbb\{P\}\[\\theta\>\\tfrac\{1\}\{2\}\]=0\.20: half the realized selection accuracy comes from problems solved less than half the time whose scattered errors leave the correct answer as the single most common one\. The plateau is the modal ceiling of Proposition[5](https://arxiv.org/html/2606.28661#Thmproposition5), on real attempts: the correct answer reaches the pool for nearly nine problems in ten, but the most common answer is correct for only four and a half\.

![Refer to caption](https://arxiv.org/html/2606.28661v1/x9.png)Figure 10:The within\-problem coverage–selection gap on a dependent\-draw log\[[20](https://arxiv.org/html/2606.28661#bib.bib20)\]: MATH\-500, Llama\-3\.2\-1B\-Instruct,256256attempts per problem in one session \(averaged over five sessions\)\. Coverage rises to0\.880\.88; self\-consistency plateaus at0\.450\.45; the shaded band is the within\-problem usable\-signal gap\.The five sessions also test the decomposition itself\. Estimatingρw\\rho\_\{w\}as the run\-to\-run dispersion of a problem’s success rate across the five independent sessions, corrected for the binomial noise of a finite session, givesρ^w\\hat\{\\rho\}\_\{w\}near zero \(ρ^w≈0\.0007\\hat\{\\rho\}\_\{w\}\\approx 0\.0007,95%95\\%CI\[0\.0005,0\.0009\]\[0\.0005,\\,0\.0009\]\): re\-running a fixed model at a fixed temperature does not move the latent success rate beyond sampling noise, so the de Finetti rateθ\\thetais nearly fixed per problem and the same\-session pooled correlation is dominated by difficulty\. The two\-stage identity then holds on real data to within0\.0010\.001: the directly pooled same\-session correlation isρ^=0\.401\\hat\{\\rho\}=0\.401, againstρ^b\+\(1−ρ^b\)​ρ^w=0\.402\\hat\{\\rho\}\_\{b\}\+\(1\-\\hat\{\\rho\}\_\{b\}\)\\hat\{\\rho\}\_\{w\}=0\.402from the separate terms \(Equation \([14](https://arxiv.org/html/2606.28661#S5.E14)\), with this log’s ownρ^b=0\.402\\hat\{\\rho\}\_\{b\}=0\.402\)\. The selection plateau in Figure[10](https://arxiv.org/html/2606.28661#S4.F10)therefore is not a between\-session effect: it is the within\-session answer collapse, the concentration of the answer distributionpqp\_\{q\}rather than any drift in the success rate, which Proposition[5](https://arxiv.org/html/2606.28661#Thmproposition5)prices as the modal\-hit rateπmode\\pi\_\{\\mathrm\{mode\}\}and Corollary[4](https://arxiv.org/html/2606.28661#Thmcorollary4)lower\-bounds byℙ​\[θ\>12\]\\mathbb\{P\}\[\\theta\>\\tfrac\{1\}\{2\}\]\.

Repeated sampling answers two questions that a single accuracy number blurs\.*Coverage*asks whether a correct answer is reachable at all, and keeps rising with the budget\.*Selection*asks whether the model’s most common answer is the correct one, and is capped at the modal\-hit rateπmode\\pi\_\{\\mathrm\{mode\}\}\. Their difference is the*identifiability gap*: the problems a model can reach but not return\.

## 5 Decomposition, allocation, and measurement

A lens earns its place only if it can be measured and acted on\. This section splits the correlation into its two sources, turns the ceiling into a compute\-allocation rule, and gives a one\-line estimator forρ\\rhothat runs on any sampling log already in hand\.

### 5\.1 Within\- and between\-problem correlation

Two sources feed the pooled correlationρ\\rho, and separating them is what tells a benchmark mean from a single answer\.*Within\-problem*correlationρw\\rho\_\{w\}is run\-to\-run dependence on a fixed problem: the dispersion of the latent rateθ\\thetaacross independent sessions in Equation \([5](https://arxiv.org/html/2606.28661#S2.E5)\)\. Under standard independent decoding this is zero by construction, each attempt an independent draw given the prompt, and Section[4\.3](https://arxiv.org/html/2606.28661#S4.SS3)measures it atρ^w≈0\.0007\\hat\{\\rho\}\_\{w\}\\approx 0\.0007\.*Between\-problem*heterogeneityρb\\rho\_\{b\}is the spread of difficulty across a benchmark: two attempts to a randomly drawn problem move together merely because they share that problem’s difficulty\. For attempts pooled over a benchmark the variance components add, so the pooled intraclass correlation \(takingρw\\rho\_\{w\}common across problems\) satisfies

ρ=ρb\+\(1−ρb\)​ρw,\\rho\\;=\\;\\rho\_\{b\}\+\(1\-\\rho\_\{b\}\)\\,\\rho\_\{w\},\(14\)the standard two\-stage design effect of clustered sampling\[[11](https://arxiv.org/html/2606.28661#bib.bib11)\]: the total same\-problem correlation is the between\-problem difficulty spread plus the within\-problem dependence left after it\. Withρw≈0\\rho\_\{w\}\\approx 0the pooledρ\\rhois essentiallyρb\\rho\_\{b\}, so the correlation ceiling of Section[3](https://arxiv.org/html/2606.28661#S3)is a statement about*estimating*a benchmark from heterogeneous problems, not about repeated tries on one problem\. The slow, power\-law coverage of a benchmark is the sameρb\\rho\_\{b\}through Proposition[4](https://arxiv.org/html/2606.28661#Thmproposition4), the regime studied bySchaeffer et al\. \[[17](https://arxiv.org/html/2606.28661#bib.bib17)\]\. The selection ceiling is a third axis, the concentration of the answer distributionpqp\_\{q\}\(Section[4\.2](https://arxiv.org/html/2606.28661#S4.SS2)\), whichρ\\rhodoes not see: difficulty heterogeneity, run\-to\-run dependence, and answer\-mode collapse are three distinct quantities the nominal sample count silently confounds\.

### 5\.2 A compute\-allocation rule

This is the practical payoff: with a fixed compute budget, the goal decides how to spend it, because the three goals reach their ceilings at different points\. Let total inference compute beC≈n​LC\\approx nLfornnsamples of reasoning lengthLL\. The levers are more samples \(raisenn\), longer reasoning \(raiseLL, which raisesss\), more problems \(for evaluation\), and decorrelation \(spread the answer distribution via temperature, nucleus \(top\-pp\) sampling, prompt diversity, or mixing models\)\[[30](https://arxiv.org/html/2606.28661#bib.bib30),[27](https://arxiv.org/html/2606.28661#bib.bib27),[7](https://arxiv.org/html/2606.28661#bib.bib7)\]\. The ceilings of Sections[3](https://arxiv.org/html/2606.28661#S3)and[4\.2](https://arxiv.org/html/2606.28661#S4.SS2)\(Corollaries[1](https://arxiv.org/html/2606.28661#Thmcorollary1)–[2](https://arxiv.org/html/2606.28661#Thmcorollary2)and Proposition[5](https://arxiv.org/html/2606.28661#Thmproposition5)\) set the stopping point for each:

To estimate a benchmark mean,n≈1/ρbn\\approx 1/\\rho\_\{b\}samples per problem suffice \(about two at the measuredρb\\rho\_\{b\}\); spend the rest on more problems\. To select an answer without a verifier, sampling helps only until the plurality stabilizes, on the order of the effective answer count1/∑apq​\(a\)21/\\sum\_\{a\}p\_\{q\}\(a\)^\{2\}, and past that it can anti\-scale\. To cover, that is to find a correct sample for a verifier, there is no within\-problem ceiling and more samples keep paying\.

The rule reframes the “think longer or sample more” question\[[3](https://arxiv.org/html/2606.28661#bib.bib3),[4](https://arxiv.org/html/2606.28661#bib.bib4),[31](https://arxiv.org/html/2606.28661#bib.bib31)\]: sampling pays without limit for coverage, stops early for selection, and stops almost at once for estimation, so the budget should follow the goal rather than a single number\. Whether decorrelation actively raises the selection plateau is the lever the protocol below tests\.

### 5\.3 Estimatingρ\\rho

The ceiling is actionable only ifρ\\rhocan be read off runs already in hand, and it can\. The correlation is measurable from any sampling log that records, for each ofMMproblems, the number correctcic\_\{i\}out ofnin\_\{i\}attempts\. The standard moment \(analysis\-of\-variance\) estimator for clustered binary data\[[11](https://arxiv.org/html/2606.28661#bib.bib11)\]compares the between\- and within\-problem sums of squares ofp^i=ci/ni\\hat\{p\}\_\{i\}=c\_\{i\}/n\_\{i\},

ρ^=MSbetween−MSwithinMSbetween\+\(n0−1\)​MSwithin,\\hat\{\\rho\}\\;=\\;\\frac\{\\mathrm\{MS\}\_\{\\text\{between\}\}\-\\mathrm\{MS\}\_\{\\text\{within\}\}\}\{\\mathrm\{MS\}\_\{\\text\{between\}\}\+\(n\_\{0\}\-1\)\\,\\mathrm\{MS\}\_\{\\text\{within\}\}\},\(15\)withn0n\_\{0\}the average cluster size; equivalently, the beta\-binomial model ofKazdan et al\. \[[18](https://arxiv.org/html/2606.28661#bib.bib18)\]has intraclass correlationρ^=1/\(α^\+β^\+1\)\\hat\{\\rho\}=1/\(\\hat\{\\alpha\}\+\\hat\{\\beta\}\+1\)by the identity in Appendix[A](https://arxiv.org/html/2606.28661#A1)\. Two cautions\. On benchmarks with many fully\-solved or never\-solved problems the within\-problem variance is degenerate and Equation \([15](https://arxiv.org/html/2606.28661#S5.E15)\) can return a small or negativeρ^\\hat\{\\rho\}, which should be clipped at zero; and a pooledρ^\\hat\{\\rho\}mixesρb\\rho\_\{b\}andρw\\rho\_\{w\}, so it states how much of the nominalnnis real only at the level \(a single problem or a whole benchmark\) at which it was estimated\. Table[1](https://arxiv.org/html/2606.28661#S4.T1)runs the estimator on the logs ofBrown et al\. \[[2](https://arxiv.org/html/2606.28661#bib.bib2)\]; because those attempts are drawn independently, what it recovers is the between\-problem termρb\\rho\_\{b\}, with a clustered \(problem\-level\) bootstrap interval that resamples problems with replacement\. The within\-problem term needs dependent draws, which Section[4\.3](https://arxiv.org/html/2606.28661#S4.SS3)reads off the single\-session log ofBeeching et al\. \[[20](https://arxiv.org/html/2606.28661#bib.bib20)\]: the seed\-to\-seed spread of a problem’s success rate is negligible there \(ρ^w≈0\.0007\\hat\{\\rho\}\_\{w\}\\approx 0\.0007\), so re\-decoding one prompt barely moves the latent rate, yet within a session the*answer*distribution still narrows to about thirteen modes and self\-consistency plateaus at0\.450\.45\. The single\-model selection ceiling is therefore set not by run\-to\-run drift but by the concentration of the answer distribution, the modal\-hit rateπmode\\pi\_\{\\mathrm\{mode\}\}of Proposition[5](https://arxiv.org/html/2606.28661#Thmproposition5)\. Genuine correlation does cap voting once the voters are distinct models rather than repeated draws of one: across nine distinct judge models the mean pairwise error correlation isρ^≈0\.39\\hat\{\\rho\}\\approx 0\.39, so the nine are worth only about two effective votes\[[14](https://arxiv.org/html/2606.28661#bib.bib14)\]\. For repeated draws of a single model, by contrast, a competition report finds high\-temperature sampling already largely decorrelates the errors\[[16](https://arxiv.org/html/2606.28661#bib.bib16)\], so what holds selection down there is not run\-to\-run correlation but the concentration of the answer mode: the handful of effective answers that makes selection saturate so early\.

The practical recommendation is one line, meant to be copied into a methods section verbatim:

Estimateρ^\\hat\{\\rho\}from the sampling log via Equation \([15](https://arxiv.org/html/2606.28661#S5.E15)\), then report the*effective number of samples*neff=n/\[1\+\(n−1\)​ρ^\]n\_\{\\mathrm\{eff\}\}=n/\[1\+\(n\-1\)\\hat\{\\rho\}\]alongside the nominal countnn, with the ceiling1/ρ^1/\\hat\{\\rho\}\.

This is the design\-effect\-corrected count ofKish \[[10](https://arxiv.org/html/2606.28661#bib.bib10)\], and it lets a reader see how much of a sampling budget is real\.

#### A pre\-registered protocol for the decoding lever\.

Section[4\.3](https://arxiv.org/html/2606.28661#S4.SS3)measures the within\-problem ceiling at one decoding configuration: the latent run rateθ\\thetais nearly fixed per problem \(ρ^w≈0\\hat\{\\rho\}\_\{w\}\\approx 0\), so the selection plateau is set by the dispersion of the modal answer within a session, not by run\-to\-run drift\. What remains to be measured is the*lever*: whether the decoding choices that decorrelate answers raise the plateau, the prediction the allocation rule rests on\. The following test is pre\-registered\. Fix a benchmark and a model\. For each ofM≥200M\\geq 200problems and each of several decoding configurations \(a sweep over temperature, nucleuspp, and prompt diversity\), drawm≥256m\\geq 256verified attempts; measure the within\-session answer\-effective count1/∑apa21/\\sum\_\{a\}p\_\{a\}^\{2\}, the plurality plateau, and the answer\-indicator intraclass correlation\. The pre\-registered prediction, from Proposition[5](https://arxiv.org/html/2606.28661#Thmproposition5), is that configurations with a lower answer correlation have a higher plurality plateau, monotone in the effective answer count; the falsifiable alternative is that the plateau is flat in the decoding configuration, under which decorrelation would not be the lever the rule names\. This requires model inference and is left to follow\-on work; it is stated now so the decoding lever is testable rather than assumed\.

### 5\.4 A practitioner’s reference

Everything needed to apply the lens fits on a page: a short checklist of what to do, a table of the formulas, and a reading of the common methods\. The checklist comes first, and these are diagnostics, not universal defaults\.

1. 1\.For evaluation, reportneffn\_\{\\mathrm\{eff\}\}, not justnn: for the benchmark mean a budget ofnnper problem is worthn/\[1\+\(n−1\)​ρb\]n/\[1\+\(n\-1\)\\rho\_\{b\}\]independent observations, at most1/ρb1/\\rho\_\{b\}, so add problems rather than samples\.
2. 2\.Estimateρb\\rho\_\{b\}from the log with Equation \([15](https://arxiv.org/html/2606.28661#S5.E15)\), clipped at zero on saturated benchmarks\.
3. 3\.Distinguish the three goals: coverage keeps improving withnn, selection plateaus atπmode\\pi\_\{\\mathrm\{mode\}\}and can anti\-scale, estimation saturates byn≈1/ρbn\\approx 1/\\rho\_\{b\}\.
4. 4\.For selection, lower the answer concentration \(temperature, nucleus sampling, prompt diversity, model mixing\) rather than raisingnn; sampling past the effective answer count buys nothing\.
5. 5\.Separate the answer ceiling from the verifier ceiling: one is whether the correct answer is the mode, the other whether the right try is identifiable by the scorer\.

A concrete pass through the checklist fixes the workflow\. Take a benchmark on which the difficulty correlation isρ^b≈0\.5\\hat\{\\rho\}\_\{b\}\\approx 0\.5and the answers concentrate onto about a dozen modes per problem, the regime of the logs in Section[4\.3](https://arxiv.org/html/2606.28661#S4.SS3)\. For evaluation, a budget of256256samples per problem is worth onlyneff≈1/ρb≈2n\_\{\\mathrm\{eff\}\}\\approx 1/\\rho\_\{b\}\\approx 2independent observations of the benchmark mean, so a confidence interval computed as if the256256were independent is too narrow by roughly elevenfold \(256/2\\sqrt\{256/2\}\); the honest move is to reportneffn\_\{\\mathrm\{eff\}\}and to spend fresh compute on fresh problems\. For deployment with a verifier, the same256256samples keep paying, since coverage carries no within\-problem ceiling\. For deployment without one, self\-consistency has captured almost all it will within a few dozen samples, a small multiple of the effective answer count, and drawing the full256256risks anti\-scaling on the problems whose mode is wrong\.

Three numbers drive the three goals: the difficulty correlationρb\\rho\_\{b\}, the effective answer count, and the verifier’s accuracy\. The first two are read off any sampling log already in hand by the estimators above, and the third is a property of the scorer\. Reading them once states how much of a budget is real for each use, which is the whole of what the lens asks of a practitioner\. Table[2](https://arxiv.org/html/2606.28661#S5.T2)collects the formulas behind the checklist, each with the behavior it explains, from the design effect that opens the argument to the modal\-hit rate that closes it\.

Table 2:Formulas for correlated test\-time scaling\. Herennis the number of samples,ssthe per\-attempt success probability, andρ\\rhothe intraclass correlation of the success indicators\.The formulas price each behavior; reading the methods as estimands then assigns each its ceiling\. Coverage escapes every ceiling because it asks only whether a correct sample exists; self\-consistency and best\-of\-nnmeet the modal\-answer wall because they read the answer distribution; and only benchmark\-mean estimation meets the correlation ceiling, the one place the design effect of Section[3](https://arxiv.org/html/2606.28661#S3)truly binds\.

Table 3:Test\-time\-scaling methods as estimands, with the ceiling that binds each\.

## 6 Conclusion

More sampling makes coverage climb while selection stalls; what separates them is the*identifiability gap*, the solvable problems whose answer a vote never returns\. Selection stalls because it meets the modal ceilingπmode\\pi\_\{\\mathrm\{mode\}\}, fixed by how often the most common answer happens to be right: on the dependent\-draw log ofBeeching et al\. \[[20](https://arxiv.org/html/2606.28661#bib.bib20)\]the256256tries per problem reduce to about thirteen, coverage reaches0\.880\.88while selection is right for only0\.450\.45, and drawing more only sharpens a confident error\. A different use meets a different limit: estimating a benchmark mean is capped by the correlation ceiling1/ρb1/\\rho\_\{b\}, so withρ^b≈0\.4\\hat\{\\rho\}\_\{b\}\\approx 0\.4–0\.60\.6on the released logs ofBrown et al\. \[[2](https://arxiv.org/html/2606.28661#bib.bib2)\], ten thousand attempts on a single problem buy the precision of about two, while the seed\-to\-seed term stays near zero \(ρ^w≈0\.0007\\hat\{\\rho\}\_\{w\}\\approx 0\.0007\), leaving coverage with no within\-problem ceiling\. One sample count thus stands in for three different things at once: difficulty spread between problems, dependence between runs, and the collapse of answers onto a mode\.

When to stop\.Because each draw costs compute and both ceilings are low, the budget that pays is small and set by the goal: about1/ρb1/\\rho\_\{b\}samples to estimate a benchmark mean, on the order of the effective number of answers to select one, and no limit for coverage where a verifier can pick the correct sample out\. Reporting the effective number of samples beside the nominalnn, a closed form solved for rather than searched\[[32](https://arxiv.org/html/2606.28661#bib.bib32)\], says how much of a budget actually counts, and where it stops paying\. The bottleneck in test\-time scaling has moved from generating a correct answer to recognizing one: coverage shows the answers are already present, the modal ceiling shows that more sampling will not surface them, and the compute that extra draws cannot use is better spent correlating answers less and choosing among them better, not drawing more\.

## References

- Wang et al\. \[2023\]Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed H\. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou\.Self\-consistency improves chain of thought reasoning in language models\.In*International Conference on Learning Representations \(ICLR\)*, 2023\.arXiv:2203\.11171\.
- Brown et al\. \[2024\]Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V\. Le, Christopher Ré, and Azalia Mirhoseini\.Large language monkeys: Scaling inference compute with repeated sampling, 2024\.arXiv:2407\.21787\.
- Snell et al\. \[2024\]Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar\.Scaling LLM test\-time compute optimally can be more effective than scaling model parameters, 2024\.arXiv:2408\.03314\.
- Wu et al\. \[2024\]Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang\.Inference scaling laws: An empirical analysis of compute\-optimal inference for problem\-solving with language models, 2024\.arXiv:2408\.00724\.
- OpenAI \[2024\]OpenAI\.Openai o1 system card, 2024\.arXiv:2412\.16720\.
- DeepSeek\-AI \[2025\]DeepSeek\-AI\.Deepseek\-r1: Incentivizing reasoning capability in LLMs via reinforcement learning, 2025\.arXiv:2501\.12948\.
- Muennighoff et al\. \[2025\]Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei\-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto\.s1: Simple test\-time scaling, 2025\.arXiv:2501\.19393\.
- Zhang et al\. \[2025\]Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, Irwin King, Xue Liu, and Chen Ma\.A survey on test\-time scaling in large language models: What, how, where, and how well?, 2025\.arXiv:2503\.24235\.
- Chen et al\. \[2021\]Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al\.Evaluating large language models trained on code\.*arXiv preprint arXiv:2107\.03374*, 2021\.arXiv:2107\.03374\.
- Kish \[1965\]Leslie Kish\.*Survey Sampling*\.John Wiley & Sons, New York, 1965\.
- Cochran \[1977\]William G\. Cochran\.*Sampling Techniques*\.John Wiley & Sons, New York, 3rd edition, 1977\.
- Ladha \[1992\]Krishna K\. Ladha\.The Condorcet jury theorem, free speech, and correlated votes\.*American Journal of Political Science*, 36\(3\):617–634, 1992\.
- Boland \[1989\]Philip J\. Boland\.Majority systems and the Condorcet jury theorem\.*Journal of the Royal Statistical Society: Series D \(The Statistician\)*, 38\(3\):181–189, 1989\.
- Kohli \[2026\]Guneet Kohli\.Nine judges, two effective votes: Correlated errors undermine LLM evaluation panels, 2026\.arXiv:2605\.29800\.
- Goel et al\. \[2025\]Shashwat Goel, Joschka Strüber, Ilze Amanda Auzina, Karuna K\. Chandra, Ponnurangam Kumaraguru, Douwe Kiela, Ameya Prabhu, Matthias Bethge, and Jonas Geiping\.Great models think alike and this undermines AI oversight, 2025\.arXiv:2502\.04313\.
- Nitarach \[2026\]Natapong Nitarach\.Model capability dominates: Inference\-time optimization lessons from AIMO 3, 2026\.arXiv:2603\.27844\.
- Schaeffer et al\. \[2025\]Rylan Schaeffer, Joshua Kazdan, John Hughes, Jordan Juravsky, Sara Price, Aengus Lynch, Erik Jones, Robert Kirk, Azalia Mirhoseini, and Sanmi Koyejo\.How do large language monkeys get their power \(laws\)?, 2025\.arXiv:2502\.17578\.
- Kazdan et al\. \[2025\]Joshua Kazdan, Rylan Schaeffer, Youssef Allouah, Colin Sullivan, Kyssen Yu, Noam Levi, and Sanmi Koyejo\.Efficient prediction of pass@k scaling in large language models, 2025\.arXiv:2510\.05197\.
- Levi \[2024\]Noam Levi\.A simple model of inference scaling laws, 2024\.arXiv:2410\.16377\.
- Beeching et al\. \[2024\]Edward Beeching, Lewis Tunstall, and Sasha Rush\.Scaling test\-time compute with open models\.Hugging Face blog, 2024\.[https://huggingface\.co/spaces/HuggingFaceH4/blogpost\-scaling\-test\-time\-compute](https://huggingface.co/spaces/HuggingFaceH4/blogpost-scaling-test-time-compute)\.
- Cobbe et al\. \[2021\]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman\.Training verifiers to solve math word problems, 2021\.arXiv:2110\.14168\.
- de Finetti \[1937\]Bruno de Finetti\.La prévision: ses lois logiques, ses sources subjectives\.*Annales de l’Institut Henri Poincaré*, 7\(1\):1–68, 1937\.
- Bay and Yearick \[2026a\]Yong Yi Bay and Kathleen A\. Yearick\.No 3D matrices: A unified tensor\-product view of matrix\-free Cartesian PDE solvers, 2026a\.arXiv:2606\.25148\.
- Skellam \[1948\]J\. G\. Skellam\.A probability distribution derived from the binomial distribution by regarding the probability of success as variable between the sets of trials\.*Journal of the Royal Statistical Society: Series B*, 10\(2\):257–261, 1948\.
- Bay and Yearick \[2024\]Yong Yi Bay and Kathleen A\. Yearick\.Machine learning vs deep learning: The generalization problem\.*arXiv preprint arXiv:2403\.01621*, 2024\.arXiv:2403\.01621\.
- Chen et al\. \[2024\]Lingjiao Chen, Jared Quincy Davis, Boris Hanin, Peter Bailis, Ion Stoica, Matei Zaharia, and James Zou\.Are more LLM calls all you need? towards scaling laws of compound inference systems, 2024\.arXiv:2403\.02419\.
- Wang et al\. \[2025\]Tianchun Wang, Zichuan Liu, Yuanzhou Chen, Jonathan Light, Weiyang Liu, Haifeng Chen, Xiang Zhang, and Wei Cheng\.On the effect of sampling diversity in scaling LLM inference, 2025\.arXiv:2502\.11027\.
- Hendrycks et al\. \[2021\]Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt\.Measuring mathematical problem solving with the MATH dataset\.In*Advances in Neural Information Processing Systems \(NeurIPS\) Datasets and Benchmarks Track*, 2021\.arXiv:2103\.03874\.
- Kirk et al\. \[2024\]Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu\.Understanding the effects of RLHF on LLM generalisation and diversity\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.arXiv:2310\.06452\.
- Holtzman et al\. \[2020\]Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi\.The curious case of neural text degeneration\.In*International Conference on Learning Representations \(ICLR\)*, 2020\.arXiv:1904\.09751\.
- Liu et al\. \[2025\]Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, and Bowen Zhou\.Can 1B LLM surpass 405B LLM? rethinking compute\-optimal test\-time scaling, 2025\.arXiv:2502\.06703\.
- Bay and Yearick \[2026b\]Yong Yi Bay and Kathleen A\. Yearick\.Solve for the hyperparameter, skip the search: Kolmogorov\-optimal scaling laws for spline regression\.*arXiv preprint arXiv:2606\.23575*, 2026b\.

## Appendix AElementary derivations

The body states the propositions and leans on these short calculations without pausing for them; each one underwrites a result above\.

#### Beta\-binomial moments and intraclass correlation\.

Take the latent rateθ∼Beta​\(α,β\)\\theta\\sim\\mathrm\{Beta\}\(\\alpha,\\beta\)with attemptsYi∣θ∼i\.i\.d\. Bernoulli​\(θ\)Y\_\{i\}\\mid\\theta\\sim\\text\{i\.i\.d\.\\ Bernoulli\}\(\\theta\)\. The Beta has mean and variance

s=𝔼​\[θ\]=αα\+β,Var⁡\(θ\)=α​β\(α\+β\)2​\(α\+β\+1\)=s​\(1−s\)α\+β\+1\.s=\\mathbb\{E\}\[\\theta\]=\\frac\{\\alpha\}\{\\alpha\+\\beta\},\\qquad\\operatorname\{Var\}\(\\theta\)=\\frac\{\\alpha\\beta\}\{\(\\alpha\+\\beta\)^\{2\}\(\\alpha\+\\beta\+1\)\}=\\frac\{s\(1\-s\)\}\{\\alpha\+\\beta\+1\}\.Two distinct attempts share only the rateθ\\theta, so their covariance is the variance of that shared rate: fori≠ji\\neq j,Cov⁡\(Yi,Yj\)=𝔼​\[θ2\]−s2=Var⁡\(θ\)\\operatorname\{Cov\}\(Y\_\{i\},Y\_\{j\}\)=\\mathbb\{E\}\[\\theta^\{2\}\]\-s^\{2\}=\\operatorname\{Var\}\(\\theta\)\. Dividing bys​\(1−s\)s\(1\-s\)gives the intraclass correlation of Equation \([5](https://arxiv.org/html/2606.28661#S2.E5)\),

ρ=Var⁡\(θ\)s​\(1−s\)=1α\+β\+1,\\rho=\\frac\{\\operatorname\{Var\}\(\\theta\)\}\{s\(1\-s\)\}=\\frac\{1\}\{\\alpha\+\\beta\+1\},which does not depend onnn\. Inverting, a target meanssand correlationρ\\rhoare realized byα=s​\(1−ρ\)/ρ\\alpha=s\(1\-\\rho\)/\\rhoandβ=\(1−s\)​\(1−ρ\)/ρ\\beta=\(1\-s\)\(1\-\\rho\)/\\rho, the map the figures use to set a Beta from\(s,ρ\)\(s,\\rho\)\.

#### Zero\-success probability and its tail\.

The chance that none ofnnattempts succeeds is the Beta average of\(1−θ\)n\(1\-\\theta\)^\{n\}, a standard beta integral:

ℙ​\[K=0\]=𝔼θ​\[\(1−θ\)n\]=∫01\(1−θ\)n​θα−1​\(1−θ\)β−1B​\(α,β\)​𝑑θ=B​\(α,β\+n\)B​\(α,β\)\.\\mathbb\{P\}\[K=0\]=\\mathbb\{E\}\_\{\\theta\}\\big\[\(1\-\\theta\)^\{n\}\\big\]=\\int\_\{0\}^\{1\}\(1\-\\theta\)^\{n\}\\frac\{\\theta^\{\\alpha\-1\}\(1\-\\theta\)^\{\\beta\-1\}\}\{B\(\\alpha,\\beta\)\}\\,d\\theta=\\frac\{B\(\\alpha,\\beta\+n\)\}\{B\(\\alpha,\\beta\)\}\.Written with gamma functions this isΓ​\(α\+β\)Γ​\(β\)⋅Γ​\(β\+n\)Γ​\(α\+β\+n\)\\frac\{\\Gamma\(\\alpha\+\\beta\)\}\{\\Gamma\(\\beta\)\}\\cdot\\frac\{\\Gamma\(\\beta\+n\)\}\{\\Gamma\(\\alpha\+\\beta\+n\)\}\. For largennthe gamma ratio decays polynomially,Γ​\(β\+n\)/Γ​\(α\+β\+n\)∼n−α\\Gamma\(\\beta\+n\)/\\Gamma\(\\alpha\+\\beta\+n\)\\sim n^\{\-\\alpha\}\(Stirling,Γ​\(x\+a\)/Γ​\(x\+b\)∼xa−b\\Gamma\(x\+a\)/\\Gamma\(x\+b\)\\sim x^\{a\-b\}\), so

ℙ​\[K=0\]∼Γ​\(α\+β\)Γ​\(β\)​n−α\(n→∞\)\.\\mathbb\{P\}\[K=0\]\\sim\\frac\{\\Gamma\(\\alpha\+\\beta\)\}\{\\Gamma\(\\beta\)\}\\,n^\{\-\\alpha\}\\qquad\(n\\to\\infty\)\.This is the power\-law tail of Proposition[4](https://arxiv.org/html/2606.28661#Thmproposition4): coverage approaches its limit polynomially, not exponentially\.

#### The selection plateau\.

Plurality returns the most frequent answer, and the answer counts overnnattempts areMultinomial​\(n,pq\)\\mathrm\{Multinomial\}\(n,p\_\{q\}\)\. By the law of large numbers the empirical frequencies converge topqp\_\{q\}almost surely, so the empirical mode converges to the true modeaq⋆=arg⁡maxa⁡pq​\(a\)a^\{\\star\}\_\{q\}=\\arg\\max\_\{a\}p\_\{q\}\(a\), and per\-problem plurality accuracy to𝟏​\{aq⋆=cq\}\\mathbf\{1\}\\\{a^\{\\star\}\_\{q\}=c\_\{q\}\\\}\. Averaging over problems gives the modal\-hit rateπmode\\pi\_\{\\mathrm\{mode\}\}of Equation \([13](https://arxiv.org/html/2606.28661#S4.E13)\)\.

The two\-answer case recovers the majority vote\. With only a correct answer and one error, writeθ\\thetafor the per\-attempt correctness; then plurality is majority andp^=K/n→θ\\hat\{p\}=K/n\\to\\thetaalmost surely, so in the limit the majority is right precisely ifθ\>12\\theta\>\\tfrac\{1\}\{2\}\. Its variance tends toVar⁡\(p^\)→ρ​s​\(1−s\)\\operatorname\{Var\}\(\\hat\{p\}\)\\to\\rho\\,s\(1\-s\), and approximatingθ\\thetaas𝒩​\(s,ρ​s​\(1−s\)\)\\mathcal\{N\}\\big\(s,\\,\\rho\\,s\(1\-s\)\\big\)gives the closed form

ℙ​\[θ\>12\]≈Φ​\(s−12ρ​s​\(1−s\)\)\\mathbb\{P\}\[\\theta\>\\tfrac\{1\}\{2\}\]\\approx\\Phi\\\!\\left\(\\frac\{s\-\\tfrac\{1\}\{2\}\}\{\\sqrt\{\\rho\\,s\(1\-s\)\}\}\\right\)of Corollary[4](https://arxiv.org/html/2606.28661#Thmcorollary4)\. Becauseθ\>12\\theta\>\\tfrac\{1\}\{2\}pushes every rival answer under12\\tfrac\{1\}\{2\}, hence under the correct one, this majority bound never exceedsπmode\\pi\_\{\\mathrm\{mode\}\}\.

#### Two\-stage design effect\.

Pool attempts over problems, and letμi\\mu\_\{i\}be the success rate of problemii, with between\-problem correlationρb=Var⁡\(μi\)/\[s​\(1−s\)\]\\rho\_\{b\}=\\operatorname\{Var\}\(\\mu\_\{i\}\)/\[s\(1\-s\)\]and a common within\-problem correlationρw\\rho\_\{w\}given the problem\. By the law of total covariance, two same\-problem attemptsj≠j′j\\neq j^\{\\prime\}have

Cov⁡\(Yi​j,Yi​j′\)=𝔼​\[ρw​μi​\(1−μi\)\]\+Var⁡\(μi\)\.\\operatorname\{Cov\}\(Y\_\{ij\},Y\_\{ij^\{\\prime\}\}\)=\\mathbb\{E\}\\\!\\big\[\\rho\_\{w\}\\,\\mu\_\{i\}\(1\-\\mu\_\{i\}\)\\big\]\+\\operatorname\{Var\}\(\\mu\_\{i\}\)\.Since𝔼​\[μi​\(1−μi\)\]=s​\(1−s\)−Var⁡\(μi\)=s​\(1−s\)​\(1−ρb\)\\mathbb\{E\}\[\\mu\_\{i\}\(1\-\\mu\_\{i\}\)\]=s\(1\-s\)\-\\operatorname\{Var\}\(\\mu\_\{i\}\)=s\(1\-s\)\(1\-\\rho\_\{b\}\), the right\-hand side iss​\(1−s\)​\[ρb\+\(1−ρb\)​ρw\]s\(1\-s\)\\big\[\\rho\_\{b\}\+\(1\-\\rho\_\{b\}\)\\rho\_\{w\}\\big\]\. Dividing bys​\(1−s\)s\(1\-s\)gives the pooled correlation

ρ=ρb\+\(1−ρb\)​ρw\\rho=\\rho\_\{b\}\+\(1\-\\rho\_\{b\}\)\\rho\_\{w\}of Equation \([14](https://arxiv.org/html/2606.28661#S5.E14)\), and the same\-problem design effect is1\+\(n−1\)​ρ1\+\(n\-1\)\\rho\[[11](https://arxiv.org/html/2606.28661#bib.bib11)\]\.

## Appendix BReproducibility

Every number, table, and figure in this work regenerates from a clean checkout\. The propositions are checked numerically byscripts/verify\_math\.py, which verifies the identities of Sections[3](https://arxiv.org/html/2606.28661#S3)–[5](https://arxiv.org/html/2606.28661#S5)\(including the variance identity \([6](https://arxiv.org/html/2606.28661#S3.E6)\), the coverage monotonicity of Proposition[2](https://arxiv.org/html/2606.28661#Thmproposition2), the modal\-answer selection ceiling \([13](https://arxiv.org/html/2606.28661#S4.E13)\) with its anti\-scaling corollary and the majority\-vote lower bound, and the two\-stage decomposition \([14](https://arxiv.org/html/2606.28661#S5.E14)\)\) against Monte Carlo simulation; all checks pass\. The five model\-based figures are generated byscripts/make\_figures\.pyfrom closed forms and fixed\-seed simulation; the correlated attempts use the de Finetti representationθ∼Beta​\(α,β\)\\theta\\sim\\mathrm\{Beta\}\(\\alpha,\\beta\),Yi∣θ∼Bernoulli​\(θ\)Y\_\{i\}\\mid\\theta\\sim\\text\{Bernoulli\}\(\\theta\), with\(α,β\)\(\\alpha,\\beta\)set from\(s,ρ\)\(s,\\rho\)by Appendix[A](https://arxiv.org/html/2606.28661#A1)\. The between\-problem figure and Table[1](https://arxiv.org/html/2606.28661#S4.T1)are produced byscripts/analyze\_brown\.py, which downloads the public sampling logs ofBrown et al\. \[[2](https://arxiv.org/html/2606.28661#bib.bib2)\], estimatesρ^b\\hat\{\\rho\}\_\{b\}with a clustered \(problem\-level\) bootstrap95%95\\%interval \(10410^\{4\}resamples, fixed seed\), computes the coverage and self\-consistency curves, and writes a small summary that the figure script reads\. The within\-problem figure and theρ^w\\hat\{\\rho\}\_\{w\},ρ^b\\hat\{\\rho\}\_\{b\}, and decomposition numbers of Section[4\.3](https://arxiv.org/html/2606.28661#S4.SS3)are produced byscripts/analyze\_rhow\.py, which downloads the five\-session best\-of\-nnlog ofBeeching et al\. \[[20](https://arxiv.org/html/2606.28661#bib.bib20)\], grades every completion withmath\-verify\(the verifier behind the dataset’s own labels, whose reported single\-sample accuracy it reproduces to about a point\), estimates the between\- and within\-problem correlations with problem\-clustered bootstrap intervals, and computes the within\-session coverage and plurality curves; the graded per\-problem counts are cached so the figure rebuilds without re\-downloading or re\-grading the multi\-hundred\-megabyte log\. The Python environment is pinned inpyproject\.toml\(uv sync\);make allregenerates everything\. Seeds are fixed, so the results are reproducible\. The code, the cached result summaries, and the manuscript source are available at[https://github\.com/bay\-yearick\-lab/sampling\-ceilings](https://github.com/bay-yearick-lab/sampling-ceilings)\.