stochastic-sampling

This paper investigates whether stochastic sampling (self-consistency) in LLMs can capture cross-question structure similar to diverse ensembles. Using a Marchenko–Pastur test, the authors find that within a single model, stochastic variation yields at most one significant dimension, while an ensemble of 24 models yields four, revealing a dimensionality gap that limits self-consistency as an ensemble substitute.