Tag
This paper investigates whether stochastic sampling (self-consistency) in LLMs can capture cross-question structure similar to diverse ensembles. Using a Marchenko–Pastur test, the authors find that within a single model, stochastic variation yields at most one significant dimension, while an ensemble of 24 models yields four, revealing a dimensionality gap that limits self-consistency as an ensemble substitute.
This paper presents a Marchenko-Pastur random matrix approach to pruning deep neural networks, offering theoretical guarantees and achieving strong accuracy retention with minimal fine-tuning on ImageNet for ViT and CNN architectures.