intrinsic-dimension

This paper proposes automatic layer selection for hallucination detection in LLMs and introduces First Effective Peak of Intrinsic Dimension (FEPoID), a training-free criterion that consistently identifies optimal intermediate layers, outperforming existing heuristics.

This paper introduces the Representation Gap, a metric for neural network generalization error with better asymptotic dynamics. Using a geometric perspective and optimal quantization theory, the authors show it is governed by the intrinsic dimension of the task, and verify this empirically on synthetic and realistic datasets.