Tag
The paper introduces GeoPair, a training-free framework for transformer compression that optimizes cross-layer factorizations while preserving activation geometries, achieving state-of-the-art results across diverse architectures.
This paper shows that the Johnson–Lindenstrauss lemma can be uninformative about retained geometry in high dimensions and derives exact limits for features like distance recovery, nearest-neighbor rankings, and covariance shape in Gaussian models.