Tag
This paper investigates fixed points and stability in randomly initialized autoencoders, introducing local and global edge-of-chaos concepts using random matrix theory and Gaussian processes.
A developer is creating a method to visually fingerprint AI models by analyzing weights and tensors using Random Matrix Theory, with data from QWEN2.5 models, and is interested in open-sourcing the tool.
This paper applies Marchenko-Pastur random matrix theory to pre-trained attention weights, separating each projection matrix into a random-like bulk and spectral outliers. Causal experiments show zeroing these outliers in Mistral-7B drives performance near random chance, revealing that spectral outliers encode dominant learned structure across 11 transformers.
This paper presents a Marchenko-Pastur random matrix approach to pruning deep neural networks, offering theoretical guarantees and achieving strong accuracy retention with minimal fine-tuning on ImageNet for ViT and CNN architectures.