neural-network-pruning

This paper introduces Squeeze-Release, an iterative pruning method that achieves exact structural minimization.

This paper presents a Marchenko-Pastur random matrix approach to pruning deep neural networks, offering theoretical guarantees and achieving strong accuracy retention with minimal fine-tuning on ImageNet for ViT and CNN architectures.

OpenAI proposes a practical L₀ regularization method for neural networks that encourages weights to become exactly zero during training, enabling network pruning for improved speed and generalization. The method uses stochastic gates and introduces the hard concrete distribution to make the non-differentiable L₀ norm optimization tractable via gradient descent.