neural-network-pruning

本文研究了端到端语音解析架构（Wav2tree）中中间神经网络单元的作用，提出了一种移除这些单元的简化架构，在减少12%参数的同时在ASR和解析任务上达到相当或更优的性能，并对法语、斯洛文尼亚语和Naija语进行了综合评估。

This paper introduces a new parameter pruning scheme for neural networks based on Fisher information distances, which outperforms existing methods on MNIST and CIFAR-10 datasets across fully-connected networks and vision transformers.

CutClean is a privacy-aware pruning method for neural networks that reduces private information leakage while increasing sparsity, using auxiliary privacy heads to quantify and mitigate privacy risks.

This paper introduces Squeeze-Release, an iterative pruning method that achieves exact structural minimization.

This paper presents a Marchenko-Pastur random matrix approach to pruning deep neural networks, offering theoretical guarantees and achieving strong accuracy retention with minimal fine-tuning on ImageNet for ViT and CNN architectures.

OpenAI proposes a practical L₀ regularization method for neural networks that encourages weights to become exactly zero during training, enabling network pruning for improved speed and generalization. The method uses stochastic gates and introduces the hard concrete distribution to make the non-differentiable L₀ norm optimization tractable via gradient descent.