Structured Neuron Pruning in Deep Neural Networks Using Multi-Armed Bandits
Summary
This paper proposes a novel structured neuron pruning framework for deep neural networks using multi-armed bandit algorithms, demonstrating effectiveness on various tasks.
View Cached Full Text
Cached at: 06/09/26, 08:51 AM
# Structured Neuron Pruning in Deep Neural Networks Using Multi-Armed Bandits Source: [https://arxiv.org/abs/2606.07615](https://arxiv.org/abs/2606.07615) [View PDF](https://arxiv.org/pdf/2606.07615) > Abstract:Deep neural networks often contain redundant hidden units\. Removing individual weights can reduce parameter count, but unstructured sparsity is not always easy to exploit in standard dense implementations\. This paper develops a structured pruning framework in which complete neurons are removed using multi\-armed bandit \(MAB\) algorithms\. Each candidate neuron is treated as an arm; pulling an arm temporarily masks that neuron, measures the change in loss on a sampled mini\-batch, restores the neuron, and updates an estimate of its safe\-removal reward\. The framework supports stochastic policies, including Epsilon\-Greedy, Softmax, UCB1 and Thompson Sampling, and multiplicative\-weight policies, including Hedge\-style multiplicative weights and EXP3\. We evaluate the method on tabular classification, tabular regression and deep neural\-network benchmarks covering image, text and reasoning tasks\. Statistical comparisons using the Friedman test followed by the Nemenyi post\-hoc test show significant differences between methods\. On tabular classification tasks, UCB1 obtains the highest mean rank among pruning policies and improves on the unpruned neural network\. On regression tasks, UCB1 obtains the highest mean rank and is statistically competitive with, or superior to, several standard regression models according to R^2\. On deep\-learning tasks, UCB1 and Thompson Sampling obtain the strongest ranks, and several MAB policies significantly outperform the unpruned model, magnitude\-based neuron pruning and greedy activation\-variation pruning\. The results show that MAB\-based neuron pruning is an effective and computationally practical approach for structured model reduction\. ## Submission history From: Salem Ameen \[[view email](https://arxiv.org/show-email/00c73b97/2606.07615)\] **\[v1\]**Fri, 29 May 2026 19:56:54 UTC \(1,522 KB\)
Similar Articles
Damage-Aware Bandit Pruning for Vision and Language Transformers
This paper proposes a damage-aware multi-armed bandit method for structured post-training pruning of vision and language transformers, showing reduced performance degradation compared to baseline approaches in experiments across various models and datasets.
Loss-Aware Feature-Map Pruning in Convolutional Neural Networks Using Multi-Armed Bandits
This paper introduces a loss-aware feature-map pruning framework for convolutional neural networks using multi-armed bandits (UCB1 and Thompson Sampling) to selectively remove redundant feature maps, reducing computational cost while preserving accuracy.
Pruning Binarized Neural Networks: A Dedicated Framework and Globally Weighted Algorithms
This paper introduces a PyTorch-based framework for pruning binarized neural networks and proposes a novel global weighting pruning method that achieves superior accuracy and pruning rates, enabling efficient deployment on edge hardware like FPGAs.
Pruning Deep Neural Networks via the Marchenko--Pastur Distribution
This paper presents a Marchenko-Pastur random matrix approach to pruning deep neural networks, offering theoretical guarantees and achieving strong accuracy retention with minimal fine-tuning on ImageNet for ViT and CNN architectures.
Adaptive Multi-Branching for Shallow Decision Tree Induction
This paper proposes the Multi-Branch Neural Decision Tree with Adaptive Pruning (MBNDT), a decision tree model that improves classification accuracy under depth constraints through adaptive multi-way splits, achieving superior performance on OpenML benchmarks.