Structured Neuron Pruning in Deep Neural Networks Using Multi-Armed Bandits
Summary
This paper proposes a novel structured neuron pruning framework for deep neural networks using multi-armed bandit algorithms, demonstrating effectiveness on various tasks.
View Cached Full Text
Cached at: 06/09/26, 08:51 AM
# Structured Neuron Pruning in Deep Neural Networks Using Multi-Armed Bandits Source: [https://arxiv.org/abs/2606.07615](https://arxiv.org/abs/2606.07615) [View PDF](https://arxiv.org/pdf/2606.07615) > Abstract:Deep neural networks often contain redundant hidden units\. Removing individual weights can reduce parameter count, but unstructured sparsity is not always easy to exploit in standard dense implementations\. This paper develops a structured pruning framework in which complete neurons are removed using multi\-armed bandit \(MAB\) algorithms\. Each candidate neuron is treated as an arm; pulling an arm temporarily masks that neuron, measures the change in loss on a sampled mini\-batch, restores the neuron, and updates an estimate of its safe\-removal reward\. The framework supports stochastic policies, including Epsilon\-Greedy, Softmax, UCB1 and Thompson Sampling, and multiplicative\-weight policies, including Hedge\-style multiplicative weights and EXP3\. We evaluate the method on tabular classification, tabular regression and deep neural\-network benchmarks covering image, text and reasoning tasks\. Statistical comparisons using the Friedman test followed by the Nemenyi post\-hoc test show significant differences between methods\. On tabular classification tasks, UCB1 obtains the highest mean rank among pruning policies and improves on the unpruned neural network\. On regression tasks, UCB1 obtains the highest mean rank and is statistically competitive with, or superior to, several standard regression models according to R^2\. On deep\-learning tasks, UCB1 and Thompson Sampling obtain the strongest ranks, and several MAB policies significantly outperform the unpruned model, magnitude\-based neuron pruning and greedy activation\-variation pruning\. The results show that MAB\-based neuron pruning is an effective and computationally practical approach for structured model reduction\. ## Submission history From: Salem Ameen \[[view email](https://arxiv.org/show-email/00c73b97/2606.07615)\] **\[v1\]**Fri, 29 May 2026 19:56:54 UTC \(1,522 KB\)
Similar Articles
Loss-Aware Feature-Map Pruning in Convolutional Neural Networks Using Multi-Armed Bandits
This paper introduces a loss-aware feature-map pruning framework for convolutional neural networks using multi-armed bandits (UCB1 and Thompson Sampling) to selectively remove redundant feature maps, reducing computational cost while preserving accuracy.
Pruning Deep Neural Networks via the Marchenko--Pastur Distribution
This paper presents a Marchenko-Pastur random matrix approach to pruning deep neural networks, offering theoretical guarantees and achieving strong accuracy retention with minimal fine-tuning on ImageNet for ViT and CNN architectures.
Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
Proposes a two-stage structured pruning framework for LLMs that jointly optimizes latency and model size using multi-objective depth pruning and parallel Bayesian optimization, achieving favorable trade-offs for edge deployment.
Sigma-Branch: Hierarchical Single-Path Network Reconstruction for Dynamic Inference with Reduced Active Parameters
Sigma-Branch restructures pretrained dense networks into a hierarchical binary tree with a shared backbone, routers, and specialized leaves, reducing per-inference active parameters by 58–60% while staying within 1.72 pp of baseline accuracy on CIFAR-100, ImageNet-1K, and ModelNet40.
TENP: Trapezoidal Expert Neuron Pruning For Mixture-of-Experts
TENP proposes a structured pruning framework for Mixture-of-Experts LLMs that retains important experts and applies neuron pruning to less important ones, achieving high sparsity with minimal accuracy loss on Qwen and DeepSeek models.