Damage-Aware Bandit Pruning for Vision and Language Transformers
Summary
This paper proposes a damage-aware multi-armed bandit method for structured post-training pruning of vision and language transformers, showing reduced performance degradation compared to baseline approaches in experiments across various models and datasets.
View Cached Full Text
Cached at: 09/10/26, 08:35 AM
# Damage-Aware Bandit Pruning for Vision and Language Transformers Source: [https://arxiv.org/abs/2609.05448](https://arxiv.org/abs/2609.05448) [View PDF](https://arxiv.org/pdf/2609.05448) > Abstract:Structured post\-training pruning of transformers requires selecting complete functional units whose suppression causes limited degradation\. We formulate structured\-unit selection for language and vision transformers as a damage\-aware multi\-armed bandit problem under a fixed candidate\-evaluation budget\. Attention heads and MLP channel groups are temporarily masked on calibration batches\. Paired damage is the masked loss minus the base loss on the same batch, reducing batch\-to\-batch variation\. A smooth bounded reward drives either a UCB\-style policy or fractional\-Beta Thompson Sampling, and the final mask is constructed sequentially by adding one unit at each step\. The selected units are functionally zeroed in the original dense checkpoint; therefore, the reported parameter effects represent effective structural suppression rather than physical compression or measured speedup\. Experiments on WikiText\-2, LAMBADA, and Imagenette cover GPT\-2, OPT, Pythia, Qwen2\.5, SmolLM2, ViT\-B/16, DeiT\-Tiny, and Swin\-Tiny, with comparisons against random, magnitude, static\-saliency, and budgeted\-greedy selection\. Across five seeds, the bandit methods usually reduce degradation relative to budgeted greedy in the paired language\-model comparisons\. Of 28 comparisons highlighted in the paper, 23 bootstrap confidence intervals exclude zero and 11 paired tests have p < 0\.05; six have q < 0\.05 after Benjamini\-Hochberg correction across the full family of 116 dataset\-wise tests\. Matched\-evaluation results for ViT\-B/16 and Swin\-Tiny indicate that their gains are not explained solely by a larger candidate\-evaluation budget\. ## Submission history From: Salem Ameen \[[view email](https://arxiv.org/show-email/05400591/2609.05448)\] **\[v1\]**Sun, 2 Aug 2026 23:57:18 UTC \(3,177 KB\)
Similar Articles
Structured Neuron Pruning in Deep Neural Networks Using Multi-Armed Bandits
This paper proposes a novel structured neuron pruning framework for deep neural networks using multi-armed bandit algorithms, demonstrating effectiveness on various tasks.
Loss-Aware Feature-Map Pruning in Convolutional Neural Networks Using Multi-Armed Bandits
This paper introduces a loss-aware feature-map pruning framework for convolutional neural networks using multi-armed bandits (UCB1 and Thompson Sampling) to selectively remove redundant feature maps, reducing computational cost while preserving accuracy.
An AI4AI Framework for Visual Token Pruning
AutoPrune is a training-free framework that uses LLMs to automatically design visual-token pruning policies for multimodal LLMs via a domain-specific language and residual search, achieving high efficiency with minimal performance loss (99% performance retained while removing 94.4% of visual tokens).
Retrieve, Don't Retrain: Extending Vision Language Action Models to New Tasks at Test Time
This paper introduces a retrieval-augmented vision-language-action policy that eliminates per-task fine-tuning by using pre-trained models with indexed demonstrations, enabling efficient cross-embodiment generalization and task adaptation at test time.
Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models
Proposes Reroute, a training-free plug-in for vision-language models that replaces irreversible visual-token pruning with recoverable routing, allowing tokens to re-enter the pipeline later to improve grounding under aggressive token reduction while maintaining VQA performance.