Unifying Depth and Width Pruning for LLMs via Binary Knapsack Optimization

# 1Introduction
Source: [https://arxiv.org/html/2608.12953](https://arxiv.org/html/2608.12953)
\\papergithub

https://github\.com/parmanu\-lcs2\\paperwebsitehttps://parmanu\.lcs2\.in\\papertitleUnifying Depth and Width Pruning for LLMs via Binary Knapsack Optimization\\papershorttitleUnifying Depth and Width Pruning for LLMs via Binary Knapsack Optimization\\papershortauthorsP\. Goel, A\. Sengupta, A\. Nambi and T\. Chakraborty\\paperauthorsPalaash Goel1Ayan Sengupta2Akshay Nambi3Tanmoy Chakraborty1,2\\paperaffil1Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi 2Department of Electrical Engineering, Indian Institute of Technology Delhi 3Microsoft Research \(MSR\) India \\paperemailsgoelpalaash@scai\.iitd\.ac\.in, ayan\.sengupta007@gmail\.com, akshay\.nambi@microsoft\.com, tanchak@iitd\.ac\.in\\paperkeywordsLLM compression, structured pruning, LLM efficiency, knapsack optimization, dual\-axis pruning\\paperabstractStructured pruning is a promising approach for compressing large language models \(LLMs\), yet existing methods rely heavily on greedy heuristics that produce myopic decisions, and often fail to precisely meet target compression budgets\. We presentSniper, a two\-stage structured pruning framework that solves a knapsack optimization over coarse\-granularity components to yield conditionally optimal parameter allocations with respect to fixed importance estimates, followed by a fine\-grained pruning stage to meet strict budget constraints\. We introduce the Compression Ratio Adherence Factor \(CRAFT\) to quantify budget fidelity, showing that while existing pruners deviate from target compression ratios by up to 33%,Sniperachieves near\-exact adherence with aCRAFTscore of 0\.98\. Evaluations across four diverse architectures over a set of 18 tasks spanning five domains demonstrateSniper’s consistent improvements in average performance retention and task\-level stability over six state\-of\-the\-art pruners\. Across all pruning configurations,Sniperachieves an excellent mean rank of 1\.25, indicating its robust cross\-architectural generalizability and excellent reliability\.\\appendixtocon\\appendixtocnameStructure of the Appendix\\makelabtitle

## 1Introduction

Large language models \(LLMs\) have become the foundation of modern natural language processing, driving advances in reasoning, generation, and comprehension across a wide range of tasks\[[19](https://arxiv.org/html/2608.12953#bib.bib18),[71](https://arxiv.org/html/2608.12953#bib.bib19),[11](https://arxiv.org/html/2608.12953#bib.bib72),[47](https://arxiv.org/html/2608.12953#bib.bib16)\]\. These capabilities, however, come at the cost of rapidly increasing parameter counts, posing significant challenges for deployment on resource\-constrained hardware such as edge devices and on\-device accelerators\[[75](https://arxiv.org/html/2608.12953#bib.bib58),[46](https://arxiv.org/html/2608.12953#bib.bib55)\]\. While techniques such as quantization\[[5](https://arxiv.org/html/2608.12953#bib.bib34),[12](https://arxiv.org/html/2608.12953#bib.bib20),[14](https://arxiv.org/html/2608.12953#bib.bib30)\]and knowledge distillation\[[21](https://arxiv.org/html/2608.12953#bib.bib74),[58](https://arxiv.org/html/2608.12953#bib.bib21)\]reduce memory footprint or training cost, they largely preserve the original model architecture and therefore do not yield proportional inference speedups\. In contrast, structured model pruning directly removes parameter groups, offering a principled route to practical acceleration\.

Despite their appeal, existing structured pruning methods suffer from fundamental limitations arising from the interplay between pruning granularity and greedy optimization\. Depth\-based pruners\[[59](https://arxiv.org/html/2608.12953#bib.bib35),[40](https://arxiv.org/html/2608.12953#bib.bib40),[60](https://arxiv.org/html/2608.12953#bib.bib36),[20](https://arxiv.org/html/2608.12953#bib.bib68)\]operate on coarse atomic units and often fail to adhere to target compression ratios, while width\-based methods\[[39](https://arxiv.org/html/2608.12953#bib.bib44),[57](https://arxiv.org/html/2608.12953#bib.bib65),[4](https://arxiv.org/html/2608.12953#bib.bib43)\]induce sparsity\-driven irregularities in weight tensor shapes, leading to minimal inference speedup\[[32](https://arxiv.org/html/2608.12953#bib.bib2),[15](https://arxiv.org/html/2608.12953#bib.bib1)\]\. Additionally, existing methods tend to rely on local and greedy heuristics that ignore inter\-component dependencies across layers and may discard components that appear weak individually but are globally important\. These design choices lead to three recurring failure modes: \(i\) calibration\-induced distributional bias\[[29](https://arxiv.org/html/2608.12953#bib.bib71)\]and high task\-wise deviations outside the calibration distribution \(Figure[1\(b\)](https://arxiv.org/html/2608.12953#S1.F1.sf2)shows deviation grows to30%30\\%\); \(ii\) irreversible local pruning decisions that cannot guarantee optimal architectural configurations, even with respect to their own importance estimates; and \(iii\) unreliable adherence to target compression ratios, undermining deployment on memory\-constrained systems \(Figure[1\(a\)](https://arxiv.org/html/2608.12953#S1.F1.sf1)highlights deviations upto33%33\\%\)\.

![Refer to caption](https://arxiv.org/html/2608.12953v1/tex/figures/budget_drift.png)\(a\)Target v/s actual compression ratio\.
![Refer to caption](https://arxiv.org/html/2608.12953v1/tex/figures/taskwise_variance.png)\(b\)Comparison of task\-wise standard deviation\.

Figure 1:\(a\) Existing structured pruners exhibit severe “capacity slack”, with the actual compression ratio deviating from the target by up to33%33\\%\.Sniperachieves near\-exact budget adherence with aCRAFTof0\.980\.98, compared to the baseline average of0\.780\.78\. Quantitative results are provided in Table[8](https://arxiv.org/html/2608.12953#S12.T8)of Appendix[12\.1](https://arxiv.org/html/2608.12953#S12.SS1)\. \(b\)Sniperreduces task\-wise standard deviation by up to60%60\\%relative to prior methods, mitigating the instability induced by existing pruning strategies\.We address these limitations in this work and summarize our contributions as follows:

- •We proposeSniper\(Structured Knapsack\-optimization\-basedPruner\), a novel dual\-axis structured pruning framework for LLMs\.Sniperfirst compresses along the depth\-axis by solving a 0/1 knapsack problem over coarse\-grained components, then applies a width\-pruning phase that lets the model hit the target compression ratio near\-perfectly\.
- •Unlike existing systems that rely on greedy and myopic pruning decisions,Sniperutilizes dynamic programming to guarantee a conditionally optimal selection of coarse\-grained components while pruning along the depth\-axis\.
- •We introduce the Compression Ratio Adherence Factor \(CRAFT\) to quantify budget fidelity, and show thatSniperachieves near\-exact compression targets, whereas existing structured pruners suffer from substantial capacity slack, thereby undermining utility\.
- •We construct a comprehensive evaluation suite spanning six modern baselines tested across 18 tasks, five domains, and four diverse architectures – including dense, reasoning\-specialized, fused\-MLP, and mixture\-of\-experts models\.Sniperconsistently achieves state\-of\-the\-art performance retention, and lower task\-level variance under multiple compression regimes\.

## 2Related Work

The escalating computational demands of Large Language Models \(LLMs\) have catalyzed extensive research into efficient deployment strategies\. Existing approaches generally fall into the categories of quantization\[[5](https://arxiv.org/html/2608.12953#bib.bib34),[33](https://arxiv.org/html/2608.12953#bib.bib33),[37](https://arxiv.org/html/2608.12953#bib.bib32),[22](https://arxiv.org/html/2608.12953#bib.bib31),[14](https://arxiv.org/html/2608.12953#bib.bib30)\], knowledge distillation\[[26](https://arxiv.org/html/2608.12953#bib.bib23),[30](https://arxiv.org/html/2608.12953#bib.bib24),[66](https://arxiv.org/html/2608.12953#bib.bib22)\], and activation sparsity\[[13](https://arxiv.org/html/2608.12953#bib.bib29),[38](https://arxiv.org/html/2608.12953#bib.bib28),[36](https://arxiv.org/html/2608.12953#bib.bib26),[74](https://arxiv.org/html/2608.12953#bib.bib25)\]\. While effective, these methods typically preserve the original model architecture\. In contrast,model pruningexplicitly removes parameters to achieve tangible inference speedups and memory reductions\.

### 2\.1Unstructured Model Pruning

Early research predominantly focused onunstructuredorsemi\-structuredsparsity, identifying individual weights for removal based on magnitude\[[24](https://arxiv.org/html/2608.12953#bib.bib42),[61](https://arxiv.org/html/2608.12953#bib.bib73)\]or second\-order gradient information\[[25](https://arxiv.org/html/2608.12953#bib.bib39),[17](https://arxiv.org/html/2608.12953#bib.bib41),[64](https://arxiv.org/html/2608.12953#bib.bib70)\]\. Although methods like SparseGPT\[[17](https://arxiv.org/html/2608.12953#bib.bib41)\]and Wanda\[[61](https://arxiv.org/html/2608.12953#bib.bib73)\]achieve high sparsity with minimal perplexity degradation, they typically produce irregular sparse matrices\. Consequently, specialized hardware accelerators or N:M sparsity support are often required to translate theoretical FLOPs reductions into actual latency gains\[[44](https://arxiv.org/html/2608.12953#bib.bib63)\]\.

### 2\.2Structured Pruning for LLMs

To circumvent the hardware dependencies of unstructured sparsity, recent work has shifted towardsstructured pruning, which removes coherent architectural units\. We categorize these approaches based on their granularity:

#### Width Pruning\.

These methods prune along the channel or embedding dimension, effectively narrowing the model’s matrices\. Techniques range from pruning individual attention heads\[[65](https://arxiv.org/html/2608.12953#bib.bib37),[42](https://arxiv.org/html/2608.12953#bib.bib38),[23](https://arxiv.org/html/2608.12953#bib.bib27)\]to removing entire neurons in MLPs\[[57](https://arxiv.org/html/2608.12953#bib.bib65)\]\. LLM\-Pruner\[[39](https://arxiv.org/html/2608.12953#bib.bib44)\]advances this by constructing a dependency graph to ensure structurally coupled parameters are pruned simultaneously\. Similarly, SliceGPT\[[4](https://arxiv.org/html/2608.12953#bib.bib43)\]projects weight matrices into a lower\-dimensional space via PCA, effectively slicing rows and columns\. Other decomposition\-based methods, such as FWSVD\[[27](https://arxiv.org/html/2608.12953#bib.bib59)\], ASVD\[[73](https://arxiv.org/html/2608.12953#bib.bib56)\], and SVD\-LLM\[[67](https://arxiv.org/html/2608.12953#bib.bib45)\], adopt greedy strategies to select and drop redundant neuron blocks from MLP and self\-attention modules\. While effective for fine\-grained compression, width pruning often struggles to fully excise large, redundant computational blocks\.

#### Depth Pruning\.

Operating at a coarser granularity, depth pruning removes entire transformer layers\[[20](https://arxiv.org/html/2608.12953#bib.bib68),[7](https://arxiv.org/html/2608.12953#bib.bib67),[8](https://arxiv.org/html/2608.12953#bib.bib66)\]\. Approaches such as ShortGPT\[[40](https://arxiv.org/html/2608.12953#bib.bib40)\]and SLEB\[[60](https://arxiv.org/html/2608.12953#bib.bib36)\]identify and discard redundant layers using block influence metrics\. ReplaceMe\[[59](https://arxiv.org/html/2608.12953#bib.bib35)\]mitigates the representation collapse caused by layer removal by substituting contiguous blocks with a learned linear projection\. Dynamic schemes, such as PuDDing\[[69](https://arxiv.org/html/2608.12953#bib.bib69)\], learn input\-conditioned routing policies for selective layer skipping at inference time\. Recent hybrid approaches\[[72](https://arxiv.org/html/2608.12953#bib.bib62),[8](https://arxiv.org/html/2608.12953#bib.bib66)\]attempt to jointly prune across neurons, heads, and layers, guided by learned or internal importance signals\.

### 2\.3Limitations of Prior Work

Most aforementioned structured approaches share a critical limitation: reliance ongreedyselection criteria\. Decisions are typically made locally \(e\.g\., layer\-by\-layer\) without an optimality\-centric view of the parameter\-performance trade\-off\. Furthermore, methods tend to not adhere to the required compression budget, introducing significant “capacity slack”, with deviations of up to33%33\\%from the target budget \(Figure[1\(a\)](https://arxiv.org/html/2608.12953#S1.F1.sf1)\)\.Sniperaddresses these gaps by pruning along two axes: first along the depth wherein component selection is driven by aconditionally optimalknapsack optimization algorithm, followed by a width\-pruning stage that ensures that the target compression budget is met with high accuracy\.

## 3Methodology

![Refer to caption](https://arxiv.org/html/2608.12953v1/tex/figures/pruner_schematic.png)Figure 2:A schematic overview ofSniperwhich performs structured pruning in two stages under a strict target compression ratio \(CR\)\.Stage 1: Coarse pruningremoves entire redundant components such as full transformer layers or attention blocks, enabling notable efficiency gains while preserving architectural coherence\.Stage 2: Fine pruningthen operates within the remaining layers, selectively pruning the internal dimensions of the model’s MLPs to precisely meet the target CR\. This dual\-axis strategy allowsSniperto combine the efficiency of depth pruning with the precision of width pruning, eliminating capacity slack and avoiding greedy, layer\-wise decisions\.In this section, we provide a detailed account of the dual\-stage pruning mechanism utilized bySniper, as illustrated in Figure[2](https://arxiv.org/html/2608.12953#S3.F2)\.

### 3\.1Problem Formulation

Consider an LLMℳ\\mathcal\{M\}consisting of a sequence ofLLtransformer layers\. We decompose the model into a set of discrete, prunable components𝒳\\mathcal\{X\}\. Specifically, we adopt amixed\-granularitydecomposition wherein each transformer layerllis decomposed into its constituent sub\-blocks: the Multi\-Head Attention block \(AlA\_\{l\}\) and the Feed\-Forward Network block \(MlM\_\{l\}\)\. Thus, the universe of items is defined as:𝒳=⋃l=1L\{Al,Ml\}∪\{Tl\}\\mathcal\{X\}=\\bigcup\_\{l=1\}^\{L\}\\\{A\_\{l\},M\_\{l\}\\\}\\cup\\\{T\_\{l\}\\\}, whereTlT\_\{l\}represents the atomic retention of theentirelayerll\. Since retainingTlT\_\{l\}effectively implies retaining bothAlA\_\{l\}andMlM\_\{l\}, it follows that for any layerll, the choice is mutually exclusive between selecting the composite itemTlT\_\{l\}or a subset of its components\{Al,Ml\}\\\{A\_\{l\},M\_\{l\}\\\}\. Each componentx∈𝒳x\\in\\mathcal\{X\}is associated with a weightw⁡\(x\)∈ℤ\+w\(x\)\\in\\mathbb\{Z\}^\{\+\}, representing its parameter count, and a valuev⁡\(x\)∈ℝv\(x\)\\in\\mathbb\{R\}, representing its contribution to model performance \(Section[3\.3](https://arxiv.org/html/2608.12953#S3.SS3)\)\. Given a target parameter budgetCC, our objective is to identify the subset𝒮∗⊆𝒳\\mathcal\{S\}^\{\*\}\\subseteq\\mathcal\{X\}that maximizes total importance while satisfying the capacity constraint:

𝒮∗=arg​max𝒮⊆𝒳∑x∈𝒮v\(x\)s\.t\.∑x∈𝒮w\(x\)≤C\\mathcal\{S\}^\{\*\}=\\argmax\_\{\\mathcal\{S\}\\subseteq\\mathcal\{X\}\}\\sum\_\{x\\in\\mathcal\{S\}\}v\(x\)\\hskip 10\.00002pt\\text\{s\.t\.\}\\hskip 10\.00002pt\\sum\_\{x\\in\\mathcal\{S\}\}w\(x\)\\leq C\(1\)
This formulation maps the coarse\-grained pruning stage directly to the 0/1 Knapsack Problem\[[50](https://arxiv.org/html/2608.12953#bib.bib57)\]\.

### 3\.2Stage 1: Coarse\-Grained Pruning via Dynamic Programming

To solve the optimization problem in Equation[1](https://arxiv.org/html/2608.12953#S3.E1), we employ a dynamic programming approach\. Let the components be indexed sequentially \(due to the inherent hierarchy in a typical transformer model, the components can be deterministically sequenced\)i=1,…,Ni=1,\\dots,N, whereN=\|𝒳\|N=\|\\mathcal\{X\}\|\. We define the state functionf⁡\(i,j\)f\(i,j\)as the maximum importance value achievable using a subset of the firstiiitems subject to a capacity limitjj\. The recurrence relation is defined as:

f⁡\(i,j\)=\{f⁡\(i−1,j\)if​w​\(xi\)\>jmax⁡\(f⁡\(i−1,j\),Δ\)if​w​\(xi\)≤jf\(i,j\)=\\begin\{cases\}f\(i\-1,j\)&\\text\{if \}w\(x\_\{i\}\)\>j\\\\ \\max\\left\(f\(i\-1,j\),\\Delta\\right\)&\\text\{if \}w\(x\_\{i\}\)\\leq j\\end\{cases\}\(2\)where,Δ=f⁡\(i−1,j−w⁡\(xi\)\)\+v⁡\(xi\)\\Delta=f\(i\-1,j\-w\(x\_\{i\}\)\)\+v\(x\_\{i\}\)\. The first case corresponds to excluding itemxix\_\{i\}due to capacity violation, while the second case evaluates the trade\-off between exclusion and inclusion\. To enforce the mutual exclusivity constraint \(i\.e\., one cannot select bothTlT\_\{l\}andAlA\_\{l\}\), we group mutually exclusive items and process them as a single decision step with multiple branches\.

Notably, while this algorithm originally runs inΘ⁡\(N⋅C\)\\Theta\(N\\cdot C\)time which is intractable whenCCis in the order of billions, we improve its tractability by discretizing all parameter counts by a discretizing factor,α\\alpha\. This reduces the algorithm’s time complexity toΘ⁡\(N​⌊Cα⌋\)\\Theta\(N\\lfloor\\frac\{C\}\{\\alpha\}\\rfloor\)\. As expected, increasingα\\alphamakes the algorithm faster at the cost of assigning less precise \(more coarsely rounded\) parameter\-count values to each component, leading to a suboptimal knapsack solution or a final compression ratio that deviates further from the target\. We provide a detailed analysis of the same in Appendix[6\.5](https://arxiv.org/html/2608.12953#S6.SS5)\.

Upon computing the terminal statef⁡\(N,⌊Cα⌋\)f\(N,\\lfloor\\frac\{C\}\{\\alpha\}\\rfloor\), we backtrack to recover the optimal set𝒮∗\\mathcal\{S\}^\{\*\}\. We note thatS∗S^\{\*\}does not represent a global optimum over all possible pruning configurations but rather aconditional optimumwith respect to fixed component\-level importance estimates\. Despite being a weaker guarantee of optimality, it surpasses what existing greedy methods provide since they do not offer any optimality guarantee whatsoever, conditional or otherwise\.

### 3\.3Iterative Importance Estimation

A critical challenge in Knapsack\-based pruning is assigning static importance valuesv⁡\(xi\)v\(x\_\{i\}\)to components that interact non\-linearly\. To address this, we propose an iterative importance estimation scheme based on marginal contribution\. Letℳ\(i−1\)\\mathcal\{M\}^\{\(i\-1\)\}denote the model pruned up to componenti−1i\-1\. We define the importance of componentxix\_\{i\}as the divergence induced by removing it fromℳ\(i−1\)\\mathcal\{M\}^\{\(i\-1\)\}\. Let𝐙∈ℝB×V\\mathbf\{Z\}\\in\\mathbb\{R\}^\{B\\times V\}be the logits produced by the full model on a calibration batch𝐗\\mathbf\{X\}, whereBBandVVare the batch size and vocabulary size of the model, respectively\. For theii\-th component, we evaluate two scenarios:

1. 1\.Retention:The component is kept\. Let𝐙retain\(i\)\\mathbf\{Z\}\_\{\\text\{retain\}\}^\{\(i\)\}be the logits of the model wherexix\_\{i\}is present\.
2. 2\.Drop:The component is pruned\. Let𝐙drop\(i\)\\mathbf\{Z\}\_\{\\text\{drop\}\}^\{\(i\)\}be the logits of the model wherexix\_\{i\}is removed\.

The importancev⁡\(xi\)v\(x\_\{i\}\)is quantified as the marginal degradation in prediction fidelity:

v⁡\(xi\)=‖𝐙−𝐙drop\(i\)‖22−‖𝐙−𝐙retain\(i\)‖22v\(x\_\{i\}\)=\\\|\\mathbf\{Z\}\-\\mathbf\{Z\}\_\{\\text\{drop\}\}^\{\(i\)\}\\\|\_\{2\}^\{2\}\-\\\|\\mathbf\{Z\}\-\\mathbf\{Z\}\_\{\\text\{retain\}\}^\{\(i\)\}\\\|\_\{2\}^\{2\}\(3\)
Unlike greedy parameter importance estimation metrics that capture the importance of individual components in isolation, Equation[3](https://arxiv.org/html/2608.12953#S3.E3)ensures that the importance scores reflect the effect of jointly pruning multiple components, thereby making the scoring function more reliable and representative\. We note thatSniperis similar to existing methods in that it utilizes heuristic proxies for component importance\. However, it greatly improves upon these baselines by not making pruning decisions greedily and instead, computingprovably optimalcompression configurations with respect to these importance scores\.

### 3\.4Stage 2: Fine\-Grained Residual Pruning

The discrete nature of components in Stage 1 often leaves a residual capacityΔ​C\>0\\Delta C\>0\. To fully utilize the budgetCC, we introduce a second stage of fine\-grained structured pruning targeting the MLPs within the transformer blocks\.

#### Budget Allocation\.

We distribute the fine\-grained pruning budgetNt​o​t​a​lN\_\{total\}\(total MLP columns to remove\) across layers inversely proportional to their coarse importance\. Let𝐮∈ℝL\\mathbf\{u\}\\in\\mathbb\{R\}^\{L\}be the vector of layer importance scores whereul=v⁡\(Tl\)u\_\{l\}=v\(T\_\{l\}\)\. We define the layer\-wise pruning ratioρl\\rho\_\{l\}via a softmax over negated importance:

ρl=exp\(−ul/τ\)∑k=1Lexp\(−uk/τ\)\\rho\_\{l\}=\\frac\{\\exp\(\-u\_\{l\}/\\tau\)\}\{\\sum\_\{k=1\}^\{L\}\\exp\(\-u\_\{k\}/\\tau\)\}\(4\)whereτ\\tau\(defaulted to 1\) is a temperature parameter\. The number of columns to prune from layerllis computed asNl=⌊ρl⋅Nt​o​t​a​l⌋N\_\{l\}=\\lfloor\\rho\_\{l\}\\cdot N\_\{total\}\\rfloor\.

#### Column Selection\.

Modern LLMs\[[19](https://arxiv.org/html/2608.12953#bib.bib18),[51](https://arxiv.org/html/2608.12953#bib.bib54)\]utilize variants of the Gated Linear Unit \(GLU\), where the MLP computation is defined as:

MLP​\(𝐗\)=\(σ⁡\(𝐗𝐆T\)⊙\(𝐗𝐔T\)\)​𝐃T\\text\{MLP\}\(\\mathbf\{X\}\)=\(\\sigma\(\\mathbf\{X\}\\mathbf\{G\}^\{T\}\)\\odot\(\\mathbf\{X\}\\mathbf\{U\}^\{T\}\)\)\\mathbf\{D\}^\{T\}\(5\)where𝐔,𝐆∈ℝdm​o​d​e​l×df​f\\mathbf\{U\},\\mathbf\{G\}\\in\\mathbb\{R\}^\{d\_\{model\}\\times d\_\{ff\}\}are the up\- and gate\-projections, respectively, and𝐃∈ℝdf​f×dm​o​d​e​l\\mathbf\{D\}\\in\\mathbb\{R\}^\{d\_\{ff\}\\times d\_\{model\}\}is the down\-projection\. Here,dm​o​d​e​ld\_\{model\}refers to the model’s hidden dimension anddf​fd\_\{ff\}is the higher dimension to which embeddings are projected within its MLPs\. To maintain structural consistency, pruning a column indexjjrequires removing thejj\-th column of𝐔\\mathbf\{U\}and𝐆\\mathbf\{G\}, and thejj\-th row of𝐃\\mathbf\{D\}\. We define the sensitivity of thejj\-th neuron index as the magnitude of its contribution to the output:

Ωj=\|𝐃j,:⋅\(𝐆:,j⊙𝐔:,j\)\|\\Omega\_\{j\}=\|\\mathbf\{D\}\_\{j,:\}\\cdot\(\\mathbf\{G\}\_\{:,j\}\\odot\\mathbf\{U\}\_\{:,j\}\)\|\(6\)Indices with the lowestΩj\\Omega\_\{j\}scores are pruned untilNlN\_\{l\}columns are removed\. This ensures that the residual capacity is filled with the least salient parameters, maxing out the available budget\.

By integrating the conditionally optimal configurations computed by the Knapsack solver with the precision of fine\-grained residual pruning,Sniperdemonstrates high retention capabilities with respect to the unpruned model, while also ensuring high\-fidelity adherence to the parameter budget, as reflected by its excellent CRAFTs \(Appendix[12\.1](https://arxiv.org/html/2608.12953#S12.SS1)\)\.

## 4Experimental Setup

#### Pruned models\.

To evaluate architectural generality, we benchmarkSniperon a diverse set of modern LLMs111All pretrained checkpoints are obtained from the HuggingFace model hub\.with different architectural paradigms\. These include the instruction\-tuned LLaMA\-3\.1\-8B\-Instruct\[[19](https://arxiv.org/html/2608.12953#bib.bib18)\], the reasoning\-oriented Qwen3\-8B\[[71](https://arxiv.org/html/2608.12953#bib.bib19)\], Phi\-4\[[1](https://arxiv.org/html/2608.12953#bib.bib17)\], which employs fused MLP projections, and the Mixture\-of\-Experts model GPT\-OSS\-20B\[[47](https://arxiv.org/html/2608.12953#bib.bib16)\]\.

#### Baselines\.

We compareSniperagainst six contemporary structured pruning baselines\. Width\-based methods include SliceGPT\[[4](https://arxiv.org/html/2608.12953#bib.bib43)\]and LLM\-Pruner\[[39](https://arxiv.org/html/2608.12953#bib.bib44)\], while depth\-based methods include ReplaceMe\[[59](https://arxiv.org/html/2608.12953#bib.bib35)\], SLEB\[[60](https://arxiv.org/html/2608.12953#bib.bib36)\], and ShortGPT\[[40](https://arxiv.org/html/2608.12953#bib.bib40)\]\. We also include 2SSP\[[56](https://arxiv.org/html/2608.12953#bib.bib3)\]since it mirrorsSnipervia its dual\-axis approach, allowing us to evaluateSniper’s effectiveness with respect to analogous pruning approaches\. Several baselines are re\-engineered to support recent LLM architectures222For instance, ReplaceMe and ShortGPT do not natively support newer LLM variants\.\. We note that vector\-space width pruners such as SliceGPT are inherently incompatible with expert routing layers and therefore cannot be applied to GPT\-OSS\. In order to ensure a fair evaluation, we adjust the compression ratios of all methods to yield pruned models of comparable sizes\. We provide all implementation details in Appendix[10](https://arxiv.org/html/2608.12953#S10)\.

#### Evaluation task suite\.

Many pruning evaluations rely on a narrow set of tasks, which can obscure degradation in a model’s linguistic capabilities, reasoning, or alignment\. For a more comprehensive assessment, we evaluate all models on a testbed of 18 tasks spanning five domains\. Generative performance is measured using log\-perplexity on WikiText\-2\[[41](https://arxiv.org/html/2608.12953#bib.bib46)\]and LAMBADA\[[48](https://arxiv.org/html/2608.12953#bib.bib64)\]\. World understanding is evaluated using PIQA\[[6](https://arxiv.org/html/2608.12953#bib.bib61)\], PROST\[[3](https://arxiv.org/html/2608.12953#bib.bib12)\], and CommonsenseQA\[[62](https://arxiv.org/html/2608.12953#bib.bib11)\]\. Domain\-specific knowledge covers STEM reasoning tasks including ARC\-Easy and ARC\-Challenge\[[10](https://arxiv.org/html/2608.12953#bib.bib14)\], mathematical reasoning tasks such as MathQA\[[2](https://arxiv.org/html/2608.12953#bib.bib15)\]and OpenBookQA\[[43](https://arxiv.org/html/2608.12953#bib.bib75)\], and medical question answering using MedQA\[[31](https://arxiv.org/html/2608.12953#bib.bib13)\]\. Natural language understanding is evaluated using BLiMP\[[68](https://arxiv.org/html/2608.12953#bib.bib10)\], BoolQ\[[9](https://arxiv.org/html/2608.12953#bib.bib9)\], Winogrande\[[55](https://arxiv.org/html/2608.12953#bib.bib60)\], and CoQA\[[53](https://arxiv.org/html/2608.12953#bib.bib48)\]\. To assess alignment preservation, we additionally evaluate safety and ethics using TruthfulQA\[[35](https://arxiv.org/html/2608.12953#bib.bib49)\], Winogender\[[54](https://arxiv.org/html/2608.12953#bib.bib50)\], and Moral Stories\[[16](https://arxiv.org/html/2608.12953#bib.bib51)\]\. A detailed description of each task is provided in Appendix[11](https://arxiv.org/html/2608.12953#S11)\.

## 5Results

Table[1](https://arxiv.org/html/2608.12953#S5.T1)comparesSniperagainst state\-of\-the\-art structured pruning methods on LLaMA\-3\.1\-8B\-Instruct and Qwen3\-8B, under two compression ratios \(25% and 35%\), both with and without recovery fine\-tuning \(RFT\)\. Acrossall configurations except one,Sniperachieves the highest average retention performance \(Avg RP\) while maintaining strong performance across different task groups\. While it is marginally outperformed by ReplaceMe and ShortGPT on Llama\-3\.1\-8B\-Instruct \(25% compression with RFT\),Snipermaintains a near\-perfect mean rank of 1\.25 across all pruning configurations, yielding competitive performance even at its worst rank of merely 3 \(Tables[1](https://arxiv.org/html/2608.12953#S5.T1)and[2](https://arxiv.org/html/2608.12953#S5.T2)\)\. In contrast, every baseline has at least one configuration in which its performance deteriorates drastically\. Unlike its baselines,Sniperneverdisintegrates\. This indicatesSniper’s unparalleled robustness and generalization capabilities across different architectures and compression regimes\. We provide detailed per\-task results in Appendix[12\.2](https://arxiv.org/html/2608.12953#S12.SS2)\.

ModelCRMethodGenerativeWorldDomainNLU & NLISafetyAvg RP \(%\)Std RP \(%\)\(Log\-PPL↓\\downarrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Score↑\\uparrow\)\(↑\\uparrow\)\(↓\\downarrow\)LLaMA\-3\.1\-\-Base1\.6966\.5556\.2878\.5255\.56\-\-8B\-Instruct25%ReplaceMe4\.91 \(2\.36\)59\.30 \(58\.31\)40\.78 \(43\.35\)54\.97 \(75\.80\)55\.62 \(54\.79\)74\.86 \(88\.90\)20\.40 \(10\.50\)SliceGPT5\.11 \(3\.60\)40\.78 \(49\.10\)26\.87 \(30\.08\)53\.75 \(62\.49\)50\.23 \(49\.50\)62\.06 \(72\.49\)20\.20 \(17\.30\)LLM\-Pruner3\.96 \(2\.54\)40\.56 \(44\.37\)32\.14 \(36\.32\)49\.85 \(62\.43\)51\.87 \(50\.99\)64\.56 \(75\.11\)22\.40 \(17\.70\)SLEB3\.37 \(2\.25\)42\.20 \(47\.52\)37\.06 \(38\.79\)50\.78 \(65\.98\)51\.01 \(50\.74\)67\.91 \(78\.45\)20\.30 \(16\.80\)ShortGPT13\.20 \(2\.46\)48\.70 \(59\.22\)31\.68 \(42\.42\)34\.50 \(74\.77\)51\.01 \(55\.34\)57\.50 \(88\.47\)26\.10 \(11\.20\)2SSP2\.77 \(2\.13\)49\.33 \(54\.23\)36\.67 \(39\.75\)68\.14 \(71\.59\)50\.93 \(51\.42\)76\.65 \(83\.19\)14\.00 \(13\.90\)Sniper3\.15 \(2\.29\)51\.81 \(57\.89\)39\.00 \(41\.71\)64\.95 \(74\.41\)53\.02 \(53\.97\)77\.03\(87\.34\)12\.70\(10\.00\)35%ReplaceMe5\.86 \(3\.19\)38\.42 \(39\.81\)29\.01 \(29\.62\)36\.09 \(50\.55\)51\.24 \(49\.58\)56\.25 \(65\.16\)27\.10 \(22\.60\)SliceGPT6\.44 \(4\.51\)38\.43 \(44\.77\)25\.68 \(28\.41\)42\.72 \(55\.22\)50\.86 \(51\.23\)56\.74 \(68\.08\)22\.50 \(18\.60\)LLM\-Pruner4\.94 \(2\.94\)37\.18 \(38\.49\)29\.61 \(33\.17\)44\.60 \(58\.50\)51\.33 \(50\.28\)59\.66 \(69\.93\)23\.70 \(19\.40\)SLEB4\.47 \(2\.71\)39\.20 \(40\.57\)30\.99 \(32\.95\)44\.13 \(57\.87\)50\.87 \(51\.29\)60\.72 \(70\.58\)24\.10 \(20\.10\)ShortGPT13\.14 \(2\.94\)40\.77 \(43\.64\)28\.99 \(32\.50\)36\.52 \(69\.91\)55\.03 \(54\.67\)56\.14 \(76\.98\)28\.00 \(18\.70\)2SSP3\.77 \(2\.45\)37\.28 \(47\.62\)29\.39 \(35\.66\)55\.55 \(68\.03\)50\.50 \(50\.19\)64\.21 \(77\.26\)19\.10 \(15\.80\)Sniper4\.70 \(2\.63\)41\.16 \(48\.52\)32\.18 \(34\.73\)53\.52 \(68\.31\)51\.29 \(50\.95\)64\.96\(77\.26\)19\.50\(14\.80\)Qwen3\-8B\-Base2\.0166\.3258\.6076\.2055\.90\-\-25%ReplaceMe4\.37 \(2\.76\)37\.40 \(43\.98\)30\.21 \(36\.83\)41\.44 \(55\.52\)50\.47 \(50\.66\)59\.92 \(72\.30\)23\.80 \(18\.60\)SliceGPT15\.72 \(10\.75\)34\.01 \(34\.48\)24\.64 \(25\.05\)29\.46 \(29\.54\)51\.00 \(51\.35\)48\.68 \(50\.07\)27\.50 \(30\.00\)LLM\-Pruner3\.20 \(2\.46\)48\.73 \(53\.44\)31\.85 \(37\.28\)63\.89 \(67\.05\)50\.99 \(52\.20\)72\.64 \(80\.11\)15\.90 \(15\.00\)SLEB3\.75 \(2\.61\)43\.11 \(50\.41\)36\.93 \(38\.38\)48\.84 \(64\.50\)51\.48 \(53\.12\)67\.53 \(78\.89\)20\.20 \(13\.80\)ShortGPT8\.98 \(3\.14\)43\.76 \(55\.87\)33\.09 \(39\.34\)46\.98 \(67\.98\)55\.02 \(54\.14\)63\.03 \(80\.21\)22\.90 \(15\.20\)2SSP6\.37 \(3\.11\)34\.79 \(50\.10\)26\.31 \(33\.86\)39\.98 \(58\.74\)52\.19 \(52\.44\)56\.09 \(72\.92\)23\.10 \(16\.30\)Sniper3\.65 \(2\.72\)50\.81 \(55\.93\)36\.21 \(41\.36\)61\.04 \(69\.31\)54\.90 \(53\.70\)74\.41\(82\.64\)15\.00\(12\.20\)35%ReplaceMe5\.99 \(4\.20\)37\.63 \(38\.12\)32\.30 \(28\.86\)40\.78 \(52\.27\)51\.47 \(52\.59\)59\.40 \(64\.00\)25\.90 \(21\.60\)SliceGPT16\.13 \(10\.82\)33\.49 \(34\.39\)24\.78 \(24\.58\)29\.53 \(29\.87\)50\.99 \(52\.44\)48\.67 \(50\.17\)27\.70 \(30\.40\)LLM\-Pruner5\.25 \(3\.46\)39\.24 \(45\.25\)26\.47 \(31\.86\)50\.05 \(55\.02\)50\.75 \(51\.39\)60\.63 \(68\.34\)21\.90 \(19\.10\)SLEB4\.94 \(3\.10\)38\.46 \(45\.15\)31\.95 \(34\.40\)38\.99 \(53\.56\)51\.55 \(52\.43\)59\.67 \(70\.47\)24\.70 \(17\.50\)ShortGPT10\.67 \(3\.98\)45\.56 \(50\.25\)30\.96 \(34\.82\)38\.09 \(59\.54\)54\.02 \(53\.11\)58\.69 \(72\.27\)27\.10 \(18\.30\)2SSP7\.42 \(3\.56\)34\.71 \(46\.37\)25\.92 \(31\.94\)36\.63 \(56\.68\)51\.57 \(51\.68\)54\.15 \(69\.23\)24\.40 \(17\.50\)Sniper5\.11 \(3\.21\)44\.04 \(50\.81\)30\.43 \(35\.45\)48\.68 \(63\.35\)52\.76 \(52\.81\)63\.71\(75\.13\)20\.10\(15\.90\)

Table 1:Comparison of different pruning methods across various compression ratios \(CR\) without and \(with\) recovery fine\-tuning \(RFT\)\. We report the scores for each task group, the average retention performance \(RP\) \(ratio of pruned and base model performances\), as well as the standard deviation across tasks for each method\. Per\-task results are highlighted in Tables[9](https://arxiv.org/html/2608.12953#S12.T9),[10](https://arxiv.org/html/2608.12953#S12.T10),[11](https://arxiv.org/html/2608.12953#S12.T11)and[12](https://arxiv.org/html/2608.12953#S12.T12)of Appendix[12\.2](https://arxiv.org/html/2608.12953#S12.SS2)\.### 5\.1Consistency and robustness

Beyond average performance,Sniperexhibits substantially lower task\-level standard deviations \(Std RP\) acrossall configurations, even including settings where it does not outperform all baselines\. For example, at 35% compression on Qwen3\-8B with RFT,Sniperreduces task\-wise variance to15\.90%,15\.90\\%,compared to 17–30% for competing pruners\. This behavior highlightsSniper’s ability to avoid a common failure mode of greedy pruning strategies, which often over\-optimize for isolated metrics \(e\.g\., perplexity\) at the expense of broader task performance\. By combining the benefits of conditional optimality during coarse\-grained pruning with that of a budget\-filling fine\-grained stage,Sniperis able to maintain superior cross\-task consistency across diverse model architectures\.

Table 2:Ranking different pruning methods across all available configurations\.Table 3:Performance analysis on larger and architecturally diverse models – Phi\-4 and GPT\-OSS\-20B with RFT\. Detailed results provided in Tables[13](https://arxiv.org/html/2608.12953#S12.T13)and[14](https://arxiv.org/html/2608.12953#S12.T14)of Appendix[12\.2](https://arxiv.org/html/2608.12953#S12.SS2)\.
### 5\.2Results on larger and diverse architectures

Table[3](https://arxiv.org/html/2608.12953#S5.T3)evaluatesSniperon larger and architecturally distinct models, i\.e, Phi\-4\-14B and GPT\-OSS\-20B, under a 35% compression regime with RFT\. Despite these departures from standard dense transformers,Sniperconsistently attains the highest Avg RP while maintaining the lowest Std RP\. On Phi\-4\-14B,Sniperattains an Avg RP of81\.58%81\.58\\%, substantially outperforming ShortGPT and 2SSP by4\.4%4\.4\\%and11\.42%11\.42\\%, respectively\.Sniperdemonstrates similar superiority on GPT\-OSS\-20B, achieving an Avg RP of86\.99%86\.99\\%\-2\.31%2\.31\\%higher than 2SSP and16\.48%16\.48\\%better than ShortGPT\. Notably, on GPT\-OSS\-20B, 2SSP operates solely via its first stage due to how it distributes the pruning budget between its two stages, effectively turning its second stage into a no\-op\. In contrast,Sniperensures that it leverages both of its stages to yield a superior compressed model\.

### 5\.3Adherence to Compression Ratio

We analyze the effect of fixing the target compression ratio for each method and observing its adherence to the same\. We observe that existing methods demonstrate significant deviations of upto 33% from the target ratio, leading to severe under\-pruning and an erosion of trust in the pruning process \(Figure[1](https://arxiv.org/html/2608.12953#S1.F1)and Appendix[12\.1](https://arxiv.org/html/2608.12953#S12.SS1)\)\. In contrast,Sniperexhibits remarkable reliability by adhering strictly to these targets with a near\-idealCRAFTof 0\.98 across a wide range of budgets\. Unlike its baselines,Snipereliminates reliance on hit\-and\-trial strategies to obtain a model of a specific size, leading to significant practical advantages\.

### 5\.4Ablation onSniper

We provide additional ablation experiments in Table[4](https://arxiv.org/html/2608.12953#S5.T4)and Figure[3](https://arxiv.org/html/2608.12953#S5.F3)and analyze the effectiveness of various components and design choices ofSniper\.

#### Dual\-Axis Pruning\.

![Refer to caption](https://arxiv.org/html/2608.12953v1/tex/figures/ablation_study.png)Figure 3:Ablation study ofSnipershowing the effect of coarse and fine pruning stages\. Combining both stages yields the best performance and stability, while removing either component leads to lower performance retention or instability\.We ablateSniper’s coarse\- and fine\-grained components, measuring Avg RP and Std RP \(Figure[3](https://arxiv.org/html/2608.12953#S5.F3)\)\. Coarse pruning alone reaches a strong Avg RP of 97\.47%, but its high Std RP \(10\.50%\) reflects a key limitation: operating on entire structural groups rather than individual neurons causes some important neurons to inevitably be pruned, destabilizing downstream performance\. Its structured removals, however, preserve weight tensor shapes, making it the primary driver of inference speedup\. The fine\-grained variant attains a marginally better Avg RP \(97\.95%\) with far lower Std RP \(2\.90%\), confirming that precise, neuron\-level interventions better preserve cross\-task consistency\. However, it produces irregularly\-shaped weight tensors, yielding little inference speedup\. The two stages are thus complementary: coarse pruning delivers hardware\-friendly speedups at the cost of consistency, while fine\-grained pruning restores consistency but lacks efficiency gains due to induced irregularities\. Additionally, random importance assignment yields a low Avg RP \(63\.24%\) and high Std RP \(26\.90%26\.90\\%\), confirming the necessity of principled importance estimation\.

Table 4:Additional ablation results forSniperobtained after compressing Qwen3\-8B by 25% \(with RFT\)\.
#### Column Selection Heuristic\.

We analyze the efficacy of our column selection heuristic \(Equation[6](https://arxiv.org/html/2608.12953#S3.E6)\) by replacing it with two popular alternatives:\(a\)magnitude\-based\[[24](https://arxiv.org/html/2608.12953#bib.bib42)\]and\(b\)activation\-aware\[[61](https://arxiv.org/html/2608.12953#bib.bib73)\]pruning, corresponding to theSniper\-MAG andSniper\-ACT variants in Table[4](https://arxiv.org/html/2608.12953#S5.T4), respectively\. Both variants perform approximately 2–4% worse thanSniperon average and exhibit 13% higher per\-task deviation, demonstrating the effectiveness of our strategy at maintaining strong and stable performance\.

#### Iterative Pruning for Importance Estimation\.

To estimate component importance,Snipercomputes the marginal degradation in performance incurred by dropping a component from a partially pruned model, whose pruning configuration is determined by our dynamic programming solver \(Section[3\.3](https://arxiv.org/html/2608.12953#S3.SS3)\)\. We compare this against the leave\-one\-out strategy employed by baselines such as SLEB\[[60](https://arxiv.org/html/2608.12953#bib.bib36)\], denotedSniper\-LeaveOneOut\. This variant incurs a significant drop of 20% in average performance and nearly twice the task\-wise standard deviation ofSniper, demonstrating that estimating component importance on iteratively optimal pruning configurations captures global inter\-component dependencies—something leave\-one\-out strategies fail to model due to their limited scope\.

#### Fine\-Grained Pruning Budget Distribution\.

To determine how many columns to drop from each MLP in Stage 2 \(Section[3\.4](https://arxiv.org/html/2608.12953#S3.SS4)\),Sniperuses the component importance scores from Section[3\.3](https://arxiv.org/html/2608.12953#S3.SS3)\(Equation[3](https://arxiv.org/html/2608.12953#S3.E3)\) to distribute the pruning budget inversely proportional to importance \(Equation[4](https://arxiv.org/html/2608.12953#S3.E4)\), pruning redundant MLPs more aggressively while compressing salient ones more conservatively\. We evaluate theSniper\-Uniform variant, which distributes the fine\-grained pruning budget uniformly across all MLPs\. As shown in Table[4](https://arxiv.org/html/2608.12953#S5.T4), this variant not only incurs a 1\.5% drop in average performance but, more critically, exhibits 12\.3% higher task\-wise standard deviation, indicating uneven performance retention across domains\.

## 6Discussion

\(a\)Calibration data sample size with Slim Orca\[[34](https://arxiv.org/html/2608.12953#bib.bib52)\]\.
\(b\)Impact of calibration dataset with sample size of 50\.

Table 5:Impact of calibration on compressed Qwen3\-8B with RFT\. Detailed results provided in Tables[15](https://arxiv.org/html/2608.12953#S12.T15)and[16](https://arxiv.org/html/2608.12953#S12.T16)of Appendix[12\.3](https://arxiv.org/html/2608.12953#S12.SS3)\.### 6\.1Impact of calibration on pruning robustness\.

We analyze the impact of calibration data on structured pruning by varying both*calibration set size*and*calibration distribution*using Qwen3\-8B with post\-pruning RFT \(Table[5](https://arxiv.org/html/2608.12953#S6.T5)\)\. As the number of calibration samples increases from 50 to 1000, 2SSP demonstrates a marked increase in its Avg RP \(72\.92%→75\.3172\.92\\%\\rightarrow 75\.31\), followed by a decline in the same \(75\.31%→74\.26%75\.31\\%\\rightarrow 74\.26\\%\), while its Std RP exhibits the reverse pattern \(16\.30%→15\.10%→15\.40%16\.30\\%\\rightarrow 15\.10\\%\\rightarrow 15\.40\\%\), indicating its relative instability and sensitivity to calibration sample count\. While ShortGPT remains comparatively stable in average performance, it consistently suffers from higher variance across task groups\. In contrast,Sniperachieves the best trade\-off between performance and consistency across all calibration sizes, reducing task\-wise standard deviation by an average of 11\.45% relative to ShortGPT\. A more noticeable trend is observed when varying calibration distributions: both ShortGPT and 2SSP show large performance fluctuations across Slim Orca\[[34](https://arxiv.org/html/2608.12953#bib.bib52)\], Alpaca\[[63](https://arxiv.org/html/2608.12953#bib.bib4)\], and C4\[[52](https://arxiv.org/html/2608.12953#bib.bib5)\], whereasSnipermaintains consistently high Avg RP and low Std RP\. These results indicate thatSniper’s pruning strategy is significantly less sensitive to calibration choices, further reinforcing its robustness with respect to its baselines\. Detailed results are provided in Appendix[12\.3](https://arxiv.org/html/2608.12953#S12.SS3)\.

Table 6:Transferability of importance scores across compression ratios\. We reuse importance scores computed at one compression ratio to prune the model at a different target ratio \(e\.g\., 35%→\\rightarrow25%\), and compare against recomputing scores from scratch \(Table[1](https://arxiv.org/html/2608.12953#S5.T1)\)\. Table[17](https://arxiv.org/html/2608.12953#S12.T17)of Appendix[12\.4](https://arxiv.org/html/2608.12953#S12.SS4)displays transferability metrics across each task\.
### 6\.2Transferability of importance scores\.

Table[6](https://arxiv.org/html/2608.12953#S6.T6)evaluates the robustness ofSniper’s importance scores when transferred across compression ratios\. AlthoughSniperis designed to recompute scores for each target budget, transferring them across ratios \(35%→\\rightarrow25% and vice versa\) results in only marginal degradation in average retention performance with a modest increase in task\-wise variance\. For example, the 35%→\\rightarrow25% transfer preserves nearly identical average RP \(74\.24% vs\. 74\.41%\), with the reverse exhibiting a similarly small drop \(63\.48% vs\. 63\.71%\)\. This stability indicates thatSniper’s importance signals reflect intrinsic structural salience rather than budget\-specific artifacts, while the slight variance increase highlights the benefit of recomputing scores for strict optimality\. Overall, transferability offers a favorable efficiency–robustness trade\-off, allowing users to bypassSniper’s most time\-intensive step, i\.e, importance computation, with minimal performance sacrifice\. We provide task\-wise results in Appendix[12\.4](https://arxiv.org/html/2608.12953#S12.SS4)\.

### 6\.3Interpretable structural salience across layers\.

![Refer to caption](https://arxiv.org/html/2608.12953v1/consistency_heatmap_colored.png)Figure 4:Layer\-wise importance scores and pruning decisions for Phi\-4\-14B across multiple compression ratios\. The substantial overlap in pruned layers across compression ratios indicates stable, interpretable importance estimates that reflect intrinsic architectural redundancy rather than budget\-specific noise\.Figure[4](https://arxiv.org/html/2608.12953#S6.F4)highlights an important qualitative property ofSniper: beyond improved performance, it yields an interpretable measure of layer\-wise importance\. The learned scores exhibit a clear structure — consistently high importance assigned to early and late transformer layers and systematically lower importance in intermediate layers — with pruned layers predominantly drawn from the middle while boundary layers are preserved\. Moreover, substantial overlap in evicted layers across compression budgets indicates that importance estimates capture stable architectural salience rather than budget\-specific noise\. This structured pattern aligns with known functional roles of transformer layers, suggesting thatSniperprovides a principled, human\-interpretable signal for understanding redundancy and capacity allocation in LLMs\.

![Refer to caption](https://arxiv.org/html/2608.12953v1/tex/figures/runtime_comparison.png)\(a\)Comparison of pruning time\.
![Refer to caption](https://arxiv.org/html/2608.12953v1/tex/figures/latency.png)\(b\)Post\-pruning inference speed comparison\.

Figure 5:Pruning runtime \(a\) and post\-pruning inference speed \(b\) for different structured pruning method\. All benchmarking conducted on Qwen3\-8B model at 25% compression\.
### 6\.4Runtime analysis, latency, and practical utility

Figure[5\(a\)](https://arxiv.org/html/2608.12953#S6.F5.sf1)compares the end\-to\-end runtime of representative depth\-, width\-, and mixed\-granularity pruning methods on Qwen3\-8B\. Greedy depth pruners such as SLEB are the fastest \(≈\\approx23 minutes\) as importance estimation is performed over a small number of coarse atomic units, whereas width\-based methods such as SliceGPT incur substantially higher runtime \(≈\\approx55 minutes\) due to computations over large sets of fine\-grained elements\.Sniperincurs a total runtime that is only about 4 minutes higher than depth\-only methods, with approximately 96% of runtime devoted to importance estimation while the actual pruning step requires only 56 seconds\. This separation enables amortization of the dominant cost across multiple pruning runs by reusing importance scores \(Appendix[6\.2](https://arxiv.org/html/2608.12953#S6.SS2)\), while requiring GPU acceleration only during importance estimation\. We further analyze post\-pruning inference speedup at 25% compression in Figure[5\(b\)](https://arxiv.org/html/2608.12953#S6.F5.sf2)\. Width pruners fall below the base model due to hardware\-unfriendly irregular tensor shapes whereas pure depth pruners achieve the largest raw speedups by eliminating entire compute blocks\. Furthermore, both dual\-axis approaches yield notable latency improvements by offsetting irregular tensor penalties through component elimination\. We note that whileSniperlags behind 2SSP in realized efficiency gains \(16\.24 vs\. 20\.26 toks/s\), it compensates with consistently strong performance and robustness across architectures and pruning regimes\. We hypothesize that 2SSP’s latency gains stem from its treatment of attention blocks as the sole coarse\-grained component, whereasSniperalso considers MLP blocks at this granularity, trading some throughput for more balanced structural coverage\.

Table 7:Quantitative analysis of the effect of different values of the discretizing factor,α\\alpha, on CRAFT and algorithm runtime\. These results were obtained on Qwen3\-8B\.
### 6\.5Analysis of Different Values of Discretizing Factor

As demonstrated in Table[7](https://arxiv.org/html/2608.12953#S6.T7),Sniperis robust to a large range of values forα\\alphaand maintains a near\-perfect CRAFT of 0\.98 for all values from 8 to 32768\. As expected, the runtime decreases asα\\alphaincreases\. However, asα\\alphareaches exceptionally high values such as 100,000, it causes large components to be discretized into the same bucket as far smaller components, leading to a notable over\-shooting of the compression budget\. Therefore, to ensureα\\alphageneralizes well across architectures while keeping runtime low, we select a conservative value ofα=32\\alpha=32\. Notably, for allα\\alphain the range 8–32768, the resulting pruned models are identical and therefore yield the same performance\.

## 7Conclusion

We presentedSniper, a dual\-axis structured pruning framework that compresses along the model’s depth via a knapsack\-based objective, guaranteeing a conditionally optimal selection of coarse\-grained components and follows that with a width\-pruning stage\. Through its two stage operation,Sniperachieves precise budget adherence, strong performance retention, and reduced task\-level variance across diverse model architectures\. Extensive experiments demonstrate consistent improvements over state\-of\-the\-art structured pruners while performing competitively even under aggressive compression\. Beyond performance gains,Sniperoffers a practical deployment advantage by enabling amortized pruning through reusable importance estimates under strict budget constraints\. Together, these results underscore the value of non\-greedy optimization as a principled foundation for reliable and deployable LLM pruning\.

## 8Limitations

Snipercurrently employs homogeneous pruning signals across all model components\. Incorporating non\-homogeneous, expert\-specific signals for Mixture\-of\-Experts \(MoE\) architectures represents a natural extension that could further enhance pruning efficacy in such settings\. Additionally, whileSniperprovides conditional optimality guarantees during component removal, extending theoretical analogous guarantees to other stages of the pipeline such as importance estimation, remains a compelling direction for future work\.

## 9Ethical Considerations

This work presentsSniper, a dual\-axis structured pruning framework for compressing LLMs\. We reflect on the ethical implications of our research below\.

#### Utilization of Public Artifacts\.

In this work, we employ several publicly available models such as Qwen3\-8B\[[71](https://arxiv.org/html/2608.12953#bib.bib19)\], LLaMA\-3\.1\-8B\-Instruct\[[19](https://arxiv.org/html/2608.12953#bib.bib18)\], Phi\-4\[[1](https://arxiv.org/html/2608.12953#bib.bib17)\], and GPT\-OSS\-20B\[[47](https://arxiv.org/html/2608.12953#bib.bib16)\]\. We also use the publicly available Slim Orca\[[34](https://arxiv.org/html/2608.12953#bib.bib52)\]dataset for pre\-compression calibration and post\-compression RFT\. Furthermore, we utilize the Alpaca\[[63](https://arxiv.org/html/2608.12953#bib.bib4)\]and C4\[[52](https://arxiv.org/html/2608.12953#bib.bib5)\]datasets to test the effect of calibration data distribution on the pruning decisions made by each method\. We strictly adhere to the terms of use of each of the artifacts used by us\.

#### Intended Use\.

Sniperis designed to prune LLMs and make them more accessible to practitioners working in resource\-constrained environments\. However, we recognize that increasing accessibility also increases the likelihood of a pruned model being misused\. Therefore, we implore users of our framework to use compressed models by adhering to the usage policies of the respective base models\.

#### Environmental Impact\.

As LLMs continue to grow in size, the resources required to run them have reached a concerning point, and will likely continue to grow\. However, through model compression, we primarily aim to limit the amount of such required resources, including electricity for powering machines that run LLMs and water for cooling them\. We believe that this work will have a positive effect on the environment by making it possible to run more economical variants of resource\-intensive LLMs\.

## Acknowledgment

T\. Chakraborty acknowledges the support of the Microsoft Research India Research Grant and the Rajiv Khemani Young Faculty Chair Professorship in Artificial Intelligence\.

## References

- \[1\]M\. Abdin, J\. Aneja, H\. Behl, S\. Bubeck, R\. Eldan, S\. Gunasekar, M\. Harrison, R\. J\. Hewett, M\. Javaheripi, P\. Kauffmann, J\. R\. Lee, Y\. T\. Lee, Y\. Li, W\. Liu, C\. C\. T\. Mendes, A\. Nguyen, E\. Price, G\. de Rosa, O\. Saarikivi, A\. Salim, S\. Shah, X\. Wang, R\. Ward, Y\. Wu, D\. Yu, C\. Zhang, and Y\. Zhang\(2024\)Phi\-4 technical report\.External Links:2412\.08905,[Link](https://arxiv.org/abs/2412.08905)Cited by:[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px1.p1.1),[§9](https://arxiv.org/html/2608.12953#S9.SS0.SSS0.Px1.p1.1)\.
- \[2\]\(2019\)MathQA: towards interpretable math word problem solving with operation\-based formalisms\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),Minneapolis, Minnesota,pp\.2357–2367\.External Links:[Link](https://aclanthology.org/N19-1245),[Document](https://dx.doi.org/10.18653/v1/N19-1245)Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px3.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[3\]S\. Aroca\-Ouellette, C\. Paik, A\. Roncone, and K\. Kann\(2021\)PROST: Physical reasoning about objects through space and time\.InFindings of the Association for Computational Linguistics: ACL\-IJCNLP 2021,C\. Zong, F\. Xia, W\. Li, and R\. Navigli \(Eds\.\),Online,pp\.4597–4608\.External Links:[Link](https://aclanthology.org/2021.findings-acl.404/),[Document](https://dx.doi.org/10.18653/v1/2021.findings-acl.404)Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[4\]S\. Ashkboos, M\. L\. Croci, M\. G\. do Nascimento, T\. Hoefler, and J\. Hensman\(2024\)SliceGPT: compress large language models by deleting rows and columns\.External Links:2401\.15024,[Link](https://arxiv.org/abs/2401.15024)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px2.p1.1)\.
- \[5\]A\. Bhandare, V\. Sripathi, D\. Karkada, V\. Menon, S\. Choi, K\. Datta, and V\. Saletore\(2019\)Efficient 8\-bit quantization of transformer neural machine language translation model\.External Links:1906\.00532,[Link](https://arxiv.org/abs/1906.00532)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p1.1),[§2](https://arxiv.org/html/2608.12953#S2.p1.1)\.
- \[6\]Y\. Bisk, R\. Zellers, R\. L\. Bras, J\. Gao, and Y\. Choi\(2020\)PIQA: reasoning about physical commonsense in natural language\.InThirty\-Fourth AAAI Conference on Artificial Intelligence,Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[7\]X\. Chen, Y\. Hu, J\. Zhang, Y\. Wang, C\. Li, and H\. Chen\(2025\)Streamlining redundant layers to compress large language models\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=IC5RJvRoMp)Cited by:[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px2.p1.1)\.
- \[8\]Y\. Chen, B\. Cheng, J\. Han, Y\. Zhang, Y\. Li, and S\. Zhang\(2025\)DLP: dynamic layerwise pruning in large language models\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=11id5ppGZ8)Cited by:[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px2.p1.1)\.
- \[9\]C\. Clark, K\. Lee, M\. Chang, T\. Kwiatkowski, M\. Collins, and K\. Toutanova\(2019\)BoolQ: exploring the surprising difficulty of natural yes/no questions\.InProceedings of the 2019 Conference of the No rth American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),J\. Burstein, C\. Doran, and T\. Solorio \(Eds\.\),Minneapolis, Minnesota,pp\.2924–2936\.External Links:[Link](https://aclanthology.org/N19-1300/),[Document](https://dx.doi.org/10.18653/v1/N19-1300)Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px4.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[10\]P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. Tafjord\(2018\)Think you have solved question answering? try arc, the ai2 reasoning challenge\.External Links:1803\.05457,[Link](https://arxiv.org/abs/1803.05457)Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px3.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[11\]DeepSeek\-AI, A\. Liu, B\. Feng, B\. Xue, B\. Wang, B\. Wu, C\. Lu, C\. Zhao, C\. Deng, C\. Zhang, C\. Ruan, D\. Dai, D\. Guo, D\. Yang, D\. Chen, D\. Ji, E\. Li, F\. Lin, F\. Dai, F\. Luo, G\. Hao, G\. Chen, G\. Li, H\. Zhang, H\. Bao, H\. Xu, H\. Wang, H\. Zhang, H\. Ding, H\. Xin, H\. Gao, H\. Li, H\. Qu, J\. L\. Cai, J\. Liang, J\. Guo, J\. Ni, J\. Li, J\. Wang, J\. Chen, J\. Chen, J\. Yuan, J\. Qiu, J\. Li, J\. Song, K\. Dong, K\. Hu, K\. Gao, K\. Guan, K\. Huang, K\. Yu, L\. Wang, L\. Zhang, L\. Xu, L\. Xia, L\. Zhao, L\. Wang, L\. Zhang, M\. Li, M\. Wang, M\. Zhang, M\. Zhang, M\. Tang, M\. Li, N\. Tian, P\. Huang, P\. Wang, P\. Zhang, Q\. Wang, Q\. Zhu, Q\. Chen, Q\. Du, R\. J\. Chen, R\. L\. Jin, R\. Ge, R\. Zhang, R\. Pan, R\. Wang, R\. Xu, R\. Zhang, R\. Chen, S\. S\. Li, S\. Lu, S\. Zhou, S\. Chen, S\. Wu, S\. Ye, S\. Ye, S\. Ma, S\. Wang, S\. Zhou, S\. Yu, S\. Zhou, S\. Pan, T\. Wang, T\. Yun, T\. Pei, T\. Sun, W\. L\. Xiao, W\. Zeng, W\. Zhao, W\. An, W\. Liu, W\. Liang, W\. Gao, W\. Yu, W\. Zhang, X\. Q\. Li, X\. Jin, X\. Wang, X\. Bi, X\. Liu, X\. Wang, X\. Shen, X\. Chen, X\. Zhang, X\. Chen, X\. Nie, X\. Sun, X\. Wang, X\. Cheng, X\. Liu, X\. Xie, X\. Liu, X\. Yu, X\. Song, X\. Shan, X\. Zhou, X\. Yang, X\. Li, X\. Su, X\. Lin, Y\. K\. Li, Y\. Q\. Wang, Y\. X\. Wei, Y\. X\. Zhu, Y\. Zhang, Y\. Xu, Y\. Xu, Y\. Huang, Y\. Li, Y\. Zhao, Y\. Sun, Y\. Li, Y\. Wang, Y\. Yu, Y\. Zheng, Y\. Zhang, Y\. Shi, Y\. Xiong, Y\. He, Y\. Tang, Y\. Piao, Y\. Wang, Y\. Tan, Y\. Ma, Y\. Liu, Y\. Guo, Y\. Wu, Y\. Ou, Y\. Zhu, Y\. Wang, Y\. Gong, Y\. Zou, Y\. He, Y\. Zha, Y\. Xiong, Y\. Ma, Y\. Yan, Y\. Luo, Y\. You, Y\. Liu, Y\. Zhou, Z\. F\. Wu, Z\. Z\. Ren, Z\. Ren, Z\. Sha, Z\. Fu, Z\. Xu, Z\. Huang, Z\. Zhang, Z\. Xie, Z\. Zhang, Z\. Hao, Z\. Gou, Z\. Ma, Z\. Yan, Z\. Shao, Z\. Xu, Z\. Wu, Z\. Zhang, Z\. Li, Z\. Gu, Z\. Zhu, Z\. Liu, Z\. Li, Z\. Xie, Z\. Song, Z\. Gao, and Z\. Pan\(2024\)DeepSeek\-v3 technical report\.External Links:2412\.19437,[Link](https://arxiv.org/abs/2412.19437)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p1.1)\.
- \[12\]T\. Dettmers, A\. Pagnoni, A\. Holtzman, and L\. Zettlemoyer\(2023\)QLORA: efficient finetuning of quantized llms\.InProceedings of the 37th International Conference on Neural Information Processing Systems,NIPS ’23,Red Hook, NY, USA\.Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p1.1)\.
- \[13\]N\. Dhar, B\. Deng, M\. R\. Islam, K\. F\. Ahmad Nasif, L\. Zhao, and K\. Suo\(2024\)Activation sparsity opportunities for compressing general large language models\.In2024 IEEE International Performance, Computing, and Communications Conference \(IPCCC\),pp\.1–9\.External Links:[Link](http://dx.doi.org/10.1109/IPCCC59868.2024.10850382),[Document](https://dx.doi.org/10.1109/ipccc59868.2024.10850382)Cited by:[§2](https://arxiv.org/html/2608.12953#S2.p1.1)\.
- \[14\]X\. Ding, X\. Liu, Z\. Tu, Y\. Zhang, W\. Li, J\. Hu, H\. Chen, Y\. Tang, Z\. Xiong, B\. Yin, and Y\. Wang\(2025\)CBQ: cross\-block quantization for large language models\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=eW4yh6HKz4)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p1.1),[§2](https://arxiv.org/html/2608.12953#S2.p1.1)\.
- \[15\]X\. Ding, R\. Sun, Y\. Zhang, X\. Yan, Y\. Zhou, K\. Huang, S\. Fu, A\. I\. Aviles\-Rivero, C\. Xie, and Y\. Zhu\(2026\)Sliding\-window merging for compacting patch\-redundant layers in llms\.Proceedings of the AAAI Conference on Artificial Intelligence40\(25\),pp\.20826–20834\.External Links:ISSN 2159\-5399,[Link](http://dx.doi.org/10.1609/aaai.v40i25.39222),[Document](https://dx.doi.org/10.1609/aaai.v40i25.39222)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p2.1)\.
- \[16\]D\. Emelin, R\. Le Bras, J\. D\. Hwang, M\. Forbes, and Y\. Choi\(2021\)Moral stories: situated reasoning about norms, intents, actions, and their consequences\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,M\. Moens, X\. Huang, L\. Specia, and S\. W\. Yih \(Eds\.\),Online and Punta Cana, Dominican Republic,pp\.698–718\.External Links:[Link](https://aclanthology.org/2021.emnlp-main.54/),[Document](https://dx.doi.org/10.18653/v1/2021.emnlp-main.54)Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px5.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[17\]E\. Frantar and D\. Alistarh\(2023\)SparseGPT: massive language models can be accurately pruned in one\-shot\.InProceedings of the 40th International Conference on Machine Learning,ICML’23\.Cited by:[§2\.1](https://arxiv.org/html/2608.12953#S2.SS1.p1.1)\.
- \[18\]L\. Gao, J\. Tow, B\. Abbasi, S\. Biderman, S\. Black, A\. DiPofi, C\. Foster, L\. Golding, J\. Hsu, A\. Le Noac’h, H\. Li, K\. McDonell, N\. Muennighoff, C\. Ociepa, J\. Phang, L\. Reynolds, H\. Schoelkopf, A\. Skowron, L\. Sutawika, E\. Tang, A\. Thite, B\. Wang, K\. Wang, and A\. Zou\(2024\)A framework for few\-shot language model evaluation\.Zenodo\.External Links:[Document](https://dx.doi.org/10.5281/zenodo.12608602),[Link](https://zenodo.org/records/12608602)Cited by:[§10](https://arxiv.org/html/2608.12953#S10.p1.1)\.
- \[19\]A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, A\. Yang, A\. Fan, A\. Goyal, A\. Hartshorn, A\. Yang, A\. Mitra, A\. Sravankumar, A\. Korenev, A\. Hinsvark, A\. Rao, A\. Zhang, A\. Rodriguez, A\. Gregerson, A\. Spataru, B\. Roziere, B\. Biron, B\. Tang, B\. Chern, C\. Caucheteux, C\. Nayak, C\. Bi, C\. Marra, C\. McConnell, C\. Keller, C\. Touret, C\. Wu, C\. Wong, C\. C\. Ferrer, C\. Nikolaidis, D\. Allonsius, D\. Song, D\. Pintz, D\. Livshits, D\. Wyatt, D\. Esiobu, D\. Choudhary, D\. Mahajan, D\. Garcia\-Olano, D\. Perino, D\. Hupkes, E\. Lakomkin, E\. AlBadawy, E\. Lobanova, E\. Dinan, E\. M\. Smith, F\. Radenovic, F\. Guzmán, F\. Zhang, G\. Synnaeve, G\. Lee, G\. L\. Anderson, G\. Thattai, G\. Nail, G\. Mialon, G\. Pang, G\. Cucurell, H\. Nguyen, H\. Korevaar, H\. Xu, H\. Touvron, I\. Zarov, I\. A\. Ibarra, I\. Kloumann, I\. Misra, I\. Evtimov, J\. Zhang, J\. Copet, J\. Lee, J\. Geffert, J\. Vranes, J\. Park, J\. Mahadeokar, J\. Shah, J\. van der Linde, J\. Billock, J\. Hong, J\. Lee, J\. Fu, J\. Chi, J\. Huang, J\. Liu, J\. Wang, J\. Yu, J\. Bitton, J\. Spisak, J\. Park, J\. Rocca, J\. Johnstun, J\. Saxe, J\. Jia, K\. V\. Alwala, K\. Prasad, K\. Upasani, K\. Plawiak, K\. Li, K\. Heafield, K\. Stone, K\. El\-Arini, K\. Iyer, K\. Malik, K\. Chiu, K\. Bhalla, K\. Lakhotia, L\. Rantala\-Yeary, L\. van der Maaten, L\. Chen, L\. Tan, L\. Jenkins, L\. Martin, L\. Madaan, L\. Malo, L\. Blecher, L\. Landzaat, L\. de Oliveira, M\. Muzzi, M\. Pasupuleti, M\. Singh, M\. Paluri, M\. Kardas, M\. Tsimpoukelli, M\. Oldham, M\. Rita, M\. Pavlova, M\. Kambadur, M\. Lewis, M\. Si, M\. K\. Singh, M\. Hassan, N\. Goyal, N\. Torabi, N\. Bashlykov, N\. Bogoychev, N\. Chatterji, N\. Zhang, O\. Duchenne, O\. Çelebi, P\. Alrassy, P\. Zhang, P\. Li, P\. Vasic, P\. Weng, P\. Bhargava, P\. Dubal, P\. Krishnan, P\. S\. Koura, P\. Xu, Q\. He, Q\. Dong, R\. Srinivasan, R\. Ganapathy, R\. Calderer, R\. S\. Cabral, R\. Stojnic, R\. Raileanu, R\. Maheswari, R\. Girdhar, R\. Patel, R\. Sauvestre, R\. Polidoro, R\. Sumbaly, R\. Taylor, R\. Silva, R\. Hou, R\. Wang, S\. Hosseini, S\. Chennabasappa, S\. Singh, S\. Bell, S\. S\. Kim, S\. Edunov, S\. Nie, S\. Narang, S\. Raparthy, S\. Shen, S\. Wan, S\. Bhosale, S\. Zhang, S\. Vandenhende, S\. Batra, S\. Whitman, S\. Sootla, S\. Collot, S\. Gururangan, S\. Borodinsky, T\. Herman, T\. Fowler, T\. Sheasha, T\. Georgiou, T\. Scialom, T\. Speckbacher, T\. Mihaylov, T\. Xiao, U\. Karn, V\. Goswami, V\. Gupta, V\. Ramanathan, V\. Kerkez, V\. Gonguet, V\. Do, V\. Vogeti, V\. Albiero, V\. Petrovic, W\. Chu, W\. Xiong, W\. Fu, W\. Meers, X\. Martinet, X\. Wang, X\. Wang, X\. E\. Tan, X\. Xia, X\. Xie, X\. Jia, X\. Wang, Y\. Goldschlag, Y\. Gaur, Y\. Babaei, Y\. Wen, Y\. Song, Y\. Zhang, Y\. Li, Y\. Mao, Z\. D\. Coudert, Z\. Yan, Z\. Chen, Z\. Papakipos, A\. Singh, A\. Srivastava, A\. Jain, A\. Kelsey, A\. Shajnfeld, A\. Gangidi, A\. Victoria, A\. Goldstand, A\. Menon, A\. Sharma, A\. Boesenberg, A\. Baevski, A\. Feinstein, A\. Kallet, A\. Sangani, A\. Teo, A\. Yunus, A\. Lupu, A\. Alvarado, A\. Caples, A\. Gu, A\. Ho, A\. Poulton, A\. Ryan, A\. Ramchandani, A\. Dong, A\. Franco, A\. Goyal, A\. Saraf, A\. Chowdhury, A\. Gabriel, A\. Bharambe, A\. Eisenman, A\. Yazdan, B\. James, B\. Maurer, B\. Leonhardi, B\. Huang, B\. Loyd, B\. D\. Paola, B\. Paranjape, B\. Liu, B\. Wu, B\. Ni, B\. Hancock, B\. Wasti, B\. Spence, B\. Stojkovic, B\. Gamido, B\. Montalvo, C\. Parker, C\. Burton, C\. Mejia, C\. Liu, C\. Wang, C\. Kim, C\. Zhou, C\. Hu, C\. Chu, C\. Cai, C\. Tindal, C\. Feichtenhofer, C\. Gao, D\. Civin, D\. Beaty, D\. Kreymer, D\. Li, D\. Adkins, D\. Xu, D\. Testuggine, D\. David, D\. Parikh, D\. Liskovich, D\. Foss, D\. Wang, D\. Le, D\. Holland, E\. Dowling, E\. Jamil, E\. Montgomery, E\. Presani, E\. Hahn, E\. Wood, E\. Le, E\. Brinkman, E\. Arcaute, E\. Dunbar, E\. Smothers, F\. Sun, F\. Kreuk, F\. Tian, F\. Kokkinos, F\. Ozgenel, F\. Caggioni, F\. Kanayet, F\. Seide, G\. M\. Florez, G\. Schwarz, G\. Badeer, G\. Swee, G\. Halpern, G\. Herman, G\. Sizov, Guangyi, Zhang, G\. Lakshminarayanan, H\. Inan, H\. Shojanazeri, H\. Zou, H\. Wang, H\. Zha, H\. Habeeb, H\. Rudolph, H\. Suk, H\. Aspegren, H\. Goldman, H\. Zhan, I\. Damlaj, I\. Molybog, I\. Tufanov, I\. Leontiadis, I\. Veliche, I\. Gat, J\. Weissman, J\. Geboski, J\. Kohli, J\. Lam, J\. Asher, J\. Gaya, J\. Marcus, J\. Tang, J\. Chan, J\. Zhen, J\. Reizenstein, J\. Teboul, J\. Zhong, J\. Jin, J\. Yang, J\. Cummings, J\. Carvill, J\. Shepard, J\. McPhie, J\. Torres, J\. Ginsburg, J\. Wang, K\. Wu, K\. H\. U, K\. Saxena, K\. Khandelwal, K\. Zand, K\. Matosich, K\. Veeraraghavan, K\. Michelena, K\. Li, K\. Jagadeesh, K\. Huang, K\. Chawla, K\. Huang, L\. Chen, L\. Garg, L\. A, L\. Silva, L\. Bell, L\. Zhang, L\. Guo, L\. Yu, L\. Moshkovich, L\. Wehrstedt, M\. Khabsa, M\. Avalani, M\. Bhatt, M\. Mankus, M\. Hasson, M\. Lennie, M\. Reso, M\. Groshev, M\. Naumov, M\. Lathi, M\. Keneally, M\. Liu, M\. L\. Seltzer, M\. Valko, M\. Restrepo, M\. Patel, M\. Vyatskov, M\. Samvelyan, M\. Clark, M\. Macey, M\. Wang, M\. J\. Hermoso, M\. Metanat, M\. Rastegari, M\. Bansal, N\. Santhanam, N\. Parks, N\. White, N\. Bawa, N\. Singhal, N\. Egebo, N\. Usunier, N\. Mehta, N\. P\. Laptev, N\. Dong, N\. Cheng, O\. Chernoguz, O\. Hart, O\. Salpekar, O\. Kalinli, P\. Kent, P\. Parekh, P\. Saab, P\. Balaji, P\. Rittner, P\. Bontrager, P\. Roux, P\. Dollar, P\. Zvyagina, P\. Ratanchandani, P\. Yuvraj, Q\. Liang, R\. Alao, R\. Rodriguez, R\. Ayub, R\. Murthy, R\. Nayani, R\. Mitra, R\. Parthasarathy, R\. Li, R\. Hogan, R\. Battey, R\. Wang, R\. Howes, R\. Rinott, S\. Mehta, S\. Siby, S\. J\. Bondu, S\. Datta, S\. Chugh, S\. Hunt, S\. Dhillon, S\. Sidorov, S\. Pan, S\. Mahajan, S\. Verma, S\. Yamamoto, S\. Ramaswamy, S\. Lindsay, S\. Lindsay, S\. Feng, S\. Lin, S\. C\. Zha, S\. Patil, S\. Shankar, S\. Zhang, S\. Zhang, S\. Wang, S\. Agarwal, S\. Sajuyigbe, S\. Chintala, S\. Max, S\. Chen, S\. Kehoe, S\. Satterfield, S\. Govindaprasad, S\. Gupta, S\. Deng, S\. Cho, S\. Virk, S\. Subramanian, S\. Choudhury, S\. Goldman, T\. Remez, T\. Glaser, T\. Best, T\. Koehler, T\. Robinson, T\. Li, T\. Zhang, T\. Matthews, T\. Chou, T\. Shaked, V\. Vontimitta, V\. Ajayi, V\. Montanez, V\. Mohan, V\. S\. Kumar, V\. Mangla, V\. Ionescu, V\. Poenaru, V\. T\. Mihailescu, V\. Ivanov, W\. Li, W\. Wang, W\. Jiang, W\. Bouaziz, W\. Constable, X\. Tang, X\. Wu, X\. Wang, X\. Wu, X\. Gao, Y\. Kleinman, Y\. Chen, Y\. Hu, Y\. Jia, Y\. Qi, Y\. Li, Y\. Zhang, Y\. Zhang, Y\. Adi, Y\. Nam, Yu, Wang, Y\. Zhao, Y\. Hao, Y\. Qian, Y\. Li, Y\. He, Z\. Rait, Z\. DeVito, Z\. Rosnbrick, Z\. Wen, Z\. Yang, Z\. Zhao, and Z\. Ma\(2024\)The llama 3 herd of models\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p1.1),[§3\.4](https://arxiv.org/html/2608.12953#S3.SS4.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px1.p1.1),[§9](https://arxiv.org/html/2608.12953#S9.SS0.SSS0.Px1.p1.1)\.
- \[20\]A\. Gromov, K\. Tirumala, H\. Shapourian, P\. Glorioso, and D\. Roberts\(2025\)The unreasonable ineffectiveness of the deeper layers\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=ngmEcEer8a)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px2.p1.1)\.
- \[21\]Y\. Gu, L\. Dong, F\. Wei, and M\. Huang\(2025\)MiniLLM: knowledge distillation of large language models\.External Links:2306\.08543,[Link](https://arxiv.org/abs/2306.08543)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p1.1)\.
- \[22\]Z\. Guan, H\. Huang, Y\. Su, H\. Huang, N\. Wong, and H\. Yu\(2024\)APTQ: attention\-aware post\-training mixed\-precision quantization for large language models\.InProceedings of the 61st ACM/IEEE Design Automation Conference,DAC ’24,pp\.1–6\.External Links:[Link](http://dx.doi.org/10.1145/3649329.3658498),[Document](https://dx.doi.org/10.1145/3649329.3658498)Cited by:[§2](https://arxiv.org/html/2608.12953#S2.p1.1)\.
- \[23\]J\. Guo, X\. Chen, Y\. Tang, and Y\. Wang\(2025\)SlimLLM: accurate structured pruning for large language models\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=2xjUkU7FDb)Cited by:[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px1.p1.1)\.
- \[24\]S\. Han, J\. Pool, J\. Tran, and W\. J\. Dally\(2015\)Learning both weights and connections for efficient neural networks\.External Links:1506\.02626,[Link](https://arxiv.org/abs/1506.02626)Cited by:[§2\.1](https://arxiv.org/html/2608.12953#S2.SS1.p1.1),[§5\.4](https://arxiv.org/html/2608.12953#S5.SS4.SSS0.Px2.p1.1)\.
- \[25\]B\. Hassibi and D\. Stork\(1992\)Second order derivatives for network pruning: optimal brain surgeon\.InAdvances in Neural Information Processing Systems,S\. Hanson, J\. Cowan, and C\. Giles \(Eds\.\),Vol\.5,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/1992/file/303ed4c69846ab36c2904d3ba8573050-Paper.pdf)Cited by:[§2\.1](https://arxiv.org/html/2608.12953#S2.SS1.p1.1)\.
- \[26\]G\. Hinton, O\. Vinyals, and J\. Dean\(2015\)Distilling the knowledge in a neural network\.External Links:1503\.02531,[Link](https://arxiv.org/abs/1503.02531)Cited by:[§2](https://arxiv.org/html/2608.12953#S2.p1.1)\.
- \[27\]Y\. Hsu, T\. Hua, S\. Chang, Q\. Lou, Y\. Shen, and H\. Jin\(2022\)Language model compression with weighted low\-rank factorization\.External Links:2207\.00112,[Link](https://arxiv.org/abs/2207.00112)Cited by:[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px1.p1.1)\.
- \[28\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen\(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=nZeVKeeFYf9)Cited by:[§10](https://arxiv.org/html/2608.12953#S10.p2.1)\.
- \[29\]Y\. Ji, Y\. Xiang, J\. Li, Q\. Xia, P\. Li, X\. Duan, Z\. Wang, and M\. Zhang\(2025\)Beware of calibration data for pruning large language models\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=x83w6yGIWb)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p2.1)\.
- \[30\]X\. Jiao, Y\. Yin, L\. Shang, X\. Jiang, X\. Chen, L\. Li, F\. Wang, and Q\. Liu\(2020\)TinyBERT: distilling BERT for natural language understanding\.InFindings of the Association for Computational Linguistics: EMNLP 2020,T\. Cohn, Y\. He, and Y\. Liu \(Eds\.\),Online,pp\.4163–4174\.External Links:[Link](https://aclanthology.org/2020.findings-emnlp.372/),[Document](https://dx.doi.org/10.18653/v1/2020.findings-emnlp.372)Cited by:[§2](https://arxiv.org/html/2608.12953#S2.p1.1)\.
- \[31\]D\. Jin, E\. Pan, N\. Oufattole, W\. Weng, H\. Fang, and P\. Szolovits\(2021\)What disease does this patient have? a large\-scale open domain question answering dataset from medical exams\.Applied Sciences11\(14\)\.External Links:[Link](https://www.mdpi.com/2076-3417/11/14/6421),ISSN 2076\-3417,[Document](https://dx.doi.org/10.3390/app11146421)Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px3.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[32\]B\. Kim, G\. Kim, T\. Kim, T\. Castells, S\. Choi, J\. Shin, and H\. Song\(2024\)Shortened llama: depth pruning for large language models with comparison of retraining methods\.External Links:2402\.02834,[Link](https://arxiv.org/abs/2402.02834)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p2.1)\.
- \[33\]S\. Kim, A\. Gholami, Z\. Yao, M\. W\. Mahoney, and K\. Keutzer\(2021\)I\-bert: integer\-only bert quantization\.InProceedings of the 38th International Conference on Machine Learning,M\. Meila and T\. Zhang \(Eds\.\),Proceedings of Machine Learning Research, Vol\.139,pp\.5506–5518\.External Links:[Link](https://proceedings.mlr.press/v139/kim21d.html)Cited by:[§2](https://arxiv.org/html/2608.12953#S2.p1.1)\.
- \[34\]W\. Lian, G\. Wang, B\. Goodson, E\. Pentland, A\. Cook, C\. Vong, and "Teknium"\(2023\)SlimOrca: an open dataset of gpt\-4 augmented flan reasoning traces, with verification\.HuggingFace\.External Links:[Link](https://huggingface.co/datasets/Open-Orca/SlimOrca)Cited by:[§10](https://arxiv.org/html/2608.12953#S10.p1.1),[Table 16](https://arxiv.org/html/2608.12953#S12.T16.3.1.5.1.1),[Table 16](https://arxiv.org/html/2608.12953#S12.T16.5.1.5.1.1),[§6\.1](https://arxiv.org/html/2608.12953#S6.SS1.p1.1),[5\(a\)](https://arxiv.org/html/2608.12953#S6.T5.st1),[5\(a\)](https://arxiv.org/html/2608.12953#S6.T5.st1.5),[5\(b\)](https://arxiv.org/html/2608.12953#S6.T5.st2.3.1.4.1.1),[§9](https://arxiv.org/html/2608.12953#S9.SS0.SSS0.Px1.p1.1)\.
- \[35\]S\. Lin, J\. Hilton, and O\. Evans\(2022\)TruthfulQA: measuring how models mimic human falsehoods\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\.3214–3252\.External Links:[Link](https://aclanthology.org/2022.acl-long.229/),[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.229)Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px5.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[36\]J\. Liu, P\. Ponnusamy, T\. Cai, H\. Guo, Y\. Kim, and B\. Athiwaratkun\(2025\)Training\-free activation sparsity in large language models\.External Links:2408\.14690,[Link](https://arxiv.org/abs/2408.14690)Cited by:[§2](https://arxiv.org/html/2608.12953#S2.p1.1)\.
- \[37\]S\. Liu, Z\. Liu, X\. Huang, P\. Dong, and K\. Cheng\(2023\)LLM\-fp4: 4\-bit floating\-point quantized transformers\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\.592–605\.External Links:[Link](http://dx.doi.org/10.18653/v1/2023.emnlp-main.39),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.39)Cited by:[§2](https://arxiv.org/html/2608.12953#S2.p1.1)\.
- \[38\]Y\. Luo, C\. Song, X\. Han, Y\. Chen, C\. Xiao, X\. Meng, L\. Deng, J\. Wei, Z\. Liu, and M\. Sun\(2025\)Sparsing law: towards large language models with greater activation sparsity\.External Links:2411\.02335,[Link](https://arxiv.org/abs/2411.02335)Cited by:[§2](https://arxiv.org/html/2608.12953#S2.p1.1)\.
- \[39\]X\. Ma, G\. Fang, and X\. Wang\(2023\)LLM\-pruner: on the structural pruning of large language models\.InProceedings of the 37th International Conference on Neural Information Processing Systems,NIPS ’23,Red Hook, NY, USA\.Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px2.p1.1)\.
- \[40\]X\. Men, M\. Xu, Q\. Zhang, B\. Wang, H\. Lin, Y\. Lu, X\. Han, and W\. Chen\(2024\)ShortGPT: layers in large language models are more redundant than you expect\.External Links:2403\.03853,[Link](https://arxiv.org/abs/2403.03853)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px2.p1.1)\.
- \[41\]S\. Merity, C\. Xiong, J\. Bradbury, and R\. Socher\(2016\)Pointer sentinel mixture models\.External Links:1609\.07843,[Link](https://arxiv.org/abs/1609.07843)Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[42\]P\. Michel, O\. Levy, and G\. Neubig\(2019\)Are sixteen heads really better than one?\.InAdvances in Neural Information Processing Systems,H\. Wallach, H\. Larochelle, A\. Beygelzimer, F\. d'Alché\-Buc, E\. Fox, and R\. Garnett \(Eds\.\),Vol\.32,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2019/file/2c601ad9d2ff9bc8b282670cdd54f69f-Paper.pdf)Cited by:[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px1.p1.1)\.
- \[43\]T\. Mihaylov, P\. Clark, T\. Khot, and A\. Sabharwal\(2018\)Can a suit of armor conduct electricity? a new dataset for open book question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,E\. Riloff, D\. Chiang, J\. Hockenmaier, and J\. Tsujii \(Eds\.\),Brussels, Belgium,pp\.2381–2391\.External Links:[Link](https://aclanthology.org/D18-1260/),[Document](https://dx.doi.org/10.18653/v1/D18-1260)Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px3.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[44\]A\. Mishra, J\. A\. Latorre, J\. Pool, D\. Stosic, D\. Stosic, G\. Venkatesh, C\. Yu, and P\. Micikevicius\(2021\)Accelerating sparse deep neural networks\.External Links:2104\.08378,[Link](https://arxiv.org/abs/2104.08378)Cited by:[§2\.1](https://arxiv.org/html/2608.12953#S2.SS1.p1.1)\.
- \[45\]S\. Mukherjee, A\. Mitra, G\. Jawahar, S\. Agarwal, H\. Palangi, and A\. Awadallah\(2023\)Orca: progressive learning from complex explanation traces of gpt\-4\.External Links:2306\.02707,[Link](https://arxiv.org/abs/2306.02707)Cited by:[§10](https://arxiv.org/html/2608.12953#S10.p1.1)\.
- \[46\]C\. V\. Nguyen, X\. Shen, R\. Aponte, Y\. Xia, S\. Basu, Z\. Hu, J\. Chen, M\. Parmar, S\. Kunapuli, J\. Barrow, J\. Wu, A\. Singh, Y\. Wang, J\. Gu, F\. Dernoncourt, N\. K\. Ahmed, N\. Lipka, R\. Zhang, X\. Chen, T\. Yu, S\. Kim, H\. Deilamsalehy, N\. Park, M\. Rimer, Z\. Zhang, H\. Yang, R\. A\. Rossi, and T\. H\. Nguyen\(2024\)A survey of small language models\.External Links:2410\.20011,[Link](https://arxiv.org/abs/2410.20011)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p1.1)\.
- \[47\]OpenAI, :, S\. Agarwal, L\. Ahmad, J\. Ai, S\. Altman, A\. Applebaum, E\. Arbus, R\. K\. Arora, Y\. Bai, B\. Baker, H\. Bao, B\. Barak, A\. Bennett, T\. Bertao, N\. Brett, E\. Brevdo, G\. Brockman, S\. Bubeck, C\. Chang, K\. Chen, M\. Chen, E\. Cheung, A\. Clark, D\. Cook, M\. Dukhan, C\. Dvorak, K\. Fives, V\. Fomenko, T\. Garipov, K\. Georgiev, M\. Glaese, T\. Gogineni, A\. Goucher, L\. Gross, K\. G\. Guzman, J\. Hallman, J\. Hehir, J\. Heidecke, A\. Helyar, H\. Hu, R\. Huet, J\. Huh, S\. Jain, Z\. Johnson, C\. Koch, I\. Kofman, D\. Kundel, J\. Kwon, V\. Kyrylov, E\. Y\. Le, G\. Leclerc, J\. P\. Lennon, S\. Lessans, M\. Lezcano\-Casado, Y\. Li, Z\. Li, J\. Lin, J\. Liss, Lily, Liu, J\. Liu, K\. Lu, C\. Lu, Z\. Martinovic, L\. McCallum, J\. McGrath, S\. McKinney, A\. McLaughlin, S\. Mei, S\. Mostovoy, T\. Mu, G\. Myles, A\. Neitz, A\. Nichol, J\. Pachocki, A\. Paino, D\. Palmie, A\. Pantuliano, G\. Parascandolo, J\. Park, L\. Pathak, C\. Paz, L\. Peran, D\. Pimenov, M\. Pokrass, E\. Proehl, H\. Qiu, G\. Raila, F\. Raso, H\. Ren, K\. Richardson, D\. Robinson, B\. Rotsted, H\. Salman, S\. Sanjeev, M\. Schwarzer, D\. Sculley, H\. Sikchi, K\. Simon, K\. Singhal, Y\. Song, D\. Stuckey, Z\. Sun, P\. Tillet, S\. Toizer, F\. Tsimpourlas, N\. Vyas, E\. Wallace, X\. Wang, M\. Wang, O\. Watkins, K\. Weil, A\. Wendling, K\. Whinnery, C\. Whitney, H\. Wong, L\. Yang, Y\. Yang, M\. Yasunaga, K\. Ying, W\. Zaremba, W\. Zhan, C\. Zhang, B\. Zhang, E\. Zhang, and S\. Zhao\(2025\)Gpt\-oss\-120b & gpt\-oss\-20b model card\.External Links:2508\.10925,[Link](https://arxiv.org/abs/2508.10925)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px1.p1.1),[§9](https://arxiv.org/html/2608.12953#S9.SS0.SSS0.Px1.p1.1)\.
- \[48\]D\. Paperno, G\. Kruszewski, A\. Lazaridou, N\. Q\. Pham, R\. Bernardi, S\. Pezzelle, M\. Baroni, G\. Boleda, and R\. Fernández\(2016\)The LAMBADA dataset: word prediction requiring a broad discourse context\.InProceedings of the 54th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),K\. Erk and N\. A\. Smith \(Eds\.\),Berlin, Germany,pp\.1525–1534\.External Links:[Link](https://aclanthology.org/P16-1144/),[Document](https://dx.doi.org/10.18653/v1/P16-1144)Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px1.p1.1),[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px4.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[49\]A\. Paszke, S\. Gross, F\. Massa, A\. Lerer, J\. Bradbury, G\. Chanan, T\. Killeen, Z\. Lin, N\. Gimelshein, L\. Antiga, A\. Desmaison, A\. Kopf, E\. Yang, Z\. DeVito, M\. Raison, A\. Tejani, S\. Chilamkurthy, B\. Steiner, L\. Fang, J\. Bai, and S\. Chintala\(2019\)PyTorch: an imperative style, high\-performance deep learning library\.InAdvances in Neural Information Processing Systems,H\. Wallach, H\. Larochelle, A\. Beygelzimer, F\. d'Alché\-Buc, E\. Fox, and R\. Garnett \(Eds\.\),Vol\.32,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf)Cited by:[§10](https://arxiv.org/html/2608.12953#S10.p1.1)\.
- \[50\]D\. Pisinger and P\. Toth\(1998\)Knapsack problems\.InHandbook of Combinatorial Optimization: Volume1–3,pp\.299–428\.Cited by:[§3\.1](https://arxiv.org/html/2608.12953#S3.SS1.p3.1)\.
- \[51\]Qwen, :, A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. Qiu\(2025\)Qwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§3\.4](https://arxiv.org/html/2608.12953#S3.SS4.SSS0.Px2.p1.1)\.
- \[52\]C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, and P\. J\. Liu\(2020\)Exploring the limits of transfer learning with a unified text\-to\-text transformer\.J\. Mach\. Learn\. Res\.21\(1\)\.External Links:ISSN 1532\-4435Cited by:[Table 16](https://arxiv.org/html/2608.12953#S12.T16.3.1.19.1.1),[Table 16](https://arxiv.org/html/2608.12953#S12.T16.5.1.19.1.1),[§6\.1](https://arxiv.org/html/2608.12953#S6.SS1.p1.1),[5\(b\)](https://arxiv.org/html/2608.12953#S6.T5.st2.3.1.10.1.1),[§9](https://arxiv.org/html/2608.12953#S9.SS0.SSS0.Px1.p1.1)\.
- \[53\]S\. Reddy, D\. Chen, and C\. D\. Manning\(2019\)CoQA: a conversational question answering challenge\.Transactions of the Association for Computational Linguistics7,pp\.249–266\.External Links:[Link](https://aclanthology.org/Q19-1016/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00266)Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px4.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[54\]R\. Rudinger, J\. Naradowsky, B\. Leonard, and B\. Van Durme\(2018\)Gender bias in coreference resolution\.InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 \(Short Papers\),M\. Walker, H\. Ji, and A\. Stent \(Eds\.\),New Orleans, Louisiana,pp\.8–14\.External Links:[Link](https://aclanthology.org/N18-2002/),[Document](https://dx.doi.org/10.18653/v1/N18-2002)Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px5.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[55\]K\. Sakaguchi, R\. L\. Bras, C\. Bhagavatula, and Y\. Choi\(2021\)Winogrande: an adversarial winograd schema challenge at scale\.Communications of the ACM64\(9\),pp\.99–106\.Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px4.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[56\]F\. Sandri, E\. Cunegatti, and G\. Iacca\(2025\)2SSP: a two\-stage framework for structured pruning of LLMs\.Transactions on Machine Learning Research\.Note:External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=Qd7LzJBg21)Cited by:[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px2.p1.1)\.
- \[57\]A\. Sengupta, S\. Chaudhary, and T\. Chakraborty\(2025\)You only prune once: designing calibration\-free model compression with policy learning\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=5RZoYIT3u6)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px1.p1.1)\.
- \[58\]A\. Sengupta, S\. Dixit, M\. S\. Akhtar, and T\. Chakraborty\(2023\)A good learner can teach better: teacher\-student collaborative knowledge distillation\.InProceedings of the The Twelfth International Conference on Learning Representations, Virtual Event,pp\.25–29\.Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p1.1)\.
- \[59\]D\. Shopkhoev, A\. Ali, M\. Zhussip, V\. Malykh, S\. Lefkimmiatis, N\. Komodakis, and S\. Zagoruyko\(2025\)ReplaceMe: network simplification via depth pruning and transformer block linearization\.External Links:2505\.02819,[Link](https://arxiv.org/abs/2505.02819)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px2.p1.1)\.
- \[60\]J\. Song, K\. Oh, T\. Kim, H\. Kim, Y\. Kim, and J\. Kim\(2024\)SLEB: streamlining LLMs through redundancy verification and elimination of transformer blocks\.InProceedings of the 41st International Conference on Machine Learning,R\. Salakhutdinov, Z\. Kolter, K\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research, Vol\.235,pp\.46136–46155\.External Links:[Link](https://proceedings.mlr.press/v235/song24f.html)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px2.p1.1),[§5\.4](https://arxiv.org/html/2608.12953#S5.SS4.SSS0.Px3.p1.1)\.
- \[61\]M\. Sun, Z\. Liu, A\. Bair, and J\. Z\. Kolter\(2024\)A simple and effective pruning approach for large language models\.External Links:2306\.11695,[Link](https://arxiv.org/abs/2306.11695)Cited by:[§2\.1](https://arxiv.org/html/2608.12953#S2.SS1.p1.1),[§5\.4](https://arxiv.org/html/2608.12953#S5.SS4.SSS0.Px2.p1.1)\.
- \[62\]A\. Talmor, J\. Herzig, N\. Lourie, and J\. Berant\(2019\)CommonsenseQA: a question answering challenge targeting commonsense knowledge\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),Minneapolis, Minnesota,pp\.4149–4158\.External Links:[Link](https://aclanthology.org/N19-1421),[Document](https://dx.doi.org/10.18653/v1/N19-1421),1811\.00937Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[63\]R\. Taori, I\. Gulrajani, T\. Zhang, Y\. Dubois, X\. Li, C\. Guestrin, P\. Liang, and T\. B\. Hashimoto\(2023\)Stanford alpaca: an instruction\-following llama model\.GitHub\.Note:[https://github\.com/tatsu\-lab/stanford\_alpaca](https://github.com/tatsu-lab/stanford_alpaca)Cited by:[Table 16](https://arxiv.org/html/2608.12953#S12.T16.3.1.12.1.1),[Table 16](https://arxiv.org/html/2608.12953#S12.T16.5.1.12.1.1),[§6\.1](https://arxiv.org/html/2608.12953#S6.SS1.p1.1),[5\(b\)](https://arxiv.org/html/2608.12953#S6.T5.st2.3.1.7.1.1),[§9](https://arxiv.org/html/2608.12953#S9.SS0.SSS0.Px1.p1.1)\.
- \[64\]T\. F\. A\. van der Ouderaa, M\. Nagel, M\. V\. Baalen, and T\. Blankevoort\(2024\)The LLM surgeon\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=DYIIRgwg2i)Cited by:[§2\.1](https://arxiv.org/html/2608.12953#S2.SS1.p1.1)\.
- \[65\]E\. Voita, D\. Talbot, F\. Moiseev, R\. Sennrich, and I\. Titov\(2019\)Analyzing multi\-head self\-attention: specialized heads do the heavy lifting, the rest can be pruned\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,A\. Korhonen, D\. Traum, and L\. Màrquez \(Eds\.\),Florence, Italy,pp\.5797–5808\.External Links:[Link](https://aclanthology.org/P19-1580/),[Document](https://dx.doi.org/10.18653/v1/P19-1580)Cited by:[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px1.p1.1)\.
- \[66\]H\. Wang, S\. Hassan, Y\. Liu, C\. Ma, Y\. Chen, Q\. Li, J\. Geng, B\. Wang, Y\. Tian, Y\. Xie, J\. Avery, L\. Hull, I\. Reid, M\. Yaqub, and G\. Carneiro\(2025\)Meta\-learned modality\-weighted knowledge distillation for robust multi\-modal learning with missing data\.External Links:2405\.07155,[Link](https://arxiv.org/abs/2405.07155)Cited by:[§2](https://arxiv.org/html/2608.12953#S2.p1.1)\.
- \[67\]X\. Wang, Y\. Zheng, Z\. Wan, and M\. Zhang\(2025\)SVD\-llm: truncation\-aware singular value decomposition for large language model compression\.External Links:2403\.07378,[Link](https://arxiv.org/abs/2403.07378)Cited by:[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px1.p1.1)\.
- \[68\]A\. Warstadt, A\. Parrish, H\. Liu, A\. Mohananey, W\. Peng, S\. Wang, and S\. R\. Bowman\(2020\)BLiMP: the benchmark of linguistic minimal pairs for english\.Transactions of the Association for Computational Linguistics8\(\),pp\.377–392\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00321),[Link](https://doi.org/10.1162/tacl_a_00321),https://doi\.org/10\.1162/tacl\_a\_00321Cited by:[§11](https://arxiv.org/html/2608.12953#S11.SS0.SSS0.Px4.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px3.p1.1)\.
- \[69\]J\. Wee, M\. Park, and J\. Lee\(2025\)Prompt\-based depth pruning of large language models\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=hRxHF1xPYB)Cited by:[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px2.p1.1)\.
- \[70\]T\. Wolf, L\. Debut, V\. Sanh, J\. Chaumond, C\. Delangue, A\. Moi, P\. Cistac, T\. Rault, R\. Louf, M\. Funtowicz, J\. Davison, S\. Shleifer, P\. von Platen, C\. Ma, Y\. Jernite, J\. Plu, C\. Xu, T\. Le Scao, S\. Gugger, M\. Drame, Q\. Lhoest, and A\. Rush\(2020\)Transformers: state\-of\-the\-art natural language processing\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,Q\. Liu and D\. Schlangen \(Eds\.\),Online,pp\.38–45\.External Links:[Link](https://aclanthology.org/2020.emnlp-demos.6/),[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-demos.6)Cited by:[§10](https://arxiv.org/html/2608.12953#S10.p1.1)\.
- \[71\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. Qiu\(2025\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p1.1),[§4](https://arxiv.org/html/2608.12953#S4.SS0.SSS0.Px1.p1.1),[§9](https://arxiv.org/html/2608.12953#S9.SS0.SSS0.Px1.p1.1)\.
- \[72\]G\. Yang, Y\. Zhou, X\. Zhang, W\. Cheng, K\. Liu, X\. Chen, T\. Y\. Zhuo, and T\. Chen\(2025\)Less is more: towards green code large language models via unified structural pruning\.External Links:2412\.15921,[Link](https://arxiv.org/abs/2412.15921)Cited by:[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px2.p1.1)\.
- \[73\]Z\. Yuan, Y\. Shang, Y\. Song, D\. Yang, Q\. Wu, Y\. Yan, and G\. Sun\(2025\)ASVD: activation\-aware singular value decomposition for compressing large language models\.External Links:2312\.05821,[Link](https://arxiv.org/abs/2312.05821)Cited by:[§2\.2](https://arxiv.org/html/2608.12953#S2.SS2.SSS0.Px1.p1.1)\.
- \[74\]Z\. Zhang, Z\. Liu, Y\. Tian, H\. Khaitan, Z\. Wang, and S\. Li\(2025\)R\-sparse: rank\-aware activation sparsity for efficient llm inference\.External Links:2504\.19449,[Link](https://arxiv.org/abs/2504.19449)Cited by:[§2](https://arxiv.org/html/2608.12953#S2.p1.1)\.
- \[75\]X\. Zhu, J\. Li, Y\. Liu, C\. Ma, and W\. Wang\(2024\)A survey on model compression for large language models\.External Links:2308\.07633,[Link](https://arxiv.org/abs/2308.07633)Cited by:[§1](https://arxiv.org/html/2608.12953#S1.p1.1)\.

\\beginappendix

## 10Implementation Details

We implementSniperusing PyTorch\[[49](https://arxiv.org/html/2608.12953#bib.bib8)\]and the HuggingFace Transformers library\[[70](https://arxiv.org/html/2608.12953#bib.bib7)\]\. All experiments are conducted on a single NVIDIA A100 GPU with 80GB of VRAM and evaluated using the Language Model Evaluation Harness\[[18](https://arxiv.org/html/2608.12953#bib.bib47)\]\. For importance estimation \(Section[3\.3](https://arxiv.org/html/2608.12953#S3.SS3)\), we use a calibration set of 50 samples randomly drawn from the SlimOrca dataset\[[34](https://arxiv.org/html/2608.12953#bib.bib52),[45](https://arxiv.org/html/2608.12953#bib.bib53)\], with a context length of 512 tokens\.

For post\-pruning recovery fine\-tuning, we train the pruned models on 2,000 SlimOrca samples with a context length of 1,024 tokens\. To maintain computational efficiency, we employ Low\-Rank Adaptation\[[28](https://arxiv.org/html/2608.12953#bib.bib6)\]rather than full fine\-tuning\. We use a LoRA rank of 64, a scaling factor of 16, and a dropout probability of 0\.05\. All models are fine\-tuned for a single epoch with a batch size of 2 and a learning rate of2×10−42\\times 10^\{\-4\}\.

## 11Description of Evaluation Tasks

We devise a suite of 18 diverse tasks to evaluate different pruning methodologies comprehensively\. We provide a description of each benchmark task in this section\.

#### Generative Performance\.

We compute log\-perplexity on the WikiText\-2\[[41](https://arxiv.org/html/2608.12953#bib.bib46)\]and LAMBADA\[[48](https://arxiv.org/html/2608.12953#bib.bib64)\]datasets to assess generative performance\. These are language modeling datasets consisting of unlabeled English texts requiring models to understand local and global context to predict subsequent words\.

#### World Understanding\.

In order to assess models’ common sense capabilities and general understanding of the physical world, we utilize the PIQA\[[6](https://arxiv.org/html/2608.12953#bib.bib61)\], PROST\[[3](https://arxiv.org/html/2608.12953#bib.bib12)\], and CommonsenseQA\[[62](https://arxiv.org/html/2608.12953#bib.bib11)\]datasets\. PIQA contains multiple\-choice questions \(MCQs\) testing a model’s general understanding of the physical world with questions such as whether the process of boiling eggs requires water or not\. PROST specifically focuses on evaluating model’s understanding of each of 10 physical reasoning concepts such as direction, mass and height\. CommonsenseQA also contains MCQs but extends its scope of evaluation to include different types of commonsense knowledge instead of specifically physical understanding\.

#### Domain\-Specific Knowledge\.

We employ the ARC\-Easy and ARC\-Challenge\[[10](https://arxiv.org/html/2608.12953#bib.bib14)\]tasks which contain grade\-school level multiple\-choice science questions and are designed to test a model’s knowledge of basic science\. As their names suggest, the former is an easier version of the task than the latter\. We evaluate mathematical capabilities using the MathQA\[[2](https://arxiv.org/html/2608.12953#bib.bib15)\]task which contains multiple\-choice mathematical word problems\. Additionally, we employ the MedQA\[[31](https://arxiv.org/html/2608.12953#bib.bib13)\]task which is a challenging dataset containing multiple\-choice medical questions from professional medical exams\. Lastly, we evaluate a model’s ability to combine its commonsense, domain knowledge and text comprehension capabilities by testing it on the OpenbookQA benchmark\[[43](https://arxiv.org/html/2608.12953#bib.bib75)\]which contains complex reasoning questions that require models to refer to a given corpus of salient facts to answer correctly\.

#### Natural Language Understanding and Inference\.

We gauge a model’s semantic understanding on natural understanding with the help of the BLIMP\[[68](https://arxiv.org/html/2608.12953#bib.bib10)\], BoolQ\[[9](https://arxiv.org/html/2608.12953#bib.bib9)\], LAMBADA\[[48](https://arxiv.org/html/2608.12953#bib.bib64)\], Winogrande\[[55](https://arxiv.org/html/2608.12953#bib.bib60)\], and CoQA\[[53](https://arxiv.org/html/2608.12953#bib.bib48)\]benchmarks\. BLIMP gauges grammatical understanding by making models choose between pairs of sentences that differ in morphology, semantics, or syntax\. On the other hand, BoolQ contains binary True/False\-style questions based on a reference text, thereby testing a model’s comprehension abilities\. While we employ LAMBADA to assess generative performance by computing log\-perplexity over it, we employ its next word prediction format that offers provides models with an incomplete passage and assesses the accuracy with which they are able to predict the next word in the passage by understanding long context\. Winogrande tests semantic understanding by providing fill\-in\-the\-blanks questions with binary options, requiring models to choose the most appropriate option given the context\. CoQA contains questions based on a multi\-turn conversation between two actors, thereby evaluating a model’s ability to understand the nuances of long conversations and how they differ from regular text passages\.

#### Safety, Bias & and Ethics\.

We assess a pruned model’s alignment to safety and ethical principles by evaluating it on the challenging Winogender\[[54](https://arxiv.org/html/2608.12953#bib.bib50)\], TruthfulQA\[[35](https://arxiv.org/html/2608.12953#bib.bib49)\], and Moral Stories\[[16](https://arxiv.org/html/2608.12953#bib.bib51)\]tasks\. Winogender assess systemic bias by offering pairs of highly similar texts that differ only by the gender of a pronoun \(e\.g\., “he” changed to “she”\)\. TruthfulQA poses questions to models which humans may answer incorrectly due to personal biases or misconceptions, assessing the ability of a model to identify and ignore these biases to answer questions truthfully\. Lastly, Moral Stories is a dataset containing narratives describing moral and immoral decisions taken by various actors with the goal of assessing whether LLMs are able to discriminate between moral and immoral actions, thereby gauging their alignment with socially acceptable norms\.

## 12Detailed Results

### 12\.1Comparison of CRAFT Across Methods

Table 8:Quantitative analysis of theCompression Adherence Factor\(CRAFT\) for each method over a diverse range of target compression ratios \(25%, 35%, 50%, 60%, 75%\)\.As depicted in Figure[1](https://arxiv.org/html/2608.12953#S1.F1), we observe significant disparities between the target and achieved compression ratios for the analyzed baselines, serving as the primary motivator for the dual\-stage mixed\-granularity design ofSniper\. We provide the quantitative results for this key observation in Table[8](https://arxiv.org/html/2608.12953#S12.T8)\. As is evident, baselines exhibit poor adherence to the target compression ratio with CRAFTs going as low as 0\.58, thereby reducing reliability and trust\. On the contrary, we demonstrate thatSniperadheres tightly to the target CRs, making it much more reliable in practical settings\. Note that since ReplaceMe, SLEB, and ShortGPT are depth pruning baselines, they prune a fixed number of layers for a given compression ratio, thereby leading to the sameΔ\\Deltaand CRAFT\.

CRMethodGenerativeWorld UnderstandingDomain\-SpecificWikitextLambadaPIQAPROSTCommonsenseQAOpenbookQAMathQAARC\-EasyARC\-ChallengeMedQA\(Log\-PPL↓\\downarrow\)\(Log\-PPL↓\\downarrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\-Base2\.161\.2280\.9641\.2877\.4043\.6039\.6079\.6755\.0363\.4725%ReplaceMe4\.475\.3668\.2335\.9673\.7133\.2027\.0753\.4537\.6352\.55SliceGPT4\.515\.7255\.3929\.8537\.1025\.8023\.5235\.6124\.8324\.59LLM\-Pruner3\.504\.4368\.4431\.4621\.7929\.6024\.9952\.5729\.1824\.35SLEB3\.443\.1770\.8929\.3326\.3734\.8026\.4060\.3533\.4530\.32ShortGPT10\.7515\.6564\.3633\.8947\.8329\.2025\.0642\.9733\.3627\.812SSP3\.272\.2768\.6630\.2849\.0633\.0027\.7157\.7435\.9228\.99Sniper3\.422\.8971\.2231\.8052\.4236\.4026\.4059\.3037\.3735\.5135%ReplaceMe6\.045\.6860\.9934\.2719\.9829\.8023\.7541\.8423\.2126\.47SliceGPT5\.197\.7054\.0830\.7530\.4727\.0021\.8431\.8222\.4425\.30LLM\-Pruner4\.275\.6162\.0228\.4821\.0528\.0025\.1343\.2224\.8326\.87SLEB4\.134\.8167\.3629\.7720\.4829\.2024\.0547\.3927\.8226\.47ShortGPT11\.8514\.4359\.5831\.7830\.9627\.0022\.2534\.2630\.1231\.342SSP3\.973\.5760\.4528\.6422\.7726\.6025\.3340\.9925\.2628\.75Sniper4\.415\.0065\.8329\.5528\.0928\.6024\.3246\.8433\.1128\.04

\(continued\)

Table 9:Task\-specific pruning results for different pruning methods on Llama\-3\.1\-8B\-Instruct across various compression ratios \(CR\) without RFT\.CRMethodGenerativeWorld UnderstandingDomain\-SpecificWikitextLambadaPIQAPROSTCommonsenseQAOpenbookQAMathQAARC\-EasyARC\-ChallengeMedQA\(Log\-PPL↓\\downarrow\)\(Log\-PPL↓\\downarrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\-Base2\.161\.2280\.9641\.2877\.4043\.6039\.6079\.6755\.0363\.4725%ReplaceMe2\.921\.8070\.7333\.6170\.6034\.4027\.9459\.3439\.5955\.46SliceGPT3\.293\.9161\.3732\.5353\.4026\.4023\.8944\.6630\.9724\.51LLM\-Pruner2\.792\.2972\.6329\.5230\.9634\.2027\.1458\.9236\.6024\.75SLEB2\.721\.7973\.5624\.7944\.2337\.6029\.6260\.5636\.8629\.30ShortGPT2\.951\.9870\.6232\.9374\.1237\.4026\.9456\.6538\.6552\.472SSP2\.641\.6373\.5629\.1859\.9534\.6031\.3662\.5440\.1930\.09Sniper2\.851\.7270\.4034\.7368\.5535\.6028\.5158\.1740\.1946\.1135%ReplaceMe3\.402\.9868\.7230\.3920\.3128\.6024\.2945\.8326\.2823\.10SliceGPT3\.655\.3758\.9829\.4845\.8627\.2022\.7839\.9427\.3024\.82LLM\-Pruner3\.072\.8167\.4125\.1322\.9332\.1026\.4351\.0531\.0625\.22SLEB3\.052\.3669\.3727\.2125\.1431\.8025\.9352\.8231\.4922\.70ShortGPT3\.342\.5466\.3829\.7534\.8129\.2024\.2647\.7732\.9428\.362SSP2\.902\.0069\.7026\.4846\.6830\.4028\.8855\.4334\.9028\.67Sniper3\.092\.1668\.7726\.9249\.8829\.4026\.3051\.3132\.0834\.56

\(continued\)

Table 10:Task\-specific pruning results for different pruning methods on Llama\-3\.1\-8B\-Instruct across various compression ratios \(CR\) with RFT\.CRMethodGenerativeWorld UnderstandingDomain\-SpecificWikitextLambadaPIQAPROSTCommonsenseQAOpenbookQAMathQAARC\-EasyARC\-ChallengeMedQA\(Log\-PPL↓\\downarrow\)\(Log\-PPL↓\\downarrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\-Base2\.501\.5277\.5342\.9078\.5441\.6049\.7581\.0656\.2364\.3425%ReplaceMe4\.114\.6462\.3028\.1321\.7931\.6023\.1542\.5925\.5128\.20SliceGPT13\.0318\.4151\.4729\.6020\.9725\.6019\.4326\.3126\.2025\.69LLM\-Pruner3\.323\.0964\.2628\.4653\.4828\.8028\.4146\.5925\.6829\.77SLEB3\.843\.6666\.7628\.7233\.8431\.8025\.8059\.7237\.1730\.17ShortGPT6\.7311\.2362\.0829\.5539\.6429\.6021\.9441\.9231\.1440\.852SSP5\.127\.6255\.5030\.6818\.1827\.8021\.1132\.4123\.2926\.94Sniper3\.553\.7563\.6630\.0458\.7230\.6026\.5752\.4434\.8136\.6135%ReplaceMe6\.125\.8763\.0630\.6719\.1732\.0023\.0548\.4929\.3528\.59SliceGPT13\.4218\.8551\.5229\.5519\.4126\.0020\.2426\.5224\.9226\.24LLM\-Pruner4\.555\.9556\.4528\.4232\.8428\.0022\.2132\.4123\.1226\.63SLEB4\.904\.9960\.9432\.7521\.7029\.6023\.7549\.5028\.2428\.67ShortGPT8\.9812\.3658\.9830\.1947\.5032\.0022\.7133\.3329\.4437\.312SSP5\.639\.2152\.6132\.4219\.0826\.0021\.9430\.2223\.7227\.73Sniper4\.455\.7759\.6832\.1540\.3027\.2022\.8842\.7227\.0532\.29

\(continued\)

Table 11:Task\-specific pruning results for different pruning methods on Qwen3\-8B across various compression ratios \(CR\) without RFT\.CRMethodGenerativeWorld UnderstandingDomain\-SpecificWikitextLambadaPIQAPROSTCommonsenseQAOpenbookQAMathQAARC\-EasyARC\-ChallengeMedQA\(Log\-PPL↓\\downarrow\)\(Log\-PPL↓\\downarrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\-Base2\.501\.5277\.5342\.9078\.5441\.6049\.7581\.0656\.2364\.3425%ReplaceMe2\.932\.6070\.4627\.0034\.4836\.2027\.7757\.0334\.7328\.44SliceGPT8\.5912\.9251\.2032\.4219\.8225\.8020\.0727\.6525\.0926\.63LLM\-Pruner2\.832\.0970\.0228\.1462\.1634\.0031\.6655\.8133\.9630\.95SLEB2\.982\.2469\.8030\.3951\.0234\.4029\.0855\.8537\.3735\.19ShortGPT3\.442\.8567\.0332\.1868\.3930\.2025\.9052\.3633\.9654\.282SSP2\.822\.0469\.7532\.2168\.9632\.2026\.7361\.3638\.1435\.98Sniper3\.042\.4168\.0132\.2967\.4931\.4031\.0962\.8838\.9942\.4235%ReplaceMe4\.154\.2561\.8629\.8122\.6928\.4022\.1439\.9427\.7326\.08SliceGPT8\.5213\.1351\.3632\.1619\.6624\.6019\.4626\.9824\.0627\.81LLM\-Pruner3\.303\.6363\.8727\.8244\.0629\.0023\.4551\.8528\.7526\.24SLEB3\.382\.8364\.9130\.8239\.7232\.0025\.9352\.8232\.6828\.59ShortGPT3\.804\.1764\.0431\.8554\.8731\.0023\.7545\.9629\.9543\.442SSP3\.523\.5964\.0428\.2346\.8529\.2023\.4547\.1430\.2929\.62Sniper3\.363\.0665\.2931\.4555\.6929\.2026\.2053\.6631\.6636\.53

\(continued\)

Table 12:Task\-specific pruning results for different pruning methods on Qwen3\-8B across various compression ratios \(CR\) with RFT\.CRMethodGenerativeWorld UnderstandingDomain\-SpecificWikitextLambadaPIQAPROSTCommonsenseQAOpenbookQAMathQAARC\-EasyARC\-ChallengeMedQA\(Log\-PPL↓\\downarrow\)\(Log\-PPL↓\\downarrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\-Base2\.051\.2581\.3454\.4674\.5345\.2047\.0772\.8156\.0662\.5335%ShortGPT3\.383\.3268\.8354\.6539\.0736\.4030\.5560\.1441\.5531\.582SSP2\.773\.3865\.5134\.8142\.5128\.0024\.9950\.4632\.3430\.95Sniper2\.772\.3373\.9434\.0066\.2637\.6032\.6366\.0842\.5841\.79

\(continued\)

Table 13:Task\-specific pruning results for different pruning methods on Phi\-4 for a CR of 35% with RFT\.CRMethodGenerativeWorld UnderstandingDomain\-SpecificWikitextLambadaPIQAPROSTCommonsenseQAOpenbookQAMathQAARC\-EasyARC\-ChallengeMedQA\(Log\-PPL↓\\downarrow\)\(Log\-PPL↓\\downarrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\-Base2\.591\.8678\.6245\.2873\.0640\.6047\.8477\.5351\.8862\.5335%ShortGPT3\.172\.9569\.6430\.5638\.3332\.0033\.8760\.4033\.7930\.092SSP3\.262\.7373\.1233\.4361\.9235\.2031\.9965\.2840\.2739\.91Sniper3\.042\.3872\.9632\.3961\.5137\.0035\.7566\.1642\.4145\.09

\(continued\)

Table 14:Task\-specific pruning results for different pruning methods on GPT\-OSS\-20B for a CR of 35% with RFT\.
### 12\.2Task\-Wise Performance

Tables[9](https://arxiv.org/html/2608.12953#S12.T9),[10](https://arxiv.org/html/2608.12953#S12.T10),[11](https://arxiv.org/html/2608.12953#S12.T11), and[12](https://arxiv.org/html/2608.12953#S12.T12)provide the detailed task\-wise results for each method on Llama\-3\.1\-8B\-Instruct and Qwen3\-8B with and without RFT at compression ratios of 25% and 35%\.Sniperdemonstrates noteworthy superiority over its baselines across the different testing configurations, repeatedly outperforming them on average and exhibiting the highest per\-task consistency among them\. Even in the instance whereSniperlags slightly behind a baseline in average performance \(Table[10](https://arxiv.org/html/2608.12953#S12.T10), 25% CR\), it still retains the lowest per\-task standard deviation of the group, providing a strong testimony to its excellent robustness to distributional bias\.

We also provide the task\-wise results for Phi\-4 and GPT\-OSS\-20B in Tables[13](https://arxiv.org/html/2608.12953#S12.T13)and[14](https://arxiv.org/html/2608.12953#S12.T14), respectively, whereSnipersurpasses ShortGPT and 2SSP on Avg RP by notable margins\. In addition to delivering excellent performance,Sniperalso maintains a considerably lower Std RP compared to them, indicating its impressive ability to adapt to unique architectures\.

### 12\.3Sensitivity to Calibration Data

\# Calibration SamplesMethodGenerativeWorld UnderstandingDomain\-SpecificWikitextLambadaPIQAPROSTCommonsenseQAOpenbookQAMathQAARC\-EasyARC\-ChallengeMedQA\(Log\-PPL↓\\downarrow\)\(Log\-PPL↓\\downarrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)50ReplaceMe2\.932\.6070\.4627\.0034\.4836\.2027\.7757\.0334\.7328\.44SliceGPT8\.5912\.9251\.2032\.4219\.8225\.8020\.0727\.6525\.0926\.63LLM\-Pruner2\.832\.0970\.0228\.1462\.1634\.0031\.6655\.8133\.9630\.95SLEB2\.982\.2469\.8030\.3951\.0234\.4029\.0855\.8537\.3735\.19ShortGPT3\.442\.8567\.0332\.1868\.3930\.2025\.9052\.3633\.9654\.282SSP2\.822\.0469\.7532\.2168\.9632\.2026\.7361\.3638\.1435\.98Sniper3\.042\.4168\.0132\.2967\.4931\.4031\.0962\.8838\.9942\.42250ReplaceMe2\.912\.5471\.0027\.0433\.9936\.0027\.9757\.7436\.5228\.12SliceGPT8\.3913\.1050\.1134\.6919\.6628\.0020\.3027\.6924\.4027\.34LLM\-Pruner2\.862\.2970\.2430\.1760\.6132\.8031\.6663\.3438\.0630\.09SLEB3\.002\.5572\.0428\.5731\.1239\.0029\.0159\.5137\.1229\.22ShortGPT3\.412\.7966\.8134\.2672\.9729\.8026\.1652\.2735\.5056\.012SSP3\.082\.9565\.8332\.1157\.0829\.8024\.2950\.8431\.9131\.27Sniper3\.012\.5069\.5932\.4361\.0232\.8028\.7462\.4638\.4838\.571000ReplaceMe2\.912\.5471\.0027\.0433\.9936\.0027\.9757\.7436\.5228\.12SliceGPT7\.9211\.9851\.0928\.9818\.7627\.2020\.3427\.6523\.3826\.24LLM\-Pruner2\.942\.3568\.5527\.7859\.5433\.6029\.0560\.9937\.8830\.56SLEB2\.982\.5870\.8929\.7737\.3535\.2029\.9560\.1934\.7331\.19ShortGPT3\.412\.7867\.1433\.2275\.4330\.2025\.6051\.0534\.9058\.052SSP2\.973\.0267\.5730\.6860\.6929\.0024\.2653\.4532\.8529\.93Sniper3\.012\.4969\.7031\.4760\.2032\.8028\.7462\.1637\.7138\.96

\(continued\)

Table 15:Detailed results for the impact of varying number of calibration samples selected from the Slim Orca dataset to identify model components to prune from Qwen3\-8B\. All results have been obtained after RFT\.Calibration DatasetMethodGenerativeWorld UnderstandingDomain\-SpecificWikitextLambadaPIQAPROSTCommonsenseQAOpenbookQAMathQAARC\-EasyARC\-ChallengeMedQA\(Log\-PPL↓\\downarrow\)\(Log\-PPL↓\\downarrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)Slim OrcaReplaceMe2\.932\.6070\.4627\.0034\.4836\.2027\.7757\.0334\.7328\.44\[[34](https://arxiv.org/html/2608.12953#bib.bib52)\]SliceGPT8\.5912\.9251\.2032\.4219\.8225\.8020\.0727\.6525\.0926\.63LLM\-Pruner2\.832\.0970\.0228\.1462\.1634\.0031\.6655\.8133\.9630\.95SLEB2\.982\.2469\.8030\.3951\.0234\.4029\.0855\.8537\.3735\.19ShortGPT3\.442\.8567\.0332\.1868\.3930\.2025\.9052\.3633\.9654\.282SSP2\.822\.0469\.7532\.2168\.9632\.2026\.7361\.3638\.1435\.98Sniper3\.042\.4168\.0132\.2967\.4931\.4031\.0962\.8838\.9942\.42AlpacaReplaceMe3\.784\.1165\.7842\.4740\.0532\.6025\.9652\.3235\.4130\.32\[[63](https://arxiv.org/html/2608.12953#bib.bib4)\]SliceGPT9\.5513\.8450\.8234\.4019\.5726\.4020\.3727\.9525\.6027\.73LLM\-Pruner3\.002\.7368\.5029\.6851\.8432\.6030\.5261\.6236\.7832\.05SLEB3\.513\.2765\.4532\.8641\.7729\.4029\.0852\.0232\.5933\.46ShortGPT3\.292\.5667\.5735\.3272\.5633\.8026\.9755\.4337\.2052\.552SSP2\.942\.4968\.2331\.0861\.0231\.0028\.9161\.0738\.7436\.61Sniper3\.012\.4970\.1932\.8661\.3432\.6030\.3261\.9539\.3338\.26C4ReplaceMe3\.784\.1165\.7842\.4740\.0532\.6025\.9652\.3235\.4130\.32\[[52](https://arxiv.org/html/2608.12953#bib.bib5)\]SliceGPT9\.2413\.9148\.3734\.0119\.5726\.2020\.5427\.6925\.0027\.73LLM\-Pruner3\.002\.7267\.4128\.7853\.1533\.8030\.2561\.6236\.3530\.48SLEB3\.212\.5769\.4836\.9255\.6134\.6031\.3962\.0039\.7633\.78ShortGPT3\.633\.6866\.0543\.0774\.4531\.2026\.5752\.6534\.0451\.932SSP3\.253\.4462\.5726\.5729\.8128\.4023\.9946\.1729\.3527\.34Sniper3\.012\.4969\.0433\.5261\.7532\.6029\.9862\.4639\.0839\.20

\(continued\)

Table 16:Detailed results for impact of calibration data distribution on post\-pruning performance of Qwen3\-8B at a compression ratio of 25%\. Sample count is kept constant at 50\. All results have been obtained after RFT\.We analyze the effect of varying calibration\-related factors, namely, dataset size and data distribution, on each method’s performance\. We provide these results in Tables[15](https://arxiv.org/html/2608.12953#S12.T15)and[16](https://arxiv.org/html/2608.12953#S12.T16), respectively\. In line with the analysis presented in Section[6](https://arxiv.org/html/2608.12953#S6),Sniperdemonstrates excellent robustness at varying calibration configurations where existing methods tend to deteriorate either in terms of average performance or per\-task consistency\.

### 12\.4Transferability of Importance Scores

CRTransfer RegimeGenerativeWorld UnderstandingDomain\-SpecificWikitextLambadaPIQAPROSTCommonsenseQAOpenbookQAMathQAARC\-EasyARC\-ChallengeMedQA\(Log\-PPL↓\\downarrow\)\(Log\-PPL↓\\downarrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)\(Acc↑\\uparrow\)25%25%→25%25\\%\\rightarrow 25\\%\(No transfer\)3\.553\.7563\.6630\.0458\.7230\.6026\.5752\.4434\.8136\.6135%→25%35\\%\\rightarrow 25\\%\(High\-to\-low\)3\.553\.7663\.4429\.9358\.4030\.6026\.8052\.5735\.0735\.5950%→25%50\\%\\rightarrow 25\\%\(High\-to\-low\)3\.553\.7164\.3129\.5254\.6330\.8026\.2755\.7233\.2832\.6035%35%→35%35\\%\\rightarrow 35\\%\(No transfer\)4\.415\.0065\.8329\.5528\.0928\.6024\.3246\.8433\.1128\.0425%→35%25\\%\\rightarrow 35\\%\(Low\-to\-high\)4\.385\.7660\.3431\.4936\.3625\.4023\.1840\.9526\.2030\.7250%→35%50\\%\\rightarrow 35\\%\(High\-to\-low\)4\.525\.6460\.1232\.5040\.6228\.8022\.5841\.5826\.4532\.7650%50%→50%50\\%\\rightarrow 50\\%\(No transfer\)7\.7415\.3955\.4432\.6121\.0528\.8020\.1731\.5225\.9427\.5725%→50%25\\%\\rightarrow 50\\%\(Low\-to\-high\)7\.4611\.6555\.8231\.4719\.2527\.0019\.3031\.8623\.9827\.1835%→50%35\\%\\rightarrow 50\\%\(Low\-to\-high\)8\.4314\.5055\.9933\.9520\.4825\.4020\.5432\.9623\.9828\.28

\(continued\)

Table 17:Detailed results forSniperunder various transfer regimes wherein importance scores computed for a particular compression ratio are utilized to prune a model to a different capacity\.Sniperdemonstrates impressive flexibility by maintaining excellent performance even when utilizing heuristics that were not computed for the target compression ratio, as evident by the detailed results in Table[17](https://arxiv.org/html/2608.12953#S12.T17)\. This characteristic enhancesSniper’s practical utility by offering an avenue for users to compute these heuristics once and re\-use them for varying target compression ratios\.