COEC: Calibrated Orthogonal-Equivalence Compensation for Structured Pruning of Large Language Models

# Calibrated Orthogonal-Equivalence Compensation for Structured Pruning of Large Language Models
Source: [https://arxiv.org/html/2608.21142](https://arxiv.org/html/2608.21142)
###### Abstract

Structured pruning reduces the size and inference cost of large language models \(LLMs\) by removing weight columns, but the resulting output error can degrade accuracy\. Existing training\-free compensation methods use an additive bias or a single orthogonal rotation on the output side of the retained weight\. These corrections leave its input singular frame unchanged and therefore limit how the retained weight can adapt after column removal\. We propose COEC \(Calibrated Orthogonal\-Equivalence Compensation\), a training\-free compensation framework that applies alternating left and right orthogonal rotations to the retained weight\. The right rotation is optimized on a reduced Stiefel manifold, while singular values are rescaled using generalized cross\-validation to select the regularization strength for each layer\. COEC further tempers the calibration Gram matrix to reduce the dominance of high\-energy activation directions and introduces an alignment penalty that preserves the geometric relation between adjacent attention projections\.All components use second\-order statistics from a small calibration set and require neither backpropagation through the LLM nor retraining of the model parameters\. COEC is independent of the column pruning criterion and can be applied to multiple structured pruning methods\. Experiments on the Llama\-3, Llama\-3\.1, and Qwen2\.5 model families across multiple structured sparsity levels show that COEC improves perplexity on every model and zero\-shot accuracy in most settings over existing compensation methods, with larger gains at higher sparsity\. These results show that post\-pruning compensation can recover part of the performance lost to column removal\.

## 1Introduction

Figure 1:Overview of COEC\. Columns are pruned using an arbitrary importance score, and the retained weightWKW\_\{K\}is compensated asW~=Qℓ​WK\(S\)​Qr\\widetilde\{W\}=Q\_\{\\ell\}\\,W\_\{K\}^\{\(S\)\}\\,Q\_\{r\}\. The left and right rotations adjust the output and input singular frames, while per\-mode rescaling adapts the singular values\. All updates use calibration statistics only, without LLM backpropagation or model retraining\. The objective combines relative reconstruction error with an alignment penalty on the attention output projection\.Large language models \(LLMs\) are increasingly deployed across a wide range of applications, yet their inference costs in computation and memory remain substantial\. Common compression techniques include quantization\([11](https://arxiv.org/html/2608.21142#bib.bib20);[25](https://arxiv.org/html/2608.21142#bib.bib21);[43](https://arxiv.org/html/2608.21142#bib.bib22)\), distillation\([21](https://arxiv.org/html/2608.21142#bib.bib23);[17](https://arxiv.org/html/2608.21142#bib.bib24)\), and pruning\([24](https://arxiv.org/html/2608.21142#bib.bib25);[20](https://arxiv.org/html/2608.21142#bib.bib26)\)\. Among them, structured pruning removes parameters at the granularity of entire weight\-matrix columns, including MLP neurons and attention key–value groups\. This produces a smaller dense model with reduced parameter count, memory usage, and computational cost on standard hardware\. Because retraining at LLM scale is expensive, structured pruning methods that preserve accuracy without additional training are particularly attractive\.

A training\-free structured pruning pipeline comprises two largely separable components: a*selection*rule that determines which columns to retain and a*compensation*mechanism that adjusts the retained weights to reduce the output error induced by pruning\. Existing methods have primarily focused on selection, while treating compensation as secondary\. Wanda\-sp\([37](https://arxiv.org/html/2608.21142#bib.bib3)\)extends Wanda’s activation\-aware importance score to column pruning but performs no compensation, leaving the output discrepancy caused by the removed columns uncorrected\. FLAP\([2](https://arxiv.org/html/2608.21142#bib.bib7)\)compensates for the mean error through an additive bias term\. This approach is simple and effective at low sparsity, but a constant bias cannot correct input\-dependent directional errors\. Direct least\-squares refitting of the retained weights can reduce calibration error, but with a limited calibration set it is prone to overfitting and may substantially distort the pretrained weights\([19](https://arxiv.org/html/2608.21142#bib.bib2)\)\. More recently, RCPU\([19](https://arxiv.org/html/2608.21142#bib.bib2)\)constrains compensation to an orthogonal rotation of the output space followed by global rescaling\. This geometry\-preserving formulation avoids the distortions introduced by unconstrained refitting and achieves state\-of\-the\-art performance\.

However, RCPU applies rotation only on the left\. Let the singular value decomposition \(SVD\) of the retained weight beWK=UK​ΣK​VK⊤W\_\{K\}=U\_\{K\}\\Sigma\_\{K\}V\_\{K\}^\{\\top\}\. A left rotationQ​WKQW\_\{K\}changes the output frameUKU\_\{K\}while leaving the input frameVKV\_\{K\}unchanged\. This one\-sided constraint is limiting because column pruning removes input coordinates and alters the correlation structure represented by the retained columns\. Effective compensation must therefore recover, through the retained columns, the output contributions previously associated with the removed columns\. Our analysis in Sec\.[5\.6](https://arxiv.org/html/2608.21142#S5.SS6)shows that pruning substantially rotates the input singular subspace across all examined layers, and neither left\-sided rotation nor reconstruction\-optimal compensation generally restores this subspace\.

Motivated by these limitations, we propose COEC \(Calibrated Orthogonal\-Equivalence Compensation\), a general post\-pruning compensation framework designed as a universal plug\-in \(Fig\.[1](https://arxiv.org/html/2608.21142#S1.F1)\)\. COEC takes the retained set produced by any column\-importance criterion and modifies only the retained weights\. It can therefore be applied after existing structured pruning methods without changing their selection rules or model architectures\. Rather than refitting the retained weights through unconstrained least squares, COEC searches within a spectrum\-preserving family that applies rotations on both sides and rescales individual singular modes\. COEC alternates between a closed\-form left rotation and an iterative right rotation optimized over a reduced Stiefel manifold\. Because the right\-rotation objective is coupled with the overall scale, the optimal scaling factor is recomputed at every iteration\. COEC further replaces global rescaling with per\-mode singular\-value rescaling, with the regularization strength selected independently for each layer through closed\-form generalized cross\-validation\. Two additional components improve the calibration objective and preserve inter\-layer geometry\. Spectral tempering of the calibration Gram matrix balances perplexity and zero\-shot accuracy, while an alignment penalty anchored to the original input frame restores geometric relations between adjacent layers that reconstruction objectives alone fail to preserve\. All components depend only on second\-moment statistics estimated from a small calibration set and require neither backpropagation through the LLM nor retraining of the model parameters\.

We evaluate COEC on the Llama\-3, Llama\-3\.1, and Qwen2\.5 model families across multiple structured sparsity levels\. Under identical column selections, COEC improves perplexity on every model and zero\-shot accuracy in most settings over existing compensation methods, with larger gains at higher pruning ratios\. When applied to the column selections produced by Wanda\-sp and FLAP, COEC recovers a substantial portion of the accuracy lost during pruning\. Our main contributions are summarized as follows\.

- •We formulate post\-pruning compensation as a two\-sided spectrum\-preserving update\. We develop an alternating solver that combines a closed\-form left rotation, a reduced\-Stiefel right rotation with in\-loop rescaling, per\-mode rescaling selected by generalized cross\-validation, spectral tempering, and an inter\-layer alignment penalty\. We isolate the effect of each component through ablation studies in Sec\.[5\.4](https://arxiv.org/html/2608.21142#S5.SS4)\.
- •We design COEC as a criterion\-agnostic compensation method that can be applied after any column\-selection rule without changing the model architecture\. We demonstrate this plug\-in capability using the selections produced by Wanda\-sp, FLAP, and RCPU in Sec\.[5\.2](https://arxiv.org/html/2608.21142#S5.SS2)\.
- •We evaluate COEC under a unified protocol on the Llama\-3, Llama\-3\.1, and Qwen2\.5 model families across multiple structured sparsity levels\. COEC improves perplexity on every model and zero\-shot accuracy in most settings over Wanda\-sp, FLAP, and RCPU in Sec\.[5\.2](https://arxiv.org/html/2608.21142#S5.SS2)\.

## 2Related Work

Pruning of LLMs\.Post\-training pruning reduces the size and inference cost of LLMs without retraining, building on classical saliency criteria\([24](https://arxiv.org/html/2608.21142#bib.bib25);[20](https://arxiv.org/html/2608.21142#bib.bib26);[9](https://arxiv.org/html/2608.21142#bib.bib27);[34](https://arxiv.org/html/2608.21142#bib.bib28)\)\. Unstructured and semi\-structured methods sparsify weights element\-wise, by per\-row least\-squares reconstruction in SparseGPT\([10](https://arxiv.org/html/2608.21142#bib.bib4)\)or activation\-aware scoring in Wanda\([37](https://arxiv.org/html/2608.21142#bib.bib3)\), but need sparse kernel support to realize speedups\. Structured methods remove entire columns, rows, or blocks and directly shrink the dense model, using gradient saliency\([28](https://arxiv.org/html/2608.21142#bib.bib5)\), masks learned with continued pretraining\([42](https://arxiv.org/html/2608.21142#bib.bib29)\), curvature information\([23](https://arxiv.org/html/2608.21142#bib.bib30);[38](https://arxiv.org/html/2608.21142#bib.bib31)\), or block\-level redundancy\([29](https://arxiv.org/html/2608.21142#bib.bib8);[36](https://arxiv.org/html/2608.21142#bib.bib32);[44](https://arxiv.org/html/2608.21142#bib.bib33)\); Wanda\-sp extends the Wanda score to columns, and FLAP\([2](https://arxiv.org/html/2608.21142#bib.bib7)\)scores channels by activation variance\. Low\-rank factorization\([45](https://arxiv.org/html/2608.21142#bib.bib34);[40](https://arxiv.org/html/2608.21142#bib.bib35)\)also operates in the singular\-value domain but truncates the spectrum, which our compensation preserves\. We work in the structured, training\-free setting\.

Post\-pruning compensation\.Methods differ in how they repair the kept weights after removal\. Wanda\-sp applies no update\. FLAP\([2](https://arxiv.org/html/2608.21142#bib.bib7)\)corrects the mean error with a bias term, which cannot fix input\-dependent directional mismatches\. Least\-squares refitting reduces calibration error further but overfits small calibration sets and damages pretrained weights\([19](https://arxiv.org/html/2608.21142#bib.bib2)\)\.RCPU\([19](https://arxiv.org/html/2608.21142#bib.bib2)\)restricts the update to an output\-frame rotation with a global scale, preserving geometry and overfitting less\. We extend this one\-sided rotation to a converged two\-sided update with per\-mode rescaling, independent of the selection criterion\.

Rotation\-based transformations\.SliceGPT\([3](https://arxiv.org/html/2608.21142#bib.bib6)\)rotates RMSNorm\-connected transformers under computational invariance and prunes in the rotated basis; QuaRot\([4](https://arxiv.org/html/2608.21142#bib.bib9)\), SpinQuant\([26](https://arxiv.org/html/2608.21142#bib.bib36)\), and DenoiseRotator\([16](https://arxiv.org/html/2608.21142#bib.bib10)\)use lossless rotations to condition weights for compression\. All of these rotate before compression under functional equivalence, whereas we rotate after pruning to compensate the removal error\.

Calibration\-aware rescaling\.Scale and bias corrections appear in FLAP and in energy\-matching heuristics\. Ours is instead a diagonal fit in the SVD frame of the rotated weight, with the ridge strength set per layer by generalized cross\-validation \(GCV\)\([13](https://arxiv.org/html/2608.21142#bib.bib11);[39](https://arxiv.org/html/2608.21142#bib.bib39)\), a classical criterion that to our knowledge has not been used for pruning compensation\. The right rotation is a weighted orthogonal Procrustes problem\([14](https://arxiv.org/html/2608.21142#bib.bib40)\)with no closed\-form solution, which we solve on the reduced Stiefel manifold\([8](https://arxiv.org/html/2608.21142#bib.bib12);[1](https://arxiv.org/html/2608.21142#bib.bib37)\)\(Sec\.[4\.2](https://arxiv.org/html/2608.21142#S4.SS2)\)\.

## 3Problem Formulation

### 3\.1Notation and Setup

For each transformer block, we prune structured channels in both the MLP and attention sub\-layers\. In the MLP, pruning removes intermediate channels by deleting the corresponding rows of the gating and value projections and the matching columns of the output projection\. In grouped\-query attention, pruning removes complete KV groups together with their associated query heads\. This corresponds to deleting the associated output rows of the query, key, and value projections and the matching input columns of the attention output projection\. The grouped\-query attention structure is preserved after pruning\.

LetW0∈ℝdout×dinW\_\{0\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\\times d\_\{\\mathrm\{in\}\}\}denote the pretrained weight matrix of a pruned sub\-layer\. LetX∈ℝdin×NX\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{in\}\}\\times N\}denote the calibration activations fromNNtoken positions, andY=W0​XY=W\_\{0\}Xthe corresponding output of the original layer\. A column\-selection rule retains an index setKKwith\|K\|=k\|K\|=k, andXK∈ℝk×NX\_\{K\}\\in\\mathbb\{R\}^\{k\\times N\}contains the rows ofXXindexed byKK\. The compensation step constructs a weight matrixW~∈ℝdout×k\\widetilde\{W\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\\times k\}acting onXKX\_\{K\}\. Its quality is measured by the relative reconstruction error:

∥Y−W~​XK∥F2∥Y∥F2=tr⁡\(W~​Gx​x​W~⊤\)−2​tr​\(W~​Gy​x⊤\)\+∥Y∥F2∥Y∥F2\.\\frac\{\\lVert Y\-\\widetilde\{W\}X\_\{K\}\\rVert\_\{F\}^\{2\}\}\{\\lVert Y\\rVert\_\{F\}^\{2\}\}=\\frac\{\\mathrm\{tr\}\(\\widetilde\{W\}G\_\{xx\}\\widetilde\{W\}^\{\\top\}\)\-2\\,\\mathrm\{tr\}\(\\widetilde\{W\}G\_\{yx\}^\{\\top\}\)\+\\lVert Y\\rVert\_\{F\}^\{2\}\}\{\\lVert Y\\rVert\_\{F\}^\{2\}\}\.The objective depends only on the second\-moment statisticsG=X​X⊤G=XX^\{\\top\},Gx​x=G⁡\[K,K\]G\_\{xx\}=G\[K,K\], andGy​x=W0G\[:,K\]G\_\{yx\}=W\_\{0\}\\,G\[:,K\], accumulated in a single calibration pass\. The token\-level activations do not need to be stored after accumulation\. In practice, 128 calibration sequences are sufficient \(Sec\.[5\.1](https://arxiv.org/html/2608.21142#S5.SS1)\)\.

### 3\.2Limitation of Least\-Squares Compensation

The unconstrained least\-squares solution isWLS=Gy​x​Gx​x−1,W^\{\\mathrm\{LS\}\}=G\_\{yx\}G\_\{xx\}^\{\-1\},whenGx​xG\_\{xx\}is invertible\. In practice, a ridge\-regularized solution is used asWridge=Gy​x​\(Gx​x\+λls​I\)−1W^\{\\mathrm\{ridge\}\}=G\_\{yx\}\(G\_\{xx\}\+\\lambda\_\{\\mathrm\{ls\}\}I\)^\{\-1\}\. Consistent with the observations reported for RCPU, we find that least\-squares refitting can fit the small calibration set too closely\. It reshapes the singular spectrum of the retained weight according to the calibration statistics and shows unstable generalization across models\.

To limit such unrestricted changes, we parameterize the compensated weight as

W~=Qℓ​WK\(S\)​Qr,Qℓ∈O⁡\(dout\),Qr∈O⁡\(k\),\\widetilde\{W\}=Q\_\{\\ell\}\\,W\_\{K\}^\{\(S\)\}\\,Q\_\{r\},\\qquad Q\_\{\\ell\}\\in O\(d\_\{\\mathrm\{out\}\}\),\\;Q\_\{r\}\\in O\(k\),whereWK\(S\)W\_\{K\}^\{\(S\)\}is obtained by applying a regularized rescaling to the singular modes of the retained weight\. The left and right orthogonal transforms adjust the output and input singular frames, respectively, while the mode\-wise rescaling controls changes to the singular spectrum\. This structured parameterization avoids the unrestricted deformation introduced by direct least\-squares refitting\. COEC estimates these components by minimizing, for each pruned module,

ℒ⁡\(W~,Qr\)=∥Y−W~​XK∥F2∥Y∥F2\+λa​𝒜​\(Qr\),\\mathcal\{L\}\(\\widetilde\{W\},Q\_\{r\}\)=\\frac\{\\lVert Y\-\\widetilde\{W\}X\_\{K\}\\rVert\_\{F\}^\{2\}\}\{\\lVert Y\\rVert\_\{F\}^\{2\}\}\+\\lambda\_\{a\}\\,\\mathcal\{A\}\(Q\_\{r\}\),\(1\)where𝒜\\mathcal\{A\}is the inter\-layer alignment penalty of Sec\.[4\.5](https://arxiv.org/html/2608.21142#S4.SS5), applied to the attention output projection withλa=0\\lambda\_\{a\}=0elsewhere\. The following sections describe how these components are estimated\.

## 4Method

### 4\.1Two\-Sided Rotation with In\-Loop Scaling

We initialize the compensated weight with the retained column slice asWK=W0\[:,K\]W\_\{K\}=W\_\{0\}\[:,K\]\. No rank truncation or singular\-value rescaling is applied, andWKW\_\{K\}represents the uncompensated pruned weight used as the reference point in App\. C of the supplementary material\. We writeW^\\widehat\{W\}for the current compensated weight, initialized asW^=WK\\widehat\{W\}=W\_\{K\}and overwritten by each block update; \([1](https://arxiv.org/html/2608.21142#S3.E1)\) is evaluated atW^\\widehat\{W\}and the currentQrQ\_\{r\}\.

For a fixedW^\\widehat\{W\}, the optimal left rotation is given by the orthogonal Procrustes solution\([35](https://arxiv.org/html/2608.21142#bib.bib19)\):Qℓ=polar⁡\(Gy​x​W^⊤\)Q\_\{\\ell\}=\\mathrm\{polar\}\(G\_\{yx\}\\,\\widehat\{W\}^\{\\top\}\), wherepolar⁡\(A\)\\mathrm\{polar\}\(A\)denotes the orthogonal factorUA​VA⊤U\_\{A\}V\_\{A\}^\{\\top\}of the SVDA=UA​ΣA​VA⊤A=U\_\{A\}\\Sigma\_\{A\}V\_\{A\}^\{\\top\}\. The left rotation minimizes the reconstruction error by aligning the outputs of the retained weight with those of the original layer on the calibration data\.

The global scale can be optimized independently of the left rotation because an orthogonal transformation preserves the norm ofW^​XK\\widehat\{W\}X\_\{K\}\. For a fixedW^\\widehat\{W\}, the optimal scale is:

s∗=⟨Gy​x,W^⟩/⟨W^,W^​Gx​x⟩,s^\{\*\}=\{\\langle G\_\{yx\},\\widehat\{W\}\\rangle\}/\{\\langle\\widehat\{W\},\\widehat\{W\}G\_\{xx\}\\rangle\},where⟨A,B⟩=tr⁡\(A​B⊤\)\\langle A,B\\rangle=\\mathrm\{tr\}\(AB^\{\\top\}\)\. The combination of a left rotation and global scaling corresponds to the compensation used by RCPU\. In COEC, it forms the left\-rotation block of the alternating compensation procedure\.

The right rotation is obtained by solving a weighted orthogonal Procrustes problem,

minQr∈O⁡\(k\)⁡∥Y−s​W^​Qr​XK∥F2\.\\min\\nolimits\_\{Q\_\{r\}\\in O\(k\)\}\\;\\lVert Y\-s\\,\\widehat\{W\}\\,Q\_\{r\}\\,X\_\{K\}\\rVert\_\{F\}^\{2\}\.Unlike the left\-rotation problem, this objective has no closed\-form solution because the rotation is coupled to both the weight matrix and the calibration covariance\. We optimizeQrQ\_\{r\}using projected gradient descent with an orthogonal retraction and initialize it asQ0=IQ\_\{0\}=I\. This identity initialization is empirically important at mild sparsity, where the optimal right rotation typically remains close to the original input frame\. Initializing from the one\-sided Procrustes solution instead performs worse, e\.g\., increasing reconstruction error on attention output projections from0\.0190\.019to0\.1590\.159\.

The right rotation is also coupled to the global scale\. We therefore recomputes∗s^\{\*\}after every update ofQrQ\_\{r\}and evaluate the gradient using the scaled residual\. The step size is normalized asη/s2\\eta/s^\{2\}to account for the scale of the objective\. This procedure allows the right rotation and global scale to converge jointly\.

The complete optimization alternates between the left rotation, scale update, right rotation, and another scale update

Qℓ→s→Qr→s\.Q\_\{\\ell\}\\rightarrow s\\rightarrow Q\_\{r\}\\rightarrow s\.We perform at most 50 alternating rounds and stop early when the change inℒ\\mathcal\{L\}falls below10−410^\{\-4\}\. The left rotation is the conditional minimizer of the reconstruction objective and leaves the alignment term unchanged, and the regularized rescale does not increase the reconstruction error in the tempered metric\. The resulting parameterization includes left\-only and right\-only compensation as special cases\.

### 4\.2Reduced\-Stiefel Solver

Directly optimizingQr∈O⁡\(k\)Q\_\{r\}\\in O\(k\)is inefficient when the compensated weight is processed in output slices\. Consider a sliceWg∈ℝb×kW\_\{g\}\\in\\mathbb\{R\}^\{b\\times k\}withb<kb<k, and letWg=Ug​Σg​Vg⊤W\_\{g\}=U\_\{g\}\\Sigma\_\{g\}V\_\{g\}^\{\\top\}, whereVg∈ℝk×bV\_\{g\}\\in\\mathbb\{R\}^\{k\\times b\}contains its right singular vectors\. The reconstruction objective depends onQrQ\_\{r\}only throughR=Vg⊤​Qr∈ℝb×kR=V\_\{g\}^\{\\top\}Q\_\{r\}\\in\\mathbb\{R\}^\{b\\times k\}\. The components ofQrQ\_\{r\}in the orthogonal complement ofVgV\_\{g\}do not affect the slice output and introduce redundant optimization directions\. We therefore optimizeRRdirectly under the constraintR​R⊤=IbRR^\{\\top\}=I\_\{b\}, which placesRRon the Stiefel manifoldSt⁡\(b,k\)\\mathrm\{St\}\(b,k\)\([8](https://arxiv.org/html/2608.21142#bib.bib12);[1](https://arxiv.org/html/2608.21142#bib.bib37)\)\.

Any feasibleR∈St⁡\(b,k\)R\\in\\mathrm\{St\}\(b,k\)can be extended to an orthogonal matrixQr∈O⁡\(k\)Q\_\{r\}\\in O\(k\)\. The reduced and full parameterizations therefore represent the same solutions for the slice objective, while the reduced formulation removes directions that do not affect the output\. Its memory and per\-step computational costs are reduced by approximately a factor ofb/kb/k\.

We use a Newton–Schulz polar retraction for rectangular frames\. For square frames, we use an exact Cayley retraction\([41](https://arxiv.org/html/2608.21142#bib.bib38)\), with the required inverse computed by Newton–Schulz iteration\. Implementation details are provided in App\. B of the supplementary material\. Optimization stops when the change in the rotation falls below a toleranceδ\\delta\. This tolerance is selected once for each model using the calibration set\.

### 4\.3GCV\-Based Singular\-Value Rescaling

A global scalar multiplies all singular values by the same factor, although the calibration statistics may favor different adjustments for different singular\-vector pairs\. Let the SVD of the current rotated weight beW^=U^​Σ^​V^⊤\\widehat\{W\}=\\widehat\{U\}\\,\\widehat\{\\Sigma\}\\,\\widehat\{V\}^\{\\top\}, whereσi=\[Σ^\]i​i\\sigma\_\{i\}=\[\\widehat\{\\Sigma\}\]\_\{ii\}is theii\-th singular value andui,viu\_\{i\},v\_\{i\}the corresponding singular vectors\. KeepingU^\\widehat\{U\}andV^\\widehat\{V\}fixed, the reconstruction\-optimal value associated with theii\-th singular\-vector pair is

sg,i⋆=ui⊤​Gy​x​vi/vi⊤​Gx​x​vi=ρi/ei\.s\_\{g,i\}^\{\\star\}=\{u\_\{i\}^\{\\top\}G\_\{yx\}v\_\{i\}\}/\{v\_\{i\}^\{\\top\}G\_\{xx\}v\_\{i\}\}=\{\\rho\_\{i\}\}/\{e\_\{i\}\}\.where the mode responseρi\\rho\_\{i\}measures the alignment between the corresponding output and input singular directions under the target cross\-covariance, while the mode energyeie\_\{i\}measures the calibration energy along the input singular direction\. When the weight has rank one, this expression reduces to the globally scaled singular values∗​σis^\{\*\}\\sigma\_\{i\}\.

Estimating each singular value independently can overfit the limited calibration data\. We therefore regularizesg,i⋆s\_\{g,i\}^\{\\star\}toward the value obtained from uniform global scaling,s∗​σis^\{\*\}\\sigma\_\{i\}\. The regularized singular value is

sg,i​\(λ\)=\(ρi\+λ​s∗​σi\)/\(ei\+λ\)\.s\_\{g,i\}\(\\lambda\)=\\left\(\{\\rho\_\{i\}\+\\lambda\\,s^\{\*\}\\sigma\_\{i\}\}\\right\)/\\left\(\{e\_\{i\}\+\\lambda\}\\right\)\.Whenλ→0\\lambda\\rightarrow 0,sg,i​\(λ\)→sg,i⋆s\_\{g,i\}\(\\lambda\)\\rightarrow s\_\{g,i\}^\{\\star\}\. Asλ\\lambdaincreases, it approaches the globally scaled values∗​σis^\{\*\}\\sigma\_\{i\}\. We selectλ\\lambdaindependently for each layer using generalized cross\-validation \(GCV\):

GCV⁡\(λ\)=∑iei​\(sg,i​\(λ\)−sg,i⋆\)2\(1−df⁡\(λ\)/M\)2,df⁡\(λ\)=∑ieiei\+λ,\\begin\{split\}\\mathrm\{GCV\}\(\\lambda\)&=\\frac\{\\sum\_\{i\}e\_\{i\}\\,\\big\(s\_\{g,i\}\(\\lambda\)\-s\_\{g,i\}^\{\\star\}\\big\)^\{2\}\}\{\\big\(1\-\\mathrm\{df\}\(\\lambda\)/M\\big\)^\{2\}\},\\\\ \\mathrm\{df\}\(\\lambda\)&=\\sum\\nolimits\_\{i\}\\frac\{e\_\{i\}\}\{e\_\{i\}\+\\lambda\},\\end\{split\}whereMMis the number of singular values included in the layer\-wise estimate\. The effective degrees of freedom penalize solutions that fit the calibration statistics too closely\. This correction is necessary because the unadjusted calibration residual is minimized atλ=0\\lambda=0\.

Asλ→0\+\\lambda\\rightarrow 0^\{\+\}, both the numerator and denominator approach zero at the same order, producing a finite limiting value\. Asλ→∞\\lambda\\rightarrow\\infty, the criterion approachesGCV⁡\(∞\)=∑iei​\(s∗​σi−sg,i⋆\)2\\mathrm\{GCV\}\(\\infty\)=\\sum\_\{i\}e\_\{i\}\\left\(s^\{\*\}\\sigma\_\{i\}\-s\_\{g,i\}^\{\\star\}\\right\)^\{2\}\. We minimize this objective over a logarithmically spaced grid for each layer\. On both validation models, the resulting layer\-specific regularization matches the performance of the best globally tuned value \(see App\. B of the supplementary material\)\.

The rescaling is computed from the SVD of the current rotated weight\. It can therefore be applied after either the left or right rotation\. We recompute the singular values within each alternating round so that the spectrum adapts jointly with the left and right singular frames\.

### 4\.4Gram Tempering

All compensation terms use the activation Gram matrixG=X​X⊤G=XX^\{\\top\},XXcontaining calibration activations\. Its diagonal entries measure activation energy, while its off\-diagonal entries capture correlations between activation dimensions\.

LetG=E​diag⁡\(μi\)​E⊤G=E\\operatorname\{diag\}\(\\mu\_\{i\}\)E^\{\\top\}be its eigendecomposition\. We temper the spectrum as:

Gα=E​diag⁡\(μiα\)​E⊤,α∈\(0,1\]\.G^\{\\alpha\}=E\\operatorname\{diag\}\(\\mu\_\{i\}^\{\\alpha\}\)E^\{\\top\},\\quad\\alpha\\in\(0,1\]\.Whenα=1\\alpha=1, the original activation statistics are preserved\. Decreasingα\\alphacompresses the spectrum, reducing the dominance of high\-energy directions and giving relatively more weight to lower\-energy directions\.

Across the evaluated models, the best values lie between0\.30\.3and0\.90\.9\. We useα=0\.9\\alpha=0\.9by default and evaluate its effect in Sec\.[5\.4](https://arxiv.org/html/2608.21142#S5.SS4)\.

### 4\.5Anchored Inter\-Layer Alignment

The preceding components minimize reconstruction error but do not explicitly preserve the geometric relation between consecutive projections\. In attention, the value projection writes the intermediate representation that is read by the attention output projection\. LetUvU\_\{v\}denote the left singular frame of the pruned value projection andVoV\_\{o\}the right singular frame of the attention output projection\. Following Weak Geometric Alignment Theorem \(WSBM\)\([27](https://arxiv.org/html/2608.21142#bib.bib1)\), their relation is characterized byVo⊤​UvV\_\{o\}^\{\\top\}U\_\{v\}\. Our analysis in App\. C of the supplementary material shows that pruning changes this relation and that reconstruction\-based compensation does not restore it\. We therefore regularize the right rotation of the attention output projection toward the original inter\-layer alignment\.

Direct comparison of singular\-vector matrices is not basis invariant because singular vectors are defined only up to sign and rotations within degenerate subspaces\. We instead compare the projectors onto their leadingrpr\_\{p\}\-dimensional subspacesΠrp​\(V\)=Vrp​Vrp⊤\\Pi\_\{r\_\{p\}\}\(V\)=V\_\{r\_\{p\}\}V\_\{r\_\{p\}\}^\{\\top\}, whereVrpV\_\{r\_\{p\}\}contains the top\-rpr\_\{p\}right singular vectors\. We userp=16r\_\{p\}=16by default, independently of the retained widthkk\.

For the attention output projection, the reduced right\-rotation variable is square, withR∈O⁡\(k\)R\\in O\(k\)\. Its rows represent the transposed right singular frame of the compensated weight\. The corresponding leading\-subspace projector isΠrp\(R\)=R\[1:rp\]⊤R\[1:rp\]\\Pi\_\{r\_\{p\}\}\(R\)=R\_\{\[1:r\_\{p\}\]\}^\{\\top\}R\_\{\[1:r\_\{p\}\]\}\. LetUv∈ℝk×rpU\_\{v\}\\in\\mathbb\{R\}^\{k\\times r\_\{p\}\}contain the leading left singular vectors of the pruned value projection, and letΠanchor=Πrp​\(Vanchor\)\\Pi\_\{\\mathrm\{anchor\}\}=\\Pi\_\{r\_\{p\}\}\(V\_\{\\mathrm\{anchor\}\}\)denote the fixed anchor projector\. We define the alignment penalty as:

𝒜⁡\(R\)=∥Uv⊤​\(Πrp​\(R\)−Πanchor\)​Uv∥F2/rp2\.\\mathcal\{A\}\(R\)=\\lVert U\_\{v\}^\{\\top\}\\left\(\\Pi\_\{r\_\{p\}\}\(R\)\-\\Pi\_\{\\mathrm\{anchor\}\}\\right\)U\_\{v\}\\rVert\_\{F\}^\{2\}/\{r\_\{p\}^\{2\}\}\.𝒜⁡\(R\)\\mathcal\{A\}\(R\)is added to the right\-rotation objective with weightλa\\lambda\_\{a\}\. OnlyUvU\_\{v\}andΠanchor\\Pi\_\{\\mathrm\{anchor\}\}are precomputed for each layer\. The analytic gradient depends only on the leadingrpr\_\{p\}rows ofRRand adds approximately0\.5%0\.5\\%to the right\-solver cost\.𝒜⁡\(R\)\\mathcal\{A\}\(R\)is applied only to the attention output projection\. The MLP compensation use the reconstruction objective alone\.

Using the right singular frame of the retained weight as the anchor makes the penalty zero at initialization and provides little regularization\. Instead, we construct the anchor from the original unpruned attention output projection\. LetVfull,rpV\_\{\\mathrm\{full\},r\_\{p\}\}contain its leadingrpr\_\{p\}right singular vectors\. We restrict these vectors to the retained coordinates and orthonormalize them asVanchor=orth\(Vfull,rp\[K,:\]\)V\_\{\\mathrm\{anchor\}\}=\\operatorname\{orth\}\\left\(V\_\{\\mathrm\{full\},r\_\{p\}\}\[K,:\]\\right\)\.

This restricted original subspace differs from the right singular subspace of the retained weight by approximately2020–30∘30^\{\\circ\}, as shown in App\. C of the supplementary material\. With the defaultλa=50\\lambda\_\{a\}=50, the alignment penalty after compensation is a median7\.2×7\.2\\timeslower than that of the uncompensated pruned weight\.

### 4\.6Full Pipeline

Figure[1](https://arxiv.org/html/2608.21142#S1.F1)illustrates the overview of COEC, and Algorithm[1](https://arxiv.org/html/2608.21142#alg1)summarizes the compensation procedure for one weight matrix\. Numerical safeguards are described in App\. B of the supplementary material\.

Algorithm 1COEC compensation of one pruned moduleInput: originalW0W\_\{0\}, kept setKK, calibration GramGG; temperα\\alpha, tolδ\\delta, align weightλa\\lambda\_\{a\} Output: Compensated retained weightW~\\widetilde\{W\}

1:

G←GαG\\leftarrow G^\{\\alpha\};

Gx​x←G⁡\[K,K\]G\_\{xx\}\\leftarrow G\[K,K\];

Gy​x←W0G\[:,K\]G\_\{yx\}\\leftarrow W\_\{0\}\\,G\[:,K\]
2:

W^←W0\[:,K\]\\widehat\{W\}\\leftarrow W\_\{0\}\[:,K\]\(kept\-column sliceWKW\_\{K\}\)

3:

Qℓ←polar⁡\(Gy​x​W^⊤\)Q\_\{\\ell\}\\leftarrow\\mathrm\{polar\}\(G\_\{yx\}\\widehat\{W\}^\{\\top\}\);

W^←Qℓ​W^\\widehat\{W\}\\leftarrow Q\_\{\\ell\}\\widehat\{W\}
4:

W^←GCV\-rescale​\(W^\)\\widehat\{W\}\\leftarrow\\text\{GCV\-rescale\}\(\\widehat\{W\}\)
5:whileround

<50<50and

\|Δ​ℒ\|≥10−4\|\\Delta\\mathcal\{L\}\|\\geq 10^\{\-4\}do

6:

Qr←Q\_\{r\}\\leftarrowReducedStiefelSolve⁡\(W^,Q0=I,δ,λa\)\\operatorname\{ReducedStiefelSolve\}\(\\widehat\{W\};Q\_\{0\}\{=\}I,\\delta,\\lambda\_\{a\}\)
7:

W^←W^​Qr\\widehat\{W\}\\leftarrow\\widehat\{W\}Q\_\{r\};

W^←GCV\-rescale​\(W^\)\\widehat\{W\}\\leftarrow\\text\{GCV\-rescale\}\(\\widehat\{W\}\)
8:

Qℓ←polar⁡\(Gy​x​W^⊤\)Q\_\{\\ell\}\\leftarrow\\mathrm\{polar\}\(G\_\{yx\}\\widehat\{W\}^\{\\top\}\);

W^←Qℓ​W^\\widehat\{W\}\\leftarrow Q\_\{\\ell\}\\widehat\{W\};

W^←GCV\-rescale​\(W^\)\\widehat\{W\}\\leftarrow\\text\{GCV\-rescale\}\(\\widehat\{W\}\)
9:endwhile

10:return

W~←W^\\widetilde\{W\}\\leftarrow\\widehat\{W\}

## 5Experiments

Table 1:WikiText\-2 perplexity\(↓\)\(\\downarrow\)at 30% column sparsity\. Each selection score is paired with its native compensation and with COEC under the same selection score and calibration data\. Bold marks the better compensation within each selection, per model\. Results at 10% and 20% sparsity are in App\. A \(supplementary material\)\.Table 2:7\-task zero\-shot accuracy%\(↑\)\(\\uparrow\)at 30% column sparsity\. Same protocol and bolding as Table[1](https://arxiv.org/html/2608.21142#S5.T1)\. Results at 10% and 20% sparsity are in App\. A \(supplementary material\)\.### 5\.1Experimental Setup

Models\.We evaluate on Llama\-3\.1\-8B and Llama\-3\-70B\([15](https://arxiv.org/html/2608.21142#bib.bib13)\)and on the Qwen2\.5 family \(7B, 14B, 32B, 72B\)\([32](https://arxiv.org/html/2608.21142#bib.bib15)\), covering scales from 7B to 72B\. Additional models are reported in the supplementary material under the same protocol\.

Pruning\.We evaluate structured sparsity levels of 10%, 20%, and 30% in every transformer block, using the pruning scope described in Sec\.[3](https://arxiv.org/html/2608.21142#S3)\. The main tables report results at 30% sparsity, where the effects of pruning and compensation are most pronounced\. Results at 10% and 20% sparsity follow the same protocol and are provided in App\. A of the supplementary material\.

Selection scores and baselines\.Columns are ranked by an activation\-aware score and the top⌈\(1−ρ\)​din⌉\\lceil\(1\-\\rho\)\\,d\_\{\\mathrm\{in\}\}\\rceilare kept for pruning ratioρ\\rho\. We use three published scores under one implementation: the Wanda\-sp score∥W:,j∥⋅∥Xj∥\\lVert W\_\{:,j\}\\rVert\\cdot\\lVert X\_\{j\}\\rVert, the WIFV score of FLAP, and the variance\-aware score∥W:,j∥⋅∥Xj∥⋅Var\(Xj\)\\lVert W\_\{:,j\}\\rVert\\cdot\\lVert X\_\{j\}\\rVert\\cdot\\mathrm\{Var\}\(X\_\{j\}\)of RCPU, which is the default selection for COEC\. All methods share the calibration data and the evaluation protocol\.

Calibration and evaluation\.We sample 128 sequences from the WikiText\-2 training set\([30](https://arxiv.org/html/2608.21142#bib.bib17)\)\. A single forward pass accumulates all Grams, and the compensation uses no evaluation data\. We report WikiText\-2 token\-level perplexity and average zero\-shot accuracy on seven tasks \(BoolQ\([5](https://arxiv.org/html/2608.21142#bib.bib41)\), RTE\([7](https://arxiv.org/html/2608.21142#bib.bib42)\), and WinoGrande\([33](https://arxiv.org/html/2608.21142#bib.bib43)\)with accuracy; HellaSwag\([46](https://arxiv.org/html/2608.21142#bib.bib44)\), ARC\-e, ARC\-c\([6](https://arxiv.org/html/2608.21142#bib.bib45)\), and OpenBookQA\([31](https://arxiv.org/html/2608.21142#bib.bib46)\)with normalized accuracy\) via the LM Evaluation Harness\([12](https://arxiv.org/html/2608.21142#bib.bib18)\)\.

### 5\.2Main Results

Tables[1](https://arxiv.org/html/2608.21142#S5.T1)and[2](https://arxiv.org/html/2608.21142#S5.T2)report the 30% setting\. Applied on top of Wanda\-sp, COEC reduces perplexity on every model, with the largest gain from13\.9013\.90to9\.649\.64on Qwen2\.5\-32B, and matches or improves zero\-shot accuracy throughout\. Against the FLAP bias, COEC gives lower perplexity on all six models, recovering the Llama\-3\.1\-8B from11\.8911\.89to10\.8610\.86, and improves accuracy on five of the six; at 10–20% the bias remains competitive on perplexity \(App\. A of the supplementary material\)\. Under the RCPU selection, COEC gives lower perplexity than the full RCPU compensation on all six models and higher zero\-shot accuracy on five\. The exception is Qwen2\.5\-14B, where RCPU is ahead by0\.70\.7points on accuracy while COEC keeps the perplexity lead\. These three comparisons cover three different column\-importance criteria \(§[5\.1](https://arxiv.org/html/2608.21142#S5.SS1)\), and a single COEC configuration improves most settings under each without per\-criterion tuning; the compensation can therefore be attached after existing pruning methods as a plug\-in\.

The gains grow with sparsity \(App\. A of the supplementary material\), as heavier pruning leaves more error for the compensation to recover\. The supplementary material repeats the same three comparisons on additional models under the same protocol\.

![Refer to caption](https://arxiv.org/html/2608.21142v1/figure/pr_calib_ppl.png)Figure 2:Perplexity versus calibration size on Llama\-3\.1\-8B under the RCPU selection score\. The selection is recomputed at eachNN, so the prune\-only baseline also varies\.
### 5\.3Effect of Calibration Size

All COEC components are estimated from calibration statistics, and the two\-sided family has more rotational freedom than the one\-sided update\. A natural concern is that this freedom overfits a small calibration set\. Figure[2](https://arxiv.org/html/2608.21142#S5.F2)tests this by varying the calibration sizeN∈\{32,64,128,256\}N\\in\\\{32,64,128,256\\\}on Llama\-3\.1\-8B at all three sparsity levels; all methods share the calibration data and the scoring rule\. COEC gives the lowest perplexity at every size and every ratio and improves monotonically withNN\. Even atN=32N\{=\}32it stays well below the prune\-only baseline; at 30% sparsity, shrinkingNNfrom256256to3232costs about0\.60\.6PPL\. RCPU has a smaller search space but is non\-monotone inNN\. The stability of the larger family comes from its regularization rather than its size: GCV shrinks the per\-mode rescale toward the uniform prior, and the rotations cannot reshape the singular spectrum to fit calibration noise\. The main results useN=128N\{=\}128, which lies on the flat part of every COEC curve\.

### 5\.4Ablation Study

Table 3:Component\-wise build\-up of COEC \(last row\), averaged over five development models at 30% sparsity\. All rows use the RCPU selection score; indented rows accumulate from top to bottom\.The ablation uses five 7–8B development models chosen for architectural diversity: Mistral\-7B\([22](https://arxiv.org/html/2608.21142#bib.bib14)\), Llama\-2\-7B, Llama\-3\.1\-8B, Qwen2\.5\-7B, and the distilled DeepSeek\-R1\-0528\-Qwen3\-8B\([18](https://arxiv.org/html/2608.21142#bib.bib16)\)\. Two of them appear in the main evaluation, and all hyperparameter defaults were selected on this set and transferred unchanged to the larger models of Tables[1](https://arxiv.org/html/2608.21142#S5.T1)and[2](https://arxiv.org/html/2608.21142#S5.T2)\. Table[3](https://arxiv.org/html/2608.21142#S5.T3)builds COEC up component by component at 30% sparsity\. The first row is the full RCPU compensation, the strongest published baseline; the indented rows then add the COEC components cumulatively\. The GCV rescale reduces perplexity from10\.0610\.06to9\.819\.81, and the converged two\-sided rotation further to9\.459\.45, supplying the input\-side adjustment that the one\-sided rotation cannot make\. The reduced\-Stiefel right solve trades perplexity for zero\-shot accuracy \(51\.5→52\.751\.5\\to 52\.7\)\. Gram tempering then recovers part of the perplexity \(9\.86→9\.659\.86\\to 9\.65\) at unchanged accuracy, and the anchored alignment penalty adds a final accuracy gain, leaving the full method at9\.659\.65perplexity and52\.952\.9accuracy\. Overall, the GCV rescale and the two\-sided rotation account for the perplexity reduction, the reduced\-Stiefel solve contributes most of the zero\-shot gain at some perplexity cost, and Gram tempering recovers most of that cost\.

### 5\.5Efficiency Analysis

Table[4](https://arxiv.org/html/2608.21142#S5.T4)reports the deployment cost at 30% column sparsity\. Structured pruning yields a smaller dense model that runs without any sparse\-kernel support: parameters, peak inference memory, and dense FLOPs per token fall by2323–29%29\\%, and the reduction grows with model size; realized wall\-clock speedup additionally depends on how well the kernels handle the pruned matrix shapes, so we report these architecture\-level quantities\. COEC rotates and rescales the kept weights in place without changing the model shape, so the deployment cost depends only on the pruning ratio and not on the compensation; the recovery reported in Tables[1](https://arxiv.org/html/2608.21142#S5.T1)and[2](https://arxiv.org/html/2608.21142#S5.T2)therefore comes at no additional cost\. Compensating a 7–8B model takes4545–8686GPU\-minutes on a single B300; the alignment penalty adds only0\.5%0\.5\\%overhead, and GCV rescaling is closed\-form\.

Table 4:Deployment cost at 30% column sparsity\. The compensation does not change the model shape, so the figures are identical for prune\-only, RCPU, and COEC\.
### 5\.6Geometric Analysis

A weight matrix reads certain input directions and writes certain output directions\. App\. C of the supplementary material measures how pruning and compensation change these directions on DeepSeek\-R1\-0528\-Qwen3\-8B at 30% sparsity\. Two observations motivate the design of COEC\. First, deleting columns rotates the input directions of every layer by about22∘22^\{\\circ\}on average\. Compensation that only minimizes reconstruction error leaves this rotation in place\. The anchored penalty of §[4\.5](https://arxiv.org/html/2608.21142#S4.SS5)targets this residual and reduces the measured mismatch by a median factor of7\.27\.2\. Second, COEC rotates the weight as a whole, so its input and output directions move together\. Their pairing within each layer stays intact \(discrepancy of1\.11\.1–1\.2%1\.2\\%\) without any explicit term\.

## 6Conclusion

We presented COEC, a training\-free compensation method for structured pruning of large language models\. Instead of refitting the retained weights freely, COEC rotates them on both sides and rescales their singular values, so the update can re\-route the signal lost with the removed columns while keeping the overall structure of the pretrained weight intact\. The strength of the rescaling is chosen automatically from calibration statistics, and a light penalty keeps the pruned layer geometrically consistent with its neighbor\. The whole procedure uses a small calibration set, requires no backpropagation through the LLM or retraining of the model parameters, and compensates a 7–8B model in under1\.51\.5GPU\-hours without changing the model architecture\.

## References

- Absilet al\.\(2008\)P\.\-A\. Absil, R\. Mahony, and R\. SepulchreBibliography\.InOptimization Algorithms on Matrix Manifolds,pp\. 201–220\.External Links:ISBN 9780691132983,[Link](http://www.jstor.org/stable/j.ctt7smmk.15)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p4.1),[§4\.2](https://arxiv.org/html/2608.21142#S4.SS2.p1.1)\.
- Anet al\.\(2023\)Y\. An, X\. Zhao, T\. Yu, M\. Tang, and J\. WangFluctuation\-based adaptive structured pruning for large language models\.External Links:2312\.11983,[Link](https://arxiv.org/abs/2312.11983)Cited by:[§1](https://arxiv.org/html/2608.21142#S1.p2.1),[§2](https://arxiv.org/html/2608.21142#S2.p1.1),[§2](https://arxiv.org/html/2608.21142#S2.p2.1)\.
- Ashkbooset al\.\(2024a\)S\. Ashkboos, M\. L\. Croci, M\. G\. do Nascimento, T\. Hoefler, and J\. HensmanSliceGPT: compress large language models by deleting rows and columns\.External Links:2401\.15024,[Link](https://arxiv.org/abs/2401.15024)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p3.1)\.
- Ashkbooset al\.\(2024b\)S\. Ashkboos, A\. Mohtashami, M\. L\. Croci, B\. Li, P\. Cameron, M\. Jaggi, D\. Alistarh, T\. Hoefler, and J\. HensmanQuaRot: outlier\-free 4\-bit inference in rotated llms\.External Links:2404\.00456,[Link](https://arxiv.org/abs/2404.00456)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p3.1)\.
- Clarket al\.\(2019\)C\. Clark, K\. Lee, M\. Chang, T\. Kwiatkowski, M\. Collins, and K\. ToutanovaBoolQ: exploring the surprising difficulty of natural yes/no questions\.External Links:1905\.10044,[Link](https://arxiv.org/abs/1905.10044)Cited by:[§5\.1](https://arxiv.org/html/2608.21142#S5.SS1.p4.1)\.
- Clarket al\.\(2018\)P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. TafjordThink you have solved question answering? try arc, the ai2 reasoning challenge\.External Links:1803\.05457,[Link](https://arxiv.org/abs/1803.05457)Cited by:[§5\.1](https://arxiv.org/html/2608.21142#S5.SS1.p4.1)\.
- Daganet al\.\(2005\)I\. Dagan, O\. Glickman, and B\. MagniniThe pascal recognising textual entailment challenge\.pp\. 177–190\.External Links:ISBN 978\-3\-540\-33427\-9,[Document](https://dx.doi.org/10.1007/11736790%5F9)Cited by:[§5\.1](https://arxiv.org/html/2608.21142#S5.SS1.p4.1)\.
- Edelmanet al\.\(1998\)A\. Edelman, T\. A\. Arias, and S\. T\. SmithThe geometry of algorithms with orthogonality constraints\.External Links:physics/9806030,[Link](https://arxiv.org/abs/physics/9806030)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p4.1),[§4\.2](https://arxiv.org/html/2608.21142#S4.SS2.p1.1)\.
- Frankle and Carbin \(2019\)J\. Frankle and M\. CarbinThe lottery ticket hypothesis: finding sparse, trainable neural networks\.External Links:1803\.03635,[Link](https://arxiv.org/abs/1803.03635)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- Frantar and Alistarh \(2023\)E\. Frantar and D\. AlistarhSparseGPT: massive language models can be accurately pruned in one\-shot\.External Links:2301\.00774,[Link](https://arxiv.org/abs/2301.00774)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- Frantaret al\.\(2023\)E\. Frantar, S\. Ashkboos, T\. Hoefler, and D\. AlistarhGPTQ: accurate post\-training quantization for generative pre\-trained transformers\.External Links:2210\.17323,[Link](https://arxiv.org/abs/2210.17323)Cited by:[§1](https://arxiv.org/html/2608.21142#S1.p1.1)\.
- Gaoet al\.\(2021\)L\. Gao, J\. Tow, S\. Biderman, S\. Black, A\. DiPofi, C\. Foster, L\. Golding, J\. Hsu, K\. McDonell, N\. Muennighoff, J\. Phang, L\. Reynolds, E\. Tang, A\. Thite, B\. Wang, K\. Wang, and A\. ZouA framework for few\-shot language model evaluation\.Zenodo\.External Links:[Document](https://dx.doi.org/10.5281/zenodo.5371629),[Link](https://doi.org/10.5281/zenodo.5371629)Cited by:[§5\.1](https://arxiv.org/html/2608.21142#S5.SS1.p4.1)\.
- Golubet al\.\(1979\)G\. H\. Golub, M\. Heath, and G\. WahbaGeneralized cross\-validation as a method for choosing a good ridge parameter\.Technometrics21\(2\),pp\. 215–223\.External Links:ISSN 00401706,[Link](http://www.jstor.org/stable/1268518)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p4.1)\.
- Gower and Dijksterhuis \(2004\)J\. C\. Gower and G\. B\. DijksterhuisProcrustes problems\.Oxford University Press\.External Links:ISBN 9780198510581,[Document](https://dx.doi.org/10.1093/acprof%3Aoso/9780198510581.001.0001),[Link](https://doi.org/10.1093/acprof:oso/9780198510581.001.0001)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p4.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafioriet al\.The llama 3 herd of models\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[§5\.1](https://arxiv.org/html/2608.21142#S5.SS1.p1.1)\.
- Guet al\.\(2025\)T\. Gu, B\. Liu, B\. Xiao, K\. Zeng, J\. Liu, and Y\. QianDenoiseRotator: enhance pruning robustness for llms via importance concentration\.External Links:2505\.23049,[Link](https://arxiv.org/abs/2505.23049)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p3.1)\.
- Guet al\.\(2026\)Y\. Gu, L\. Dong, F\. Wei, and M\. HuangMiniLLM: on\-policy distillation of large language models\.External Links:2306\.08543,[Link](https://arxiv.org/abs/2306.08543)Cited by:[§1](https://arxiv.org/html/2608.21142#S1.p1.1)\.
- Guoet al\.\(2025\)D\. Guoet al\.DeepSeek\-r1 incentivizes reasoning in llms through reinforcement learning\.Nature645\(8081\),pp\. 633–638\.External Links:ISSN 1476\-4687,[Link](http://dx.doi.org/10.1038/s41586-025-09422-z),[Document](https://dx.doi.org/10.1038/s41586-025-09422-z)Cited by:[§5\.4](https://arxiv.org/html/2608.21142#S5.SS4.p1.1)\.
- Harutaet al\.\(2026\)S\. Haruta, K\. Matsumoto, Z\. Li, Y\. Wang, and M\. KurokawaRCPU: rotation\-constrained error compensation for structured pruning of large language models\.External Links:2510\.07782,[Link](https://arxiv.org/abs/2510.07782)Cited by:[§1](https://arxiv.org/html/2608.21142#S1.p2.1),[§2](https://arxiv.org/html/2608.21142#S2.p2.1)\.
- Hassibi and Stork \(1992\)B\. Hassibi and D\. StorkSecond order derivatives for network pruning: optimal brain surgeon\.InAdvances in Neural Information Processing Systems,S\. Hanson, J\. Cowan, and C\. Giles \(Eds\.\),Vol\.5,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/1992/file/303ed4c69846ab36c2904d3ba8573050-Paper.pdf)Cited by:[§1](https://arxiv.org/html/2608.21142#S1.p1.1),[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- Hintonet al\.\(2015\)G\. Hinton, O\. Vinyals, and J\. DeanDistilling the knowledge in a neural network\.External Links:1503\.02531,[Link](https://arxiv.org/abs/1503.02531)Cited by:[§1](https://arxiv.org/html/2608.21142#S1.p1.1)\.
- Jianget al\.\(2023\)A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. R\. Lavaud, M\. Lachaux, P\. Stock, T\. L\. Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. E\. SayedMistral 7b\.External Links:2310\.06825,[Link](https://arxiv.org/abs/2310.06825)Cited by:[§5\.4](https://arxiv.org/html/2608.21142#S5.SS4.p1.1)\.
- Kurticet al\.\(2023\)E\. Kurtic, E\. Frantar, and D\. AlistarhZipLM: inference\-aware structured pruning of language models\.External Links:2302\.04089,[Link](https://arxiv.org/abs/2302.04089)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- LeCunet al\.\(1989\)Y\. LeCun, J\. Denker, and S\. SollaOptimal brain damage\.InAdvances in Neural Information Processing Systems,D\. Touretzky \(Ed\.\),Vol\.2,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/1989/file/6c9882bbac1c7093bd25041881277658-Paper.pdf)Cited by:[§1](https://arxiv.org/html/2608.21142#S1.p1.1),[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- Linet al\.\(2026\)J\. Lin, J\. Tang, H\. Tang, S\. Yang, W\. Chen, W\. Wang, G\. Xiao, X\. Dang, C\. Gan, and S\. HanAWQ: activation\-aware weight quantization for llm compression and acceleration\.External Links:2306\.00978,[Link](https://arxiv.org/abs/2306.00978)Cited by:[§1](https://arxiv.org/html/2608.21142#S1.p1.1)\.
- Liuet al\.\(2025\)Z\. Liu, C\. Zhao, I\. Fedorov, B\. Soran, D\. Choudhary, R\. Krishnamoorthi, V\. Chandra, Y\. Tian, and T\. BlankevoortSpinQuant: llm quantization with learned rotations\.External Links:2405\.16406,[Link](https://arxiv.org/abs/2405.16406)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p3.1)\.
- Liuet al\.\(2026\)Z\. Liu, W\. Wang, J\. Wang, P\. Wang, X\. Sui, C\. Ruan, N\. Ling, and W\. JiangGeometric and spectral alignment for deep neural network i\.External Links:2605\.02108,[Link](https://arxiv.org/abs/2605.02108)Cited by:[§4\.5](https://arxiv.org/html/2608.21142#S4.SS5.p1.1)\.
- Maet al\.\(2023\)X\. Ma, G\. Fang, and X\. WangLLM\-pruner: on the structural pruning of large language models\.External Links:2305\.11627,[Link](https://arxiv.org/abs/2305.11627)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- Menet al\.\(2024\)X\. Men, M\. Xu, Q\. Zhang, B\. Wang, H\. Lin, Y\. Lu, X\. Han, and W\. ChenShortGPT: layers in large language models are more redundant than you expect\.External Links:2403\.03853,[Link](https://arxiv.org/abs/2403.03853)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- Merityet al\.\(2016\)S\. Merity, C\. Xiong, J\. Bradbury, and R\. SocherPointer sentinel mixture models\.External Links:1609\.07843,[Link](https://arxiv.org/abs/1609.07843)Cited by:[§5\.1](https://arxiv.org/html/2608.21142#S5.SS1.p4.1)\.
- Mihaylovet al\.\(2018\)T\. Mihaylov, P\. Clark, T\. Khot, and A\. SabharwalCan a suit of armor conduct electricity? a new dataset for open book question answering\.External Links:1809\.02789,[Link](https://arxiv.org/abs/1809.02789)Cited by:[§5\.1](https://arxiv.org/html/2608.21142#S5.SS1.p4.1)\.
- Qwenet al\.\(2025\)Qwen, :, A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. QiuQwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§5\.1](https://arxiv.org/html/2608.21142#S5.SS1.p1.1)\.
- Sakaguchiet al\.\(2019\)K\. Sakaguchi, R\. L\. Bras, C\. Bhagavatula, and Y\. ChoiWinoGrande: an adversarial winograd schema challenge at scale\.External Links:1907\.10641,[Link](https://arxiv.org/abs/1907.10641)Cited by:[§5\.1](https://arxiv.org/html/2608.21142#S5.SS1.p4.1)\.
- Sanhet al\.\(2020\)V\. Sanh, T\. Wolf, and A\. M\. RushMovement pruning: adaptive sparsity by fine\-tuning\.External Links:2005\.07683,[Link](https://arxiv.org/abs/2005.07683)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- Schönemann \(1966\)P\. H\. SchönemannA generalized solution of the orthogonal procrustes problem\.Psychometrika31\(1\),pp\. 1–10\.External Links:[Document](https://dx.doi.org/10.1007/BF02289451)Cited by:[§4\.1](https://arxiv.org/html/2608.21142#S4.SS1.p2.1)\.
- Songet al\.\(2024\)J\. Song, K\. Oh, T\. Kim, H\. Kim, Y\. Kim, and J\. KimSLEB: streamlining llms through redundancy verification and elimination of transformer blocks\.External Links:2402\.09025,[Link](https://arxiv.org/abs/2402.09025)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- Sunet al\.\(2024\)M\. Sun, Z\. Liu, A\. Bair, and J\. Z\. KolterA simple and effective pruning approach for large language models\.External Links:2306\.11695,[Link](https://arxiv.org/abs/2306.11695)Cited by:[§1](https://arxiv.org/html/2608.21142#S1.p2.1),[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- van der Ouderaaet al\.\(2024\)T\. F\. A\. van der Ouderaa, M\. Nagel, M\. van Baalen, Y\. M\. Asano, and T\. BlankevoortThe llm surgeon\.External Links:2312\.17244,[Link](https://arxiv.org/abs/2312.17244)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- Wahba \(1978\)P\. WahbaSmoothing noisy data with spline functions\. estimating the correct degree of smoothing by the method of generalized cross\-validation\.\.Numerische Mathematik31,pp\. 377–404\.External Links:[Link](http://eudml.org/doc/132586)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p4.1)\.
- Wanget al\.\(2025\)X\. Wang, Y\. Zheng, Z\. Wan, and M\. ZhangSVD\-llm: truncation\-aware singular value decomposition for large language model compression\.External Links:2403\.07378,[Link](https://arxiv.org/abs/2403.07378)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- Wen and Yin \(2012\)Z\. Wen and W\. YinA feasible method for optimization with orthogonality constraints\.Mathematical Programming142,pp\. 397 – 434\.External Links:[Link](https://api.semanticscholar.org/CorpusID:10155814)Cited by:[§4\.2](https://arxiv.org/html/2608.21142#S4.SS2.p3.1)\.
- Xiaet al\.\(2024\)M\. Xia, T\. Gao, Z\. Zeng, and D\. ChenSheared llama: accelerating language model pre\-training via structured pruning\.External Links:2310\.06694,[Link](https://arxiv.org/abs/2310.06694)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- Xiaoet al\.\(2024\)G\. Xiao, J\. Lin, M\. Seznec, H\. Wu, J\. Demouth, and S\. HanSmoothQuant: accurate and efficient post\-training quantization for large language models\.External Links:2211\.10438,[Link](https://arxiv.org/abs/2211.10438)Cited by:[§1](https://arxiv.org/html/2608.21142#S1.p1.1)\.
- Yanget al\.\(2024\)Y\. Yang, Z\. Cao, and H\. ZhaoLaCo: large language model pruning via layer collapse\.External Links:2402\.11187,[Link](https://arxiv.org/abs/2402.11187)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- Yuanet al\.\(2025\)Z\. Yuan, Y\. Shang, Y\. Song, D\. Yang, Q\. Wu, Y\. Yan, and G\. SunASVD: activation\-aware singular value decomposition for compressing large language models\.External Links:2312\.05821,[Link](https://arxiv.org/abs/2312.05821)Cited by:[§2](https://arxiv.org/html/2608.21142#S2.p1.1)\.
- Zellerset al\.\(2019\)R\. Zellers, A\. Holtzman, Y\. Bisk, A\. Farhadi, and Y\. ChoiHellaSwag: can a machine really finish your sentence?\.External Links:1905\.07830,[Link](https://arxiv.org/abs/1905.07830)Cited by:[§5\.1](https://arxiv.org/html/2608.21142#S5.SS1.p4.1)\.