COEC: Calibrated Orthogonal-Equivalence Compensation for Structured Pruning of Large Language Models

COEC: Calibrated Orthogonal-Equivalence Compensation for Structured Pruning of Large Language Models Peiqi Yu1 , Nam Ling1 , Wei Wang2 , Wei Jiang2

arXiv:2608.21142v1 [cs.LG] 21 Aug 2026

1

School of Engineering, Santa Clara University, Santa Clara, CA, USA 2 Futurewei Technologies, Inc., USA pyu@scu.edu

Abstract Structured pruning reduces the size and inference cost of large language models (LLMs) by removing weight columns, but the resulting output error can degrade accuracy. Existing training-free compensation methods use an additive bias or a single orthogonal rotation on the output side of the retained weight. These corrections leave its input singular frame unchanged and therefore limit how the retained weight can adapt after column removal. We propose COEC (Calibrated Orthogonal-Equivalence Compensation), a training-free compensation framework that applies alternating left and right orthogonal rotations to the retained weight. The right rotation is optimized on a reduced Stiefel manifold, while singular values are rescaled using generalized cross-validation to select the regularization strength for each layer. COEC further tempers the calibration Gram matrix to reduce the dominance of high-energy activation directions and introduces an alignment penalty that preserves the geometric relation between adjacent attention projections.All components use second-order statistics from a small calibration set and require neither backpropagation through the LLM nor retraining of the model parameters. COEC is independent of the column pruning criterion and can be applied to multiple structured pruning methods. Experiments on the Llama-3, Llama-3.1, and Qwen2.5 model families across multiple structured sparsity levels show that COEC improves perplexity on every model and zero-shot accuracy in most settings over existing compensation methods, with larger gains at higher sparsity. These results show that post-pruning compensation can recover part of the performance lost to column removal.

1

Introduction

Large language models (LLMs) are increasingly deployed across a wide range of applications, yet their inference costs in computation and memory remain substantial. Common compression techniques include quantization (Frantar et al. 2023; Lin et al. 2026; Xiao et al. 2024), distillation (Hinton, Vinyals, and Dean 2015; Gu et al. 2026), and pruning (LeCun, Denker, and Solla 1989; Hassibi and Stork 1992). Among them, structured pruning removes parameters at the granularity of entire weight-matrix columns, including MLP neurons and attention key–value groups. This produces a smaller dense model with reduced parameter count, memory usage, and computational cost on standard hardware. Because retraining at LLM scale is expensive, structured Copyright © 2027, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

pruning methods that preserve accuracy without additional training are particularly attractive. A training-free structured pruning pipeline comprises two largely separable components: a selection rule that determines which columns to retain and a compensation mechanism that adjusts the retained weights to reduce the output error induced by pruning. Existing methods have primarily focused on selection, while treating compensation as secondary. Wanda-sp (Sun et al. 2024) extends Wanda’s activation-aware importance score to column pruning but performs no compensation, leaving the output discrepancy caused by the removed columns uncorrected. FLAP (An et al. 2023) compensates for the mean error through an additive bias term. This approach is simple and effective at low sparsity, but a constant bias cannot correct input-dependent directional errors. Direct least-squares refitting of the retained weights can reduce calibration error, but with a limited calibration set it is prone to overfitting and may substantially distort the pretrained weights (Haruta et al. 2026). More recently, RCPU (Haruta et al. 2026) constrains compensation to an orthogonal rotation of the output space followed by global rescaling. This geometry-preserving formulation avoids the distortions introduced by unconstrained refitting and achieves state-of-the-art performance. However, RCPU applies rotation only on the left. Let the singular value decomposition (SVD) of the retained weight be WK = UK ΣK VK⊤ . A left rotation QWK changes the output frame UK while leaving the input frame VK unchanged. This one-sided constraint is limiting because column pruning removes input coordinates and alters the correlation structure represented by the retained columns. Effective compensation must therefore recover, through the retained columns, the output contributions previously associated with the removed columns. Our analysis in Sec. 5.6 shows that pruning substantially rotates the input singular subspace across all examined layers, and neither left-sided rotation nor reconstructionoptimal compensation generally restores this subspace. Motivated by these limitations, we propose COEC (Calibrated Orthogonal-Equivalence Compensation), a general post-pruning compensation framework designed as a universal plug-in (Fig. 1). COEC takes the retained set produced by any column-importance criterion and modifies only the retained weights. It can therefore be applied after existing structured pruning methods without changing their selection

COEC objective

f XK ∥2 / ∥Y ∥2 + λa A(Qr ) L = ∥Y − W F F

W0

alternated to convergence

X

Qℓ

column selection (any importance score)

f W

WK

×

Calibration data Y = W0 X

(training-free)

left rotation (closed form)

(S)

×

WK

per-mode rescale (GCV-selected λ)

Qr

right rotation (reduced Stiefel)

f = Qℓ W (S) Qr W K

c QXK ∥2 Qr = arg minQ∈O(k) ∥Y − s W F

c⊤ ) Qℓ = polar(Gyx W

Figure 1: Overview of COEC. Columns are pruned using an arbitrary importance score, and the retained weight WK is f = Qℓ W (S) Qr . The left and right rotations adjust the output and input singular frames, while per-mode compensated as W K rescaling adapts the singular values. All updates use calibration statistics only, without LLM backpropagation or model retraining. The objective combines relative reconstruction error with an alignment penalty on the attention output projection. rules or model architectures. Rather than refitting the retained weights through unconstrained least squares, COEC searches within a spectrum-preserving family that applies rotations on both sides and rescales individual singular modes. COEC alternates between a closed-form left rotation and an iterative right rotation optimized over a reduced Stiefel manifold. Because the right-rotation objective is coupled with the overall scale, the optimal scaling factor is recomputed at every iteration. COEC further replaces global rescaling with per-mode singular-value rescaling, with the regularization strength selected independently for each layer through closed-form generalized cross-validation. Two additional components improve the calibration objective and preserve inter-layer geometry. Spectral tempering of the calibration Gram matrix balances perplexity and zero-shot accuracy, while an alignment penalty anchored to the original input frame restores geometric relations between adjacent layers that reconstruction objectives alone fail to preserve. All components depend only on second-moment statistics estimated from a small calibration set and require neither backpropagation through the LLM nor retraining of the model parameters. We evaluate COEC on the Llama-3, Llama-3.1, and Qwen2.5 model families across multiple structured sparsity levels. Under identical column selections, COEC improves perplexity on every model and zero-shot accuracy in most settings over existing compensation methods, with larger gains at higher pruning ratios. When applied to the column selections produced by Wanda-sp and FLAP, COEC recovers a substantial portion of the accuracy lost during pruning. Our main contributions are summarized as follows. • We formulate post-pruning compensation as a two-sided spectrum-preserving update. We develop an alternating solver that combines a closed-form left rotation, a reduced-Stiefel right rotation with in-loop rescaling, permode rescaling selected by generalized cross-validation, spectral tempering, and an inter-layer alignment penalty. We isolate the effect of each component through ablation studies in Sec. 5.4. • We design COEC as a criterion-agnostic compensation method that can be applied after any column-selection

rule without changing the model architecture. We demonstrate this plug-in capability using the selections produced by Wanda-sp, FLAP, and RCPU in Sec. 5.2. • We evaluate COEC under a unified protocol on the Llama3, Llama-3.1, and Qwen2.5 model families across multiple structured sparsity levels. COEC improves perplexity on every model and zero-shot accuracy in most settings over Wanda-sp, FLAP, and RCPU in Sec. 5.2.

2

Related Work

Pruning of LLMs. Post-training pruning reduces the size and inference cost of LLMs without retraining, building on classical saliency criteria (LeCun, Denker, and Solla 1989; Hassibi and Stork 1992; Frankle and Carbin 2019; Sanh, Wolf, and Rush 2020). Unstructured and semi-structured methods sparsify weights element-wise, by per-row leastsquares reconstruction in SparseGPT (Frantar and Alistarh 2023) or activation-aware scoring in Wanda (Sun et al. 2024), but need sparse kernel support to realize speedups. Structured methods remove entire columns, rows, or blocks and directly shrink the dense model, using gradient saliency (Ma, Fang, and Wang 2023), masks learned with continued pretraining (Xia et al. 2024), curvature information (Kurtic, Frantar, and Alistarh 2023; van der Ouderaa et al. 2024), or block-level redundancy (Men et al. 2024; Song et al. 2024; Yang, Cao, and Zhao 2024); Wanda-sp extends the Wanda score to columns, and FLAP (An et al. 2023) scores channels by activation variance. Low-rank factorization (Yuan et al. 2025; Wang et al. 2025) also operates in the singular-value domain but truncates the spectrum, which our compensation preserves. We work in the structured, training-free setting. Post-pruning compensation. Methods differ in how they repair the kept weights after removal. Wanda-sp applies no update. FLAP (An et al. 2023) corrects the mean error with a bias term, which cannot fix input-dependent directional mismatches. Least-squares refitting reduces calibration error further but overfits small calibration sets and damages pretrained weights (Haruta et al. 2026).RCPU (Haruta et al. 2026) restricts the update to an output-frame rotation with a global scale, preserving geometry and overfitting less. We

extend this one-sided rotation to a converged two-sided update with per-mode rescaling, independent of the selection criterion. Rotation-based transformations. SliceGPT (Ashkboos et al. 2024a) rotates RMSNorm-connected transformers under computational invariance and prunes in the rotated basis; QuaRot (Ashkboos et al. 2024b), SpinQuant (Liu et al. 2025), and DenoiseRotator (Gu et al. 2025) use lossless rotations to condition weights for compression. All of these rotate before compression under functional equivalence, whereas we rotate after pruning to compensate the removal error. Calibration-aware rescaling. Scale and bias corrections appear in FLAP and in energy-matching heuristics. Ours is instead a diagonal fit in the SVD frame of the rotated weight, with the ridge strength set per layer by generalized crossvalidation (GCV) (Golub, Heath, and Wahba 1979; Wahba 1978/79), a classical criterion that to our knowledge has not been used for pruning compensation. The right rotation is a weighted orthogonal Procrustes problem (Gower and Dijksterhuis 2004) with no closed-form solution, which we solve on the reduced Stiefel manifold (Edelman, Arias, and Smith 1998; Absil, Mahony, and Sepulchre 2008) (Sec. 4.2).

3 3.1

Problem Formulation

Notation and Setup

For each transformer block, we prune structured channels in both the MLP and attention sub-layers. In the MLP, pruning removes intermediate channels by deleting the corresponding rows of the gating and value projections and the matching columns of the output projection. In grouped-query attention, pruning removes complete KV groups together with their associated query heads. This corresponds to deleting the associated output rows of the query, key, and value projections and the matching input columns of the attention output projection. The grouped-query attention structure is preserved after pruning. Let W0 ∈ Rdout ×din denote the pretrained weight matrix of a pruned sub-layer. Let X ∈ Rdin ×N denote the calibration activations from N token positions, and Y = W0 X the corresponding output of the original layer. A column-selection rule retains an index set K with |K| = k, and XK ∈ Rk×N contains the rows of X indexed by K. The compensation f ∈ Rdout ×k acting on XK . step constructs a weight matrix W Its quality is measured by the relative reconstruction error:

2 f Gxx W f ⊤ ) − 2 tr(W f G⊤ f XK ∥2 tr(W ∥Y − W yx ) + ∥Y ∥F F = . ∥Y ∥2F ∥Y ∥2F

The objective depends only on the second-moment statistics G = XX ⊤ , Gxx = G[K, K], and Gyx = W0 G[:, K], accumulated in a single calibration pass. The token-level activations do not need to be stored after accumulation. In practice, 128 calibration sequences are sufficient (Sec. 5.1).

3.2

Limitation of Least-Squares Compensation

The unconstrained least-squares solution is W LS = Gyx G−1 xx , when Gxx is invertible. In practice, a ridgeregularized solution is used as W ridge = Gyx (Gxx +

λls I)−1 . Consistent with the observations reported for RCPU, we find that least-squares refitting can fit the small calibration set too closely. It reshapes the singular spectrum of the retained weight according to the calibration statistics and shows unstable generalization across models. To limit such unrestricted changes, we parameterize the compensated weight as f = Qℓ W (S) Qr , W K

Qℓ ∈ O(dout ), Qr ∈ O(k),

(S) where WK is obtained by applying a regularized rescal-

ing to the singular modes of the retained weight. The left and right orthogonal transforms adjust the output and input singular frames, respectively, while the mode-wise rescaling controls changes to the singular spectrum. This structured parameterization avoids the unrestricted deformation introduced by direct least-squares refitting. COEC estimates these components by minimizing, for each pruned module, f , Qr ) = L(W

f XK ∥2 ∥Y − W F + λa A(Qr ), ∥Y ∥2F

(1)

where A is the inter-layer alignment penalty of Sec. 4.5, applied to the attention output projection with λa = 0 elsewhere. The following sections describe how these components are estimated.

4 4.1

Method

Two-Sided Rotation with In-Loop Scaling

We initialize the compensated weight with the retained column slice as WK = W0 [:, K]. No rank truncation or singularvalue rescaling is applied, and WK represents the uncompensated pruned weight used as the reference point in App. C of c for the current comthe supplementary material. We write W c = WK and overwritten by pensated weight, initialized as W c and the current Qr . each block update; (1) is evaluated at W c For a fixed W , the optimal left rotation is given by the orthogonal Procrustes solution (Schönemann 1966): Qℓ = c ⊤ ), where polar(A) denotes the orthogonal polar(Gyx W factor UA VA⊤ of the SVD A = UA ΣA VA⊤ . The left rotation minimizes the reconstruction error by aligning the outputs of the retained weight with those of the original layer on the calibration data. The global scale can be optimized independently of the left rotation because an orthogonal transformation preserves c XK . For a fixed W c , the optimal scale is: the norm of W c ⟩/⟨W c, W c Gxx ⟩, s∗ = ⟨Gyx , W

where ⟨A, B⟩ = tr(AB ⊤ ). The combination of a left rotation and global scaling corresponds to the compensation used by RCPU. In COEC, it forms the left-rotation block of the alternating compensation procedure. The right rotation is obtained by solving a weighted orthogonal Procrustes problem, c Qr XK ∥2F . minQr ∈O(k) ∥Y − s W

Unlike the left-rotation problem, this objective has no closedform solution because the rotation is coupled to both the

weight matrix and the calibration covariance. We optimize Qr using projected gradient descent with an orthogonal retraction and initialize it as Q0 = I. This identity initialization is empirically important at mild sparsity, where the optimal right rotation typically remains close to the original input frame. Initializing from the one-sided Procrustes solution instead performs worse, e.g., increasing reconstruction error on attention output projections from 0.019 to 0.159. The right rotation is also coupled to the global scale. We therefore recompute s∗ after every update of Qr and evaluate the gradient using the scaled residual. The step size is normalized as η/s2 to account for the scale of the objective. This procedure allows the right rotation and global scale to converge jointly. The complete optimization alternates between the left rotation, scale update, right rotation, and another scale update

c=U bΣ b Vb ⊤ , where σi = of the current rotated weight be W b [Σ]ii is the i-th singular value and ui , vi the corresponding b and Vb fixed, the reconstructionsingular vectors. Keeping U optimal value associated with the i-th singular-vector pair is ⊤ s⋆g,i = u⊤ i Gyx vi /vi Gxx vi = ρi /ei . where the mode response ρi measures the alignment between the corresponding output and input singular directions under the target cross-covariance, while the mode energy ei measures the calibration energy along the input singular direction. When the weight has rank one, this expression reduces to the globally scaled singular value s∗ σi . Estimating each singular value independently can overfit the limited calibration data. We therefore regularize s⋆g,i toward the value obtained from uniform global scaling, s∗ σi . The regularized singular value is

Qℓ → s → Qr → s.

sg,i (λ) = (ρi + λ s∗ σi ) / (ei + λ) .

We perform at most 50 alternating rounds and stop early when the change in L falls below 10−4 . The left rotation is the conditional minimizer of the reconstruction objective and leaves the alignment term unchanged, and the regularized rescale does not increase the reconstruction error in the tempered metric. The resulting parameterization includes left-only and right-only compensation as special cases.

4.2

Reduced-Stiefel Solver

Directly optimizing Qr ∈ O(k) is inefficient when the compensated weight is processed in output slices. Consider a slice Wg ∈ Rb×k with b < k, and let Wg = Ug Σg Vg⊤ , where Vg ∈ Rk×b contains its right singular vectors. The reconstruction objective depends on Qr only through R = Vg⊤ Qr ∈ Rb×k . The components of Qr in the orthogonal complement of Vg do not affect the slice output and introduce redundant optimization directions. We therefore optimize R directly under the constraint RR⊤ = Ib , which places R on the Stiefel manifold St(b, k) (Edelman, Arias, and Smith 1998; Absil, Mahony, and Sepulchre 2008). Any feasible R ∈ St(b, k) can be extended to an orthogonal matrix Qr ∈ O(k). The reduced and full parameterizations therefore represent the same solutions for the slice objective, while the reduced formulation removes directions that do not affect the output. Its memory and per-step computational costs are reduced by approximately a factor of b/k. We use a Newton–Schulz polar retraction for rectangular frames. For square frames, we use an exact Cayley retraction (Wen and Yin 2012), with the required inverse computed by Newton–Schulz iteration. Implementation details are provided in App. B of the supplementary material. Optimization stops when the change in the rotation falls below a tolerance δ. This tolerance is selected once for each model using the calibration set.

4.3

GCV-Based Singular-Value Rescaling

A global scalar multiplies all singular values by the same factor, although the calibration statistics may favor different adjustments for different singular-vector pairs. Let the SVD

When λ → 0, sg,i (λ) → s⋆g,i . As λ increases, it approaches the globally scaled value s∗ σi . We select λ independently for each layer using generalized cross-validation (GCV):
P ⋆ 2 i ei sg,i (λ) − sg,i GCV(λ) =
2 , 1 − df(λ)/M X ei df(λ) = , i ei + λ where M is the number of singular values included in the layer-wise estimate. The effective degrees of freedom penalize solutions that fit the calibration statistics too closely. This correction is necessary because the unadjusted calibration residual is minimized at λ = 0. As λ → 0+ , both the numerator and denominator approach zero at the same order, producing a finite limiting value. As λ → ∞, the criterion approaches GCV(∞) =
P ∗ ⋆ 2 i ei s σi − sg,i . We minimize this objective over a logarithmically spaced grid for each layer. On both validation models, the resulting layer-specific regularization matches the performance of the best globally tuned value (see App. B of the supplementary material). The rescaling is computed from the SVD of the current rotated weight. It can therefore be applied after either the left or right rotation. We recompute the singular values within each alternating round so that the spectrum adapts jointly with the left and right singular frames.

4.4

Gram Tempering

All compensation terms use the activation Gram matrix G = XX ⊤ , X containing calibration activations. Its diagonal entries measure activation energy, while its off-diagonal entries capture correlations between activation dimensions. Let G = E diag(µi )E ⊤ be its eigendecomposition. We temper the spectrum as: ⊤ Gα = E diag(µα i )E ,

α ∈ (0, 1].

When α = 1, the original activation statistics are preserved. Decreasing α compresses the spectrum, reducing the dominance of high-energy directions and giving relatively more weight to lower-energy directions.

Across the evaluated models, the best values lie between 0.3 and 0.9. We use α = 0.9 by default and evaluate its effect in Sec. 5.4.

4.5

Anchored Inter-Layer Alignment

The preceding components minimize reconstruction error but do not explicitly preserve the geometric relation between consecutive projections. In attention, the value projection writes the intermediate representation that is read by the attention output projection. Let Uv denote the left singular frame of the pruned value projection and Vo the right singular frame of the attention output projection. Following Weak Geometric Alignment Theorem (WSBM) (Liu et al. 2026), their relation is characterized by Vo⊤ Uv . Our analysis in App. C of the supplementary material shows that pruning changes this relation and that reconstruction-based compensation does not restore it. We therefore regularize the right rotation of the attention output projection toward the original inter-layer alignment. Direct comparison of singular-vector matrices is not basis invariant because singular vectors are defined only up to sign and rotations within degenerate subspaces. We instead compare the projectors onto their leading rp -dimensional subspaces Πrp (V ) = Vrp Vr⊤p , where Vrp contains the top-rp right singular vectors. We use rp = 16 by default, independently of the retained width k. For the attention output projection, the reduced rightrotation variable is square, with R ∈ O(k). Its rows represent the transposed right singular frame of the compensated weight. The corresponding leading-subspace projector ⊤ is Πrp (R) = R[1:r R[1:rp ] . Let Uv ∈ Rk×rp contain the p] leading left singular vectors of the pruned value projection, and let Πanchor = Πrp (Vanchor ) denote the fixed anchor projector. We define the alignment penalty as:
A(R) = ∥Uv⊤ Πrp (R) − Πanchor Uv ∥2F /rp2 .

A(R) is added to the right-rotation objective with weight λa . Only Uv and Πanchor are precomputed for each layer. The analytic gradient depends only on the leading rp rows of R and adds approximately 0.5% to the right-solver cost. A(R) is applied only to the attention output projection. The MLP compensation use the reconstruction objective alone. Using the right singular frame of the retained weight as the anchor makes the penalty zero at initialization and provides little regularization. Instead, we construct the anchor from the original unpruned attention output projection. Let Vfull,rp contain its leading rp right singular vectors. We restrict these vectors to the retained coordinates
and orthonormalize them as Vanchor = orth Vfull,rp [K, :] . This restricted original subspace differs from the right singular subspace of the retained weight by approximately 20– 30◦ , as shown in App. C of the supplementary material. With the default λa = 50, the alignment penalty after compensation is a median 7.2× lower than that of the uncompensated pruned weight.

4.6

Full Pipeline

Figure 1 illustrates the overview of COEC, and Algorithm 1 summarizes the compensation procedure for one weight ma-

Algorithm 1 COEC compensation of one pruned module Input: original W0 , kept set K, calibration Gram G; temper α, tol δ, align weight λa f Output: Compensated retained weight W α 1: G ← G ; Gxx ← G[K, K]; Gyx ← W0 G[:, K] c ← W0 [:, K] 2: W (kept-column slice WK ) c ⊤ ); W c ← Qℓ W c 3: Qℓ ← polar(Gyx W c ← GCV-rescale(W c) 4: W 5: while round < 50 and |∆L| ≥ 10−4 do c ; Q0 =I, δ, λa ) 6: Qr ← ReducedStiefelSolve(W c c c c) 7: W ← W Qr ; W ← GCV-rescale(W c ⊤ ); W c ← Qℓ W c; W c ← 8: Qℓ ← polar(Gyx W c GCV-rescale(W ) 9: end while f←W c 10: return W trix. Numerical safeguards are described in App. B of the supplementary material.

5 5.1

Experiments

Experimental Setup

Models. We evaluate on Llama-3.1-8B and Llama-3-70B (Grattafiori et al. 2024) and on the Qwen2.5 family (7B, 14B, 32B, 72B) (Qwen et al. 2025), covering scales from 7B to 72B. Additional models are reported in the supplementary material under the same protocol. Pruning. We evaluate structured sparsity levels of 10%, 20%, and 30% in every transformer block, using the pruning scope described in Sec. 3. The main tables report results at 30% sparsity, where the effects of pruning and compensation are most pronounced. Results at 10% and 20% sparsity follow the same protocol and are provided in App. A of the supplementary material. Selection scores and baselines. Columns are ranked by an activation-aware score and the top ⌈(1 − ρ) din ⌉ are kept for pruning ratio ρ. We use three published scores under one implementation: the Wanda-sp score ∥W:,j ∥ · ∥Xj ∥, the WIFV score of FLAP, and the variance-aware score ∥W:,j ∥ · ∥Xj ∥ · Var(Xj ) of RCPU, which is the default selection for COEC. All methods share the calibration data and the evaluation protocol. Calibration and evaluation. We sample 128 sequences from the WikiText-2 training set (Merity et al. 2016). A single forward pass accumulates all Grams, and the compensation uses no evaluation data. We report WikiText-2 token-level perplexity and average zero-shot accuracy on seven tasks (BoolQ (Clark et al. 2019), RTE (Dagan, Glickman, and Magnini 2005), and WinoGrande (Sakaguchi et al. 2019) with accuracy; HellaSwag (Zellers et al. 2019), ARC-e, ARC-c (Clark et al. 2018), and OpenBookQA (Mihaylov et al. 2018) with normalized accuracy) via the LM Evaluation Harness (Gao et al. 2021).

Selection

Comp.

Llama-3.1-8B

Llama-3-70B

Qwen2.5-7B

Qwen2.5-14B

Qwen2.5-32B

Qwen2.5-72B

Wanda-sp

– COEC

14.35 10.89

6.20 5.61

13.03 9.09

15.15 13.26

13.90 9.64

6.02 5.17

FLAP

bias COEC

11.89 10.86

5.86 5.58

8.65 8.34

8.82 7.98

6.85 6.71

5.29 5.25

RCPU

RCPU COEC

11.57 10.91

5.69 5.58

8.98 8.91

7.98 7.63

7.03 6.74

5.23 5.21

Table 1: WikiText-2 perplexity (↓) at 30% column sparsity. Each selection score is paired with its native compensation and with COEC under the same selection score and calibration data. Bold marks the better compensation within each selection, per model. Results at 10% and 20% sparsity are in App. A (supplementary material). Selection

Comp.

Llama-3.1-8B

Llama-3-70B

Qwen2.5-7B

Qwen2.5-14B

Qwen2.5-32B

Qwen2.5-72B

Wanda-sp

– COEC

47.4 48.9

68.7 70.4

49.1 56.0

53.7 53.8

57.5 61.0

71.1 72.5

FLAP

bias COEC

44.8 47.1

68.8 71.7

52.6 54.5

55.9 59.5

65.6 65.3

70.5 72.5

RCPU

RCPU COEC

47.0 50.2

71.3 71.7

54.4 56.1

59.2 58.5

63.8 64.2

71.7 72.0

Table 2: 7-task zero-shot accuracy % (↑) at 30% column sparsity. Same protocol and bolding as Table 1. Results at 10% and 20% sparsity are in App. A (supplementary material).

5.2

Main Results

Tables 1 and 2 report the 30% setting. Applied on top of Wanda-sp, COEC reduces perplexity on every model, with the largest gain from 13.90 to 9.64 on Qwen2.5-32B, and matches or improves zero-shot accuracy throughout. Against the FLAP bias, COEC gives lower perplexity on all six models, recovering the Llama-3.1-8B from 11.89 to 10.86, and improves accuracy on five of the six; at 10–20% the bias remains competitive on perplexity (App. A of the supplementary material). Under the RCPU selection, COEC gives lower perplexity than the full RCPU compensation on all six models and higher zero-shot accuracy on five. The exception is Qwen2.5-14B, where RCPU is ahead by 0.7 points on accuracy while COEC keeps the perplexity lead. These three comparisons cover three different column-importance criteria (§5.1), and a single COEC configuration improves most settings under each without per-criterion tuning; the compensation can therefore be attached after existing pruning methods as a plug-in. The gains grow with sparsity (App. A of the supplementary material), as heavier pruning leaves more error for the compensation to recover. The supplementary material repeats the same three comparisons on additional models under the same protocol.

5.3

Configuration

PPL ↓

ZS ↑

Prune + RCPU compensation + GCV per-mode rescale + converged two-sided rotation + reduced-Stiefel right solve + Gram tempering (α=0.9) + anchored align (λa =50)

10.06 9.81 9.45 9.86 9.65 9.65

50.9 51.3 51.5 52.7 52.7 52.9

Table 3: Component-wise build-up of COEC (last row), averaged over five development models at 30% sparsity. All rows use the RCPU selection score; indented rows accumulate from top to bottom.

the calibration data and the scoring rule. COEC gives the lowest perplexity at every size and every ratio and improves monotonically with N . Even at N =32 it stays well below the prune-only baseline; at 30% sparsity, shrinking N from 256 to 32 costs about 0.6 PPL. RCPU has a smaller search space but is non-monotone in N . The stability of the larger family comes from its regularization rather than its size: GCV shrinks the per-mode rescale toward the uniform prior, and the rotations cannot reshape the singular spectrum to fit calibration noise. The main results use N =128, which lies on the flat part of every COEC curve.

Effect of Calibration Size

All COEC components are estimated from calibration statistics, and the two-sided family has more rotational freedom than the one-sided update. A natural concern is that this freedom overfits a small calibration set. Figure 2 tests this by varying the calibration size N ∈ {32, 64, 128, 256} on Llama-3.1-8B at all three sparsity levels; all methods share

5.4

Ablation Study

The ablation uses five 7–8B development models chosen for architectural diversity: Mistral-7B (Jiang et al. 2023), Llama-2-7B, Llama-3.1-8B, Qwen2.5-7B, and the distilled DeepSeek-R1-0528-Qwen3-8B (Guo et al. 2025). Two of them appear in the main evaluation, and all hyperparameter

Figure 2: Perplexity versus calibration size on Llama-3.1-8B under the RCPU selection score. The selection is recomputed at each N , so the prune-only baseline also varies. defaults were selected on this set and transferred unchanged to the larger models of Tables 1 and 2. Table 3 builds COEC up component by component at 30% sparsity. The first row is the full RCPU compensation, the strongest published baseline; the indented rows then add the COEC components cumulatively. The GCV rescale reduces perplexity from 10.06 to 9.81, and the converged two-sided rotation further to 9.45, supplying the input-side adjustment that the one-sided rotation cannot make. The reduced-Stiefel right solve trades perplexity for zero-shot accuracy (51.5 → 52.7). Gram tempering then recovers part of the perplexity (9.86 → 9.65) at unchanged accuracy, and the anchored alignment penalty adds a final accuracy gain, leaving the full method at 9.65 perplexity and 52.9 accuracy. Overall, the GCV rescale and the two-sided rotation account for the perplexity reduction, the reduced-Stiefel solve contributes most of the zero-shot gain at some perplexity cost, and Gram tempering recovers most of that cost.

5.5

Efficiency Analysis

Table 4 reports the deployment cost at 30% column sparsity. Structured pruning yields a smaller dense model that runs without any sparse-kernel support: parameters, peak inference memory, and dense FLOPs per token fall by 23–29%, and the reduction grows with model size; realized wall-clock speedup additionally depends on how well the kernels handle the pruned matrix shapes, so we report these architecturelevel quantities. COEC rotates and rescales the kept weights in place without changing the model shape, so the deployment cost depends only on the pruning ratio and not on the compensation; the recovery reported in Tables 1 and 2 therefore comes at no additional cost. Compensating a 7–8B model takes 45–86 GPU-minutes on a single B300; the alignment penalty adds only 0.5% overhead, and GCV rescaling is closed-form.

5.6

Geometric Analysis

A weight matrix reads certain input directions and writes certain output directions. App. C of the supplementary material

Params (B)

Mem (GB)

GFLOPs/tok

Model

dense 30% dense 30% dense

Qwen2.5-7B Llama-3-70B

7.62 70.6

30%

5.70 15.0 11.5 15.2 11.4 50.6 132.5 95.7 141.1 101.2

Table 4: Deployment cost at 30% column sparsity. The compensation does not change the model shape, so the figures are identical for prune-only, RCPU, and COEC.

measures how pruning and compensation change these directions on DeepSeek-R1-0528-Qwen3-8B at 30% sparsity. Two observations motivate the design of COEC. First, deleting columns rotates the input directions of every layer by about 22◦ on average. Compensation that only minimizes reconstruction error leaves this rotation in place. The anchored penalty of §4.5 targets this residual and reduces the measured mismatch by a median factor of 7.2. Second, COEC rotates the weight as a whole, so its input and output directions move together. Their pairing within each layer stays intact (discrepancy of 1.1–1.2%) without any explicit term.

6

Conclusion

We presented COEC, a training-free compensation method for structured pruning of large language models. Instead of refitting the retained weights freely, COEC rotates them on both sides and rescales their singular values, so the update can re-route the signal lost with the removed columns while keeping the overall structure of the pretrained weight intact. The strength of the rescaling is chosen automatically from calibration statistics, and a light penalty keeps the pruned layer geometrically consistent with its neighbor. The whole procedure uses a small calibration set, requires no backpropagation through the LLM or retraining of the model parameters, and compensates a 7–8B model in under 1.5 GPU-hours without changing the model architecture.

References Absil, P.-A.; Mahony, R.; and Sepulchre, R. 2008. Bibliography, 201–220. Princeton University Press. ISBN 9780691132983. An, Y.; Zhao, X.; Yu, T.; Tang, M.; and Wang, J. 2023. Fluctuation-based Adaptive Structured Pruning for Large Language Models. arXiv:2312.11983. Ashkboos, S.; Croci, M. L.; do Nascimento, M. G.; Hoefler, T.; and Hensman, J. 2024a. SliceGPT: Compress Large Language Models by Deleting Rows and Columns. arXiv:2401.15024. Ashkboos, S.; Mohtashami, A.; Croci, M. L.; Li, B.; Cameron, P.; Jaggi, M.; Alistarh, D.; Hoefler, T.; and Hensman, J. 2024b. QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs. arXiv:2404.00456. Clark, C.; Lee, K.; Chang, M.-W.; Kwiatkowski, T.; Collins, M.; and Toutanova, K. 2019. BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions. arXiv:1905.10044. Clark, P.; Cowhey, I.; Etzioni, O.; Khot, T.; Sabharwal, A.; Schoenick, C.; and Tafjord, O. 2018. Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge. arXiv:1803.05457. Dagan, I.; Glickman, O.; and Magnini, B. 2005. The PASCAL Recognising Textual Entailment Challenge, 177–190. Springer. ISBN 978-3-540-33427-9. Edelman, A.; Arias, T. A.; and Smith, S. T. 1998. The Geometry of Algorithms with Orthogonality Constraints. arXiv:physics/9806030. Frankle, J.; and Carbin, M. 2019. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. arXiv:1803.03635. Frantar, E.; and Alistarh, D. 2023. SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot. arXiv:2301.00774. Frantar, E.; Ashkboos, S.; Hoefler, T.; and Alistarh, D. 2023. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. arXiv:2210.17323. Gao, L.; Tow, J.; Biderman, S.; Black, S.; DiPofi, A.; Foster, C.; Golding, L.; Hsu, J.; McDonell, K.; Muennighoff, N.; Phang, J.; Reynolds, L.; Tang, E.; Thite, A.; Wang, B.; Wang, K.; and Zou, A. 2021. A framework for few-shot language model evaluation. Golub, G. H.; Heath, M.; and Wahba, G. 1979. Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter. Technometrics, 21(2): 215–223. Gower, J. C.; and Dijksterhuis, G. B. 2004. Procrustes Problems. Oxford University Press. ISBN 9780198510581. Grattafiori, A.; et al. 2024. The Llama 3 Herd of Models. arXiv:2407.21783. Gu, T.; Liu, B.; Xiao, B.; Zeng, K.; Liu, J.; and Qian, Y. 2025. DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration. arXiv:2505.23049. Gu, Y.; Dong, L.; Wei, F.; and Huang, M. 2026. MiniLLM: On-Policy Distillation of Large Language Models. arXiv:2306.08543.

Guo, D.; et al. 2025. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature, 645(8081): 633–638. Haruta, S.; Matsumoto, K.; Li, Z.; Wang, Y.; and Kurokawa, M. 2026. RCPU: Rotation-Constrained Error Compensation for Structured Pruning of Large Language Models. arXiv:2510.07782. Hassibi, B.; and Stork, D. 1992. Second order derivatives for network pruning: Optimal Brain Surgeon. In Hanson, S.; Cowan, J.; and Giles, C., eds., Advances in Neural Information Processing Systems, volume 5. Morgan-Kaufmann. Hinton, G.; Vinyals, O.; and Dean, J. 2015. Distilling the Knowledge in a Neural Network. arXiv:1503.02531. Jiang, A. Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D. S.; de las Casas, D.; Bressand, F.; Lengyel, G.; Lample, G.; Saulnier, L.; Lavaud, L. R.; Lachaux, M.-A.; Stock, P.; Scao, T. L.; Lavril, T.; Wang, T.; Lacroix, T.; and Sayed, W. E. 2023. Mistral 7B. arXiv:2310.06825. Kurtic, E.; Frantar, E.; and Alistarh, D. 2023. ZipLM: Inference-Aware Structured Pruning of Language Models. arXiv:2302.04089. LeCun, Y.; Denker, J.; and Solla, S. 1989. Optimal Brain Damage. In Touretzky, D., ed., Advances in Neural Information Processing Systems, volume 2. Morgan-Kaufmann. Lin, J.; Tang, J.; Tang, H.; Yang, S.; Chen, W.-M.; Wang, W.C.; Xiao, G.; Dang, X.; Gan, C.; and Han, S. 2026. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration. arXiv:2306.00978. Liu, Z.; Wang, W.; Wang, J.; Wang, P.; Sui, X.; Ruan, C.; Ling, N.; and Jiang, W. 2026. Geometric and Spectral Alignment for Deep Neural Network I. arXiv:2605.02108. Liu, Z.; Zhao, C.; Fedorov, I.; Soran, B.; Choudhary, D.; Krishnamoorthi, R.; Chandra, V.; Tian, Y.; and Blankevoort, T. 2025. SpinQuant: LLM quantization with learned rotations. arXiv:2405.16406. Ma, X.; Fang, G.; and Wang, X. 2023. LLM-Pruner: On the Structural Pruning of Large Language Models. arXiv:2305.11627. Men, X.; Xu, M.; Zhang, Q.; Wang, B.; Lin, H.; Lu, Y.; Han, X.; and Chen, W. 2024. ShortGPT: Layers in Large Language Models are More Redundant Than You Expect. arXiv:2403.03853. Merity, S.; Xiong, C.; Bradbury, J.; and Socher, R. 2016. Pointer Sentinel Mixture Models. arXiv:1609.07843. Mihaylov, T.; Clark, P.; Khot, T.; and Sabharwal, A. 2018. Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering. arXiv:1809.02789. Qwen; :; Yang, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Li, C.; Liu, D.; Huang, F.; Wei, H.; Lin, H.; Yang, J.; Tu, J.; Zhang, J.; Yang, J.; Yang, J.; Zhou, J.; Lin, J.; Dang, K.; Lu, K.; Bao, K.; Yang, K.; Yu, L.; Li, M.; Xue, M.; Zhang, P.; Zhu, Q.; Men, R.; Lin, R.; Li, T.; Tang, T.; Xia, T.; Ren, X.; Ren, X.; Fan, Y.; Su, Y.; Zhang, Y.; Wan, Y.; Liu, Y.; Cui, Z.; Zhang, Z.; and Qiu, Z. 2025. Qwen2.5 Technical Report. arXiv:2412.15115.

Sakaguchi, K.; Bras, R. L.; Bhagavatula, C.; and Choi, Y. 2019. WinoGrande: An Adversarial Winograd Schema Challenge at Scale. arXiv:1907.10641. Sanh, V.; Wolf, T.; and Rush, A. M. 2020. Movement Pruning: Adaptive Sparsity by Fine-Tuning. arXiv:2005.07683. Schönemann, P. H. 1966. A Generalized Solution of the Orthogonal Procrustes Problem. Psychometrika, 31(1): 1–10. Song, J.; Oh, K.; Kim, T.; Kim, H.; Kim, Y.; and Kim, J.-J. 2024. SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks. arXiv:2402.09025. Sun, M.; Liu, Z.; Bair, A.; and Kolter, J. Z. 2024. A Simple and Effective Pruning Approach for Large Language Models. arXiv:2306.11695. van der Ouderaa, T. F. A.; Nagel, M.; van Baalen, M.; Asano, Y. M.; and Blankevoort, T. 2024. The LLM Surgeon. arXiv:2312.17244. Wahba, P., G.and Craven. 1978/79. Smoothing Noisy Data with Spline Functions. Estimating the Correct Degree of Smoothing by the Method of Generalized Cross-Validation. Numerische Mathematik, 31: 377–404. Wang, X.; Zheng, Y.; Wan, Z.; and Zhang, M. 2025. SVDLLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression. arXiv:2403.07378. Wen, Z.; and Yin, W. 2012. A feasible method for optimization with orthogonality constraints. Mathematical Programming, 142: 397 – 434. Xia, M.; Gao, T.; Zeng, Z.; and Chen, D. 2024. Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning. arXiv:2310.06694. Xiao, G.; Lin, J.; Seznec, M.; Wu, H.; Demouth, J.; and Han, S. 2024. SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models. arXiv:2211.10438. Yang, Y.; Cao, Z.; and Zhao, H. 2024. LaCo: Large Language Model Pruning via Layer Collapse. arXiv:2402.11187. Yuan, Z.; Shang, Y.; Song, Y.; Yang, D.; Wu, Q.; Yan, Y.; and Sun, G. 2025. ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models. arXiv:2312.05821. Zellers, R.; Holtzman, A.; Bisk, Y.; Farhadi, A.; and Choi, Y. 2019. HellaSwag: Can a Machine Really Finish Your Sentence? arXiv:1905.07830.

A

Results at 10% and 20% Column Sparsity

Tables 1 and 2 report the 10% and 20% settings under the protocol of §5.1 of the main paper; the 30% setting is in Tables 1 and 2 of the main paper. Under the Wanda-sp selection, COEC lowers perplexity on all six models at both ratios. Under the RCPU selection it lowers perplexity on five of six models at 10% (matching on the sixth) and on all six at 20%, by margins that grow with sparsity: the mean gap over RCPU is 0.06 at 10%, 0.20 at 20%, and 0.25 at 30%. Under the FLAP selection the bias compensation is ahead on perplexity on five of six models at both ratios, and the two split evenly on zero-shot accuracy; that advantage is gone at 30%, where COEC leads on all six models (Table 1 of the main paper). A mean correction absorbs the small output shift that mild pruning produces, but not the directional error that dominates once a substantial fraction of the columns is removed, which is also the regime worth compensating: removing 10% of the columns takes correspondingly little off the inference cost to begin with. Selection

Comp.

Llama-3.1-8B

Llama-3-70B

Qwen2.5-7B

Qwen2.5-14B

Qwen2.5-32B

Qwen2.5-72B

10% column sparsity Wanda-sp

– COEC

7.32 6.86

3.87 3.72

6.87 6.78

7.04 6.06

6.33 5.43

4.17 4.09

FLAP

bias COEC

6.89 7.41

3.50 3.71

6.77 6.82

6.12 6.07

5.24 5.41

4.01 4.09

RCPU

RCPU COEC

6.93 6.93

3.75 3.72

6.77 6.74

6.20 6.01

5.58 5.50

4.07 4.06

20% column sparsity Wanda-sp

– COEC

10.74 8.95

5.59 4.93

11.49 8.35

13.76 11.27

12.56 9.20

5.46 4.73

FLAP

bias COEC

8.63 9.96

4.72 4.87

7.57 8.36

7.26 7.18

6.05 6.22

4.65 4.76

RCPU

RCPU COEC

9.19 8.97

5.05 4.89

8.33 8.25

7.51 6.97

6.33 6.25

4.85 4.76

Table 1: WikiText-2 perplexity (↓) at 10% and 20% column sparsity. Protocol and bolding as Table 1 of the main paper.

B

Implementation Details

Retractions. For rectangular Stiefel frames (b < k, the MLP case) we use a Newton–Schulz polar retraction (Higham 2008). For square frames (b = k, attention output projections) we use a Cayley retraction (Wen and Yin 2012): the Cayley transform of the skew part of the projected gradient, with the matrix inverse computed by Newton–Schulz iteration in low precision on the GPU. GCV grid. The ridge λ of §4.3 of the main paper is minimized over a 120-point logarithmic grid spanning [10−4 , 105 ], scaled by the median mode energy ei so that the grid is dimensionless. Each grid evaluation is closed form. The resulting effective ridge reproduces the best globally swept value on both models where we checked it: 0.79 versus 0.75 on Llama-3, and 0.00 versus 0 on the distilled Qwen. Per-mode dose band. On near-degenerate spectra the per-mode least-squares value can fall far from the uniform one (isolated tail values up to ∼104 × were measured), which under the iterated frames inflates the largest singular value and destabilizes a This is an anonymized submission for review purposes only. Distribution, citation, or public sharing of this manuscript is strictly prohibited. Copyright and publication details will appear in the final version if accepted.

Selection

Comp.

Llama-3.1-8B

Llama-3-70B

Qwen2.5-7B

Qwen2.5-14B

Qwen2.5-32B

Qwen2.5-72B

10% column sparsity Wanda-sp

– COEC

61.4 63.6

73.4 73.5

69.3 69.7

68.4 69.1

68.3 70.0

73.3 74.3

FLAP

bias COEC

59.9 60.7

73.0 74.0

67.8 70.0

69.0 67.5

72.1 71.2

74.8 74.5

RCPU

RCPU COEC

62.6 62.9

74.0 73.9

69.1 69.2

65.3 65.9

71.2 71.3

74.5 74.4

Wanda-sp

– COEC

52.6 52.2

70.8 71.8

56.7 59.9

57.1 56.8

57.8 64.2

71.3 73.3

FLAP

bias COEC

53.4 48.4

71.6 73.3

59.1 59.2

65.7 63.6

70.0 68.0

72.8 72.9

RCPU

RCPU COEC

52.6 52.8

72.6 73.4

58.1 59.6

62.5 62.5

66.5 66.2

71.4 72.3

20% column sparsity

Table 2: 7-task zero-shot accuracy % (↑) at 10% and 20% column sparsity. Protocol and bolding as Table 1 of the main paper. later SVD. Every sg,i is therefore clamped to within a factor κ = 10 of the uniform value s∗ σi , in all rescale branches including GCV. The healthy per-mode deviation is about 2%, so the band is inactive on well-conditioned modes. Numerical safeguards. The scale update is applied only when its denominator is positive: on some Qwen attention layers the calibration covariance is indefinite (e.g., ⟨R, S 2 RGxx ⟩ = −3.9 · 108 ), and a clamped denominator produced scales on the order of 1037 . The retraction step is capped so that ∥step · G∥ ≈ 1, since degenerate layers can otherwise produce steps on the order of 1036 . An SVD watchdog converts cuSOLVER hangs on degenerate slices into recoverable failures through a timed fallback chain from gesdd to gesvd to CPU double precision. Hyperparameters. All reported results use one configuration: α = 0.9 (swept over [0.3, 1.0]); λa = 50 on the attention output projection and 0 on the MLP (swept over {0, 10, 50, 100, 500, 1000}); rp = 16; δ = 7·10−4 (swept over {10−2 , 10−3 , 7·10−4 }); at most 50 alternation rounds with early stop at 10−4 relative change, where raising the cap to 100 moves every development model by at most 0.02 PPL; κ = 10. Each value was chosen by best mean combined gain over RCPU on the five development models of §5.4 of the main paper. Calibration uses 128 WikiText-2 training sequences of length 2048; perplexity is evaluated at sequence length 4096. Randomness. The compensation is deterministic given the calibration Grams: the left rotation and the rescale are closed form, and the right solve starts from Qr = I. The only stochastic component is the sampling of the 128 calibration sequences, drawn with a fixed seed (0, also set for NumPy and PyTorch). Across three seeds on Llama-3.1-8B the per-cell perplexity spread is a median of 0.08 (maximum 0.89) for COEC, against 0.23 for RCPU and 0.09 for score-only pruning. Compute environment. All reported cells ran on NVIDIA B300 nodes (SXM6, ∼275 GB; Intel Xeon 6737P, 3 TB RAM); NVIDIA L40S nodes (46 GB; AMD EPYC 7513, 512 GB RAM) were used for diagnostics only. Software: Ubuntu 22.04, Python 3.9, PyTorch 2.8.0 (CUDA 12.9), transformers 4.57.6, lm-eval 0.4.9.1. A same-configuration cross-check of Qwen2.5-7B at 30% sparsity on L40S reproduced the B300 perplexity to within 0.001 but shifted zero-shot accuracy by 0.55 points, so the L40S diagnostics are restricted to reconstruction-side quantities.

C

Geometric Measurements

All measurements are taken on DeepSeek-R1-0528-Qwen3-8B at 30% sparsity (36 layers, Wanda-sp selection). We write O for the original read frame restricted to the kept coordinates, P for the right singular frame of the pruned weight, and C for the frame after full COEC compensation. Angles are principal angles between top-rp frames; alignment values use the projector of §4.5 of the main paper. The read–write alignment Vo⊤ Uv is the inter-layer quantity of the WSBM framework (Liu et al. 2026), whose spectral properties the family of §3 of the main paper preserves by construction, leaving the frames as the open question. Effect of pruning. The angle between O and P averages 22.2◦ on attention output projections (at rp = 16; per-module values span 8–48◦ ) and 22.6◦ on MLP down projections (at r = 64; span 2–32◦ ). The 20–30◦ figure quoted in §4.5 of the main paper refers to these module means. Effect of compensation. After full compensation the mean angle between O and C is 23.8◦ for attention and 23.0◦ for MLP, larger than the corresponding O–P angle on 54 of the 72 module-layers. The reconstruction error makes no reference to the original frame, so nothing pulls the frame back toward it. Intra-layer alignment. The update Qℓ WK Qr maps UK → Qℓ UK and VK → Q⊤ r VK , so the two frames move together and only the per-mode rescale could break their pairing. It does not: the frame-change discrepancy |Qu − Qv |/|Qu |, with Qu = UP⊤ UC

and Qv = VP⊤ VC , averages 1.1–1.2% at rp = 16 with a maximum of 3.9%. Choice of metric. A layer can show a mean O–P rotation of 22◦ (maximum 87◦ ) while its projector penalty is only about 5 · 10−7 (median; 10−4 at worst), because for any O ∈ O(rp ) the projector satisfies Πrp (Vrp O) = Vrp OO⊤ Vr⊤p = Πrp (Vrp ) and is blind to rotation inside the retained subspace, whereas principal angles are not. An S 2 -weighted variant gives values of the same order. One caveat: the rank cut is itself near-degenerate in these models (σrp /σrp +1 ≈ 1.00 at rp = 16 and 64), so rp should be placed at a spectral gap where one exists. Effect of the penalty. The penalty at the pruned point is nonzero (10−8 –10−4 by layer). At λa = 1 it barely moves; at the default λa = 50 it falls by a median factor of 7.2 (up to 975) on the attention projections. The O–C angles do not decrease and remain at 22–24◦ : the remaining rotation lies in the within-subspace component the projector ignores, and closing it would require an angle-aware objective.

D

Additional Model Families: Llama-2 and Qwen3

Tables 3 and 4 repeat the three comparisons of §5.2 of the main paper on the Llama-2 (7B/13B/70B) (Touvron et al. 2023) and Qwen3 (8B/14B/32B) (Yang et al. 2025) families at 10%/20%/30% column sparsity, under the identical protocol. Under Wanda-sp, COEC lowers perplexity on all six models at 10% and 20% and on five of six at 30%; the largest recovery is Qwen3-32B at 30%, from 11.86 to 8.94 perplexity and +6.6 zero-shot points. Under FLAP, COEC lowers perplexity on every model at all three ratios, including the 10–20% range where the bias compensation leads on the main-text models (App. A). Under RCPU the two are closest: COEC leads on five of six models at 10%, four at 20%, and three at 30%, with its gains concentrated on the Qwen3 models. Selection

Comp.

Llama-2-7B

Llama-2-13B

Llama-2-70B

Qwen3-8B

Qwen3-14B

Qwen3-32B

10% column sparsity Wanda-sp

– COEC

5.65 5.43

5.10 4.85

3.45 3.38

9.89 9.68

8.99 8.51

7.62 7.47

FLAP

bias COEC

5.64 5.54

5.05 4.99

3.28 3.26

10.02 9.90

8.48 8.37

7.27 7.22

RCPU

RCPU COEC

5.43 5.42

4.87 4.86

3.38 3.37

9.97 9.82

8.51 8.57

7.52 7.51

Wanda-sp

– COEC

6.72 6.43

5.58 5.22

3.94 3.76

12.75 11.32

11.28 9.68

10.13 8.53

FLAP

bias COEC

6.71 6.40

5.80 5.63

3.64 3.60

12.04 11.34

9.62 9.27

7.74 7.62

RCPU

RCPU COEC

6.02 5.95

5.44 5.46

3.78 3.76

11.20 11.12

9.53 9.80

8.47 8.25

20% column sparsity

30% column sparsity Wanda-sp

– COEC

8.88 8.34

7.44 7.50

4.32 4.14

14.56 12.18

12.95 10.60

11.86 8.94

FLAP

bias COEC

8.53 7.81

6.83 6.50

4.13 4.06

14.78 12.95

10.94 10.24

8.66 8.37

RCPU

RCPU COEC

7.18 7.21

6.54 6.67

4.17 4.16

12.23 11.94

10.16 10.20

9.32 8.77

Table 3: WikiText-2 perplexity (↓) on the Llama-2 and Qwen3 families across 10–30% column sparsity. Same protocol and bolding as Table 1 of the main paper.

Selection

Comp.

Llama-2-7B

Llama-2-13B

Llama-2-70B

Qwen3-8B

Qwen3-14B

Qwen3-32B

10% column sparsity Wanda-sp

– COEC

62.3 63.0

67.3 67.8

71.6 72.1

64.7 66.7

70.3 71.2

71.5 71.3

FLAP

bias COEC

58.8 58.6

63.0 62.8

71.3 71.0

63.3 63.4

68.9 69.1

59.0 72.2

RCPU

RCPU COEC

62.5 62.4

67.4 67.1

71.3 71.9

67.8 67.9

70.8 72.2

71.4 71.7

Wanda-sp

– COEC

58.7 59.6

64.8 64.8

71.6 71.4

54.8 55.9

62.6 65.4

61.1 68.2

FLAP

bias COEC

53.1 53.0

59.0 58.6

71.0 70.7

55.4 56.4

64.0 64.4

56.2 68.8

RCPU

RCPU COEC

59.6 60.0

64.3 64.5

71.4 71.2

58.6 58.2

63.6 63.8

66.1 67.6

20% column sparsity

30% column sparsity Wanda-sp

– COEC

52.0 52.2

55.0 48.8

69.6 69.6

48.8 50.2

55.2 57.8

59.5 66.1

FLAP

bias COEC

51.4 51.8

55.7 55.1

69.2 68.9

48.8 50.5

56.2 57.2

55.5 65.3

RCPU

RCPU COEC

53.0 53.5

57.9 57.7

70.6 70.7

53.7 54.4

55.0 55.5

63.3 64.7

Table 4: 7-task zero-shot accuracy % (↑) on the Llama-2 and Qwen3 families across 10–30% column sparsity. Same protocol and bolding as Table 1 of the main paper.

References Higham, N. J. 2008. Functions of Matrices. Society for Industrial and Applied Mathematics. Liu, Z.; Wang, W.; Wang, J.; Wang, P.; Sui, X.; Ruan, C.; Ling, N.; and Jiang, W. 2026. Geometric and Spectral Alignment for Deep Neural Network I. arXiv:2605.02108. Touvron, H.; et al. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv:2307.09288. Wen, Z.; and Yin, W. 2012. A feasible method for optimization with orthogonality constraints. Mathematical Programming, 142: 397 – 434. Yang, A.; Li, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Gao, C.; Huang, C.; Lv, C.; Zheng, C.; Liu, D.; Zhou, F.; Huang, F.; Hu, F.; Ge, H.; Wei, H.; Lin, H.; Tang, J.; Yang, J.; Tu, J.; Zhang, J.; Yang, J.; Yang, J.; Zhou, J.; Zhou, J.; Lin, J.; Dang, K.; Bao, K.; Yang, K.; Yu, L.; Deng, L.; Li, M.; Xue, M.; Li, M.; Zhang, P.; Wang, P.; Zhu, Q.; Men, R.; Gao, R.; Liu, S.; Luo, S.; Li, T.; Tang, T.; Yin, W.; Ren, X.; Wang, X.; Zhang, X.; Ren, X.; Fan, Y.; Su, Y.; Zhang, Y.; Zhang, Y.; Wan, Y.; Liu, Y.; Wang, Z.; Cui, Z.; Zhang, Z.; Zhou, Z.; and Qiu, Z. 2025. Qwen3 Technical Report. arXiv:2505.09388.

Related documents

• COEC: Calibrated Orthogonal-Equivalence Compensation for Structured Pruning of Large Language Models#662308
• COEC: Calibrated Orthogonal-Equivalence Compensation for Structured Pruning of Large Language Models#661258
• CORDIAL: Calibrating Ordinal LLM Outputs from Few Labels#1077734
• CALIBURN: Self-Calibrated LLM Unlearning Alignment#821781
• LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry#998715